engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

qa(retrieval): mind/brain-tech synonym group for intent-question queries

Browse source 
Closes the 2026-05-01 retrieval gap fox surfaced on:

  Q: what technology are currently or soon available which may enable
     one person to reconstruct and understand some or a portion of
     another persons thoughts or ideas without speaking or sign language?

  Pre-fix: top-K = [Videoconferencing, Telepathy, Identity, PKM,
           Intrapersonal]; only Videoconferencing used (BM25 won on
           "person/language/speak" token density). Verifier returned
           EVIDENCE-WARRANTED-PARTIAL 2/5 with Videoconferencing
           cited as evidence for "currently no technology...". Telepathy
           was retrieved but unused — title-relevance filter dropped
           it to body-density accept path with a low score.

  Post-fix: same top-K but Telepathy now SURVIVES title-relevance
            (its title token "telepathy" is in the accept set via
            synonym expansion). Title-token-boost lifts it.
            EVIDENCE-WARRANTED-PARTIAL 6/10 with Telepathy E4 now
            cited alongside Videoconferencing — sources used 5/5.

Adds a 7th SYNONYM_GROUPS entry mixing concept tokens (thoughts,
mind, cognition, consciousness) with specialized entity tokens
(telepathy, neurotechnology, BCI, fMRI, EEG, TMS, transcranial).
A query naming either side expands to the whole family, so the
title-relevance accept path admits brain-tech pages even when the
question is phrased in lay vocabulary.

Limitation accepted: this only affects title-relevance filtering &
title-token rerank, not the FTS5 BM25 retrieval that produces the
top-K. Pages whose BM25 score is below the top-K cutoff (e.g.
Neurotechnology at #4 in fox's manual-hint Run 2) still won't
surface unless FTS5-side synonym expansion is added too. Tracked
as a follow-on ticket — for now, query-time retrieval hints
("transcranial knowledge acquisition") continue to be the manual
escape hatch when intent vocabulary doesn't lexically anchor.

Tests: 624 passed (no regression).
This commit is contained in:
russell@unturf.com 2026-05-01 20:55:23 -04:00
parent 00528456ff
commit bde1bd61f4
No known key found for this signature in database
1 changed files with 19 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

19
aborist/qa/concepts.py
View file

@ -47,6 +47,25 @@ SYNONYM_GROUPS: list[frozenset[str]] = [
frozenset({"windows", "microsoft", "win32", "winnt", "win9x"}),
# Linux family
frozenset({"linux", "gnu", "ubuntu", "debian", "fedora", "redhat", "kernel"}),
# Mind-reading / brain-computer-interface family. Closes the
# 2026-05-01 intent-question gap where "what technology can
# reconstruct another person's thoughts" pulled Videoconferencing
# via "person/language/speak" BM25 density and left Telepathy /
# Neurotechnology pages unused at #2 / #4. The concept tokens
# (thoughts, mind, cognition) sit alongside the specialized
# entity tokens (telepathy, BCI, fMRI, TMS) so a query naming
# either side expands to the whole family. Title-token-boost
# in `_rerank_by_title` then promotes the brain-tech pages above
# the lexical-decoy pages whose body shares only generic verbs.
frozenset({
"telepathy", "telepathic", "neurotechnology", "neuroscience",
"neuroimaging", "neural", "neuron", "neurons",
"brain", "brains", "mind", "minds", "cognition", "cognitive",
"consciousness", "thoughts", "thinking", "thought",
"psychic", "psychokinesis", "esp", "clairvoyance",
"fmri", "eeg", "fnirs", "meg", "ecog",
"tms", "transcranial", "bci",
}),
]