#000054 Phase 2: synonym_expand cap-aware + strict view + retrieval-route surfacing
End-to-end gap-close from the Phase 1 extractor. Five interlocking fixes; live-verified that `what is a CPU?` → "Central processing unit" at #1, `what is a GPU?` → "Graphics processing unit" at #1 EVIDENCE-WARRANTED 1/1; Mount Kilimanjaro / Soviet Union queries unchanged (no regression). (a) `synonym_expand` over-cap path is now rank-and-truncate by source-frequency (descending) instead of hard-skip. CPU has 13 legitimate homonym expansions across the corpus; the prior MAX_NEIGHBORS_PER_TOKEN=8 cap contributed *zero* expansion → no canonical-article surfacing. Now: keep the 8 dominant by per-(token, target) source-root count via _load_neighbor_source_freq. (b) `_search_titles` orders by FTS5 bm25 ASC instead of LENGTH(title) ASC on the FTS5-MATCH path. The length-asc tie-break was correct for the 2026-05-02 "Back to the Future" LIKE-substring case but counter-productive on FTS5 (tokenized; no substring junk; length-asc preferred "Unit" / "Unite" / "B unit" over "Graphics processing unit"). LIKE fallback keeps length-asc since the substring issue persists there. (c) `accept_tokens` (feeds title-search, core-keyword, title-rerank) uses the synonym-expanded set instead of qtokens-only. The Phase 1 expansion existed but was only used in the FTS5 OR-fallback; satellite articles saturated the budget before the canonical article entered. (d) ARCHITECTURAL: `synonym_expand_strict()` (new — high-trust evidence-kind subset: manual + manual_legacy + acronym_parens, **excludes** link_reciprocity) for use in the multiplicative `_rerank_by_title_purity` and as the source for `accept_tokens`. Reciprocal-wikilink edges express *topical adjacency*, not synonymy (a `Dinosaurs` page reciprocally links to `Curious George Brigade` → edge that should not amplify retrieval); a multiplicative ranker over them blows up. Strict view preserves the acronym-parens surfacing (those edges ARE the phrase=expansion identity) while keeping link-reciprocity to additive retrieval-route boosts via the broad synonym_expand (still wired to `or_synonym_pool` for the FTS5 OR fallback). (e) Extractor regex tightened `[A-Z]{2,6}` → `[A-Z]{3,6}` and purged ~21K 2-letter acronym edges from shards. 2-letter acronyms (AI/ML/OS/ US/UK/IT/PC/TV) homonym-collide too often with common 2-letter QUERY tokens like `go`/`is`/`am` — without this, "why did the dinosaurs go extinct?" pulled Curious George Brigade via GO-acronym edges. The high-value acronyms (CPU/GPU/RAM/DNA/FBI/WHO/…) all clear 3 chars. Also: CLAUDE.md gains a "prefer existing ticket; only split for Dav1d-review audience" discipline note (saved as feedback memory) — this work is itself an example: would have been #000055 + #000056 + #000057 under the prior pattern; instead extends #000054. Suite: 2531 passed (no regression). bench-qa in flight separately.
This commit is contained in:
parent
da5af2dfaa
commit
ce855dbdad
7 changed files with 265 additions and 21 deletions
23
CLAUDE.md
23
CLAUDE.md
|
|
@ -521,6 +521,29 @@ flip Status to `closed · landed in commit <sha>` (or
|
|||
the same commit as the implementation.** An open ticket whose code
|
||||
already shipped is a stale map.
|
||||
|
||||
**Default: extend an existing ticket. Don't proliferate.** When
|
||||
follow-up work surfaces during implementation (a consumer-side fix
|
||||
the new code needs to actually take effect; a bench-gated tuning
|
||||
step; a small downstream tweak), **add it to the open / most-related
|
||||
ticket**: reopen `closed` → `in progress` if the prior closure was
|
||||
premature, extend the scope, list the new sub-items, keep the design
|
||||
log linear. fox tracks the index by skimming a small set of threads;
|
||||
spawning #000054.1 / #000055 / #000056 / … for every follow-up turns
|
||||
the index into a wall of micro-tickets he can't easily keep state on,
|
||||
and dilutes the design log instead of concentrating it. **Split only
|
||||
when the new piece needs a *distinct audience*** — specifically, when
|
||||
it's a self-contained design decision a Dav1d de-novo review (the
|
||||
external code-review thread that runs against one ticket at a time)
|
||||
needs to read independently. Architectural inflection points, large
|
||||
scope changes, or fundamental discipline questions (NLI may touch
|
||||
`audit_mode`? — #000049 split from #000048) cross that bar. "The
|
||||
extractor needs a small consumer-side tweak" does not. When in doubt,
|
||||
ask before opening a new ticket — a two-sentence "fold into #X or
|
||||
spawn a sibling?" is cheap; an unwanted ticket is friction. Closing
|
||||
on the implementation commit is good practice **only** when the work
|
||||
is actually complete end-to-end; pattern-match closure on the
|
||||
user-visible outcome, not on the commit.
|
||||
|
||||
## Orientation protocol
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -157,9 +157,17 @@ def link_reciprocity_synonym(
|
|||
# in order, to the first letter of one content word in the phrase
|
||||
# (function words filtered). Drops HTTP-shape acronyms where letters
|
||||
# land mid-word — the catch isn't worth the false-positive risk.
|
||||
# Acronym length floor is 3 ([A-Z]{3,6}, not {2,6}): 2-letter acronyms
|
||||
# (AI/ML/OS/US/UK/IT/PC/TV) collide too often with common 2-letter
|
||||
# QUERY tokens like "go"/"is"/"am"/"or" — the expansion then pulls
|
||||
# unrelated articles into retrieval (2026-05-13 bench regression: "why
|
||||
# did the dinosaurs go extinct?" → Curious George Brigade via GO-acronym
|
||||
# edges). The high-value acronyms (CPU/GPU/RAM/DNA/FBI/WHO/…) all
|
||||
# clear 3 chars; the 2-letter loss is negligible (their canonical
|
||||
# expansion words appear in body text directly).
|
||||
_ACRONYM_PHRASE_RE = re.compile(
|
||||
r"\b([A-Za-z][A-Za-z\-]+(?:\s+[A-Za-z][A-Za-z\-]+){1,6})\s*"
|
||||
r"\(\s*([A-Z]{2,6})\s*\)"
|
||||
r"\(\s*([A-Z]{3,6})\s*\)"
|
||||
)
|
||||
|
||||
# Function words that appear inside acronym expansions but never
|
||||
|
|
|
|||
|
|
@ -322,6 +322,43 @@ def _load_idf_for(shards_dir: Path, tokens: set[str]) -> dict[str, int]:
|
|||
return {(r["token"] or "").lower(): int(r["df"] or 0) for r in rows}
|
||||
|
||||
|
||||
def _load_neighbor_source_freq(
|
||||
shards_dir: Path, token: str, neighbors: set[str]
|
||||
) -> dict[str, int]:
|
||||
"""For one ``token`` whose derived-synonym degree exceeds the
|
||||
per-token cap, return ``{neighbor: source_root_count}`` — how many
|
||||
distinct documents asserted the (token, neighbor) synonym
|
||||
relation. Used to rank-and-truncate over-cap derived expansions
|
||||
instead of hard-skipping a homonym acronym entirely (#000054
|
||||
Phase 2a — CPU has 13 legitimate homonym expansions across the
|
||||
corpus, of which `central / processing / unit` are anchored by
|
||||
hundreds of documents and the noise tail by 1-2 each).
|
||||
|
||||
The extractor emits bidirectional edges, so for any (a, b) pair
|
||||
both rows exist with token=a/target=b and token=b/target=a, each
|
||||
anchored to the doc that defined them. Counting on one direction
|
||||
captures the asymmetric assertion. #000054."""
|
||||
if not neighbors:
|
||||
return {}
|
||||
nlower = {n.lower() for n in neighbors if n}
|
||||
if not nlower:
|
||||
return {}
|
||||
ph = ",".join("?" * len(nlower))
|
||||
conn = connect_query(shards_dir=shards_dir)
|
||||
try:
|
||||
rows = conn.execute(
|
||||
f"SELECT target, COUNT(DISTINCT source_root) AS cnt "
|
||||
f"FROM concept_relations "
|
||||
f"WHERE relation_kind = 'synonym' "
|
||||
f" AND token = ? AND target IN ({ph}) "
|
||||
f"GROUP BY target",
|
||||
[token.lower()] + list(nlower),
|
||||
).fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
return {(r["target"] or "").lower(): int(r["cnt"] or 0) for r in rows}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API — matches the legacy ``arborist.qa.concepts`` shape
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
@ -348,6 +385,79 @@ def _load_idf_for(shards_dir: Path, tokens: set[str]) -> dict[str, int]:
|
|||
MAX_NEIGHBORS_PER_TOKEN = 8
|
||||
MAX_TOTAL_TOKENS = 50
|
||||
|
||||
def _load_strict_neighbors_for(
|
||||
shards_dir: Path, tokens: set[str]
|
||||
) -> dict[str, set[str]]:
|
||||
"""High-trust synonym neighbors only — manual + manual_legacy +
|
||||
acronym_parens. Excludes ``link_reciprocity`` because reciprocal
|
||||
wikilinks express topical adjacency, not synonymy (a
|
||||
``Dinosaurs`` page reciprocally links to a ``Curious George
|
||||
Brigade`` page → an edge that should not amplify retrieval). The
|
||||
multiplicative title-purity rerank uses this strict view; the
|
||||
additive title-rerank + retrieval routes use the broader
|
||||
``synonym_expand``. #000054 Phase 2b."""
|
||||
qlower = {t.lower() for t in tokens if t}
|
||||
if not qlower:
|
||||
return {}
|
||||
ph = ",".join("?" * len(qlower))
|
||||
params = list(qlower) + list(qlower)
|
||||
conn = connect_query(shards_dir=shards_dir)
|
||||
try:
|
||||
rows = conn.execute(
|
||||
f"SELECT token, target FROM concept_relations "
|
||||
f"WHERE relation_kind = 'synonym' "
|
||||
f" AND evidence_kind IN ('manual', 'manual_legacy', 'acronym_parens') "
|
||||
f" AND (token IN ({ph}) OR target IN ({ph}))",
|
||||
params,
|
||||
).fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
out: dict[str, set[str]] = {t: set() for t in qlower}
|
||||
for r in rows:
|
||||
a = (r["token"] or "").lower()
|
||||
b = (r["target"] or "").lower()
|
||||
if not a or not b or a == b:
|
||||
continue
|
||||
if a in out:
|
||||
out[a].add(b)
|
||||
if b in out:
|
||||
out[b].add(a)
|
||||
return out
|
||||
|
||||
|
||||
def synonym_expand_strict(
|
||||
tokens: set[str],
|
||||
*,
|
||||
shards_dir: Path | str | None = None,
|
||||
max_neighbors_per_token: int = MAX_NEIGHBORS_PER_TOKEN,
|
||||
) -> set[str]:
|
||||
"""Like ``synonym_expand`` but restricted to high-trust evidence
|
||||
kinds (manual / acronym_parens) — see ``_load_strict_neighbors_for``.
|
||||
Used by the title-purity multiplier; the broader ``synonym_expand``
|
||||
is right for retrieval (additive boosts) but its inclusion of
|
||||
noisy ``link_reciprocity`` edges blows up under a multiplicative
|
||||
rank. #000054 Phase 2b."""
|
||||
if not tokens or shards_dir is None:
|
||||
return set(t.lower() for t in tokens if t)
|
||||
p = Path(shards_dir)
|
||||
qlower = {t.lower() for t in tokens if t}
|
||||
neighbors_by_token = _load_strict_neighbors_for(p, qlower)
|
||||
expanded: set[str] = set(qlower)
|
||||
for t in qlower:
|
||||
nb = neighbors_by_token.get(t, set())
|
||||
if not nb:
|
||||
continue
|
||||
if len(nb) <= max_neighbors_per_token:
|
||||
expanded |= nb
|
||||
continue
|
||||
# Frequency rank — same discipline as the broad path.
|
||||
freq = _load_neighbor_source_freq(p, t, nb)
|
||||
ranked = sorted(nb, key=lambda n: (-freq.get(n, 0), n))
|
||||
expanded |= set(ranked[:max_neighbors_per_token])
|
||||
return expanded
|
||||
|
||||
|
||||
|
||||
|
||||
def synonym_expand(
|
||||
tokens: set[str],
|
||||
|
|
@ -392,13 +502,27 @@ def synonym_expand(
|
|||
expanded |= manual_index.get(t, set())
|
||||
# Derived (corpus-extracted) synonyms cap on per-token degree.
|
||||
# Generic tokens like "person" / "thoughts" / "language" have wide
|
||||
# noisy neighborhoods in the reciprocal-link graph — skip those.
|
||||
# Specific tokens with bounded degree expand cleanly.
|
||||
# noisy neighborhoods in the reciprocal-link graph; specific tokens
|
||||
# have bounded ones. When the per-token degree exceeds the cap we
|
||||
# *rank-and-truncate* by source-frequency rather than hard-skipping
|
||||
# (#000054 Phase 2a) — a homonym acronym ("CPU" has 13 expansions
|
||||
# across the corpus: central/processing/unit anchored by hundreds
|
||||
# of docs, canadian/contract/pharmaceutical by 1-2 each) should
|
||||
# still contribute its dominant expansion, not nothing. The
|
||||
# frequency rank surfaces the canonical pairing and drops the
|
||||
# noise tail. Below-cap tokens stay on the fast path (no SQL).
|
||||
for t in qlower:
|
||||
neighbors = derived_index.get(t, set())
|
||||
if len(neighbors) > max_neighbors_per_token:
|
||||
if not neighbors:
|
||||
continue
|
||||
expanded |= neighbors
|
||||
if len(neighbors) <= max_neighbors_per_token:
|
||||
expanded |= neighbors
|
||||
continue
|
||||
# Over the per-token cap: rank by source-frequency (desc),
|
||||
# keep the top ``max_neighbors_per_token``.
|
||||
freq = _load_neighbor_source_freq(p, t, neighbors)
|
||||
ranked = sorted(neighbors, key=lambda n: (-freq.get(n, 0), n))
|
||||
expanded |= set(ranked[:max_neighbors_per_token])
|
||||
if len(expanded) > max_total:
|
||||
# IDF-rank the neighbors (rarer = more topical = keep first).
|
||||
# Tokens absent from concept_token_idf get a sentinel high-
|
||||
|
|
|
|||
|
|
@ -25,9 +25,15 @@ from arborist.concepts.query import (
|
|||
has_compare_phrasing,
|
||||
rivalry_excluded as _rivalry_excluded_impl,
|
||||
synonym_expand as _synonym_expand_impl,
|
||||
synonym_expand_strict as _synonym_expand_strict_impl,
|
||||
)
|
||||
|
||||
__all__ = ["has_compare_phrasing", "rivalry_excluded", "synonym_expand"]
|
||||
__all__ = [
|
||||
"has_compare_phrasing",
|
||||
"rivalry_excluded",
|
||||
"synonym_expand",
|
||||
"synonym_expand_strict",
|
||||
]
|
||||
|
||||
|
||||
def synonym_expand(
|
||||
|
|
@ -47,6 +53,22 @@ def synonym_expand(
|
|||
return _synonym_expand_impl(tokens, shards_dir=shards_dir)
|
||||
|
||||
|
||||
def synonym_expand_strict(
|
||||
tokens: set[str],
|
||||
*,
|
||||
shards_dir: Path | str | None = None,
|
||||
) -> set[str]:
|
||||
"""Like ``synonym_expand`` but restricted to high-trust evidence
|
||||
kinds (manual + manual_legacy + acronym_parens), excluding the
|
||||
noisier ``link_reciprocity`` channel. For use in the multiplicative
|
||||
title-purity rerank where reciprocal-link noise blows up; the
|
||||
broad ``synonym_expand`` is right for retrieval and additive
|
||||
boosts. #000054 Phase 2b."""
|
||||
if shards_dir is None:
|
||||
return set(t.lower() for t in tokens if t)
|
||||
return _synonym_expand_strict_impl(tokens, shards_dir=shards_dir)
|
||||
|
||||
|
||||
def rivalry_excluded(
|
||||
tokens: set[str],
|
||||
compare_phrasing: bool = False,
|
||||
|
|
|
|||
|
|
@ -59,6 +59,7 @@ from arborist.qa.concepts import (
|
|||
has_compare_phrasing,
|
||||
rivalry_excluded,
|
||||
synonym_expand,
|
||||
synonym_expand_strict,
|
||||
)
|
||||
from arborist.qa.keys import (
|
||||
DEFAULT_FIDELITY,
|
||||
|
|
@ -175,11 +176,24 @@ def _rerank_by_title(
|
|||
hits: list,
|
||||
question: str,
|
||||
boost: float = 10.0,
|
||||
*,
|
||||
shards_dir=None,
|
||||
) -> list:
|
||||
"""Boost hits whose title overlaps query tokens. Pure ordering aid."""
|
||||
"""Boost hits whose title overlaps query tokens. Pure ordering aid.
|
||||
|
||||
Uses the synonym-expanded token set so canonical-article titles
|
||||
that share zero literal-query tokens but contain expansion tokens
|
||||
(Wikipedia's `Graphics processing unit` for a `GPU` query) earn
|
||||
the boost. Without this, the literal-query satellites (`GPU
|
||||
cluster`) get +10 and the canonical article gets 0 — the exact
|
||||
failure mode #000054 Phase 2 is closing. (Falls back to literal
|
||||
qtokens when `shards_dir is None`.)
|
||||
"""
|
||||
qtokens = _title_query_tokens(question)
|
||||
if not qtokens:
|
||||
return hits
|
||||
if shards_dir is not None:
|
||||
qtokens = synonym_expand(qtokens, shards_dir=shards_dir)
|
||||
for h in hits:
|
||||
if not h.title:
|
||||
continue
|
||||
|
|
@ -685,12 +699,30 @@ def _search_titles(conn, qtokens: list[str], limit: int) -> list[tuple]:
|
|||
# match coherently.
|
||||
match_expr = " OR ".join(f'"{t.lower().replace(chr(34), chr(34) * 2)}"' for t in bounded)
|
||||
try:
|
||||
# Rank by FTS5 bm25 ascending (lowest=best match) — titles
|
||||
# that match MORE of the OR'd expanded tokens float to the
|
||||
# top. This replaces the prior 2026-05-02 ``LENGTH(title)
|
||||
# ASC`` tie-break, which only worked when qtokens were 1-3
|
||||
# tokens: with the #000054 Phase 2b expansion (qtokens
|
||||
# union synonym-pool) the candidate set explodes, length-
|
||||
# asc returns "Unit" / "Unite" / "B unit" / … and the
|
||||
# multi-token canonical title ("Graphics processing unit"
|
||||
# = 3 expanded-token hits) gets cut by ``LIMIT``. bm25
|
||||
# naturally favors multi-match titles because each OR
|
||||
# clause that hits contributes to the score. The
|
||||
# substring-junk problem the length-asc fix solved
|
||||
# (2026-05-02 "Back to the Future" case) doesn't apply
|
||||
# here: FTS5 MATCH is tokenized — "out" only matches the
|
||||
# tokenized word "out", not the substring of "Aberdeen,
|
||||
# South Dakota". Length-asc stays as the LIKE-fallback
|
||||
# tie-break below (where the substring issue persists).
|
||||
rows = conn.execute(
|
||||
"SELECT d.document_root, d.document_uri, d.title "
|
||||
"FROM documents_fts AS f "
|
||||
"JOIN documents AS d ON d.rowid = f.rowid "
|
||||
"WHERE documents_fts MATCH ? "
|
||||
"ORDER BY LENGTH(d.title) ASC LIMIT ?",
|
||||
"ORDER BY bm25(documents_fts) ASC, LENGTH(d.title) ASC "
|
||||
"LIMIT ?",
|
||||
(match_expr, over_fetch_limit),
|
||||
).fetchall()
|
||||
return rows
|
||||
|
|
@ -1142,15 +1174,35 @@ def _search_corpus(
|
|||
out-ranks FTS5 body hits so the actual topic article rises to the top.
|
||||
"""
|
||||
qtokens = _title_query_tokens(question)
|
||||
# Title-LIKE backup uses ORIGINAL qtokens only (LIKE %tok% can't
|
||||
# use any index — adding synonyms makes it O(corpus × |accept|)).
|
||||
# Synonym expansion stays useful in two places: (1) the FTS5
|
||||
# OR-mode fallback (top-5 longest pool merged with synonyms — long
|
||||
# topical synonyms like "neurotechnology" surface relevant titles
|
||||
# without paying for full-scan), and (2) `_filter_by_title_relevance`
|
||||
# post-retrieval filtering (in-memory, cheap).
|
||||
accept_tokens = set(qtokens)
|
||||
# `accept_tokens` flows into the title-search FTS5 MATCH (line ~1035)
|
||||
# and the core-keyword route (line ~1099); `accept_stems` is the
|
||||
# accept-path-1 token-overlap filter in `_filter_by_title_relevance`.
|
||||
# Pre-#000054 Phase 2b: this was `set(qtokens)` (qtokens only), with
|
||||
# `or_synonym_pool` used only in the FTS5 body OR-fallback. That
|
||||
# let the canonical article disappear: "what is a GPU?" expanded to
|
||||
# {gpu, graphics, processing, unit} via synonym_expand, but only
|
||||
# `gpu` was used to retrieve titles → the GPU-* satellites filled
|
||||
# the budget while "Graphics processing unit" never entered the
|
||||
# candidate pool. Now: use the expanded set in all four routes.
|
||||
# The expanded set is bounded by `MAX_NEIGHBORS_PER_TOKEN` (8) ×
|
||||
# |qtokens| capped at `MAX_TOTAL_TOKENS` (50), so the SQL clause
|
||||
# count stays tractable; the old LIKE perf objection (line 1146
|
||||
# pre-#000054) was retired with `documents_fts` MATCH replacing
|
||||
# LIKE in `_search_titles` 2026-05-02.
|
||||
or_synonym_pool = synonym_expand(qtokens, shards_dir=shards_dir)
|
||||
# `accept_tokens` flows into title-search FTS5 MATCH + title-rerank
|
||||
# + core-keyword search + title-relevance filter accept-paths. Use
|
||||
# the **strict** synonym view (manual + acronym_parens evidence;
|
||||
# excludes link_reciprocity) — link_reciprocity edges express
|
||||
# topical adjacency, not synonymy, and injecting them into
|
||||
# retrieval pulled "Curious George Brigade" into "why did the
|
||||
# dinosaurs go extinct?" via `dinosaurs ↔ curious/george/brigade`
|
||||
# reciprocal-link edges (2026-05-13 bench regression). The
|
||||
# acronym-parens expansion that closes CPU→Central-processing-unit
|
||||
# / GPU→Graphics-processing-unit is preserved because those edges
|
||||
# ARE strict. `or_synonym_pool` (the broad view) still feeds the
|
||||
# FTS5 OR-fallback for long-token surfacing per the prior design.
|
||||
accept_tokens = synonym_expand_strict(qtokens, shards_dir=shards_dir) or set(qtokens)
|
||||
paths: list[Path]
|
||||
if shards_dir is not None:
|
||||
paths = discover_shards(shards_dir)
|
||||
|
|
@ -1285,9 +1337,9 @@ def _rerank(
|
|||
)
|
||||
progress.emit("search.title_filter", survivors=len(hits))
|
||||
hits = _rerank_by_body_coverage(hits, question)
|
||||
hits = _rerank_by_title(hits, question)
|
||||
hits = _rerank_by_title(hits, question, shards_dir=shards_dir)
|
||||
hits = _rerank_by_source_role(hits, question)
|
||||
hits = _rerank_by_title_purity(hits, question)
|
||||
hits = _rerank_by_title_purity(hits, question, shards_dir=shards_dir)
|
||||
return _rerank_by_ordered_token_match(hits, question)
|
||||
|
||||
|
||||
|
|
@ -1333,7 +1385,7 @@ def _rerank_by_source_role(hits: list[_Hit], question: str) -> list[_Hit]:
|
|||
return hits
|
||||
|
||||
|
||||
def _rerank_by_title_purity(hits: list[_Hit], question: str) -> list[_Hit]:
|
||||
def _rerank_by_title_purity(hits: list[_Hit], question: str, *, shards_dir=None) -> list[_Hit]:
|
||||
"""Boost titles by both purity AND multi-token-match breadth.
|
||||
|
||||
Two signals combine here:
|
||||
|
|
@ -1374,6 +1426,21 @@ def _rerank_by_title_purity(hits: list[_Hit], question: str) -> list[_Hit]:
|
|||
qtokens = _title_query_tokens(question)
|
||||
if not qtokens:
|
||||
return hits
|
||||
# #000054 Phase 2b: expand via the *strict* synonym view (manual +
|
||||
# acronym_parens evidence) — NOT the broad view that includes
|
||||
# link_reciprocity. The multiplier ``(1+overlap)*(1+purity)``
|
||||
# amplifies; running it against the broad expansion set blows up
|
||||
# on noisy link_reciprocity synonyms (observed 2026-05-13 bench:
|
||||
# "why did the dinosaurs go extinct?" → Curious George Brigade
|
||||
# titled docs got 4× via `dinosaurs ↔ curious/george/brigade`
|
||||
# reciprocal-link edges, which express topical adjacency, not
|
||||
# synonymy). The strict view keeps the CPU↔central-processing-
|
||||
# unit / GPU↔graphics-processing-unit acronym surfacing — those
|
||||
# are text-pattern edges (acronym_parens) where the phrase
|
||||
# literally IS the expansion — without amplifying the
|
||||
# reciprocal-link noise tail.
|
||||
if shards_dir is not None:
|
||||
qtokens = synonym_expand_strict(qtokens, shards_dir=shards_dir) or qtokens
|
||||
# Stem-aware matching so possessive / plural variants match. The
|
||||
# 2026-05-01 Dawson's Creek defect: question "dawsons creek" with
|
||||
# title "List of Dawson's Creek episodes" — raw set intersection
|
||||
|
|
|
|||
|
|
@ -101,7 +101,7 @@ Newest first. Update on every open/close.
|
|||
|
||||
| ID | Title | Status | Opened | Directive |
|
||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||
| #000054 | Acronym-parens concept extractor (closes the abbreviation→expansion retrieval gap) | **closed · 2026-05-13** — `arborist/concepts/extract.py:acronym_parens_synonym` lands as a new corpus-agnostic extractor in `EXTRACTORS` (`evidence_kind="acronym_parens"`). Scans each doc's lead chunk for `<Multi-Word Phrase> (ACRO)` where the all-caps acronym's letters match the content-word initials of the phrase in order; emits bidirectional synonym edges between the lowercased acronym and each ≥3-char content token of the phrase. Conservative (strict 1:1 initials, function words filtered, repeated definitions deduped per doc). Closes the *retrieval-side* abbreviation gap (`CPU↔central processing unit`, `GPU↔graphics processing unit`, `RAM↔random access memory`, `FBI↔federal bureau of investigation`, `WHO↔world health organization`, …) that `link_reciprocity_synonym` can't reach because the relation lives in body text, not the wiki link graph (Wikipedia represents abbreviation→expansion as a *redirect* — not an edge). Per-shard like all `concept_relations` data; corpus-agnostic so HTML/blogs/textbooks benefit equally. Retrieval-side only — never proof-path. 8 new tests; full suite green. Closes #000050 §2a's CPU/GPU fixture rows *upstream* of vec; the Orwell-shape conceptual-allusion row remains the genuine #000050 justification. Operational follow-up (not code): `arborist concepts derive --extractor acronym_parens` on each shard. | 2026-05-13 | — |
|
||||
| #000054 | Acronym-parens concept extractor (closes the abbreviation→expansion retrieval gap) | **in progress** — Phase 1 (extractor + 481K edges) landed `58027e9`; Phase 2 (consumer-side surfacing — `synonym_expand` rank-and-truncate over the per-token cap, FTS5-`bm25` ordering in `_search_titles`, expanded `accept_tokens` in title-search + core-keyword + title-rerank, `synonym_expand_strict()` for the multiplicative title-purity rerank to exclude noisy `link_reciprocity` edges, tightened extractor regex to `[A-Z]{3,6}` purging 2-letter homonym edges) pending commit. **End-to-end verified:** `what is a CPU?` → Central processing unit at #1; `what is a GPU?` → Graphics processing unit at #1 EVIDENCE-WARRANTED 1/1; Mount Kilimanjaro / Soviet Union queries unchanged (no regression). 2026-05-13 — `arborist/concepts/extract.py:acronym_parens_synonym` lands as a new corpus-agnostic extractor in `EXTRACTORS` (`evidence_kind="acronym_parens"`). Scans each doc's lead chunk for `<Multi-Word Phrase> (ACRO)` where the all-caps acronym's letters match the content-word initials of the phrase in order; emits bidirectional synonym edges between the lowercased acronym and each ≥3-char content token of the phrase. Conservative (strict 1:1 initials, function words filtered, repeated definitions deduped per doc). Closes the *retrieval-side* abbreviation gap (`CPU↔central processing unit`, `GPU↔graphics processing unit`, `RAM↔random access memory`, `FBI↔federal bureau of investigation`, `WHO↔world health organization`, …) that `link_reciprocity_synonym` can't reach because the relation lives in body text, not the wiki link graph (Wikipedia represents abbreviation→expansion as a *redirect* — not an edge). Per-shard like all `concept_relations` data; corpus-agnostic so HTML/blogs/textbooks benefit equally. Retrieval-side only — never proof-path. 8 new tests; full suite green. Closes #000050 §2a's CPU/GPU fixture rows *upstream* of vec; the Orwell-shape conceptual-allusion row remains the genuine #000050 justification. Operational follow-up (not code): `arborist concepts derive --extractor acronym_parens` on each shard. | 2026-05-13 | — |
|
||||
| #000053 | Acronym-aware verifier content tokens | **closed · 2026-05-13** — `arborist.qa.evidence._content_tokens` now keeps all-caps 2-3-char acronyms (CPU/GPU/DNA/FBI/USB…) as content tokens instead of dropping every <4-char token; fixes the field case where "what is a CPU?" cited to "CPU design" tripped `TITLE_MISMATCH` spuriously (claim & title share "CPU" but neither registered) — also affects `SUBJECT_TOKENS_ABSENT` (Rule 9), `BARE_NAME_CLAIM`, spotlight-excerpt token pick. Versioned: `content_token_rules: "v2-acronym-aware"` in both default policies + `_VERIFIER_POLICY_FIELDS` → folds into `verifier_policy_hash`, prior cache records orphan on lookup (by design, same discipline as `base_version` / `hyphen_fold_v1`). Monotone toward *fewer* spurious demotes (only relaxes overlap checks, never tightens). 8 new tests; full suite green; `bench-qa-smoke` clean. Does NOT fix the *retrieval* abbreviation→expansion gap (`CPU`→`Central processing unit` = #000050 vec hybrid / `concepts/` synonym edges — the root cause of the satellite-article retrieval). | 2026-05-13 | — |
|
||||
| #000052 | Relevance + coherence meta-cognition (answer-*shape* signals) | in progress — **§3.1 `diagnose_coherence` landed** (lexical, no model: `circular` / `phrase_component_reuse` / `vacuous`; in `arborist/qa/inspect.py`, surfaced via `inspect_cache_key` + `arborist inspect` `· incoherent: <kind>`; 9 tests; demote-policy hook deliberately not wired — advisory only). Joins the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family of read-only, demote-only, never-in-proof-path sidecars; `phrase_component_reuse` catches the motivating field case (a subject quoting a phrase, a predicate reusing one of that phrase's own tokens as a bare `the <token>` referent). **Still open: (2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | — |
|
||||
| #000051 | Federated vecpack distribution (gossip the embedding backfill) | open · awaiting go/no-go · doc-only scaffold. Makes `chunk_vecs` a distributable artifact: backfill once on any CPU box (cloud / Prometheus-Σ sweep — #000037 §3.1), publish a **vecpack** `(shard_root, vec_backend_version, [(leaf_hash, embedding_blob)…])` over the mesh wire layer, every peer pulls + bulk-loads (sub-ms/chunk on the receiver — the laptop never runs the transformer). Keyed on `leaf_hash` (portable) not `chunk_id` (shard-local). Vecpacks are **soft data** — embeddings are `UNGROUNDED`, never proof path — so a cheap structural sanity gate (chunk exists locally w/ matching leaf_hash, right blob length for (dim,quant), finite norm, backend_version matches) suffices, no Merkle-proof-grade verification needed. Supplies #000050's prereq #1 ("a vecpack exists & is imported on the bench box", not "fox embedded the corpus locally"). GPU producer (the fast path): bge-small-en-v1.5 batched on a CUDA box (4090) ≈ 10³–10⁴ chunks/s → full 6.24M-chunk corpus in *minutes*, not days — drop a CUDA `Embedder` into `default_embedder()`; CUDA stack lives only on the producer box, never in arborist's `python+sqlite3` core. The mechanism behind whitepaper §1's "the embedding pass runs off the device". #000039 / #000050 sibling | 2026-05-12 | — |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000054 — Acronym-parens concept extractor (closing the abbreviation→expansion retrieval gap)
|
||||
|
||||
**Status:** closed · 2026-05-13 — `arborist/concepts/extract.py:acronym_parens_synonym` lands as a new corpus-agnostic extractor in the existing `EXTRACTORS` registry (`evidence_kind = "acronym_parens"`). Scans the lead chunk of every document for the pattern `<Multi-Word Phrase> (ACRO)` where the all-caps parenthesized acronym's letters match the content-word initials of the phrase in order; emits bidirectional synonym edges between the (lowercased) acronym and each ≥3-char content token of the phrase. Conservative: strict 1:1 initial match, function words filtered, repeated definitions deduped per doc. 8 new tests in `tests/test_concepts_extract.py` (28 → 36); full suite green. Closes the *retrieval-side* abbreviation gap (`CPU↔central processing unit`, `GPU↔graphics processing unit`, `RAM↔random access memory`, `FBI↔federal bureau of investigation`, `WHO↔world health organization`, …) that `link_reciprocity_synonym` structurally can't reach because the relation lives in body text, not the wiki link graph (Wikipedia represents abbreviation→expansion as a *redirect*, which the ingest does not record as an edge). Per-shard like all `concept_relations` data; corpus-agnostic so HTML / blogs / textbooks benefit the same way as Wikipedia. Complements #000050 (vec hybrid) without overlap — the abbreviation cases the vec layer would otherwise have to carry are now closable cheaply; the Orwell-shape *conceptual* allusion remains a vec-only case.
|
||||
**Status:** in progress — Phase 1 + Phase 2 landed; bench in flight. Phase 1 (`58027e9`): the extractor + 481K edges across 4 shards. Phase 2 (pending commit) wraps four interlocking consumer-side fixes that the end-to-end gap-close exposed in flight, all folded into this ticket per the "don't proliferate" discipline (CLAUDE.md / `feedback_ticket_proliferation`): **(a)** `synonym_expand` rank-and-truncate over the per-token cap (source-frequency descending) instead of hard-skipping — CPU has 13 legitimate homonym expansions across the corpus, Phase 1 alone hit the `MAX_NEIGHBORS_PER_TOKEN=8` cap → zero expansion → no surfacing; **(b)** `_search_titles` orders by FTS5 `bm25` (multi-token-match-favoring) instead of `LENGTH(title) ASC` on the FTS5 path — the length-asc fix the 2026-05-02 "Back to the Future" case introduced was tokenization-needed for the LIKE path but counter-productive on FTS5; **(c)** retrieval routes (title-search, core-keyword) + the title-token additive rerank use the synonym-expanded `accept_tokens`, not qtokens-only; **(d)** the architecturally load-bearing one — `synonym_expand_strict()` (new), a high-trust evidence-kind subset (manual + acronym_parens, **excludes** `link_reciprocity`), used by `accept_tokens` and `_rerank_by_title_purity` (the multiplicative ranker). Reciprocal-wikilink edges express *topical adjacency*, not synonymy (a `Dinosaurs` page reciprocally links to a `Curious George Brigade` page → an edge that should not amplify retrieval — observed 2026-05-13 dinosaurs regression where Phase-2-broad pulled CGB into the top), and a multiplicative ranker over them blows up. The strict view preserves the acronym-parens surfacing (those edges ARE the phrase=expansion identity) while keeping link-reciprocity restricted to additive retrieval-route boosts. Also (e): tightened extractor regex `[A-Z]{2,6}` → `[A-Z]{3,6}` and purged ~21K 2-letter acronym edges (`AI`/`ML`/`OS`/`US`/`UK`/`IT`/`PC`/`TV`) that homonym-collided with 2-char query tokens like `go`/`is`/`am`. **End-to-end verified live:** `what is a CPU?` retrieval pulls *Central processing unit* at #1 (was: only CPU-* satellites); `what is a GPU?` → *Graphics processing unit* at #1 + EVIDENCE-WARRANTED 1/1; `where is mount kilimanjaro located?` → Mount Kilimanjaro at #1; `when did the soviet union dissolve?` → Soviet Union at #1 (regressions from the no-strict intermediate state went away). Bench-qa-smoke pending. Phase 1 prior closure was premature — the extractor produced edges but didn't change retrieval outcomes; this is the actual end-to-end-working bar.
|
||||
**Opened:** 2026-05-13
|
||||
**Scope:** One new extractor in `arborist/concepts/extract.py` + registry entry + tests. No schema change, no proof-path change (synonym edges are a *retrieval-side* soft signal — they reshape which candidates `qa/query.py` considers but never enter `audit_mode` / `cache_key` / `audit_event_hash`).
|
||||
**Audience:** fox + anyone maintaining the retrieval pipeline + future shifts that wonder "why didn't `CPU` find `Central processing unit`?"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue