The Hank-Scorpio-vs-Mr.-Burns problem: body BM25 alone outranks the
canonical primary-source article ("Homer Simpson") below sibling
articles that incidentally mention the same tokens. The sidecar
pipeline already had title-boost-with-extras-penalty; lift it into
a shared utility that the orchestrator applies to ANY adapter's
fts_body output.
arborist/qa/corpus.py:apply_title_boost(hits, query, *,
higher_is_better)
Stems both sides (possessive + plural collapse), numeral-expands
(7↔VII), accent-folds (é→e). Effective bonus per hit:
max(0, overlap - extras/2) * boost
where extras = title tokens NOT in query. Score direction honors
each adapter's convention (BM25 negative → subtract; sidecar BM25
positive → add).
arborist/qa/corpus_query.py:run_query
Now oversamples fts_body by 4× and reranks via apply_title_boost
before slicing to top_k. Without oversampling the right primary
can sit at rank 7-15 in the body BM25 output and get cut before
the rerank sees it.
bench/three_way_bench.py: NEW — drives same fixture through
- `arborist query` (legacy 2000-line pipeline)
- `arborist corpus-query` (new, local shards via Corpus + run_query)
- `arborist cloud query` (new, sidecar via Corpus + run_query)
prints 3-column table + flags primary-source disagreements.
Smoke fixture (5 questions, all paths):
4/5 all-paths agree on primary source (was 2/5 pre-fix)
1/5 disagrees: dinosaur-extinction
legacy: UNGROUNDED · Edwina (children's book) ← wrong
corpus: STRICT · Dinosaur (main article) ← right
cloud: HYBRID · Edwina (children's book) ← wrong
The new title-boost lifted corpus-query above the legacy here;
cloud still picks Edwina because SidecarReader.search applies
title-boost INTERNALLY (it predates the shared util), so the
extras-penalty stacks weirdly when run_query applies it again.
Next fix: disable internal boost in SidecarReader once shared
util is the single source of truth.
26 corpus/wallet/sidecar tests still green.
russell@unturf.com
2b8777c947