Two structural defects in the corpus retrieval path were causing
the Intel-CPU query (and any query whose terms hit many article
titles generically) to surface 80486-era legacy articles instead
of contemporary topic articles:
1. _search_titles entries arrived with a 50-point baseline score
(50 + 10*overlap) plus _rerank_by_title doubled the overlap
bonus on top. A single-token title match against a generic
word like "intel" landed at score 70-80 standalone, swamping
FTS5 BM25 hits that scored 5-15. Result: every Intel_*-prefixed
article in the corpus rose to the top of every "intel" query,
regardless of whether the article was actually about a fast CPU.
Drop the 50-point baseline. Title overlap now scores `10*overlap`
from this path, with `_rerank_by_title` adding another `10*overlap`
for any hit (FTS5 or title) whose title overlaps query tokens.
Net effect: title-only-overlap=1 hits score ~10-20, FTS5 hits with
title-overlap score ~25-45, FTS5-only-no-title-overlap score ~5-25.
Body relevance now beats single-token title coincidence.
2. _body_density_passes counted ANY query token's mentions toward
`min_mentions`, so Intel_8086 (with 30 "Intel" mentions and zero
"fastest" or "cpu" mentions) passed the gate for a query of
{intel, fastest, cpu}. Tighten to require breadth: at least
half the query tokens must each appear at least once, AND total
mentions must clear min_mentions. For 1-token queries this is
identical to the prior behavior.
Verified end-to-end on the fresh 2010-11 enwiki corpus (3.47M docs,
no distill cores yet). Pre-fix: top-K all Intel_8086/80286/80386/etc.,
Hermes refused to identify a fastest CPU. Post-fix: Intel Core i7
in top-K, Hermes correctly identifies Core i7-980X Extreme Edition
Gulftown (Intel's 2010 flagship — 32nm, 6 cores, 3.33 GHz, 12 MB L3).
Same 2003-corpus AMD-query result preserved (already had Athlon /
Athlon_XP_3200+ in top-K via FTS5 BM25 + multi-token TF-IDF cores).
125 tests + 1 skipped, no regressions.
russell@unturf.com
d608c39880