russell@unturf.com
b573c592d8
Second MEASURED fold-search win, and the instrument correcting my own
premature call. accent-fold ON vs OFF on the mined accent fixture:
recall@1 55% -> 85% (+30pp), rank-1 22/40 -> 34/40. recall@8 was
flat (95->98) — a too-lenient k nearly got a real lever wrongly
reverted; @1/@3 is the resolution that drives primary-source
selection. _accent_fold_variants: ASCII-fold then re-tokenise so a
diacritic title ("Béla Bartók", which _TITLE_TOKEN_RE otherwise
fragments to junk) matches the ASCII form a user types. Additive+
symmetric, no-op on pure-ASCII (zero effect on non-accent
queries/titles), mirrors _hyphen_fold_variants (#000007).
Also fixes a defect I shipped in a3ac653: an orphaned duplicate
body left as dead code after `return base` in _title_query_tokens
(unreachable — numeral-fold behaviour/measurement were valid — but
cruft; removed).
Fold-search factory, fanned out across the full survey backlog
(deterministic recall, no LLM, parallel — serial-by-caution was
halting in disguise):
- recall_at_k.py: returns rank -> recall@1/@3/@k from one retrieval
(verified offline). A coarse k hides rank-only lifts.
- mine_questions.py: numeral/accent/hyphen/honorific/amp/brit
ground-truth classes; fixtures committed.
- Measured @1 headroom verdicts: accent SHIP (this commit);
honorific 45% / brit 50% = real headroom (build next); hyphen
90% = existing #000007 already delivers, NOTHING to build (the
measure-the-unmeasured-thing check pays off); amp 82% = no fold
needed (prevalence-overranked, instrument kills it cheaply).
CLAUDE.md bench-maxing: two measured lessons codified — report
recall@1/@3/@k (a lenient k hides rank lifts; prevalence != miss-
rate), and fan out independent measurements (serial-by-caution is
halting). Full suite 2488 passed, 0 regressions (accent-fold is
hot-path in _title_query_tokens); real-path test (FakeSource->
ingest->query()->real _Hit).
43 lines
1.4 KiB
Text
43 lines
1.4 KiB
Text
# AUTO-MINED (hyphen class) from corpus titles via bench/mine_questions.py — ground-truth-carrying.
|
|
# Graded by deterministic retrieval recall@k (bench/recall_at_k.py), NOT audit_mode. Not adversarial; complements (never replaces) qa_questions.txt.
|
|
|
|
what is The Amazing Spider Man (comic book)?
|
|
what is Augustin Jean Fresnel?
|
|
what is André Marie Ampère?
|
|
what is Aster CT 80?
|
|
what is Anti globalization movement?
|
|
what is Abd ar Rahman I?
|
|
what is Abd ar Rahman V?
|
|
what is Anti Ballistic Missile Treaty?
|
|
what is AGM 88 HARM?
|
|
what is Lockheed AC 130?
|
|
what is CIM 10 Bomarc?
|
|
what is North American B 25 Mitchell?
|
|
what is Bain marie?
|
|
what is Cross dressing?
|
|
what is Call of Cthulhu (role playing game)?
|
|
what is Context sensitive?
|
|
what is Boeing C 17 Globemaster III?
|
|
what is Chiang Kai shek?
|
|
what is Context free language?
|
|
what is C* algebra?
|
|
what is Computer generated imagery?
|
|
what is Lockheed C 130 Hercules?
|
|
what is Covenant breaker?
|
|
what is Comprehensive Nuclear Test Ban Treaty?
|
|
what is Double slit experiment?
|
|
what is Double ended queue?
|
|
what is Cost push inflation?
|
|
what is Eductor jet pump?
|
|
what is Eight ball?
|
|
what is Élisabeth Louise Vigée Le Brun?
|
|
what is Evidence based medicine?
|
|
what is E Prime?
|
|
what is Boeing E 3 Sentry?
|
|
what is Emperor Go En'yū?
|
|
what is Field programmable gate array?
|
|
what is Five spice powder?
|
|
what is Four poster?
|
|
what is Flip flop (electronics)?
|
|
what is Guinea Bissau?
|
|
what is Politics of Guinea Bissau?
|