russell@unturf.com
b573c592d8
Second MEASURED fold-search win, and the instrument correcting my own
premature call. accent-fold ON vs OFF on the mined accent fixture:
recall@1 55% -> 85% (+30pp), rank-1 22/40 -> 34/40. recall@8 was
flat (95->98) — a too-lenient k nearly got a real lever wrongly
reverted; @1/@3 is the resolution that drives primary-source
selection. _accent_fold_variants: ASCII-fold then re-tokenise so a
diacritic title ("Béla Bartók", which _TITLE_TOKEN_RE otherwise
fragments to junk) matches the ASCII form a user types. Additive+
symmetric, no-op on pure-ASCII (zero effect on non-accent
queries/titles), mirrors _hyphen_fold_variants (#000007).
Also fixes a defect I shipped in a3ac653: an orphaned duplicate
body left as dead code after `return base` in _title_query_tokens
(unreachable — numeral-fold behaviour/measurement were valid — but
cruft; removed).
Fold-search factory, fanned out across the full survey backlog
(deterministic recall, no LLM, parallel — serial-by-caution was
halting in disguise):
- recall_at_k.py: returns rank -> recall@1/@3/@k from one retrieval
(verified offline). A coarse k hides rank-only lifts.
- mine_questions.py: numeral/accent/hyphen/honorific/amp/brit
ground-truth classes; fixtures committed.
- Measured @1 headroom verdicts: accent SHIP (this commit);
honorific 45% / brit 50% = real headroom (build next); hyphen
90% = existing #000007 already delivers, NOTHING to build (the
measure-the-unmeasured-thing check pays off); amp 82% = no fold
needed (prevalence-overranked, instrument kills it cheaply).
CLAUDE.md bench-maxing: two measured lessons codified — report
recall@1/@3/@k (a lenient k hides rank lifts; prevalence != miss-
rate), and fan out independent measurements (serial-by-caution is
halting). Full suite 2488 passed, 0 regressions (accent-fold is
hot-path in _title_query_tokens); real-path test (FakeSource->
ingest->query()->real _Hit).
242 lines
8.2 KiB
JSON
242 lines
8.2 KiB
JSON
[
|
|
{
|
|
"question": "what is The Amazing Spider Man (comic book)?",
|
|
"target_title": "The Amazing Spider-Man (comic book)",
|
|
"target_root": "d53601e2e4f7660788840ffc97520b3c8f8fd9cbe72e868274096ab16e08baee",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Augustin Jean Fresnel?",
|
|
"target_title": "Augustin-Jean Fresnel",
|
|
"target_root": "afa063eca8b1cc23f2d8b474a79e0f7abab1613e5a6156cf9ab3ab79805f3c2f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is André Marie Ampère?",
|
|
"target_title": "André-Marie Ampère",
|
|
"target_root": "46c0ca49bbebb1895e66a621afeb28f207766d76be1a2d40d7ffd567fa757478",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Aster CT 80?",
|
|
"target_title": "Aster CT-80",
|
|
"target_root": "3bd6a0d66582ca9aafc97acc0e4e8c41608baa22fd218ea21d7cf83e9a90b4e9",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Anti globalization movement?",
|
|
"target_title": "Anti-globalization movement",
|
|
"target_root": "e0a5d7170fce542cae4f00c1b98acb98b6fe7b8f6033140419fa023318ae65bd",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Abd ar Rahman I?",
|
|
"target_title": "Abd ar-Rahman I",
|
|
"target_root": "1e62aee7c9143bcf070585b0af30e0ea6d3fd3e434b6a6c4fc1c37dfe2fde651",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Abd ar Rahman V?",
|
|
"target_title": "Abd ar-Rahman V",
|
|
"target_root": "de82c6f4adfa9b8721e708c5056a0fd6a4c4755711b60e9c16ae1d02e3e224db",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Anti Ballistic Missile Treaty?",
|
|
"target_title": "Anti-Ballistic Missile Treaty",
|
|
"target_root": "b68d02841d328befe2c4cba80d60d3d45130718925b284c2941d2521e37937b9",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is AGM 88 HARM?",
|
|
"target_title": "AGM-88 HARM",
|
|
"target_root": "cc0bf6f28be2f070cb8514ae32ef439a5f7eda8e00aa2903ebbb29754bbb8e35",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Lockheed AC 130?",
|
|
"target_title": "Lockheed AC-130",
|
|
"target_root": "5c1118c0c162c30a7cc774f069c4b631c19a7572146cc6afce724efdbabf4ed3",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is CIM 10 Bomarc?",
|
|
"target_title": "CIM-10 Bomarc",
|
|
"target_root": "cde6d4f22963a409a2aab12f784c97915c48fb40fd5bff8f3c5ca569322e192d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is North American B 25 Mitchell?",
|
|
"target_title": "North American B-25 Mitchell",
|
|
"target_root": "76551e89d1e598a858947a01e843506ae8c46c73734a05f91e75355303a06971",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Bain marie?",
|
|
"target_title": "Bain-marie",
|
|
"target_root": "bbe25b304971c272e0414cbf48d3cc1fd864b808cc681ab95dac9280c1d000a3",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Cross dressing?",
|
|
"target_title": "Cross-dressing",
|
|
"target_root": "f888c381efe454debf20d54f19dc8963ae259725570e46e8d786043ddd23617f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Call of Cthulhu (role playing game)?",
|
|
"target_title": "Call of Cthulhu (role-playing game)",
|
|
"target_root": "c1ac04f507ac1f7794ef465ae5091d39eddb030c0329a20e5bb3576c3bd6dc36",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Context sensitive?",
|
|
"target_title": "Context-sensitive",
|
|
"target_root": "daf91b754de7c52fce3c8f23741a4595df935403ddff744ab3fd00f500d0da1a",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Boeing C 17 Globemaster III?",
|
|
"target_title": "Boeing C-17 Globemaster III",
|
|
"target_root": "abcaff97ccfc0bdcccf3ac107ddd4e4b9931ad4a8696987c589e79f1b1444e29",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Chiang Kai shek?",
|
|
"target_title": "Chiang Kai-shek",
|
|
"target_root": "28b23bbba487cb03e004896c67af97fec6ae2ef92dc2becea94f0bb2bd392c3c",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Context free language?",
|
|
"target_title": "Context-free language",
|
|
"target_root": "ecbf405046a393e614f37cc3a0135238e554011bb314c457cb088a581e28f342",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is C* algebra?",
|
|
"target_title": "C*-algebra",
|
|
"target_root": "f286b4b5bc856209b3f6daea90947ac70f0cc3a5f8e719cb5d00bd0c8daa4006",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Computer generated imagery?",
|
|
"target_title": "Computer-generated imagery",
|
|
"target_root": "107e71d0d1efc8c3ffecb5baff42e1a340aa8c2a65c1e14d410d7caffe679f24",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Lockheed C 130 Hercules?",
|
|
"target_title": "Lockheed C-130 Hercules",
|
|
"target_root": "f55294ffdfdaedad52f197b098a8c5684e5534a60d31c40f94ba3102645ea311",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Covenant breaker?",
|
|
"target_title": "Covenant-breaker",
|
|
"target_root": "1602c6273b5239feca1ed84bc62e1730b1d3004d03428235ff46de62d2ce19e1",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Comprehensive Nuclear Test Ban Treaty?",
|
|
"target_title": "Comprehensive Nuclear-Test-Ban Treaty",
|
|
"target_root": "a9e1899f647960cb33f437b5a2be04cfc739aac3da9bbc6a3a724864390a66d4",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Double slit experiment?",
|
|
"target_title": "Double-slit experiment",
|
|
"target_root": "8e9293f1bb3a10a8266ceecad25ef30227f029b03aa7681ba5ceb219780e374d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Double ended queue?",
|
|
"target_title": "Double-ended queue",
|
|
"target_root": "b1d5f50ef8b84ecf3a20600f9c5394cda71d2eb3f5b9de11ea54b02b33534889",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Cost push inflation?",
|
|
"target_title": "Cost-push inflation",
|
|
"target_root": "890e9c85d9567ab08aa9aebb5249a805b135d2d9db9e41133d5802cac55c474e",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Eductor jet pump?",
|
|
"target_title": "Eductor-jet pump",
|
|
"target_root": "135f3206dda2874696ba3417514338a05b2bc187de3076524446864d59b3242b",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Eight ball?",
|
|
"target_title": "Eight-ball",
|
|
"target_root": "cf8dfa5eb3a35215e5555de9e695bd4b142bb8cc9efa1757655859c42f0dd22b",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Élisabeth Louise Vigée Le Brun?",
|
|
"target_title": "Élisabeth-Louise Vigée-Le Brun",
|
|
"target_root": "73e60587c35c69d31312f9585ae1a43dcf0e2b9ba100ce573fcbd95fc98bb778",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Evidence based medicine?",
|
|
"target_title": "Evidence-based medicine",
|
|
"target_root": "6db654c484d855d8c24dcc9d2ac163709ec054c8083bb333e4bdea6e7edd361f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is E Prime?",
|
|
"target_title": "E-Prime",
|
|
"target_root": "20c3570fa3140fc3429f4a95695fca2afeae1cf686d15fadb4929bca5fd9eb91",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Boeing E 3 Sentry?",
|
|
"target_title": "Boeing E-3 Sentry",
|
|
"target_root": "efc1f15bc037ec39561326cc240100b3f39528ca782b7a634408c80f2599bb77",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Emperor Go En'yū?",
|
|
"target_title": "Emperor Go-En'yū",
|
|
"target_root": "1eda22a89b6650822efc097067dbc87e5d74a42b3f93dd1184c9f25f97e0e6a3",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Field programmable gate array?",
|
|
"target_title": "Field-programmable gate array",
|
|
"target_root": "a000892caf082200a3fcc3a6f1b3126f2c5d4151fe189f99808fde02b632aa11",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Five spice powder?",
|
|
"target_title": "Five-spice powder",
|
|
"target_root": "bf0de4cc5ed75d9c655f008e3a9114409e8cfabec49b84aa95d0832ff529c584",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Four poster?",
|
|
"target_title": "Four-poster",
|
|
"target_root": "a1c0787760f6eb2b01d3f681b39b132227de485f18cc64914282829cdacfea8d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Flip flop (electronics)?",
|
|
"target_title": "Flip-flop (electronics)",
|
|
"target_root": "56d5cb2d7ff9fcf3206af52f7d7d86e0c30478b940988725c89e6a863d63dc98",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Guinea Bissau?",
|
|
"target_title": "Guinea-Bissau",
|
|
"target_root": "1b76ef7f4e81065dd11292e4399fddc8facf1711fc93a962096e3082713d20a2",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Politics of Guinea Bissau?",
|
|
"target_title": "Politics of Guinea-Bissau",
|
|
"target_root": "d8a22dbea22e127ce9bd141dbf2996b8b9efe94f632d0b9c2cf2f47e497f0106",
|
|
"shard": "000.db"
|
|
}
|
|
]
|