russell@unturf.com
b573c592d8
Second MEASURED fold-search win, and the instrument correcting my own
premature call. accent-fold ON vs OFF on the mined accent fixture:
recall@1 55% -> 85% (+30pp), rank-1 22/40 -> 34/40. recall@8 was
flat (95->98) — a too-lenient k nearly got a real lever wrongly
reverted; @1/@3 is the resolution that drives primary-source
selection. _accent_fold_variants: ASCII-fold then re-tokenise so a
diacritic title ("Béla Bartók", which _TITLE_TOKEN_RE otherwise
fragments to junk) matches the ASCII form a user types. Additive+
symmetric, no-op on pure-ASCII (zero effect on non-accent
queries/titles), mirrors _hyphen_fold_variants (#000007).
Also fixes a defect I shipped in a3ac653: an orphaned duplicate
body left as dead code after `return base` in _title_query_tokens
(unreachable — numeral-fold behaviour/measurement were valid — but
cruft; removed).
Fold-search factory, fanned out across the full survey backlog
(deterministic recall, no LLM, parallel — serial-by-caution was
halting in disguise):
- recall_at_k.py: returns rank -> recall@1/@3/@k from one retrieval
(verified offline). A coarse k hides rank-only lifts.
- mine_questions.py: numeral/accent/hyphen/honorific/amp/brit
ground-truth classes; fixtures committed.
- Measured @1 headroom verdicts: accent SHIP (this commit);
honorific 45% / brit 50% = real headroom (build next); hyphen
90% = existing #000007 already delivers, NOTHING to build (the
measure-the-unmeasured-thing check pays off); amp 82% = no fold
needed (prevalence-overranked, instrument kills it cheaply).
CLAUDE.md bench-maxing: two measured lessons codified — report
recall@1/@3/@k (a lenient k hides rank lifts; prevalence != miss-
rate), and fan out independent measurements (serial-by-caution is
halting). Full suite 2488 passed, 0 regressions (accent-fold is
hot-path in _title_query_tokens); real-path test (FakeSource->
ingest->query()->real _Hit).
242 lines
8.4 KiB
JSON
242 lines
8.4 KiB
JSON
[
|
|
{
|
|
"question": "what is Heckler and Koch?",
|
|
"target_title": "Heckler & Koch",
|
|
"target_root": "978210367f209c583186c0c826506da54b9ca1414390325960b6c9535d9c34e5",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Science and Environmental Policy Project?",
|
|
"target_title": "Science & Environmental Policy Project",
|
|
"target_root": "0ea78023a0dd8f2c10334fd1e5aa659c50b1ed2b21d0565186969d9a55a194e2",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Texas A and M University?",
|
|
"target_title": "Texas A&M University",
|
|
"target_root": "996f9a9e7841fc4b27b8b58f08feedaefcf8be72337abc267526390c21d10554",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Pratt and Whitney?",
|
|
"target_title": "Pratt & Whitney",
|
|
"target_root": "dc90f35ad29a9c39d2885005ebaff3877bc4d5715d6e82e92ac0ff332682fc10",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Ernst and Young?",
|
|
"target_title": "Ernst & Young",
|
|
"target_root": "6a1a662c541927efcee315bc87a20875debbadbfc3ff976e6e146697a3025f54",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Waterloo and City line?",
|
|
"target_title": "Waterloo & City line",
|
|
"target_root": "afd255f827b62f26d8ed616ef73a8d5761ac3957a3095dd1a60216983f363d29",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Duany Plater-Zyberk and Company?",
|
|
"target_title": "Duany Plater-Zyberk & Company",
|
|
"target_root": "b8b97a7f838ac2116c6de410b628bc9cc8ef7e6aecc9e1edbbd3d9d482ba7d71",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is The Sandman: Fables and Reflections?",
|
|
"target_title": "The Sandman: Fables & Reflections",
|
|
"target_root": "2a7e6bcd2a102495062337590b910d2cc5231b970304658bfc246c4e2df43bdf",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Rape, Abuse and Incest National Network?",
|
|
"target_title": "Rape, Abuse & Incest National Network",
|
|
"target_root": "95954855b7ffc3a93eb567ec14f992fa42e8208777a1abc1a1bd19c41541c721",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Hilton Hotels and Resorts?",
|
|
"target_title": "Hilton Hotels & Resorts",
|
|
"target_root": "f06e50b70959ea4df96f549200fc94c7602443905a3b8deee32942f6b7f53b2f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is North Walsham and Dilham Canal?",
|
|
"target_title": "North Walsham & Dilham Canal",
|
|
"target_root": "3c43a24f38fb9ab784094bf8683c79de6fb0db425ec6d6729785472424e93812",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Bill and Melinda Gates Foundation?",
|
|
"target_title": "Bill & Melinda Gates Foundation",
|
|
"target_root": "ff90a5fd4baca71f65c1d1886bb7aafbea7808b9fbdc47d423c8f077f612f4b1",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Question Mark and the Mysterians?",
|
|
"target_title": "Question Mark & the Mysterians",
|
|
"target_root": "140e8733583e32753ac5284c3d9fa19a83f99c779e3fb7eac6ce24678d3eebf5",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is A and B?",
|
|
"target_title": "A&B",
|
|
"target_root": "89792d9631cb85464c18a217608029866f263e7fef32a802b4a16ccaca0daff5",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Chivalry and Sorcery?",
|
|
"target_title": "Chivalry & Sorcery",
|
|
"target_root": "b54eb248a89978066bae0f64111f02124132d8a52b3c60669f7cc0055d8ced62",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is II and III?",
|
|
"target_title": "II & III",
|
|
"target_root": "2c044b0037189e81caa0c3ba5d65fc5bce0c00e4538eba57b02a221dc005d203",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Standard and Poor's?",
|
|
"target_title": "Standard & Poor's",
|
|
"target_root": "49a1135e45aa1ceccdc4c6b1f40b76032ac8753801960aa5bdcc63a613d343ef",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Cheech and Chong?",
|
|
"target_title": "Cheech & Chong",
|
|
"target_root": "22d3997ec9786757c58879ca7d16c6662b1ea4c2dc0f8a6104025b5a645b0c7f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Sasha and John Digweed?",
|
|
"target_title": "Sasha & John Digweed",
|
|
"target_root": "f23eeac55103a33e8ed53dda226e96428ee043b0fe8f6b34447a8432d19742ff",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Murat and Jose?",
|
|
"target_title": "Murat & Jose",
|
|
"target_root": "7edb97a9c375502a60e93a7d8fa4317e12895339aab516d21d219cafa0336655",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Valleys and Cardiff Local Routes?",
|
|
"target_title": "Valleys & Cardiff Local Routes",
|
|
"target_root": "01f5e16c6b6ef71f60d798b4c8bb79dedbaca8fa9746289859a48120738f46dd",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is The College of William and Mary?",
|
|
"target_title": "The College of William & Mary",
|
|
"target_root": "2686b673858f3310b59052e230c381ac7ae687632041409368cb23c628c556ca",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Industrial Light and Magic?",
|
|
"target_title": "Industrial Light & Magic",
|
|
"target_root": "bef0cc77efa128c3c89796f1a8f0f26c1f491e34fbd13440ddc9b3b3193883f0",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Barnes and Noble?",
|
|
"target_title": "Barnes & Noble",
|
|
"target_root": "94b4e7e483c8ca38521e8f85e4c7bdd31d9f8b1b51a599d303eea2f362b8206f",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Law and Order: Special Victims Unit?",
|
|
"target_title": "Law & Order: Special Victims Unit",
|
|
"target_root": "ab5b911577a527a4e526613d35d2f7dfd5bd6fb44dc1fb513ddffa8fee30c0eb",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Grammy Award for Best R and B Song?",
|
|
"target_title": "Grammy Award for Best R&B Song",
|
|
"target_root": "2ea1b11647fbdb0f9918000c159750e4220cf9fdf219b0ef3434be86148ed8af",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Ike and Tina Turner?",
|
|
"target_title": "Ike & Tina Turner",
|
|
"target_root": "9ed9c3e0d8fd1b42b94647ef22f70b3bf4b5a21ec90aa604d26c4892e126accc",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is H and M?",
|
|
"target_title": "H&M",
|
|
"target_root": "cf790fda061f316a16d9fd07e5db1af8a0ca8bc82bed84e9eb0fd2b573017845",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Yesterday and Today?",
|
|
"target_title": "Yesterday & Today",
|
|
"target_root": "ff1cc6c11ec37effc4a0e202b1197ec4c670b5359b5b8c8f0dfb36a655ba8d6d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Open Fire (Y and T album)?",
|
|
"target_title": "Open Fire (Y&T album)",
|
|
"target_root": "ea2873e3ae0ba0c5b8efbef26795eafc6b94b5354caea5a6c1fb800cca6b951d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Funk and Wagnalls?",
|
|
"target_title": "Funk & Wagnalls",
|
|
"target_root": "2e420c61ea81778cb0ad6df0d24edc5383f3d0c12528f5fd93d609cbf9ca4a27",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is South Park: Bigger, Longer and Uncut?",
|
|
"target_title": "South Park: Bigger, Longer & Uncut",
|
|
"target_root": "46dc76de50c40c2597ad010bd21890889ddaa9b83cfb2f693703202a94323490",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Lewis and Clark College?",
|
|
"target_title": "Lewis & Clark College",
|
|
"target_root": "5f52f2383a2eadf7fcc1835a18fd4de66a8a7eaec547356bc82242268efa064c",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Love and Pop?",
|
|
"target_title": "Love & Pop",
|
|
"target_root": "417083ad66a3ce0972933d83191921b94101001abf159b49c8e055233aa22b2c",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Barnes and Barnes?",
|
|
"target_title": "Barnes & Barnes",
|
|
"target_root": "a9251d47e8449b5bc94f4b1ebe5bbe0aaf95e9f1df0ec760546ecdaba1208de3",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Sky (UK and Ireland)?",
|
|
"target_title": "Sky (UK & Ireland)",
|
|
"target_root": "eeeed19fb1fbe2ffd96abce62ba11b50846960a3e7039350d2d92ffa5db73f7a",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Sergio and The Ladies?",
|
|
"target_title": "Sergio & The Ladies",
|
|
"target_root": "931c0417680eeef901977d8b03b82b9476bd03d7c16b388612e748f3f88ec1b8",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is B and B?",
|
|
"target_title": "B&B",
|
|
"target_root": "bbf6e2ce9136d564c504c596b648094a8672e75f7e99b3c990630c00736c5f41",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Starwood Hotels and Resorts Worldwide?",
|
|
"target_title": "Starwood Hotels & Resorts Worldwide",
|
|
"target_root": "ba0402f6975a9ab58bbcd23b6ae94ac824c648852a1070ec06814d56a377bd26",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is FRANC 2D and 3D?",
|
|
"target_title": "FRANC 2D&3D",
|
|
"target_root": "6af75043e187533640266428c432cf209a3895b19ad85d9c3b090069adfd9e8a",
|
|
"shard": "000.db"
|
|
}
|
|
]
|