russell@unturf.com
b573c592d8
Second MEASURED fold-search win, and the instrument correcting my own
premature call. accent-fold ON vs OFF on the mined accent fixture:
recall@1 55% -> 85% (+30pp), rank-1 22/40 -> 34/40. recall@8 was
flat (95->98) — a too-lenient k nearly got a real lever wrongly
reverted; @1/@3 is the resolution that drives primary-source
selection. _accent_fold_variants: ASCII-fold then re-tokenise so a
diacritic title ("Béla Bartók", which _TITLE_TOKEN_RE otherwise
fragments to junk) matches the ASCII form a user types. Additive+
symmetric, no-op on pure-ASCII (zero effect on non-accent
queries/titles), mirrors _hyphen_fold_variants (#000007).
Also fixes a defect I shipped in a3ac653: an orphaned duplicate
body left as dead code after `return base` in _title_query_tokens
(unreachable — numeral-fold behaviour/measurement were valid — but
cruft; removed).
Fold-search factory, fanned out across the full survey backlog
(deterministic recall, no LLM, parallel — serial-by-caution was
halting in disguise):
- recall_at_k.py: returns rank -> recall@1/@3/@k from one retrieval
(verified offline). A coarse k hides rank-only lifts.
- mine_questions.py: numeral/accent/hyphen/honorific/amp/brit
ground-truth classes; fixtures committed.
- Measured @1 headroom verdicts: accent SHIP (this commit);
honorific 45% / brit 50% = real headroom (build next); hyphen
90% = existing #000007 already delivers, NOTHING to build (the
measure-the-unmeasured-thing check pays off); amp 82% = no fold
needed (prevalence-overranked, instrument kills it cheaply).
CLAUDE.md bench-maxing: two measured lessons codified — report
recall@1/@3/@k (a lenient k hides rank lifts; prevalence != miss-
rate), and fan out independent measurements (serial-by-caution is
halting). Full suite 2488 passed, 0 regressions (accent-fold is
hot-path in _title_query_tokens); real-path test (FakeSource->
ingest->query()->real _Hit).
242 lines
7.8 KiB
JSON
242 lines
7.8 KiB
JSON
[
|
||
{
|
||
"question": "what is Casa Batllo?",
|
||
"target_title": "Casa Batlló",
|
||
"target_root": "c7cb48984f93abd73b6d9166fae8adfa1c11cf93240a2a5673c7006d729366f6",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Andre-Marie Ampere?",
|
||
"target_title": "André-Marie Ampère",
|
||
"target_root": "46c0ca49bbebb1895e66a621afeb28f207766d76be1a2d40d7ffd567fa757478",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Antoni Gaudi?",
|
||
"target_title": "Antoni Gaudí",
|
||
"target_root": "70ca4151b7cb1b6c5d9198ca53cb60d2ea932cfd2db67ab0e196ec95b27c9ffe",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Alcobaca (Portugal)?",
|
||
"target_title": "Alcobaça (Portugal)",
|
||
"target_root": "1bf97449bb4d7932e7b0ede1048dc59ff032d85ffc320c55635a78f3f3eee848",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Bifrost?",
|
||
"target_title": "Bifröst",
|
||
"target_root": "5866b76e99adc35eb76df8369ffc816af9c3731bfbe253e4f2773792bacf322a",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Bootes?",
|
||
"target_title": "Boötes",
|
||
"target_root": "f4666faa11a94258c05853bc4fa77d95f4934b0db93072aa1ffe44436351910c",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Bela Bartok?",
|
||
"target_title": "Béla Bartók",
|
||
"target_root": "9c32c437787b598486b73ca1c3d163e8c1078d371e44a2342b4dff00a8da9b2c",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Blue Oyster Cult?",
|
||
"target_title": "Blue Öyster Cult",
|
||
"target_root": "f1164863229cba552651c4e21efabe7f8036a40292128a2353909438b7a4a240",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Bjorn Borg?",
|
||
"target_title": "Björn Borg",
|
||
"target_root": "7e42aee6fc7f3e1f8a768c8675165263d9e38f0adf1540944750638b19a6d259",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Barıs Manco?",
|
||
"target_title": "Barış Manço",
|
||
"target_root": "38403d8732dcb022ebe2d09710ac90f0616bb6710b8033ca8ed1f1770ef5803b",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Transport in Cote d'Ivoire?",
|
||
"target_title": "Transport in Côte d'Ivoire",
|
||
"target_root": "2e3411d102c03f245e9906ef61da6ff3fede9600feaac53c39df1f69afd5790c",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Crannog?",
|
||
"target_title": "Crannóg",
|
||
"target_root": "12b46b289c0db835b8d95c7485f6b86e2b7b104b7c39e597ec532458345550a8",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emily Bronte?",
|
||
"target_title": "Emily Brontë",
|
||
"target_root": "78c53cd81b568539d6e38454482a0ba97fd86bfa4a590db8670c99dc215c2b58",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Evariste Galois?",
|
||
"target_title": "Évariste Galois",
|
||
"target_root": "3da640319defb1ffe7b178a4a424de2b283d96fc62672d7c36e91a3ce9943036",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Elblag?",
|
||
"target_title": "Elbląg",
|
||
"target_root": "b5ec92e119576a7c04a7600fb60ac6058edcd971d29e798699215065404a3580",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Erwin Schrodinger?",
|
||
"target_title": "Erwin Schrödinger",
|
||
"target_root": "0ead5314753e1894bc2eaa220bf2cc93c639997128dac3a517d8b4661be228cc",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Elisabeth-Louise Vigee-Le Brun?",
|
||
"target_title": "Élisabeth-Louise Vigée-Le Brun",
|
||
"target_root": "73e60587c35c69d31312f9585ae1a43dcf0e2b9ba100ce573fcbd95fc98bb778",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Shomu?",
|
||
"target_title": "Emperor Shōmu",
|
||
"target_root": "61bc5d5b019b07bb92f9572dab03afedc236dfad43f87d9c0381718f243afd2d",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Yomei?",
|
||
"target_title": "Emperor Yōmei",
|
||
"target_root": "42559a42ca8213a5a87187b7c2a05d44ea979bc0bdb8e891ad8b8d5b138db69c",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Kotoku?",
|
||
"target_title": "Emperor Kōtoku",
|
||
"target_root": "4aeed736732080574b552f04ec71097b4889c0bef3c33de5a74593a7f3e90c49",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Empress Koken?",
|
||
"target_title": "Empress Kōken",
|
||
"target_root": "52a095549a02229caa4bde6bcd2c564173219329b36929cb9507d74ea103f49a",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Go-En'yu?",
|
||
"target_title": "Emperor Go-En'yū",
|
||
"target_root": "1eda22a89b6650822efc097067dbc87e5d74a42b3f93dd1184c9f25f97e0e6a3",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Koan?",
|
||
"target_title": "Emperor Kōan",
|
||
"target_root": "511757327322ff07b858641fdb97a561bf5601c43bafcff5ee56c42fc84d20d9",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Emperor Chuai?",
|
||
"target_title": "Emperor Chūai",
|
||
"target_root": "eb7ebfeb5117d4982f657d52604321eb95b6aec001a7d44e081769471285a160",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Francisco Alvares?",
|
||
"target_title": "Francisco Álvares",
|
||
"target_root": "06cb19fc595ef25636dcb7a25354109e82322a59d732dfab8d2104bbabda5a3f",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Felix Guattari?",
|
||
"target_title": "Félix Guattari",
|
||
"target_root": "53ce58083d07db46e928cfc6021a3e8af2e8d6a3e04d93a530ac25dbec672d17",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Gota Canal?",
|
||
"target_title": "Göta Canal",
|
||
"target_root": "be181ec1af0cd06845a315b1454c766a8b70d5755b432733ee313cf3cf7e222f",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Goran Bregovic?",
|
||
"target_title": "Goran Bregović",
|
||
"target_root": "671c9e75c1d1f3b7139bfa328b2b45dd0bbfbaca8b8e26e6a3e4dae806f25164",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Godel's completeness theorem?",
|
||
"target_title": "Gödel's completeness theorem",
|
||
"target_root": "8b5321e865294e0aa60353714ad65687ffd1a9afa4eb35ab351bfbf17dc8b2c7",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Gilbert Arthur a Beckett?",
|
||
"target_title": "Gilbert Arthur à Beckett",
|
||
"target_root": "973b6bfbe56a78a864b821048f8068b4dcfe602d97de264b21e40e23bc73b9f7",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is GEANT?",
|
||
"target_title": "GÉANT",
|
||
"target_root": "68a6935d77e47fab693eff9f5cbf1ee85ccb88ce0f6f9296b3904457f5e057dc",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Herge?",
|
||
"target_title": "Hergé",
|
||
"target_root": "6e442f3603465ea8802aed2bcdeb4cf6843fb3d313c4c469a3e4b24ee611a081",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Ismet Inonu?",
|
||
"target_title": "İsmet İnönü",
|
||
"target_root": "0eee497696dccd84a7da7b6eb30437f8f16815213f115e7a0bbfd49252336052",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Gyula Andrassy?",
|
||
"target_title": "Gyula Andrássy",
|
||
"target_root": "669fc54d24191a8d02430773239c69e8c5e96219dc895edaec8bfcf3d60aa316",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Jean-Francois Millet?",
|
||
"target_title": "Jean-François Millet",
|
||
"target_root": "9f7f942db3b566855a28ec765cf10ff66302dc26e2716788e4db54977827e9ea",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is House of Karađorđevic?",
|
||
"target_title": "House of Karađorđević",
|
||
"target_root": "6cf69d71367fa69d540fe16d85a25f5a3ef5763108dfb9e0974fd0171d1f9e5a",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Kunstlerroman?",
|
||
"target_title": "Künstlerroman",
|
||
"target_root": "3185b8bc72db6fc984585ae896fe090cc58528c8f7d6f93a26f15201cbfbcbc2",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Musee du Louvre?",
|
||
"target_title": "Musée du Louvre",
|
||
"target_root": "f41c5777f3ca280ce9fc74cd5d2b58e4da947db5e1d10e5ca30a801943224100",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is Lubeck?",
|
||
"target_title": "Lübeck",
|
||
"target_root": "be78b002f1ccd5ab64f738ed8e550f5439a11827d8ccb9d84081c4383201e4ab",
|
||
"shard": "000.db"
|
||
},
|
||
{
|
||
"question": "what is La Tene culture?",
|
||
"target_title": "La Tène culture",
|
||
"target_root": "494f1dc71c13f5fbeb6aa114377104f1bcbbb824df1776fd1d5c424754570ae3",
|
||
"shard": "000.db"
|
||
}
|
||
]
|