russell@unturf.com
b573c592d8
Second MEASURED fold-search win, and the instrument correcting my own
premature call. accent-fold ON vs OFF on the mined accent fixture:
recall@1 55% -> 85% (+30pp), rank-1 22/40 -> 34/40. recall@8 was
flat (95->98) — a too-lenient k nearly got a real lever wrongly
reverted; @1/@3 is the resolution that drives primary-source
selection. _accent_fold_variants: ASCII-fold then re-tokenise so a
diacritic title ("Béla Bartók", which _TITLE_TOKEN_RE otherwise
fragments to junk) matches the ASCII form a user types. Additive+
symmetric, no-op on pure-ASCII (zero effect on non-accent
queries/titles), mirrors _hyphen_fold_variants (#000007).
Also fixes a defect I shipped in a3ac653: an orphaned duplicate
body left as dead code after `return base` in _title_query_tokens
(unreachable — numeral-fold behaviour/measurement were valid — but
cruft; removed).
Fold-search factory, fanned out across the full survey backlog
(deterministic recall, no LLM, parallel — serial-by-caution was
halting in disguise):
- recall_at_k.py: returns rank -> recall@1/@3/@k from one retrieval
(verified offline). A coarse k hides rank-only lifts.
- mine_questions.py: numeral/accent/hyphen/honorific/amp/brit
ground-truth classes; fixtures committed.
- Measured @1 headroom verdicts: accent SHIP (this commit);
honorific 45% / brit 50% = real headroom (build next); hyphen
90% = existing #000007 already delivers, NOTHING to build (the
measure-the-unmeasured-thing check pays off); amp 82% = no fold
needed (prevalence-overranked, instrument kills it cheaply).
CLAUDE.md bench-maxing: two measured lessons codified — report
recall@1/@3/@k (a lenient k hides rank lifts; prevalence != miss-
rate), and fan out independent measurements (serial-by-caution is
halting). Full suite 2488 passed, 0 regressions (accent-fold is
hot-path in _title_query_tokens); real-path test (FakeSource->
ingest->query()->real _Hit).
242 lines
8.8 KiB
JSON
242 lines
8.8 KiB
JSON
[
|
|
{
|
|
"question": "what is Finnish defense Forces?",
|
|
"target_title": "Finnish Defence Forces",
|
|
"target_root": "6114cd5bf6cfcd9b69fdb7dbc07842d6e610c3e2dced64cf61c8f25c6cbc5948",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Hopewell center, Hong Kong?",
|
|
"target_title": "Hopewell Centre, Hong Kong",
|
|
"target_root": "1e6245dfd251badf7caf1022a673a9935f036853b958c58f76e4432dd353bf91",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is labor economics?",
|
|
"target_title": "Labour economics",
|
|
"target_root": "ad7a31d34d70fb673a30deb8a458a6e924fc3c41970b30752ce5d14d55996260",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is World organization for Animal Health?",
|
|
"target_title": "World Organisation for Animal Health",
|
|
"target_root": "e0e1f1c1d3e206192d3e74278426af1587c055ab12d35211fbefa356b827e3f4",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Papua New Guinea defense Force?",
|
|
"target_title": "Papua New Guinea Defence Force",
|
|
"target_root": "1d6da0aa77ad422ce9f23baba3832c67547b09ac6945733c4d484491f96ca3a6",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is programr?",
|
|
"target_title": "Programmer",
|
|
"target_root": "f7109a928748c48b88bd213a59bcf1205777c7aaed49209d5c57f4884d578c10",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Tonga defense Services?",
|
|
"target_title": "Tonga Defence Services",
|
|
"target_root": "f5754650b7149e5e5c3cbf23d8876eafb5070c3571c6f8dae6322c75e09f8342",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Trinidad and Tobago defense Force?",
|
|
"target_title": "Trinidad and Tobago Defence Force",
|
|
"target_root": "a816b09a0aab2bcf777039b3ac5a26bbc20fb812b2a45bebb317333d21aa5670",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Zambian defense Force?",
|
|
"target_title": "Zambian Defence Force",
|
|
"target_root": "0a81b1428df734fefcf16d56d4bc33e9121720974e03238eff2c2d0d3b7aaaad",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is defense Signals Directorate?",
|
|
"target_title": "Defence Signals Directorate",
|
|
"target_root": "d9abb78507855e0318888f23e1a5f513cb3c3e5710649577e0874d6b54039725",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is English Renaissance theater?",
|
|
"target_title": "English Renaissance theatre",
|
|
"target_root": "dd17095f643aa6fcbe309f3d28df76cea80817c5a26c77f3934caaa04ee7d650",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is labor Party (Netherlands)?",
|
|
"target_title": "Labour Party (Netherlands)",
|
|
"target_root": "76aec0d0f059d87a91d121b83a16b4e39ecb4e84974f2490871e417603e83286",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is The Luzhin defense?",
|
|
"target_title": "The Luzhin Defence",
|
|
"target_root": "cc09adf74f7a1c960b1051af7685e9e9510a7400cb878ba05185a9872d170996",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Asia-Pacific Network Information center?",
|
|
"target_title": "Asia-Pacific Network Information Centre",
|
|
"target_root": "1eb9e5765f58ae5c3dca51be14997c998dab3b8faaae727d97f17cc934dd0345",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Soyuz program?",
|
|
"target_title": "Soyuz programme",
|
|
"target_root": "acf6a8b2a5f9ec258861702edeaa7f3e4cc7af298a84fa3a6294e356bc47a44a",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is center Georges Pompidou?",
|
|
"target_title": "Centre Georges Pompidou",
|
|
"target_root": "3b7f02b2b5ddf5dac61a054aef90fabd44be2629de79d8b4165eaa7ecaba1201",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Amphitheater?",
|
|
"target_title": "Amphitheatre",
|
|
"target_root": "c63e6c7a60bbaf19dc65d2b869d86274ba1268bd3790366fdf0a14476907a7eb",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Australian defense Force?",
|
|
"target_title": "Australian Defence Force",
|
|
"target_root": "566a69ad2196a5e5d5f084a375dee9231f0344e0d4c085a79990b2291265edd0",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is center County, Pennsylvania?",
|
|
"target_title": "Centre County, Pennsylvania",
|
|
"target_root": "80487e3ec0141594504f4bc68cd858d67d6ef085c10125e40fc5533a234c5c94",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is organization internationale de la Francophonie?",
|
|
"target_title": "Organisation internationale de la Francophonie",
|
|
"target_root": "8b02358ebc82c033b55c85972344cc54310b34a5a2a0a921d7f97f59677377e2",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Luna program?",
|
|
"target_title": "Luna programme",
|
|
"target_root": "add3b8760cbe907a2a3076b91466afa7f0e5957d47c56fdcf7e72d9e37e7c1ba",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Man and the Biosphere program?",
|
|
"target_title": "Man and the Biosphere Programme",
|
|
"target_root": "c92585fe422ad5ff0467a57f0c854319985e15b66bb5fd56ac0664b19650ec70",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is centerville, Maryland?",
|
|
"target_title": "Centreville, Maryland",
|
|
"target_root": "148a7ab7213c794ebdc8875f6ff9e5697a1699bc1923e08857e81f133ea99bbd",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Boggs Township, center County, Pennsylvania?",
|
|
"target_title": "Boggs Township, Centre County, Pennsylvania",
|
|
"target_root": "ff94137ff22e6330cfc72972ff949b6f45a56e590d08251ec6d14c8341cd8e2d",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Ferguson Township, center County, Pennsylvania?",
|
|
"target_title": "Ferguson Township, Centre County, Pennsylvania",
|
|
"target_root": "b73fc3e57cc5a9d6574f772f3fb1c921a04b08518b0ff725ddaa8ba983496ede",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Harris Township, center County, Pennsylvania?",
|
|
"target_title": "Harris Township, Centre County, Pennsylvania",
|
|
"target_root": "37bc6558d805cbc30ef085cacc0c407142745e994a38daa0f0d64b9397e5eac6",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Huston Township, center County, Pennsylvania?",
|
|
"target_title": "Huston Township, Centre County, Pennsylvania",
|
|
"target_root": "e633a8476cb1d547718cf6952b235762c2762a0d49a4dc8c0e313e19c3ad0d2e",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Patton Township, center County, Pennsylvania?",
|
|
"target_title": "Patton Township, Centre County, Pennsylvania",
|
|
"target_root": "5746ee9bb4d96fed1695555c3e0c910b1f2f701046ebdaf229aaeb035b41e513",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Union Township, center County, Pennsylvania?",
|
|
"target_title": "Union Township, Centre County, Pennsylvania",
|
|
"target_root": "a16566edef459502e4a188d8edb01710d9644732ab8b69b0baa47fdb556042d6",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Worth Township, center County, Pennsylvania?",
|
|
"target_title": "Worth Township, Centre County, Pennsylvania",
|
|
"target_root": "3e2050e2372af027615ddc47e8999aa179f58b92cc0bc992f25b444b7daf7cc7",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is center Township, Perry County, Pennsylvania?",
|
|
"target_title": "Centre Township, Perry County, Pennsylvania",
|
|
"target_root": "f9e0aef04d0933d69a7c4e67d3f48d784ee6e7c05c9a1f2d5d9485930f15e84b",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is center Township, Pennsylvania?",
|
|
"target_title": "Centre Township, Pennsylvania",
|
|
"target_root": "afb66d096c4dce06b655923fd2f05f124d95651f93d9c02e5589a5b9720a24bd",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Tricorn center?",
|
|
"target_title": "Tricorn Centre",
|
|
"target_root": "1b35fe0e9b63e317624587f1adf3da28084e6dbe922767c778c609521002b130",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Living color?",
|
|
"target_title": "Living Colour",
|
|
"target_root": "3f19efdb859270607248ae4d4402d2ba8b382159085c1e9e51763b2e098d9d14",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Philipsburg, center County, Pennsylvania?",
|
|
"target_title": "Philipsburg, Centre County, Pennsylvania",
|
|
"target_root": "7febde5ddf0b0a33830b22ae4c46f9f86531c9128c61f9708f5260317de87f37",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is German labor Front?",
|
|
"target_title": "German Labour Front",
|
|
"target_root": "1603c2b80ed5c6795e940d20de1ab6db4fcf8bac2ae37371f91c5674521347a1",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Rogers center?",
|
|
"target_title": "Rogers Centre",
|
|
"target_root": "091f7ff1f01402924989f7f93165a257f25c8708e39820d94c2b03803be4a3a8",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Ulster defense Association?",
|
|
"target_title": "Ulster Defence Association",
|
|
"target_root": "5cb5157f5c4ee9f910b29c49303b8b2f51c644b30de4c39b0104665d5e6bef87",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Guiana Space center?",
|
|
"target_title": "Guiana Space Centre",
|
|
"target_root": "50db2798791be79ef25a6ed0082909ea268f6bad25763ffd4c6e60feb19d8c30",
|
|
"shard": "000.db"
|
|
},
|
|
{
|
|
"question": "what is Community theater?",
|
|
"target_title": "Community theatre",
|
|
"target_root": "5c745645b88927ed021df3b3ee2de14926efc8be020f0fc85b534126cb653bc4",
|
|
"shard": "000.db"
|
|
}
|
|
]
|