russell@unturf.com
12bb6dbb6f
fox: 'make the n huge huge, check in every ~7 turns'. - stale fixture re-mined to the FULL pool: 386 questions (180 pres / 119 PM / 34 premier / 20 gov-gen / 17 chancellor / 15 CM / 1 FM), deterministic. - control_sweep.py rebuilt: ThreadPoolExecutor work-unit-per-item (per-unit qa_db -> zero SQLite write contention; shards read-only safe under concurrent readers), incremental flushed JSONL, and a --report-only PATH mode that aggregates a PARTIAL file with ZERO LLM/judge spend (the interim check-in path). - huge N goes on the CONTROL (solo x3 models x3 framings) where the open statistical question lives; Arborist A/B = fixed --arborist-n (default 40), not re-measured 386x (power belongs on the control, and query() over ~40GB shards is the heaviest call). - bench-maxing doctrine applied: independent hermetic judge calls fanned out, 'serial-by-caution is halting in disguise'. N=3 (prior run) already shows the coherent, review-relevant story: plain -> all 3 models confidently assert (0% abstain); source_relative -> abstain jumps (hermes 33 / qwen-think 67 / qwen-nothink 100%); as_of_corpus -> ~100% correct all models. The gap is largely framing+snapshot, not a universal capability deficit.
389 lines
16 KiB
Text
389 lines
16 KiB
Text
# AUTO-MINED (stale class) from corpus titles via bench/mine_questions.py — ground-truth-carrying.
|
|
# #000057 FABRICATION-BAIT: present-tense questions on time-varying offices; the ingested ~2010-2011 corpus (verified from the artifact, NOT the 2003 dump CLAUDE.md names) is the temporal adversary against a ~2023 model. Graded by the external Opus judge vs the fixed corpus-vintage article gold (bench/control_ab.py + bench/judge.py), NOT recall@k. Adversarial by construction.
|
|
|
|
who is the prime minister of Australia?
|
|
who is the president of France?
|
|
who is the prime minister of Poland?
|
|
who is the president of India?
|
|
who is the president of Dáil Éireann?
|
|
who is the president of Iceland?
|
|
who is the president of Yemen Arab Republic?
|
|
who is the president of Finland?
|
|
who is the prime minister of Iraq?
|
|
who is the president of Iraq?
|
|
who is the president of Austria?
|
|
who is the prime minister of Portugal?
|
|
who is the chancellor of the Duchy of Lancaster?
|
|
who is the president of the Swiss Confederation?
|
|
who is the president of China?
|
|
who is the president of Guatemala?
|
|
who is the prime minister of Denmark?
|
|
who is the premier of the Republic of China?
|
|
who is the president of Liberia?
|
|
who is the prime minister of Croatia?
|
|
who is the president of the Quorum of the Twelve Apostles?
|
|
who is the president of Afghanistan?
|
|
who is the prime minister of Russia?
|
|
who is the prime minister of Singapore?
|
|
who is the president of Armenia?
|
|
who is the president of Algeria?
|
|
who is the president of the Church?
|
|
who is the president of the Czech Republic?
|
|
who is the premier of New Brunswick?
|
|
who is the chief minister of Gibraltar?
|
|
who is the governor-general of Barbados?
|
|
who is the prime minister of Bangladesh?
|
|
who is the prime minister of the Cook Islands?
|
|
who is the prime minister of Malta?
|
|
who is the governor-general of Jamaica?
|
|
who is the premier of Alberta?
|
|
who is the governor-general of Fiji?
|
|
who is the premier of the Northwest Territories?
|
|
who is the president of Suriname?
|
|
who is the president of Cuba?
|
|
who is the president of Portugal?
|
|
who is the prime minister of the Faroe Islands?
|
|
who is the prime minister of Angola?
|
|
who is the prime minister of Turkey?
|
|
who is the prime minister of Afghanistan?
|
|
who is the prime minister of Yemen?
|
|
who is the prime minister of Tuvalu?
|
|
who is the prime minister of Yemen Arab Republic?
|
|
who is the governor-general of Tuvalu?
|
|
who is the prime minister of Zambia?
|
|
who is the president of Serbia?
|
|
who is the president of Serbia and Montenegro?
|
|
who is the chancellor of Justice?
|
|
who is the president of Spain?
|
|
who is the prime minister of Iran?
|
|
who is the prime minister of Uzbekistan?
|
|
who is the premier of the Cayman Islands?
|
|
who is the chancellor of Uppsala University?
|
|
who is the prime minister of Kosovo?
|
|
who is the president of Hawaii?
|
|
who is the president of Mongolia?
|
|
who is the prime minister of Mauritius?
|
|
who is the prime minister of Somalia?
|
|
who is the president of the European Central Bank?
|
|
who is the prime minister of the Palestinian National Authority?
|
|
who is the president of the Indian National Congress?
|
|
who is the president of the Sahrawi Arab Democratic Republic?
|
|
who is the prime minister of the Philippines?
|
|
who is the president of the Maldives?
|
|
who is the chief minister of the Isle of Man?
|
|
who is the premier of the Northern Cape?
|
|
who is the prime minister of Azerbaijan?
|
|
who is the prime minister of Cambodia?
|
|
who is the prime minister of Kazakhstan?
|
|
who is the governor-general of the West Indies Federation?
|
|
who is the premier of the Turks and Caicos Islands?
|
|
who is the president of Palau?
|
|
who is the prime minister of Solomon Islands?
|
|
who is the president of Abkhazia?
|
|
who is the chancellor of the Exchequer of Ireland?
|
|
who is the prime minister of Yugoslavia?
|
|
who is the president of the Canary Islands?
|
|
who is the president of the University of Southern California?
|
|
who is the governor-general of the Federation of Rhodesia and Nyasaland?
|
|
who is the prime minister of Zimbabwe?
|
|
who is the president of the Chechen Republic?
|
|
who is the prime minister of Crimea?
|
|
who is the prime minister of Saudi Arabia?
|
|
who is the president of Chile?
|
|
who is the prime minister of China?
|
|
who is the president of French Polynesia?
|
|
who is the president of the Associação Brasileira de Estatística?
|
|
who is the prime minister of Serbia and Montenegro?
|
|
who is the president of the Federation of Bosnia and Herzegovina?
|
|
who is the chief minister of Singapore?
|
|
who is the president of the University of Richmond?
|
|
who is the chief minister of Sindh?
|
|
who is the president of the Autonomous Region of Bougainville?
|
|
who is the prime minister of Seychelles?
|
|
who is the president of Ingushetia?
|
|
who is the president of the Komi Republic?
|
|
who is the president of the Sakha Republic?
|
|
who is the president of the Republic of Buryatia?
|
|
who is the president of Dagestan?
|
|
who is the president of Karachay-Cherkessia?
|
|
who is the prime minister of the West Indies Federation?
|
|
who is the president of the National Congress of Honduras?
|
|
who is the president of Lombardy?
|
|
who is the prime minister of Iraqi Kurdistan?
|
|
who is the chancellor of the University of Cape Town?
|
|
who is the prime minister of Lower Saxony?
|
|
who is the governor-general of Australia?
|
|
who is the president of the United States?
|
|
who is the prime minister of India?
|
|
who is the premier of Western Australia?
|
|
who is the prime minister of Japan?
|
|
who is the president of Poland?
|
|
who is the chancellor of the Exchequer?
|
|
who is the president of the People's Republic of China?
|
|
who is the prime minister of Northern Ireland?
|
|
who is the prime minister of France?
|
|
who is the prime minister of the United States?
|
|
who is the president of the Republic?
|
|
who is the governor-general of India?
|
|
who is the first minister of Scotland?
|
|
who is the prime minister of Greece?
|
|
who is the prime minister of Italy?
|
|
who is the president of Nauru?
|
|
who is the president of Croatia?
|
|
who is the premier of Ontario?
|
|
who is the president of Malta?
|
|
who is the president of Georgia?
|
|
who is the president of Ghana?
|
|
who is the president of Bangladesh?
|
|
who is the prime minister of Vietnam?
|
|
who is the prime minister of Mongolia?
|
|
who is the president of Uganda?
|
|
who is the prime minister of Antigua and Barbuda?
|
|
who is the governor-general of Belize?
|
|
who is the prime minister of Lithuania?
|
|
who is the premier of Manitoba?
|
|
who is the premier of Prince Edward Island?
|
|
who is the prime minister of Fiji?
|
|
who is the prime minister of Armenia?
|
|
who is the governor-general of the Philippines?
|
|
who is the prime minister of Burma?
|
|
who is the president of Haiti?
|
|
who is the president of North Korea?
|
|
who is the president of Rhodesia?
|
|
who is the prime minister of Dominica?
|
|
who is the prime minister of the Sahrawi Arab Democratic Republic?
|
|
who is the prime minister of Cameroon?
|
|
who is the president of the Government?
|
|
who is the president of Harvard University?
|
|
who is the prime minister of Estonia?
|
|
who is the president of Azerbaijan?
|
|
who is the chancellor of Florence?
|
|
who is the prime minister of Egypt?
|
|
who is the president of Moldova?
|
|
who is the president of the Palestinian National Authority?
|
|
who is the president of the German Bundesrat?
|
|
who is the president of Kyrgyzstan?
|
|
who is the prime minister of Kyrgyzstan?
|
|
who is the president of Zanzibar?
|
|
who is the prime minister of Rwanda?
|
|
who is the president of Turkmenistan?
|
|
who is the president of Belarus?
|
|
who is the prime minister of Ghana?
|
|
who is the president of the Institutional Revolutionary Party?
|
|
who is the president of Italy?
|
|
who is the president of Kiribati?
|
|
who is the prime minister of Samoa?
|
|
who is the president of Nagorno-Karabakh Republic?
|
|
who is the president of Uzbekistan?
|
|
who is the president of the RSUA?
|
|
who is the chancellor of the University of St Andrews?
|
|
who is the president of Northern Cyprus?
|
|
who is the chief minister of Khyber Pakhtunkhwa?
|
|
who is the president of the United Arab Emirates?
|
|
who is the president of Burma?
|
|
who is the president of Madagascar's Small Grants Scheme?
|
|
who is the president of Savannah State University?
|
|
who is the chancellor of Keele University?
|
|
who is the prime minister of the Republic of Adygea?
|
|
who is the prime minister of Zimbabwe Rhodesia?
|
|
who is the premier of Niue?
|
|
who is the president of Veneto?
|
|
who is the president of Grinnell College?
|
|
who is the prime minister of South Ossetia?
|
|
who is the premier of Dominica?
|
|
who is the president of the Dewan Negara?
|
|
who is the chief minister of Sri Lanka?
|
|
who is the chancellor of the University of Glasgow?
|
|
who is the president of Bashkortostan?
|
|
who is the president of the Marshall Islands?
|
|
who is the president of Iraqi Kurdistan?
|
|
who is the prime minister of Slovenia?
|
|
who is the prime minister of Curaçao?
|
|
who is the prime minister of Sint Maarten?
|
|
who is the chief minister of the Northern Territory?
|
|
who is the premier of Victoria?
|
|
who is the president of Ireland?
|
|
who is the president of South Korea?
|
|
who is the president of the Irish Republic?
|
|
who is the president of Israel?
|
|
who is the prime minister of Iceland?
|
|
who is the president of Tajikistan?
|
|
who is the president of Bolivia?
|
|
who is the prime minister of Belgium?
|
|
who is the prime minister of Finland?
|
|
who is the president of Iran?
|
|
who is the president of Russia?
|
|
who is the president of the Philippines?
|
|
who is the prime minister of Thailand?
|
|
who is the prime minister of the Netherlands?
|
|
who is the president of Nicaragua?
|
|
who is the prime minister of Jamaica?
|
|
who is the president of South Africa?
|
|
who is the president of Sri Lanka?
|
|
who is the premier of China?
|
|
who is the president of Sierra Leone?
|
|
who is the prime minister of Pakistan?
|
|
who is the president of Fiji?
|
|
who is the premier of Nunavut?
|
|
who is the prime minister of Latvia?
|
|
who is the president of Trinidad and Tobago?
|
|
who is the prime minister of Georgia?
|
|
who is the prime minister of South Korea?
|
|
who is the president of Madrid?
|
|
who is the president of the Confederate States of America?
|
|
who is the prime minister of Rhodesia?
|
|
who is the president of the Federated States of Micronesia?
|
|
who is the president of the European Parliament?
|
|
who is the president of Kosovo?
|
|
who is the governor-general of Taiwan?
|
|
who is the president of East Timor?
|
|
who is the prime minister of East Timor?
|
|
who is the president of South Ossetia?
|
|
who is the president of Princeton University?
|
|
who is the president of Vanuatu?
|
|
who is the prime minister of Moldova?
|
|
who is the premier of the Eastern Cape?
|
|
who is the premier of Mpumalanga?
|
|
who is the prime minister of Bahrain?
|
|
who is the prime minister of Kenya?
|
|
who is the chancellor of the Order of the Thistle?
|
|
who is the president of Tynwald?
|
|
who is the president of the Municipal Chamber?
|
|
who is the prime minister of Brunei?
|
|
who is the prime minister of Vanuatu?
|
|
who is the president of the Gaelic Athletic Association?
|
|
who is the president of Montenegro?
|
|
who is the president of the Family Division?
|
|
who is the prime minister of Cuba?
|
|
who is the president of the Geological Society of London?
|
|
who is the premier of Nevis?
|
|
who is the prime minister of Tajikistan?
|
|
who is the chancellor of the Tang Dynasty?
|
|
who is the chancellor of Justice of Finland?
|
|
who is the president of Crimea?
|
|
who is the president of the Republic of Adygea?
|
|
who is the president of the Chamber of Deputies of Romania?
|
|
who is the president of Nepal?
|
|
who is the president of the Republic of Sierra Leone?
|
|
who is the president of the Government of Vojvodina?
|
|
who is the president of Ukraine?
|
|
who is the chief minister of the Australian Capital Territory?
|
|
who is the president of Mexico?
|
|
who is the premier of Gauteng?
|
|
who is the premier of KwaZulu-Natal?
|
|
who is the president of the Parliament of Montenegro?
|
|
who is the president of Arellano University?
|
|
who is the prime minister of Canada?
|
|
who is the prime minister of the United Kingdom?
|
|
who is the prime minister of Israel?
|
|
who is the president of Germany?
|
|
who is the chancellor of Austria?
|
|
who is the president of the Republic of China?
|
|
who is the prime minister of New Zealand?
|
|
who is the governor-general of the Irish Free State?
|
|
who is the premier of the People's Republic of China?
|
|
who is the president of Turkey?
|
|
who is the prime minister of Sweden?
|
|
who is the prime minister of Spain?
|
|
who is the prime minister of Norway?
|
|
who is the president of Argentina?
|
|
who is the president of Colombia?
|
|
who is the prime minister of Romania?
|
|
who is the president of Brazil?
|
|
who is the president of Romania?
|
|
who is the governor-general of Finland?
|
|
who is the prime minister of Papua New Guinea?
|
|
who is the prime minister of South Africa?
|
|
who is the president of Slovenia?
|
|
who is the premier of British Columbia?
|
|
who is the president of Egypt?
|
|
who is the president of the Continental Congress?
|
|
who is the president of Singapore?
|
|
who is the prime minister of Sri Lanka?
|
|
who is the governor-general of Papua New Guinea?
|
|
who is the premier of Quebec?
|
|
who is the president of Angola?
|
|
who is the governor-general of Kenya?
|
|
who is the prime minister of Algeria?
|
|
who is the president of Pakistan?
|
|
who is the president of Mauritius?
|
|
who is the president of El Salvador?
|
|
who is the president of Vietnam?
|
|
who is the premier of Nova Scotia?
|
|
who is the premier of Saskatchewan?
|
|
who is the premier of Newfoundland and Labrador?
|
|
who is the premier of Yukon?
|
|
who is the governor-general of Pakistan?
|
|
who is the prime minister of Malaysia?
|
|
who is the president of Greece?
|
|
who is the president of Nigeria?
|
|
who is the prime minister of Abkhazia?
|
|
who is the prime minister of Serbia?
|
|
who is the president of Paraguay?
|
|
who is the governor-general of the Dutch East Indies?
|
|
who is the governor-general of Solomon Islands?
|
|
who is the prime minister of Uganda?
|
|
who is the prime minister of Ukraine?
|
|
who is the president of the Republic of Macedonia?
|
|
who is the president of Estonia?
|
|
who is the prime minister of Namibia?
|
|
who is the president of Cyprus?
|
|
who is the prime minister of Senegal?
|
|
who is the premier of the Kingdom of Viti?
|
|
who is the president of Ichkeria?
|
|
who is the prime minister of Madagascar?
|
|
who is the governor-general of Ceylon?
|
|
who is the president of Laos?
|
|
who is the prime minister of Tunisia?
|
|
who is the prime minister of Aruba?
|
|
who is the chief minister of Anguilla?
|
|
who is the chief minister of Jersey?
|
|
who is the premier of North West?
|
|
who is the chief minister of Papua and New Guinea?
|
|
who is the prime minister of Laos?
|
|
who is the prime minister of Tonga?
|
|
who is the chief minister of Montserrat?
|
|
who is the prime minister of the Republic of Macedonia?
|
|
who is the governor-general of New Zealand?
|
|
who is the prime minister of Nagorno-Karabakh Republic?
|
|
who is the president of Transnistria?
|
|
who is the chief minister of Balochistan?
|
|
who is the prime minister of Montenegro?
|
|
who is the president of the Legislative Yuan?
|
|
who is the president of Bucknell University?
|
|
who is the president of the Queen's Bench Division?
|
|
who is the chief minister of Guernsey?
|
|
who is the prime minister of Northern Cyprus?
|
|
who is the president of Grande Comore?
|
|
who is the president of Anjouan?
|
|
who is the president of Mohéli?
|
|
who is the president of the Bundestag?
|
|
who is the president of the Moldovan Parliament?
|
|
who is the chief minister of Punjab?
|
|
who is the president of Indonesia?
|
|
who is the president of Cantabria?
|
|
who is the president of the University of Michigan?
|
|
who is the chancellor of the University of Edinburgh?
|
|
who is the prime minister of the Federation of Rhodesia and Nyasaland?
|
|
who is the president of the Congress of Colombia?
|
|
who is the premier of the Western Cape?
|
|
who is the president of Congress?
|
|
who is the president of Zimbabwe Rhodesia?
|
|
who is the president of the World Esperanto Association?
|
|
who is the prime minister of Turkmenistan?
|
|
who is the chancellor of Germany?
|
|
who is the president of the States of Alderney?
|
|
who is the president of the Government of New Caledonia?
|
|
who is the chancellor of the University of Dallas?
|
|
who is the president of the Naval War College?
|
|
who is the president of the University of Southern Maine?
|
|
who is the president of Kabardino-Balkaria?
|
|
who is the president of the University of the Philippines?
|
|
who is the president of Honduras?
|
|
who is the premier of the Free State?
|
|
who is the premier of Limpopo?
|
|
who is the president of Uruguay?
|
|
who is the president of the Xunta of Galicia?
|
|
who is the president of Umbria?
|
|
who is the prime minister of Hyderabad?
|
|
who is the premier of Montserrat?
|