arborist/bench
russell@unturf.com 2c98fc964e
feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF
Three workstreams, full suite 2482 passed, experimental paths default-OFF.

#000055 — Windows quickstart without make
  tasks.py (pure-stdlib runner) + make.bat shim + .gitattributes;
  README Windows section rewritten. Quickstart needs only Python
  3.10+ (no make/bzip2/curl/bash). Mirrors the Makefile quickstart
  subset; drift-pinned by tests/test_tasks_runner.py.

#000001 §7 Phase 0 — deterministic cross-language guard
  arborist/qa/crosslang.py: non-English signal (¿/¡/non-ASCII) + an
  es function-word stoppack. Fail-closed to UNGROUNDED before
  retrieval/LLM (mirrors the quantifier reject-DAG) when no content
  token survives, else strips es stopwords from the retrieval query
  only. English path byte-identical by construction. Default OFF
  (crosslang_guard_enabled). Measured: the anarcocapitalismo field
  case 10.4s -> 1.6s.

#000056 — Operation Sandwich (cross-language grounding)
  arborist/qa/mt/: opus-mt es/fr/ru<->en, lazy per-pair memoised
  singleton (fixes the 88%-engine-error concurrency defect),
  manifest-pinned, [mt] extra; entity_mask wrapper. Sandwich =
  translate query in (retrieval + LLM prompt) -> English answer ->
  UNTOUCHED verifier grounds English-vs-English -> translate the
  verified answer out as display-only (banner-labelled, zero
  grounding). question_hash + verifier_policy_hash invariant; MT
  engine identity binds into RetrievalPlan, not governance. CLI
  --crosslang-translate / make XLANG_MT=1. Default OFF; entity_mask
  default OFF (measured net-negative at bench scale). Fan-out bench
  (bench/*.py): Spanish ~0% -> 71% grounded vs the real no-support
  baseline; the round-trip predictor was tried and refuted; the
  entity-mask lever failed at scale (corpus-title anchoring untried).

CLAUDE.md: cross-language bright-line convention + module map.
Pre-existing modified diagram files are intentionally excluded.
2026-05-18 12:12:23 -04:00
..
batteries #000025 §10.11 + §10.13 + §10.14 — close the 5F battery 2026-05-11 07:41:37 -04:00
fixtures #000052 §3.2.2 step 3 part B: real-context MIS-CITE fixtures (n=20) — bge-large catches 100% of mis-cite at strictly 0% real-STRICT FP (the Zionist failure mode is fully covered with margin) 2026-05-13 15:27:58 -04:00
results #000052 §3.2.2 step 3 part C: multi-signal combination experiment — does NOT lift precision/recall on §3.2's failure shapes; clean architectural finding that lexical sidecars + relevance reranker cover NON-OVERLAPPING failure modes 2026-05-13 15:30:39 -04:00
scripts #000052 §3.2.2 refinement: claim-lattice metadata cleaning — STRICT FP drops universally (-0.5 to -5.6 pts), L-2 down to 1.5% 2026-05-13 15:01:50 -04:00
emergent_log.jsonl #000006 — +30 emergent cycles (2026-05-12); verifier-ladder health re-confirmed 2026-05-12 11:28:57 -04:00
es_delta.py feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
es_join_patterns.py feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
es_roundtrip_analysis.py feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
make_es_questions.py feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
make_lang_questions.py feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
prometheus_sigma_trigger_probe.py #000012 Phase 1c follow-through: wire #000037 §12 Trigger 1 probe to fork_score_branches 2026-05-11 06:56:09 -04:00
qa_questions.txt aborist/arborist 2026-05-07 09:31:49 -04:00
qa_questions_canonical_witness_npower.txt three-thread session output: stale TODOs, N-power probe, ForkScore Phase 1c 2026-05-10 07:46:35 -04:00
qa_questions_es.txt feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
qa_questions_es_map.json feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
qa_questions_fr.txt feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
qa_questions_fr_map.json feat: cross-language Q&A (Operation Sandwich) + Windows quickstart — all default-OFF 2026-05-18 12:12:23 -04:00
qa_questions_metacog_subset.txt qa(#000011 + 4 more): SOFT_PREFLIGHT_HINT impl + 5-task fan-out 2026-05-03 23:00:56 -04:00
qa_questions_progressive_and.txt bench: progressive-AND fixture + 2026-05-09 A/B baseline report 2026-05-10 06:35:18 -04:00
qa_questions_quantifier_baseline.txt bench(#000008): harness extension — FC rate, violation kinds, raw brackets 2026-05-02 18:35:08 -04:00
qa_questions_quantifier_subset.txt ticket(#000008): §12 dry-run bench findings + --policy harness flag 2026-05-03 08:39:20 -04:00
qa_questions_smoke.txt speed: pytest-xdist, bench smoke, concurrency default; UTF surrogate fix 2026-05-02 09:29:40 -04:00
qa_questions_warrant_chain_aggressive.txt bench: aggressive warrant fixture confirms Phase 3 is rescue-only, not default-path 2026-05-10 09:53:21 -04:00
qa_questions_warrant_chain_paraphrase.txt bench: Phase 3 paraphrase fixture investigation — empirically dormant on current corpus 2026-05-10 10:06:23 -04:00
qa_questions_warrant_chain_probe.txt bench: #000031 Phase 3 A/B finds mechanism dormant on warrant-targeted fixture 2026-05-10 09:37:27 -04:00
qa_sweep.py bench/qa_sweep: scrub lone surrogates from the NLI-shadow answer_text/context fields before json.dumps 2026-05-12 17:04:33 -04:00
run.sh aborist/arborist 2026-05-07 09:31:49 -04:00