arborist/bench
russell@unturf.com ff1752c383
bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037 → #000025)
Closes the controller → 5F battery loop fox designed: #000037 Phase
1 produces FalsificationFixtureProposal records from high-divergence
providence_cache rows; this commit lands the harvester that turns
those proposals into a real 5F fixture pack the falsification
battery exercises every test run.

bench/scripts/harvest_falsification_proposals.py — reads qa.db,
filters live rows with witness_divergence = (n_unverified / n_quotes)
>= 0.5, stratifies by audit_mode, picks 20 HYBRID + 20 UNGROUNDED
top-by-cache_key for determinism, writes embedded-mode fixtures
to bench/fixtures/5f/falsification-harvested-v1.jsonl.

Each fixture carries `_harvest_meta` with the source cache_key,
divergence at harvest time, audit_mode at harvest time, and the
ticket reference (#000037 §13 step 11). Embedded mode — uses
`observed_violations` directly without calling verify_quotes
again; `answer_text` preserved verbatim for debugging.

Pack composition (initial harvest 2026-05-10):

  20 UNGROUNDED         (expected_reason: UNGROUNDED)
   9 HYBRID_QUOTE       (NEW motif — not in falsification-v1.jsonl)
   7 HYBRID_CLAIM_LATTICE (NEW motif — not in falsification-v1.jsonl)
   4 HYBRID_PARAPHRASE  (already covered in v1)

Two harness tests pin the pack:
- test_5f_falsification_harvested_pack_runs_clean: 100% pass-rate,
  every fixture carries traceable _harvest_meta.
- test_5f_falsification_harvested_pack_widens_motif_coverage: the
  HYBRID_QUOTE / HYBRID_CLAIM_LATTICE motifs surface from real
  corpus (loud-fail if harvest rotation drops them).

Makefile: `make bench-5f-harvest` re-runs the harvester. Parameters
exposed: HARVEST_QA_DB, HARVEST_OUT, HARVEST_THRESHOLD,
HARVEST_SAMPLE_PER_BUCKET.

Full suite: 2328 passed, 37 skipped (+2 from the two new pins).
2026-05-10 17:47:07 -04:00
..
batteries ticket #000032: combinatorics@v1 π* (pure-integer counting kernel) 2026-05-09 13:20:29 -04:00
fixtures bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037 → #000025) 2026-05-10 17:47:07 -04:00
results #000037 Phase 1 continued: §14 row 4 + §13 step 11 + weight-tunable gates 2026-05-10 16:56:18 -04:00
scripts bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037 → #000025) 2026-05-10 17:47:07 -04:00
emergent_log.jsonl ticket(#000006): 300-cycle update — zero false-positive STRICT post-hardening 2026-05-03 23:16:45 -04:00
prometheus_sigma_trigger_probe.py #000037 Phase 0: §12 trigger probe + Phase 1 test scaffolding 2026-05-10 07:24:59 -04:00
qa_questions.txt aborist/arborist 2026-05-07 09:31:49 -04:00
qa_questions_canonical_witness_npower.txt three-thread session output: stale TODOs, N-power probe, ForkScore Phase 1c 2026-05-10 07:46:35 -04:00
qa_questions_metacog_subset.txt qa(#000011 + 4 more): SOFT_PREFLIGHT_HINT impl + 5-task fan-out 2026-05-03 23:00:56 -04:00
qa_questions_progressive_and.txt bench: progressive-AND fixture + 2026-05-09 A/B baseline report 2026-05-10 06:35:18 -04:00
qa_questions_quantifier_baseline.txt bench(#000008): harness extension — FC rate, violation kinds, raw brackets 2026-05-02 18:35:08 -04:00
qa_questions_quantifier_subset.txt ticket(#000008): §12 dry-run bench findings + --policy harness flag 2026-05-03 08:39:20 -04:00
qa_questions_smoke.txt speed: pytest-xdist, bench smoke, concurrency default; UTF surrogate fix 2026-05-02 09:29:40 -04:00
qa_questions_warrant_chain_aggressive.txt bench: aggressive warrant fixture confirms Phase 3 is rescue-only, not default-path 2026-05-10 09:53:21 -04:00
qa_questions_warrant_chain_paraphrase.txt bench: Phase 3 paraphrase fixture investigation — empirically dormant on current corpus 2026-05-10 10:06:23 -04:00
qa_questions_warrant_chain_probe.txt bench: #000031 Phase 3 A/B finds mechanism dormant on warrant-targeted fixture 2026-05-10 09:37:27 -04:00
qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
run.sh aborist/arborist 2026-05-07 09:31:49 -04:00