arborist/bench/scripts
russell@unturf.com ff1752c383
bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037 → #000025)
Closes the controller → 5F battery loop fox designed: #000037 Phase
1 produces FalsificationFixtureProposal records from high-divergence
providence_cache rows; this commit lands the harvester that turns
those proposals into a real 5F fixture pack the falsification
battery exercises every test run.

bench/scripts/harvest_falsification_proposals.py — reads qa.db,
filters live rows with witness_divergence = (n_unverified / n_quotes)
>= 0.5, stratifies by audit_mode, picks 20 HYBRID + 20 UNGROUNDED
top-by-cache_key for determinism, writes embedded-mode fixtures
to bench/fixtures/5f/falsification-harvested-v1.jsonl.

Each fixture carries `_harvest_meta` with the source cache_key,
divergence at harvest time, audit_mode at harvest time, and the
ticket reference (#000037 §13 step 11). Embedded mode — uses
`observed_violations` directly without calling verify_quotes
again; `answer_text` preserved verbatim for debugging.

Pack composition (initial harvest 2026-05-10):

  20 UNGROUNDED         (expected_reason: UNGROUNDED)
   9 HYBRID_QUOTE       (NEW motif — not in falsification-v1.jsonl)
   7 HYBRID_CLAIM_LATTICE (NEW motif — not in falsification-v1.jsonl)
   4 HYBRID_PARAPHRASE  (already covered in v1)

Two harness tests pin the pack:
- test_5f_falsification_harvested_pack_runs_clean: 100% pass-rate,
  every fixture carries traceable _harvest_meta.
- test_5f_falsification_harvested_pack_widens_motif_coverage: the
  HYBRID_QUOTE / HYBRID_CLAIM_LATTICE motifs surface from real
  corpus (loud-fail if harvest rotation drops them).

Makefile: `make bench-5f-harvest` re-runs the harvester. Parameters
exposed: HARVEST_QA_DB, HARVEST_OUT, HARVEST_THRESHOLD,
HARVEST_SAMPLE_PER_BUCKET.

Full suite: 2328 passed, 37 skipped (+2 from the two new pins).
2026-05-10 17:47:07 -04:00
..
__init__.py #000026 Phase 2: real-shard baseline + lazy concept loading 2026-05-08 13:10:58 -04:00
demo_plot.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00
harvest_falsification_proposals.py bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037 → #000025) 2026-05-10 17:47:07 -04:00
phi_alignment_probe.py ticket #000034 Phase 1a: φ_linear alignment probe + KAT fixture 2026-05-10 10:23:38 -04:00
prometheus_sigma_sweep_dryrun.py #000037 Phase 1 continued: §14 row 4 + §13 step 11 + weight-tunable gates 2026-05-10 16:56:18 -04:00
real_shard_baseline.py #000026 Phase 2: real-shard baseline + lazy concept loading 2026-05-08 13:10:58 -04:00
t3_bound_calculator.py ticket #000036 Phase 1: T3 per-window covert-channel bound + calculator 2026-05-10 10:32:50 -04:00
textbooks_manifest.py ticket #000031 follow-ups B-1 + B-2: alias attribution + source-side title author 2026-05-10 09:35:49 -04:00
witness_sweep.py fan-out: close #000030 · composition fixtures · witness end-to-end 2026-05-09 13:29:59 -04:00
witness_sweep_cron.sh fan-out: witness automation · #000018 analysis · #000033 pillar VII (Claude) 2026-05-09 14:36:39 -04:00
witness_to_5f.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00