arborist/bench/scripts
russell@unturf.com 02f8dfec07
#000049 §7 #21: candidate-clause restriction in ShadowNLI.check — helps, doesn't close it
candidate_clauses() — NLI now runs only on the top-N source clauses by
content-token overlap with the answer claim (max_candidate_clauses=6),
not the whole context; records n_candidate_clauses / best_clause_overlap
/ recombination_risk. Synthetic sweep unchanged (28/28 recombination,
0/26 legit FP, mean 1.45 candidate clauses/record). Real-traffic smoke
re-run: STRICT would-demote 30% → 20%, overall 47% → 33% — better, not
fixed; recombination-risk split doesn't separate either. Residual STRICT
false-contras at ~0.83-0.92 → θc would need ≈ 0.90 (vs the clean-set
0.5); at θc=0.90 the data in hand gives 27/28 synthetic recall, 0/26
legit FP, 0/10 smoke STRICT FP — but n=10 is too small to set on.
Next: a fuller ARBORIST_NLI_SHADOW=1 bench-qa run → sweep θc on hundreds
of STRICT cells → confirm → set it. θc stays 0.5; runtime NLI demotion
stays off. Production verifier unchanged; falsification-hard stays 10/12.
2026-05-12 14:34:44 -04:00
..
__init__.py #000026 Phase 2: real-shard baseline + lazy concept loading 2026-05-08 13:10:58 -04:00
demo_plot.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00
fivef_threshold_calibration.py #000047 — close: delta_aggregator knob on ForkScore (Option D) 2026-05-11 08:27:38 -04:00
harvest_falsification_proposals.py #000037 follow-through: persist falsification proposals + harvest live stream 2026-05-10 19:40:00 -04:00
nli_shadow_sweep.py #000049 §7 #21: candidate-clause restriction in ShadowNLI.check — helps, doesn't close it 2026-05-12 14:34:44 -04:00
phi_alignment_probe.py ticket #000034 Phase 1a: φ_linear alignment probe + KAT fixture 2026-05-10 10:23:38 -04:00
prometheus_sigma_sweep_dryrun.py #000037 §22: dry-run swap to sweep_weights() + §15.4 added 2026-05-10 19:22:03 -04:00
real_shard_baseline.py #000026 Phase 2: real-shard baseline + lazy concept loading 2026-05-08 13:10:58 -04:00
selfmodel_chain_snapshot.py #000025 §10.11 + §10.13 + §10.14 — close the 5F battery 2026-05-11 07:41:37 -04:00
t3_bound_calculator.py ticket #000036 Tier-2: dav1d Option B (conservative B1 envelope) applied in v1 2026-05-11 07:06:50 -04:00
textbooks_manifest.py ticket #000031 follow-ups B-1 + B-2: alias attribution + source-side title author 2026-05-10 09:35:49 -04:00
witness_sweep.py fan-out: close #000030 · composition fixtures · witness end-to-end 2026-05-09 13:29:59 -04:00
witness_sweep_cron.sh fan-out: witness automation · #000018 analysis · #000033 pillar VII (Claude) 2026-05-09 14:36:39 -04:00
witness_to_5f.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00