arborist/bench/results
russell@unturf.com e1ac7c2ea2
bench/full-bench-qa: no regression after substrate+Phase 3 sprint (675 cells)
Full QA-quality sweep, 75 questions × 3 modes × n=3 = 675 cells
against Hermes-3 8B at concurrency=4. No mode regressed past
the 5pp signal floor:

  quote                 53.8% STRICT  vs 54% baseline = -0pp (stable)
  claim_lattice_pointer 23.1% STRICT  vs 20% baseline = +3pp (within floor)
  claim_lattice (JSON)  46.7% STRICT  vs 42% baseline = +5pp (at floor — marginal positive)

JSON-mode +5pp is right at the noise threshold per
docs/bench-maxing.md — could be the 2026-05-10 substrate work
(92/92 warrant chains + 18 textbook ingests + cascade tuning +
Phase 3 wiring) translating to retrieval-quality lift, OR
sample variance. Follow-up bench in 1-2 days disambiguates.

Format-collapse 0/225 in every mode. Latency stable
(10.6-11.1s median). Lattice-mode directive coverage 100%.

Phase 3 per-claim warrant-chain tail still dormant on this
fixture (rescue-only by design; no warrant-shape question
retrieves a chain-backed chunk while failing lexical anchor —
parallel shift's same finding). Source-level warrants tail
fires on claim-pack-targeted retrievals as documented in
phase3-live-validation-2026-05-10.md.

Substrate + Phase 3 sprint shipped clean.
2026-05-10 13:42:12 -04:00
..
citation-aliases-batch-2026-05-10.md citation-aliases batch: 18 -> 35 / 92 records resolve (38%) 2026-05-10 06:58:27 -04:00
claim-pack-retrieval-lift-2026-05-09.md bench/results: claim-pack retrieval lift measurement (2026-05-09) 2026-05-09 13:38:04 -04:00
full-bench-qa-2026-05-10.md bench/full-bench-qa: no regression after substrate+Phase 3 sprint (675 cells) 2026-05-10 13:42:12 -04:00
full-warrant-resolution-2026-05-10.md docs: refresh stale alias counts (residual 40/54 → 74) 2026-05-10 12:27:23 -04:00
grinstead-snell-and-cascade-tuning-2026-05-10.md Grinstead-Snell ingest + cascade tuning batch: 68 -> 77/92 (84%) 2026-05-10 08:03:11 -04:00
hilbert-pillar-iv-fully-resolved-2026-05-09.md ticket #000042: 4 geometry aliases resolve all 18 Hilbert pillar IV records 2026-05-09 20:18:58 -04:00
lambda-and-pm-additions-2026-05-10.md PLFA + SF-LF + PM Vol 1 ingest: 62 -> 68/92 (74%) — pillar IX nearly closed 2026-05-10 07:45:40 -04:00
medium-confidence-aliases-and-peano-2026-05-10.md medium-conf aliases + Peano ingest + title backfill: 35 -> 54/92 (59%) 2026-05-10 07:14:44 -04:00
phase3-live-validation-2026-05-10.md Phase 3 live validation: 6/7 probes reach EVIDENCE-WARRANTED 2026-05-10 09:46:10 -04:00
phase3-warrant-chain-A-B-2026-05-10.md bench: #000031 Phase 3 A/B finds mechanism dormant on warrant-targeted fixture 2026-05-10 09:37:27 -04:00
phase3-warrant-chain-A-B-aggressive-2026-05-10.md bench: aggressive warrant fixture confirms Phase 3 is rescue-only, not default-path 2026-05-10 09:53:21 -04:00
phase3-warrant-chain-paraphrase-investigation-2026-05-10.md bench: Phase 3 paraphrase fixture investigation — empirically dormant on current corpus 2026-05-10 10:06:23 -04:00
progressive-and-2026-05-09.md bench: progressive-AND fixture + 2026-05-09 A/B baseline report 2026-05-10 06:35:18 -04:00
prometheus-sigma-triggers-2026-05-10-npower.md three-thread session output: stale TODOs, N-power probe, ForkScore Phase 1c 2026-05-10 07:46:35 -04:00
prometheus-sigma-triggers-2026-05-10.md #000037 Phase 0: §12 trigger probe + Phase 1 test scaffolding 2026-05-10 07:24:59 -04:00
real-shard-baseline.json modified: .gitignore 2026-05-08 16:38:09 -04:00
real-shard-baseline.md modified: .gitignore 2026-05-08 16:38:09 -04:00
textbook-base-knowledge-upgrade-2026-05-09.md textbook ingest: 4 base-knowledge additions for pillars I/II/III/IX 2026-05-09 20:38:32 -04:00
three-more-textbooks-2026-05-10.md 3 more textbooks (Russell PoM + Laplace + Dedekind): 54 -> 62/92 (67%) 2026-05-10 07:31:40 -04:00
warrant-chain-render-2026-05-09.md bench/results: warrant-chain render tail measurement (2026-05-09) 2026-05-09 18:00:36 -04:00
witness-sweep.json fan-out: witness automation · #000018 analysis · #000033 pillar VII (Claude) 2026-05-09 14:36:39 -04:00