Rolling research log gains a 2026-05-09 amend for the
multi-witness pipeline (#000028) running against live Hermes for
the first time. Qualitatively different from the random-word
triangulation amends above — those measure honesty under no
ground truth; this measures agreement under available ground
truth (kernel IS the ground truth on canonical-shape questions).
First sweep: 8 canonical-shape questions, 5 LLM-DIVERGED, 3
STRICT-WITNESSED. Three distinct failure shapes captured:
1. Wrong arithmetic on float-shape input
0.1 + 0.2 → kernel 3/10 vs Hermes 1/10
(off by 2/10; possibly trained on the IEEE-754 trap as
the "answer" itself rather than recognizing the kernel
returns the exact rational)
2. Implication-tautology error
A IMPL B → kernel (NOT A OR B) vs Hermes TRUE
(NOT B) IMPL (NOT A) → same divergence
(Hermes treats the formula as a tautology rather than
canonicalizing to CNF; A OR NOT A — the genuine tautology
— correctly returns TRUE)
3. Symbolic erasure on algebra-shape input
(x+1)**2 → kernel x²+2x+1 vs Hermes 1
x**2 + 2*x + 1 → same divergence
(Hermes collapses to a constant — possibly evaluating at
x=0 — instead of returning the canonical expanded
polynomial)
The amend frames witness-sweep as the *capability-failure*
signal source, distinct from random-word emergent's
*honesty-failure* signal. Different signals, different repair
paths:
- Verifier ladder hardening / warrant-tier sharpening
addresses honesty failures (false STRICTs).
- Prompt engineering / fine-tune-on-divergence-corpus
addresses capability failures.
The verifier ladder cannot help with witness-divergence — the
LLM fundamentally produced a wrong answer that no number of
citation checks recovers. This is the calibration-data stream
the original #000028 ticket imagined.
End-to-end validates: canonical persistence (#000027),
multi-witness fan-out (#000028), audit-event chain — 0 breaks on
chain-check-shards post-sweep, 5 providence_canonical_witness
events appended, capital ledger captured 5 canonical_witness
op_type rows.
Pattern documented for future witness-sweep amends as the
divergence corpus grows (cron + commit harness in
bench/scripts/witness_sweep_cron.sh handles the unattended
collection).
Bench artifact: bench/results/witness-sweep.json.
Calibration corpus: bench/fixtures/5f/falsification-witness-v1.jsonl
(10 rows after two extraction passes today).
russell@unturf.com
2acbc0e691