bench: #000031 Phase 3 A/B finds mechanism dormant on warrant-targeted fixture

Bench follow-up to a57b194 ("ticket #000031 Phase 3: warrant-chain-
aware verifier suppresses WARRANT_MISSING"): land the warrant-
targeted probe fixture + capture an A/B comparing pre-Phase 3
(a57b194~1) vs HEAD on 7 questions × 3 modes × n=3.

bench/qa_questions_warrant_chain_probe.txt (new fixture)
========================================================

7 questions targeting claim-pack-resolved topics: Hilbert geometry
(2), Russell classes (1), Cantor diagonal (1), De Morgan (1),
Boolean absorption (1), pigeonhole in Bogart (1). Picked because
their topical match against the 18 textbook substrates +
warrant-resolver derivations should — in principle — drive
retrieval to chain-root chunks where Phase 3's suppression logic
fires.

bench/results/phase3-warrant-chain-A-B-2026-05-10.md (A/B report)
=================================================================

Findings:

- **21/21 cache_keys match BEFORE/AFTER.** Phase 3 doesn't fold
  into governance_policy_hash (warrant_chain_roots is a runtime
  parameter, not a policy field), so retrieval + prompt + LLM
  inputs are byte-identical between conditions. Any STRICT-rate
  movement is real verifier signal, not LLM dice.
- **`warrant_proven_claim_idxs` is non-empty in 0 / 63 AFTER cells.**
  Phase 3's suppression path never fires on this fixture.
- **STRICT-rate: 42/63 → 43/63 (+1 cell, +1.6pp).** Four cells
  move (2 up, 2 down); within n=3 LLM noise.

Why dormant despite warrant-targeted topic selection:

1. Pre-bench probe via `--dry-run --json` showed only 2/7 questions
   retrieve any chain-root sources — the claim-pack textbooks have
   far fewer chunks than Wikipedia, so retrieval defaults to the
   bulkier corpus on general queries.
2. Even on the 2 questions that DO retrieve chain roots, the LLM
   cites the larger Wikipedia chunks more often.
3. On the rare cell where the LLM cites a chain-root chunk, lexical
   `warrant_check` already passes — Phase 3 has nothing to suppress.

Calibration finding, not a defect:

- Phase 3 mechanism is correct (no regression on cells where it
  doesn't fire; cache_keys identical → can't move retrieval).
- The data condition for Phase 3 to actually fire requires a
  fixture where (a) retrieval surfaces chain-root chunks AND
  (b) the LLM cites them AND (c) the cited claim's lexical
  warrant_check fails.
- Future fixture would need very specific terminology that
  biases retrieval AWAY from Wikipedia (exact theorem labels,
  axiom names, etc.).

Same calibration shape as yesterday's progressive-AND smoke A/B
finding: a targeted change can be dormant on the fixture you're
benching against. The bench-fixture-design discipline that ticket
#000037 §12 codifies — "measured trigger, not calendar date" —
applies here too: until a fixture lands that exercises Phase 3,
the suppression mechanism's value is theoretical.

bench/qa_questions_warrant_chain_probe.txt

@@ -0,0 +1,23 @@
+# Warrant-chain probe fixture — questions targeting claim-pack-resolved
+# textbooks (Hilbert geometry, Cantor set theory, Russell logic, De Morgan,
+# Boole, Peano arithmetic, Dedekind reals, Judson abstract algebra).
+#
+# Purpose: exercise the post-#000031-Phase 3 verifier path that suppresses
+# WARRANT_MISSING when a cited chunk's document_root has a warrant-resolver
+# derivation row. Smoke fixture retrieval lands zero chain hits; this
+# fixture's questions explicitly target the 92 chain-resolved sources.
+
+# Pillar IV — Hilbert geometry (18 records)
+what does Hilbert's first axiom of incidence state?
+what does Hilbert mean by the axiom of order between three points?
+
+# Pillar I — set theory / classes (13 records, Russell + Cantor)
+what is Russell's definition of a class?
+what does Cantor's diagonal argument prove about real numbers?
+
+# Pillar II — logic (10 records, De Morgan + Boole)
+what are De Morgan's laws?
+in Boolean algebra, what is the absorption law?
+
+# Pillar VII — combinatorics (14 records, Bogart + Levin)
+what is the pigeonhole principle in Bogart?

bench/results/phase3-warrant-chain-A-B-2026-05-10.md

@@ -0,0 +1,110 @@
+# #000031 Phase 3 verifier — A/B (2026-05-10)
+
+**Change**: commit `a57b194` ("ticket #000031 Phase 3:
+warrant-chain-aware verifier suppresses WARRANT_MISSING") landed
+2026-05-10 09:17 EDT.
+
+**Mechanism**: when a claim's lexical `warrant_check` fails AND any
+cited evidence's `source_root` is in `warrant_chain_lookup()` (the
+92 `document_root` values across all shards that have a derivation
+row with `process_id LIKE 'warrant-resolver-v1%'`), the verifier
+suppresses the `WARRANT_MISSING` violation and tracks the claim_idx
+on a new `warrant_proven_claim_idxs` field. Render layer surfaces
+this as `· warrant proven via chain ×N` in the audit-line tail.
+
+**Bench question**: does Phase 3 actually move STRICT-rate on real
+questions? Or is it structurally dormant?
+
+## Method
+
+- Fixture: `bench/qa_questions_warrant_chain_probe.txt` — 7 questions
+  targeting claim-pack-resolved topics (Hilbert geometry, Russell
+  classes, Cantor diagonal, De Morgan, Boolean absorption, pigeonhole
+  in Bogart).
+- Pre-bench probe via `arborist query --dry-run --json`: only **2/7
+  questions retrieve any chain-root sources** (De Morgan: 1 source;
+  Bogart pigeonhole: 2 sources). The other 5 questions retrieve
+  Wikipedia chunks for their topics — the claim-pack textbooks have
+  vastly fewer chunks than Wikipedia, so retrieval defaults to the
+  bulkier corpus.
+- Conditions: BEFORE = `a57b194~1` (verify.py / runner.py / query.py
+  reverted via `git show`); AFTER = HEAD.
+- Modes: quote, claim_lattice_pointer, claim_lattice. n=3 with
+  `--burn` between samples.
+
+## Cache-key parity
+
+**21/21 cells match BEFORE/AFTER.** Phase 3 doesn't fold into
+`governance_policy_hash` (the new `warrant_chain_roots` parameter is
+runtime-passed, not policy-folded), so retrieval + prompt are
+byte-identical between conditions. Any STRICT-rate movement is real
+verifier-side signal, not LLM dice.
+
+## Phase 3 fire rate
+
+**`warrant_proven_claim_idxs` is non-empty in 0 / 63 AFTER cells.**
+The Phase 3 suppression path never triggers on this fixture.
+
+## STRICT-rate
+
+| | BEFORE | AFTER | Δ |
+|---|---|---|---|
+| overall | 42/63 (66.7%) | 43/63 (68.3%) | +1 cell (+1.6pp) |
+
+Four cells move at the per-cell level — 2 up (Russell quote +2; De
+Morgan pointer +1; Bogart pointer +1) and 2 down (Cantor quote −2;
+Hilbert-incidence pointer −1). Net +1, within LLM noise on n=3.
+
+Per-cell parity reads as: cache_keys identical, Phase 3 mechanism
+not firing, so the deltas are LLM sampling variance — exactly the
+same shape as yesterday's smoke A/B finding.
+
+## Verdict
+
+Phase 3 is **mechanistically landed but currently dormant on this
+fixture**. The data condition (claim cites chain-root chunk AND
+lexical `warrant_check` fails) is not met by:
+
+1. Questions whose retrieval doesn't surface any chain-root chunks
+   (5/7 in this fixture).
+2. Questions that DO retrieve chain-root chunks but where the LLM
+   cites the bulkier Wikipedia chunks instead (the remaining 2/7).
+3. Questions where the LLM does cite a chain-root chunk but the
+   lexical `warrant_check` already passes — Phase 3 has nothing to
+   suppress.
+
+This is not a defect — it's a calibration finding. The mechanism is
+correct + non-regressing on cells where it doesn't fire. To actually
+exercise Phase 3, a future fixture needs questions where:
+
+- Specific terminology biases retrieval AWAY from Wikipedia toward
+  claim-pack chunks (e.g. exact theorem labels: "Hilbert pillar IV
+  axiom of incidence").
+- Claims about those topics phrase themselves in ways that don't
+  pass lexical anchors but DO have warrant-chain backing.
+
+## Reproducing
+
+```bash
+# AFTER (HEAD)
+.venv/bin/python bench/qa_sweep.py \
+    --questions bench/qa_questions_warrant_chain_probe.txt \
+    --shards-dir ~/.arborist/shards \
+    --out-dir bench/qa_results_phase3_AB/after \
+    --top-k 8 \
+    --modes quote,claim_lattice_pointer,claim_lattice \
+    --n 3 --concurrency 4
+
+# BEFORE (Phase 3 reverted)
+git show a57b194~1:arborist/qa/verify.py > arborist/qa/verify.py
+git show a57b194~1:arborist/qa/runner.py > arborist/qa/runner.py
+git show a57b194~1:arborist/qa/query.py > arborist/qa/query.py
+# (re-run the same sweep with --out-dir bench/qa_results_phase3_AB/before)
+git checkout HEAD -- arborist/qa/verify.py arborist/qa/runner.py arborist/qa/query.py
+```
+
+## Source files
+
+- Fixture: `bench/qa_questions_warrant_chain_probe.txt`
+- BEFORE results: `bench/qa_results_phase3_AB/before/`
+- AFTER results: `bench/qa_results_phase3_AB/after/`