# #000031 Phase 3 verifier — A/B (2026-05-10)

**Change**: commit `a57b194` ("ticket #000031 Phase 3:
warrant-chain-aware verifier suppresses WARRANT_MISSING") landed
2026-05-10 09:17 EDT.

**Mechanism**: when a claim's lexical `warrant_check` fails AND any
cited evidence's `source_root` is in `warrant_chain_lookup()` (the
92 `document_root` values across all shards that have a derivation
row with `process_id LIKE 'warrant-resolver-v1%'`), the verifier
suppresses the `WARRANT_MISSING` violation and tracks the claim_idx
on a new `warrant_proven_claim_idxs` field. Render layer surfaces
this as `· warrant proven via chain ×N` in the audit-line tail.

**Bench question**: does Phase 3 actually move STRICT-rate on real
questions? Or is it structurally dormant?

## Method

- Fixture: `bench/qa_questions_warrant_chain_probe.txt` — 7 questions
  targeting claim-pack-resolved topics (Hilbert geometry, Russell
  classes, Cantor diagonal, De Morgan, Boolean absorption, pigeonhole
  in Bogart).
- Pre-bench probe via `arborist query --dry-run --json`: only **2/7
  questions retrieve any chain-root sources** (De Morgan: 1 source;
  Bogart pigeonhole: 2 sources). The other 5 questions retrieve
  Wikipedia chunks for their topics — the claim-pack textbooks have
  vastly fewer chunks than Wikipedia, so retrieval defaults to the
  bulkier corpus.
- Conditions: BEFORE = `a57b194~1` (verify.py / runner.py / query.py
  reverted via `git show`); AFTER = HEAD.
- Modes: quote, claim_lattice_pointer, claim_lattice. n=3 with
  `--burn` between samples.

## Cache-key parity

**21/21 cells match BEFORE/AFTER.** Phase 3 doesn't fold into
`governance_policy_hash` (the new `warrant_chain_roots` parameter is
runtime-passed, not policy-folded), so retrieval + prompt are
byte-identical between conditions. Any STRICT-rate movement is real
verifier-side signal, not LLM dice.

## Phase 3 fire rate

**`warrant_proven_claim_idxs` is non-empty in 0 / 63 AFTER cells.**
The Phase 3 suppression path never triggers on this fixture.

## STRICT-rate

| | BEFORE | AFTER | Δ |
|---|---|---|---|
| overall | 42/63 (66.7%) | 43/63 (68.3%) | +1 cell (+1.6pp) |

Four cells move at the per-cell level — 2 up (Russell quote +2; De
Morgan pointer +1; Bogart pointer +1) and 2 down (Cantor quote −2;
Hilbert-incidence pointer −1). Net +1, within LLM noise on n=3.

Per-cell parity reads as: cache_keys identical, Phase 3 mechanism
not firing, so the deltas are LLM sampling variance — exactly the
same shape as yesterday's smoke A/B finding.

## Verdict

Phase 3 is **mechanistically landed but currently dormant on this
fixture**. The data condition (claim cites chain-root chunk AND
lexical `warrant_check` fails) is not met by:

1. Questions whose retrieval doesn't surface any chain-root chunks
   (5/7 in this fixture).
2. Questions that DO retrieve chain-root chunks but where the LLM
   cites the bulkier Wikipedia chunks instead (the remaining 2/7).
3. Questions where the LLM does cite a chain-root chunk but the
   lexical `warrant_check` already passes — Phase 3 has nothing to
   suppress.

This is not a defect — it's a calibration finding. The mechanism is
correct + non-regressing on cells where it doesn't fire. To actually
exercise Phase 3, a future fixture needs questions where:

- Specific terminology biases retrieval AWAY from Wikipedia toward
  claim-pack chunks (e.g. exact theorem labels: "Hilbert pillar IV
  axiom of incidence").
- Claims about those topics phrase themselves in ways that don't
  pass lexical anchors but DO have warrant-chain backing.

## Reproducing

```bash
# AFTER (HEAD)
.venv/bin/python bench/qa_sweep.py \
    --questions bench/qa_questions_warrant_chain_probe.txt \
    --shards-dir ~/.arborist/shards \
    --out-dir bench/qa_results_phase3_AB/after \
    --top-k 8 \
    --modes quote,claim_lattice_pointer,claim_lattice \
    --n 3 --concurrency 4

# BEFORE (Phase 3 reverted)
git show a57b194~1:arborist/qa/verify.py > arborist/qa/verify.py
git show a57b194~1:arborist/qa/runner.py > arborist/qa/runner.py
git show a57b194~1:arborist/qa/query.py > arborist/qa/query.py
# (re-run the same sweep with --out-dir bench/qa_results_phase3_AB/before)
git checkout HEAD -- arborist/qa/verify.py arborist/qa/runner.py arborist/qa/query.py
```

## Source files

- Fixture: `bench/qa_questions_warrant_chain_probe.txt`
- BEFORE results: `bench/qa_results_phase3_AB/before/`
- AFTER results: `bench/qa_results_phase3_AB/after/`
