bench: aggressive warrant fixture confirms Phase 3 is rescue-only, not default-path

Followup to dee6623 ("bench: #000031 Phase 3 A/B finds mechanism
dormant on warrant-targeted fixture") — the initial A/B reported
Phase 3 dormant on a 7-question fixture where only 2/7 questions
retrieved any chain-root sources. Open question: was the dormancy
a fixture inadequacy or a structural property of Phase 3?

This commit answers it. Aggressive fixture (13 questions using
exact claim-pack title phrasing — "What is the Pigeonhole
Principle?", "Define the Axiom of Extensionality", etc.; titles
sampled directly from the 92-document chain-root set on shard
000.db) drives retrieval to chain-root chunks on **12/13
questions** (24 chain sources retrieved total).

A/B vs a57b194~1, n=3, 117 cells per condition:

  cache_key parity:                          39/39 cells match
  Phase 3 fire rate (warrant_proven_idxs):   0 / 117 cells
  STRICT-rate:                               85/117 → 85/117 (Δ +0)
  per-cell movement (LLM dice):              4 ↑, 4 ↓

Phase 3 fires zero times even though retrieval does land on chain
roots on 92% of questions. The structural reason now reads as:

  Phase 3 is a rescue mechanism, not a default path. It fires only
  when (a) a cited chunk has warrant-resolver chain backing AND
  (b) the lexical warrant_check would otherwise fail. On
  well-formed axiom questions, the LLM's answer contains the
  technical terminology that the lexical anchor heuristic looks
  for — so warrant_check passes via the cheap lexical path and
  Phase 3 has nothing to suppress.

Two readings of this finding:

1. **Positive for the lexical layer.** It's strong enough on
   clean axiom questions that the chain-fallback isn't needed.
   The mechanism that lands warrants on cheap lexical evidence
   continues to do most of the work.

2. **Phase 3 calibration needs a different fixture shape.** To
   exercise the rescue path empirically, a future fixture would
   need to elicit lexical-fail-but-chain-pass claims —
   paraphrase-heavy questions, disputed-terminology topics,
   questions where the LLM naturally drifts from chunk vocabulary.

Mechanism is correct + non-regressing (cache_keys identical → no
retrieval movement; 0/117 fire rate confirms defensive-only
behavior). No further changes warranted to the implementation.

C — separate finding (refactor sweep verification): a57b194 has
zero v7/v8 patterns. Final exhaustive sweep across arborist/ +
tests/ shows only the intentional historical-note comments in
arborist/cli.py (Was \`arborist v8 score\`...) and
arborist/substrate/__init__.py (explaining why v-dirs were
retired). Refactor cleanup is complete.

B — separate finding (Makefile bench-fork-score smoke):
make bench-fork-baseline + make bench-fork-score runs end-to-end
under the renamed CLI. Verdict MARGINAL (score 0.0 for self-
comparison), exit 0. Yesterday's Makefile:430 fix verified live.

bench/qa_questions_warrant_chain_aggressive.txt

@@ -0,0 +1,40 @@
+# Aggressive warrant-chain fixture (2026-05-10).
+#
+# Phase 3 verifier (a57b194) suppresses WARRANT_MISSING when a cited
+# chunk has a warrant-resolver derivation row. The first probe fixture
+# (qa_questions_warrant_chain_probe.txt) found Phase 3 dormant —
+# retrieval landed on Wikipedia chunks instead of claim-pack chunks
+# even on warrant-targeted topics.
+#
+# This fixture uses **exact claim-pack title phrasing** to maximize
+# title-token boost in the retrieval pipeline. Each question's noun
+# phrase matches a specific claim-pack record's title verbatim, which
+# should drive retrieval to surface that claim-pack chunk in the top
+# slots.
+#
+# Source: titles sampled from documents where document_root ∈
+# warrant_chain_lookup() (the 92 chain-root set) on shard 000.db
+# 2026-05-10.
+
+# Pillar VII — Combinatorics (titles match claim-pack records)
+What is the Pigeonhole Principle?
+What is Pascal's Rule?
+What is the Multiplication Principle in combinatorics?
+
+# Pillar I — First-order logic (axiom titles)
+What is the Axiom of Implication Introduction?
+What is the Axiom of Reflexivity of Equality?
+What is the Axiom of Universal Specification?
+
+# Pillar II — Set theory (ZFC axioms)
+Define the Axiom of Extensionality.
+What is the Axiom of Empty Set?
+What is the Axiom of Power Set?
+Define the Axiom of Foundation.
+
+# Pillar III — Peano arithmetic
+What is the Axiom of Successor?
+Define the Axiom of Addition (Zero).
+
+# Pillar IV — Hilbert geometry
+State the Archimedean Axiom.

bench/results/phase3-warrant-chain-A-B-aggressive-2026-05-10.md

@@ -0,0 +1,110 @@
+# #000031 Phase 3 verifier — aggressive A/B (2026-05-10)
+
+**Followup to**: `phase3-warrant-chain-A-B-2026-05-10.md` (initial
+A/B which found Phase 3 dormant on a 7-question warrant fixture).
+The initial probe showed only 2/7 questions retrieved any chain-
+root sources; this followup uses **exact claim-pack title phrasing**
+(13 questions; titles sampled directly from the 92-document
+chain-root set on shard 000.db) to maximize title-token boost
+in retrieval. Goal: confirm whether dormancy was a fixture
+inadequacy or a structural property of Phase 3.
+
+## Pre-bench probe
+
+```
+12/13 questions hit chain ✓     24 chain sources retrieved total
+```
+
+Aggressive fixture works as designed: title-token-aligned questions
+drive retrieval to claim-pack chunks. The single miss (Archimedean
+Axiom) retrieves 8 sources but none are chain-root — likely the
+Wikipedia article on the Archimedean property dominates.
+
+## A/B method
+
+- BEFORE: `a57b194~1` (verify.py / runner.py / query.py reverted
+  via `git show`).
+- AFTER: HEAD (Phase 3 active).
+- 13 questions × 3 modes (quote / claim_lattice_pointer /
+  claim_lattice) × n=3 = 117 cells per condition.
+- `--burn` between samples.
+
+## Cache-key parity
+
+**39/39 cells match BEFORE/AFTER.** Phase 3 doesn't fold into
+`governance_policy_hash` — retrieval + prompt + LLM input are
+byte-identical between conditions. Same clean A/B shape as the
+first probe.
+
+## Phase 3 fire rate
+
+**`warrant_proven_claim_idxs` is non-empty in 0 / 117 AFTER cells.**
+
+Phase 3's suppression path **never fires** even though the
+aggressive fixture drives retrieval to chain-root chunks on 92% of
+questions.
+
+## STRICT-rate
+
+| | BEFORE | AFTER | Δ |
+|---|---|---|---|
+| overall | 85/117 (72.6%) | 85/117 (72.6%) | **+0** (exactly identical aggregate) |
+
+8 cells move at the per-cell level — 4 up, 4 down. All within LLM
+n=3 noise; cache_keys match so retrieval+prompt are byte-identical
+and the variance is purely in LLM sampling.
+
+## Refined verdict
+
+The first A/B's "Phase 3 is structurally dormant" reading was
+correct in fact but incomplete in *why*. Now, with retrieval driving
+chain-root chunks into the source set and Phase 3 still firing zero
+times, the structural reason becomes clear:
+
+**Phase 3 is a rescue mechanism, not a default path.** It fires only
+when:
+
+1. A cited chunk has warrant-resolver chain backing, AND
+2. The lexical `warrant_check` would otherwise fail on the
+   claim citing that chunk.
+
+On well-formed axiom questions, the LLM's answer typically contains
+the technical terminology that the lexical anchor heuristic looks
+for — so `warrant_check` passes via the cheap lexical path and
+Phase 3 has nothing to suppress.
+
+This is a **positive** finding for the lexical layer: it's strong
+enough on clean axiom questions that the chain-fallback isn't
+needed. Phase 3's value will surface on questions where:
+
+- The LLM paraphrases its claim away from the chunk's exact
+  terminology (lexical anchor heuristic fails).
+- BUT the chunk has chain backing (Phase 3 rescue available).
+
+That intersection is empirically rare on well-named axiom
+questions where the model naturally produces well-anchored output.
+Future Phase 3 calibration could target it explicitly with prompts
+that encourage paraphrase / questions about disputed-terminology
+topics where the LLM is likely to drift.
+
+## Concrete recommendation
+
+Phase 3 mechanism is **correct + non-regressing**; cache_keys
+identical → cannot move retrieval; 0/117 fire rate confirms
+defensive-only behavior. No further changes warranted to the
+mechanism itself.
+
+For ticket #000031 closure: the warrant-chain rescue path is
+landed and unit-tested (see `tests/test_warrant_chain.py` from
+a57b194). Live-bench measurement of the rescue's effect awaits
+a fixture designed to elicit lexical-fail-but-chain-pass claims —
+which is itself a research-design problem orthogonal to whether
+the mechanism works.
+
+## Source files
+
+- Aggressive fixture: `bench/qa_questions_warrant_chain_aggressive.txt`
+- Probe fixture (initial): `bench/qa_questions_warrant_chain_probe.txt`
+- Initial A/B report: `bench/results/phase3-warrant-chain-A-B-2026-05-10.md`
+- BEFORE results: `bench/qa_results_phase3_aggressive/before/`
+- AFTER results: `bench/qa_results_phase3_aggressive/after/`