From 8ba123dc17c24858f6070cb5801074743fd5a6ba Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Sat, 9 May 2026 18:00:36 -0400 Subject: [PATCH] bench/results: warrant-chain render tail measurement (2026-05-09) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six-question Q&A sweep against the augmented shard cluster post-#000031 Phase 3 commit (69e0a95). Looking for the new `· warrant: N proven` tail surfaced by _render_warrant_chain_tail when an answer's cited sources include a claim-pack record with a warrant-resolver-v1 derivations row. Result: 1 of 6 questions lit up the warrant tail. Pasch's axiom → STRICT · via quote · warrant: 1 proven ✓ Hilbert geometry axioms → STRICT · via quote ✗ axiom of betweenness → HYBRID · via paraphrase ✗ Bayes theorem → STRICT · via quote ✗ modus tollens → UNGROUNDED · via quote ✗ binomial coefficient id. → HYBRID · via paraphrase ✗ 1/6 matches the data substrate: 11 of 92 records have warrant chains, all pillar-IV Hilbert axioms. The 5 unsigned cases are honest: - Hilbert / betweenness queries cited Wikipedia + crawled textbook surfaces, NOT the claim-pack records — no claim-pack record cited → no warrant chain to surface. - Bayes theorem: even if cited, Kolmogorov isn't surface- ingested → would stay at ANCHOR-WARRANTED ceiling. - modus tollens UNGROUNDED: pre-warrant verifier issue, orthogonal to this ticket. - binomial-coefficient: claim-pack pillar VII cites Brualdi / Stanley / Knuth, none surface-ingested. No false positives observed — the render tail correctly stays silent when chain-of-custody data is absent. Roadmap: Adding any cited textbook to the manifest → re-running warrant-resolve → more records resolve, more queries light up the tail. The 74 still-unresolved records are a prioritized acquisition list (Mendelson ~20, Enderton ~10, Stanley ~5, Brualdi ~3, Knuth ~3). The substrate writes its own roadmap. --- .../warrant-chain-render-2026-05-09.md | 116 ++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 bench/results/warrant-chain-render-2026-05-09.md diff --git a/bench/results/warrant-chain-render-2026-05-09.md b/bench/results/warrant-chain-render-2026-05-09.md new file mode 100644 index 0000000..741d8b5 --- /dev/null +++ b/bench/results/warrant-chain-render-2026-05-09.md @@ -0,0 +1,116 @@ +# Warrant chain render tail bench — 2026-05-09 + +**Date:** 2026-05-09 (UTC) +**Author:** blackops + fox +**Commits:** `f0e6baf` (Phase 2 resolver + writer) + `69e0a95` +(Phase 3 render tail + theorem-name resolver) + +## Setup + +- Shard cluster: `~/.arborist/shards/000-003.db` — 3.47M docs + including 92 claim-pack records (78 v2 + 14 pillar VII) +- Crawl shards: `~/.arborist/crawl/textbook_*.db` — 9 surface + textbooks across 6 g4 pillars +- 11 `derivations.process_id='warrant-resolver-v1'` rows in + shard 000.db, all binding pillar-IV Hilbert axioms to chunks + in the surface-ingested Hilbert *Foundations of Geometry* + +## Method + +Six representative questions through `arborist query --top-k 3` +(full QA pipeline: retrieval + body-coverage rerank + title- +relevance + concept-relations + verifier + render layer): + +```bash +arborist --shards-dir ~/.arborist/shards query "$q" --top-k 3 +``` + +Looking for the new `· warrant: N proven` tail surfaced by +`_render_warrant_chain_tail` (commit `69e0a95`) when the +answer's cited sources include a claim-pack record with a +`warrant-resolver-v1` derivations row. + +## Results + +| # | Question | Audit line | Warrant tail? | +|---|---|---|---| +| 1 | `Pasch's axiom` | STRICT · via quote · **warrant: 1 proven** 1/1 2.9s | ✓ | +| 2 | `Hilbert geometry axioms` | STRICT · via quote 13/13 13.9s | ✗ | +| 3 | `axiom of betweenness` | HYBRID · via paraphrase 3/4 4.1s | ✗ | +| 4 | `Bayes theorem` | STRICT · via quote 1/1 8.1s | ✗ | +| 5 | `modus tollens` | UNGROUNDED · via quote 0/1 4.0s | ✗ | +| 6 | `binomial coefficient identity` | HYBRID · via paraphrase 5/7 5.4s | ✗ | + +**1 of 6 questions** lit up the warrant tail. That's the +honest signal — only **11 of 92** claim-pack records (~12%) +have warrant chains, and only the question that retrieved a +Hilbert-pillar-IV record AS its cited source surfaced the tail. + +## Why each case landed where it did + +- **#1 Pasch's axiom (warrant: 1 proven)** — The retrieval + pipeline cited *both* the Wikipedia "Pasch's axiom" article + AND the claim-pack pillar-IV record. The claim-pack record + has a `warrant-resolver-v1` derivations row binding it to + chunk 1 of the Hilbert textbook. The render layer surfaces + this chain. +- **#2 Hilbert geometry axioms (no tail)** — STRICT verdict, + but the cited sources were Wikipedia + crawled Hilbert + surface — NOT the claim-pack records. The claim-pack records + ARE in the cluster but didn't make the top-k for this + multi-axiom question. No claim-pack record cited → no warrant + chain to surface. +- **#3 axiom of betweenness (no tail)** — Same as #2 — claim- + pack record not in the cited set. +- **#4 Bayes theorem (no tail)** — Different reason: even if + the claim-pack Bayes record HAD been cited, Kolmogorov isn't + surface-ingested, so its `source_reference` doesn't resolve + to a Merkle-bound surface chunk. ANCHOR-WARRANTED ceiling per + #000031 §1 honest-ceiling clause. +- **#5 modus tollens (UNGROUNDED, no tail)** — The verifier + couldn't substring-match the answer's quotes against any + cited source. Pre-warrant-chain issue; orthogonal to this + ticket's scope. +- **#6 binomial coefficient identity (no tail)** — Same as #4 + — claim-pack pillar VII records cite Brualdi / Stanley / + Knuth, none of which are surface-ingested under the current + manifest. + +## What this measurement actually shows + +The render-layer tail correctly surfaces when a chain-of-custody +exists from answer → cited source → primary-source surface +chunk via Merkle-proven derivations. It correctly stays silent +in absence of data: + +1. **Honest signal floor.** 1/6 lift in the broad sweep matches + the data substrate (11/92 records have chains). No false + positives observed on Bayes / Combinatorics queries whose + cited textbooks aren't ingested. +2. **Targeted lift would be much higher.** A bench specifically + on the 11 Hilbert-pillar-IV axioms with warrants, asked + against the augmented shard, would light up the tail on + ~all of them (modulo which records the retrieval pipeline + chooses to cite). +3. **The path to higher coverage is content acquisition, not + code.** Adding Mendelson + Enderton would resolve ~20 more + records (logic pillar I); adding Stanley + Brualdi + Knuth + would resolve ~7 pillar VII records; adding Jech would + resolve ~7 set-theory records. The resolver code is done; + the manifest needs more textbooks. + +## Reproducibility + +```bash +# Clear + re-populate derivations +sqlite3 ~/.arborist/shards/000.db \ + "DELETE FROM derivations WHERE process_id='warrant-resolver-v1';" +arborist --shards-dir ~/.arborist/shards warrant-resolve --write +# → records_total=92, records_resolved=11, derivations_written=11 + +# Run targeted query +arborist --shards-dir ~/.arborist/shards query "Pasch's axiom" --top-k 3 +# Look for: "· warrant: 1 proven" on the audit line +``` + +arborist commit at measurement: `69e0a95`.