# Claim-pack retrieval lift — 2026-05-09

**Date:** 2026-05-09 (UTC)
**Author:** blackops + fox
**Method:** apples-to-apples FTS5 search comparison on a single shard (`~/.arborist/shards/000.db`) before vs after ingesting the claim-pack source (#000029) bundles
**Shard:** ~867 K Wikipedia surface docs (one quarter of the 4-shard cluster)
**Augmentation:** + 78 claim-pack documents (55 axioms + 23 theorems across 7 pillars from `axiomsg4-v2.json` + `theoremsg4-v2.json`)

## Why this measurement

Closing-note honest ceiling from #000029: every claim-pack record
lands `kind='surface'`. Question pending until measured: does the
pre-distilled CORE-shape content earn its 1.6%-ish storage tax in
practice, or does Wikipedia's existing coverage dominate retrieval
already? This bench answers the question for four representative
queries against the same shard.

## Method

```bash
# Baseline copy (Wikipedia only).
cp ~/.arborist/shards/000.db /tmp/baseline-shard.db

# Augmented copy (Wikipedia + claim-pack).
cp ~/.arborist/shards/000.db /tmp/claim-pack-test.db
arborist --db /tmp/claim-pack-test.db ingest \
    --source claim_pack \
    --bundle ~/Downloads/axiomsg4-v2.json \
    --bundle ~/Downloads/theoremsg4-v2.json \
    --quiet
# → 78 docs inserted, 14 deduped pillar-reference edges, 78 audit events.

# Compare top-3 FTS5 hits per query.
for q in "modus tollens" "law of excluded middle" \
         "Bayes theorem" "associativity of addition"; do
  arborist --db /tmp/baseline-shard.db search "$q" | head
  arborist --db /tmp/claim-pack-test.db search "$q" | head
done
```

FTS5 search is the right tool here: fast, deterministic, no LLM
call, no audit chain. Compares retrieval-only behavior; downstream
verifier and audit_mode behavior is unchanged by content-source.

## Results

| Question | Baseline top-3 | Augmented top-3 | Claim-pack lift |
|---|---|---|---|
| `modus tollens` | Modus ponendo tollens / List of mathematics articles (M) / Proving a negative | Modus ponendo tollens / List of mathematics articles (M) / **Modus Tollens** (claim-pack, BM25 31.77) | ✓ #3 |
| `law of excluded middle` | Diaconescu's theorem / Brouwer–Hilbert controversy / Propositional formula | Diaconescu's theorem / Brouwer–Hilbert controversy / **Axiom of Double Negation and Implication** (claim-pack, BM25 18.84) | ✓ #3 (related axiom; "Law of Excluded Middle" theorem is at #5) |
| `Bayes theorem` | Bayes / Bayes rule / Admissible decision rule | Bayes / Bayes rule / Admissible decision rule | ✗ no lift in top-3 (claim-pack record exists but BM25 below the cutoff because Wikipedia's "Bayes" + "Bayes rule" articles match strongly) |
| `associativity of addition` | Addition / Modular Addition / Semifield | **Associativity of Addition** (claim-pack, BM25 8.72) / Addition / Modular Addition | ✓ **#1** (displaces Wikipedia's general "Addition" article) |

**Lift rate:** 3/4 queries show claim-pack record in top-3.
**Strongest case:** "associativity of addition" — claim-pack record
takes #1, displacing Wikipedia's general "Addition" article. The
title-token overlap on a narrow technical query is exactly where
pre-distilled CORE-shape content shines.

## Why Bayes theorem didn't lift

Wikipedia's "Bayes" and "Bayes rule" articles have stronger
BM25 scores (27.1, 26.6) than the claim-pack record's likely
score because:

1. The claim-pack record's title is `Bayes' Theorem` (with
   apostrophe); the existing FTS5 tokenization may split it
   differently than the Wikipedia articles.
2. The Wikipedia "Bayes rule" article matches "rule" via
   redirect/disambig, raising its score.
3. Wikipedia's stub-style articles for `Bayes` and `Bayes rule`
   are short, which artificially boosts BM25 (short-doc bias).

**Mitigation already in flight (#000026 retrieval pipeline):**
the `body-coverage sqrt rerank` (`arborist/qa/query.py`) counters
BM25's short-doc bias; with that rerank, the claim-pack record
likely surfaces. This bench measures FTS5-only retrieval; the
full QA pipeline runs more stages.

## Honest verdict

**Claim-pack ingest is earning its storage tax for 3/4 of these
representative queries.** The lift is asymmetric — strongest on
narrow technical phrases that match a specific axiom/theorem
title (e.g. "associativity of addition"), weaker on phrases
where Wikipedia already has a focused article (Bayes / Bayes
rule). The four-rung audit ladder still caps at
`ANCHOR-WARRANTED` per #000029's honest ceiling; surface-ingest
of cited textbooks (#000031) is required to push toward
`EVIDENCE-WARRANTED`.

**Next step:** Run the same comparison through the full QA
pipeline (`arborist query`) instead of FTS5-only, so the
body-coverage rerank, title-relevance filter, and concept-
relations layer all contribute. That's a separate, larger bench
that consumes LLM tokens; this read-only FTS5 comparison was
the right first signal.

## Storage cost

Single-shard storage tax measured before vs after (claim-pack only,
not including any concept_relations updates):

```
baseline (000.db):  ~870 MB
augmented:          ~870 MB  (claim-pack delta is sub-MB; below
                              filesystem allocation granularity)
```

78 documents × ~512-token chunks = a single tiny chunk per
document → negligible storage delta. The dominant cost stays the
existing Wikipedia content; claim-pack rides essentially for
free.

## Reproducibility

- Source bundles: `~/Downloads/axiomsg4-v2.json` +
  `~/Downloads/theoremsg4-v2.json`
- arborist commit at measurement: `7b7ac38`
  (combinatorics@v1 landed; claim-pack source unchanged since
  `b38f4b8`)
- SymPy version irrelevant (FTS5-only path; no kernel involved)
