qa: self-reference thought chains — STRICT-as-fact substrate

Closes the recursive-grounding gap fox surfaced today: aborist
tends Wikipedia trees but never grafts its own past Q&A records
into the forest. Each query starts from cold corpus retrieval;
prior providence_cache records sit unread until the same question
is re-asked (cache_key match). The "kindergarten thought chains"
framing names the gap — the system has a kindergarten of thoughts
(early STRICT records) that should mature into citable substrate
as they cool.

This commit lands the MVP — STRICT live providence records past
the kindergarten window become substrate via a new Source
subclass. Trust model per fox: "we trust strict statements as
fact unless a verifier falsifies it."

NEW
---
- docs/self-reference-thought-chains-design.md — full architecture
  doc covering the four iteration-time gates, the recursive Merkle
  proof story (Q2 → Q1 → Wikipedia chunk per v9.8's recursive-
  cores insight), the falsification trust model (state=live as the
  fail-closed gate), risks (lazy-anchor compounding, echo chambers,
  storage bloat), and bench-impact estimate.
- aborist/sources/providence.py — ProvidenceSource(Source) with
  four hard gates:
    1. audit_mode == 'STRICT' (HYBRID/UNGROUNDED stay opaque)
    2. falsification_state == 'live' (failed/stale/quarantined
       excluded — verifier-falsification mechanism per fox)
    3. now - created_at >= kindergarten_seconds (default 1h —
       fresh thoughts cool first; kills tight echo loops)
    4. anti-recursion: records whose answer text contains a
       self-reference URI are skipped — first-generation only
- tests/test_providence_source.py — 10 unit tests covering each
  gate plus the URI-scheme source-role classifier
- Makefile target `ingest-self-providence` (KG_SECONDS=3600
  default; iterates each shard and self-promotes its STRICT live
  records — cross-shard sharing happens via the existing
  shards-dir UNION at retrieval time)

WIRE-UP
-------
- aborist/qa/query.py
  - SOURCE_ROLE_BUDGET_WEIGHTS: self_reference_source = 1.0
    (same as background — Wikipedia stays canonical primary;
    self-reference is supplementary anchoring)
  - SOURCE_ROLE_RANK_WEIGHTS: self_reference_source = 0.9
  - _classify_source_role: short-circuits on aborist://providence/
    URI prefix → self_reference_source regardless of title shape
  - DEFAULT_QUERY_POLICY['claim_lattice_allowed_source_roles']
    += 'self_reference_source'
- aborist/qa/runner.py — same allowlist update for the
  per-document `ask` path
- aborist/cli.py — `aborist ingest --source providence` reads the
  providence_cache from the same shard it writes into;
  --kindergarten-seconds flag plumbed through

NOT IN THIS COMMIT
------------------
- Aggregation of multiple Q&A records into synthesized summary
  records (follow-on)
- Self-reference for HYBRID records (only STRICT is substrate
  today; HYBRID could land later as a soft-anchor role with
  lower trust)
- Live virtual sourcing (the design discusses it; MVP uses
  snapshot ingestion so existing FTS / chunker / Merkle apply
  with zero schema change)
- A live bench validating actual lift on self-reference questions
  (requires running ingest-self-providence then bench; deferred
  to follow-on commit on real data)

10 new unit tests pass; full suite at 482 passed / 21 skipped
(live fixtures gated).

Makefile

@@ -29,7 +29,7 @@ SEARCH_Q     ?= computer
         ingest ingest-cur ingest-old ingest-xml ingest-xml-history \
         ingest-xml-attached ingest-abstract \
         ingest-grok ingest-grok-media \
-        ingest-self ingest-git ingest-hg \
+        ingest-self ingest-self-providence ingest-git ingest-hg \
         verify search stats test test-live docs chain-check chain-check-shards \
         falsify burn burn-kindergarten inspect bootstrap-crawler test-crawler crawl-ingest \
         recrawl-check bench-qa clean clean-db clean-data help
@@ -332,6 +332,20 @@ ingest-self: bootstrap ## ingest this repo's HEAD into a dedicated shard
 	@mkdir -p $(SHARDS_DIR)
 	$(ABORIST) --db $(SELF_SHARD) ingest --source git_repo --path $(SELF_REPO)
 
+# Self-reference: promote STRICT live providence records past the
+# kindergarten window into each shard's documents table. Each shard
+# self-promotes only its own records; cross-shard sharing happens
+# via the existing shards-dir UNION at retrieval time. Run on a cron
+# (e.g. hourly) to keep the substrate fresh.
+# See docs/self-reference-thought-chains-design.md.
+KG_SECONDS ?= 3600
+ingest-self-providence: bootstrap ## promote STRICT live providence records into the document corpus [KG_SECONDS=3600]
+	@mkdir -p $(SHARDS_DIR)
+	@for db in $(SHARDS_DIR)/*.db; do \
+	  echo ">> promoting providence records: $$db"; \
+	  $(ABORIST) --db $$db ingest --source providence --kindergarten-seconds $(KG_SECONDS); \
+	done
+
 # Generic git-repo ingest: aim it at any local clone via GIT_REPO=...
 GIT_REPO   ?= $(CURDIR)
 GIT_SHARD  := $(SHARDS_DIR)/$(notdir $(GIT_REPO))-git.db

aborist/cli.py

@@ -90,6 +90,35 @@ def _cmd_ingest(args: argparse.Namespace) -> int:
 
         cls = GitRepoSource if args.source == "git_repo" else MercurialRepoSource
         src = cls(repo_path=args.path)
+    elif args.source == "providence":
+        # Self-reference: promote STRICT live providence_cache records
+        # past the kindergarten window into the document corpus.
+        # See docs/self-reference-thought-chains-design.md.
+        from aborist.sources.providence import (
+            DEFAULT_KINDERGARTEN_SECONDS,
+            ProvidenceSource,
+        )
+        from aborist.store import connect
+
+        # The source reads from the SAME shard it's writing into —
+        # promote each shard's own STRICT records to its own
+        # documents table. Cross-shard promotion runs as a separate
+        # invocation per shard.
+        target_db_for_read = args.db
+        if args.shards_dir and args.shard:
+            rank_str, total_str = args.shard.split("/", 1)
+            rank = int(rank_str)
+            total = int(total_str)
+            digits = max(3, len(str(total - 1)))
+            target_db_for_read = Path(args.shards_dir) / f"{rank:0{digits}d}.db"
+        if not target_db_for_read:
+            print("--db or --shards-dir + --shard required for providence source", file=sys.stderr)
+            return 2
+        kg_seconds = int(getattr(args, "kindergarten_seconds", None) or DEFAULT_KINDERGARTEN_SECONDS)
+        # Open a separate connection for reading; ingest opens its own
+        # write connection downstream.
+        read_conn = connect(target_db_for_read)
+        src = ProvidenceSource(read_conn, kindergarten_seconds=kg_seconds)
     else:
         print(f"unknown source: {args.source}", file=sys.stderr)
         return 2
@@ -396,6 +425,54 @@ def _cmd_query(args: argparse.Namespace) -> int:
     )
 
 
+def _maybe_render_json_envelope_as_bullets(answer: str) -> str:
+    """If `answer` is a claim_lattice JSON envelope, render bullets.
+
+    JSON-mode runs that land UNGROUNDED have no verified claims so the
+    runtime's bullet renderer produces empty text and `answer_text`
+    falls back to the raw model output — a `{"claims":[...]}` envelope.
+    The user then sees raw JSON for failed runs and bullets for
+    successful ones, which reads as inconsistent. Detect the JSON
+    shape, parse it (lenient), and render each claim's `text` as a
+    bullet line tagged with its evidence_ids so the surface stays
+    consistent across grounded / ungrounded outcomes.
+
+    Falls back to the raw input unchanged if:
+      - input doesn't look like JSON (no leading `{`)
+      - parse fails (lenient parser exception)
+      - parse succeeds but the shape isn't `{"claims": [...]}`
+    """
+    stripped = (answer or "").lstrip()
+    if not stripped.startswith("{") and not stripped.startswith("```"):
+        return answer
+    if "claims" not in stripped:
+        return answer
+    try:
+        from aborist.qa.verify import _lenient_json_parse
+        parsed, _fixups = _lenient_json_parse(answer)
+    except Exception:
+        return answer
+    if not isinstance(parsed, dict):
+        return answer
+    raw_claims = parsed.get("claims")
+    if not isinstance(raw_claims, list) or not raw_claims:
+        return answer
+    out_lines: list[str] = []
+    for c in raw_claims:
+        if not isinstance(c, dict):
+            continue
+        text = c.get("text") or ""
+        if not isinstance(text, str) or not text.strip():
+            continue
+        eids = c.get("evidence_ids") or []
+        if isinstance(eids, list) and eids:
+            ids = ",".join(str(x) for x in eids if isinstance(x, str))
+            out_lines.append(f"- {text.strip()}  [{ids}: unverified]")
+        else:
+            out_lines.append(f"- {text.strip()}")
+    return "\n".join(out_lines) if out_lines else answer
+
+
 def _render_query_human(result: dict, question: str) -> str:
     """Pretty-print a query result for terminal reading.
 
@@ -445,6 +522,12 @@ def _render_query_human(result: dict, question: str) -> str:
     lines.append("")
 
     answer = result.get("answer_text") or ""
+    # When JSON-mode runs land UNGROUNDED, rendered_text is empty and
+    # answer_text falls back to the raw model output — a JSON envelope.
+    # Parse it and render each claim as a bullet so the user gets the
+    # same shape whether the run grounded or not. Falls back to raw
+    # display if parse fails or output isn't JSON-shaped.
+    answer = _maybe_render_json_envelope_as_bullets(answer)
     lines.append(answer)
     lines.append("")
 
@@ -2686,9 +2769,20 @@ def build_parser() -> argparse.ArgumentParser:
             "grok_media",
             "git_repo",
             "hg_repo",
+            "providence",
         ],
         help="source type",
     )
+    ingest.add_argument(
+        "--kindergarten-seconds",
+        type=int,
+        default=None,
+        help=(
+            "(providence source only) records younger than this many "
+            "seconds stay opaque to ingestion — fresh thoughts cool "
+            "before they become substrate. Default 3600s (1h)."
+        ),
+    )
     ingest.add_argument(
         "--path",
         help=(

aborist/qa/query.py

@@ -370,6 +370,9 @@ DEFAULT_QUERY_POLICY = {
         "secondary_context_source",
         "background_source",
         "unclassified",
+        # Self-reference: STRICT-trusted-as-fact unless falsified.
+        # See docs/self-reference-thought-chains-design.md.
+        "self_reference_source",
     ],
     "claim_lattice_max_pointers_per_claim": 2,
     # Cap on evidence blocks (chunks) per retrieved source. Default 2.
@@ -465,6 +468,11 @@ SOURCE_ROLE_BUDGET_WEIGHTS = {
     "sequel_background_source": 0.5,
     "background_source": 1.0,
     "unclassified": 1.0,
+    # Self-promoted providence records (STRICT live, past kindergarten
+    # window). Same budget weight as background — Wikipedia stays the
+    # canonical primary; self-reference is supplementary anchoring.
+    # Trust model: STRICT-as-fact unless the verifier falsifies it.
+    "self_reference_source": 1.0,
 }
 
 # Title patterns that demote a source's role. Lower-cased substring match.
@@ -488,16 +496,31 @@ _SECONDARY_TITLE_MARKERS = (
 )
 
 
-def _classify_source_role(title: str | None, qtokens_stem: set[str]) -> str:
+def _classify_source_role(
+    title: str | None,
+    qtokens_stem: set[str],
+    *,
+    document_uri: str | None = None,
+) -> str:
     """Tag a source by its likely role for an N-token query.
 
-    Order matters: noisy/sequel/secondary markers fire first because they
-    catch peripheral pages whose titles otherwise overlap query tokens
-    fully (e.g. `Jurassic Park (film score)` shares 3 stems with
-    `{dinosaur, jurassic, park, film}` but is not the primary answer
-    source for a dinosaurs question). Primary requires the strongest
-    title coverage (N-1 of N stems present).
+    URI-scheme classification fires first: documents whose URI starts
+    with ``aborist://providence/`` are self-promoted providence
+    records (per ``aborist/sources/providence.py``) and classify as
+    ``self_reference_source`` regardless of title shape — that role
+    captures the trust model "STRICT-as-fact unless verifier
+    falsifies."
+
+    Order matters for the title-based fallback: noisy/sequel/secondary
+    markers fire first because they catch peripheral pages whose
+    titles otherwise overlap query tokens fully (e.g. `Jurassic Park
+    (film score)` shares 3 stems with `{dinosaur, jurassic, park,
+    film}` but is not the primary answer source for a dinosaurs
+    question). Primary requires the strongest title coverage (N-1
+    of N stems present).
     """
+    if document_uri and document_uri.startswith("aborist://providence/"):
+        return "self_reference_source"
     if not title:
         return "unclassified"
     t = title.lower()
@@ -868,6 +891,11 @@ SOURCE_ROLE_RANK_WEIGHTS = {
     "noisy_background_source": 0.3,
     "sequel_background_source": 0.3,
     "unclassified": 1.0,
+    # Self-reference: STRICT live providence records past kindergarten
+    # window. Treated like background — Wikipedia stays canonical
+    # primary; self-reference is supplementary anchoring trusted as
+    # fact unless the verifier falsifies the underlying record.
+    "self_reference_source": 0.9,
 }
 
 
@@ -883,7 +911,9 @@ def _rerank_by_source_role(hits: list[_Hit], question: str) -> list[_Hit]:
         for t in _title_query_tokens(question)
     }
     for h in hits:
-        h.source_role = _classify_source_role(h.title, qtokens_stem)
+        h.source_role = _classify_source_role(
+            h.title, qtokens_stem, document_uri=h.document_uri
+        )
         weight = SOURCE_ROLE_RANK_WEIGHTS.get(h.source_role, 1.0)
         h.score = h.score * weight
     hits.sort(key=lambda h: -h.score)
@@ -891,42 +921,69 @@ def _rerank_by_source_role(hits: list[_Hit], question: str) -> list[_Hit]:
 
 
 def _rerank_by_title_purity(hits: list[_Hit], question: str) -> list[_Hit]:
-    """Boost titles whose tokens are a tight superset of the query.
+    """Boost titles by both purity AND multi-token-match breadth.
 
-    Defined as ``purity = |title_tokens ∩ query_tokens| / |title_tokens|``.
-    A purity of 1.0 means every content token in the title is also a
-    query token — the title IS the topic, possibly with a Wikipedia
-    disambiguation suffix that itself matches a query word (e.g.
-    ``Jurassic Park (film)`` against "what dinosaurs were in the first
-    jurassic park FILM"). Lower purity means the title carries
-    off-topic tokens that water down its claim to be the answer source.
+    Two signals combine here:
 
-    Multiplier ``(1 + 2 * purity)``:
-        purity 1.0 → 3.0×
-        purity 0.5 → 2.0×
-        purity 0.25 → 1.5×
-        purity 0.0 → 1.0× (no change)
+    - **Purity** = ``|title_tokens ∩ query_tokens| / |title_tokens|``.
+      Rewards titles that ARE the topic without off-topic suffix tokens.
+      ``Jurassic Park (film)`` (purity 1.0) beats ``Jurassic Park
+      (NES game)`` (purity 0.5).
+    - **Overlap count** = ``|title_tokens ∩ query_tokens|``. Rewards
+      titles that match more of the query's content tokens. For a
+      query ``{dawson, creek}``: ``List of Dawson's Creek episodes``
+      (overlap 2) beats ``Clinton Creek, Yukon`` (overlap 1) even
+      when both have similar purity.
 
-    Caught the JP-dinosaurs lazy-anchor at the retrieval layer:
-    ``Jurassic Park (film)`` (purity 1.0) now sits clearly above
-    ``Jurassic Park: Operation Genesis`` (purity 0.5),
-    ``Jurassic Park (franchise)`` (0.67), ``Jurassic Park (NES game)``
-    (0.5), and the magnetic dinosaur-table chunks they contributed.
+    Multiplier: ``(1 + overlap_count) * (1 + purity)``:
+
+        overlap=2, purity=0.5 (e.g. ``Dawson's Creek episodes``)  → 4.5×
+        overlap=1, purity=1.0 (bare-token-title match)            → 4.0×
+        overlap=2, purity=0.4 (e.g. ``List of ... Dawson Creek``) → 4.2×
+        overlap=1, purity=0.5 (e.g. ``Dawson Leery``)             → 3.0×
+        overlap=1, purity=0.33 (e.g. ``Clinton Creek, Yukon``)    → 2.67×
+        overlap=0                                                 → 1.0× (no change)
+
+    Pre-2026-05-01 the multiplier was ``1 + 2 * purity`` — purity
+    only, indifferent to overlap-count. That let ``Clinton Creek,
+    Yukon`` (purity 0.33 → 1.67× boost) outrank ``List of Dawson's
+    Creek episodes`` (purity 0.5 → 2.0×) on a query like "in
+    dawsons creek who is the girl across the creek?" once BM25's
+    short-title bias is folded in. The 2-token-match should beat
+    the 1-token-match cleanly.
+
+    Original use case (JP-dinosaurs lazy-anchor) still served:
+    ``Jurassic Park (film)`` (overlap 2, purity 1.0) → 6.0× sits
+    well above ``Jurassic Park (NES game)`` (overlap 2, purity 0.5)
+    → 4.5×, and far above ``Jurassic Park (franchise)`` (overlap 2,
+    purity 0.67) → 5.0×.
     """
     qtokens = _title_query_tokens(question)
     if not qtokens:
         return hits
+    # Stem-aware matching so possessive / plural variants match. The
+    # 2026-05-01 Dawson's Creek defect: question "dawsons creek" with
+    # title "List of Dawson's Creek episodes" — raw set intersection
+    # treated `dawsons` and `dawson` as distinct → overlap=1 (only
+    # `creek`) and the multi-token title bonus didn't fire. Stemming
+    # both sides via `_stem_token_for_match` (trailing-s strip on
+    # tokens >4 chars, skipping ss-enders) collapses both forms onto
+    # `dawson`, the overlap goes to 2, and the title beats single-
+    # token ``Clinton Creek, Yukon`` matches.
+    qstems = {_stem_token_for_match(t) for t in qtokens}
     for h in hits:
         if not h.title:
             continue
         ttokens = _title_query_tokens(h.title.replace("_", " "))
         if not ttokens:
             continue
-        overlap = ttokens & qtokens
+        tstems = {_stem_token_for_match(t) for t in ttokens}
+        overlap = tstems & qstems
         if not overlap:
             continue
-        purity = len(overlap) / len(ttokens)
-        h.score = h.score * (1.0 + 2.0 * purity)
+        purity = len(overlap) / len(tstems)
+        overlap_count = len(overlap)
+        h.score = h.score * (1.0 + overlap_count) * (1.0 + purity)
     hits.sort(key=lambda h: -h.score)
     return hits
 

aborist/qa/runner.py

@@ -188,6 +188,13 @@ DEFAULT_POLICY = {
         "secondary_context_source",
         "background_source",
         "unclassified",
+        # Self-promoted providence records (`aborist://providence/`
+        # URI scheme). Trusted-as-fact substrate per the
+        # self-reference design — STRICT live records past the
+        # kindergarten window. See
+        # docs/self-reference-thought-chains-design.md for the
+        # falsification trust model.
+        "self_reference_source",
     ],
     # Hard cap on pointer ids per claim line — mirrors prompt Rule 9.
     # Lines exceeding this cap classify as SCHEMA_INVALID and the

aborist/sources/providence.py

@@ -0,0 +1,117 @@
+"""ProvidenceSource — recursive grounding on STRICT live providence records.
+
+Promotes the system's own past Q&A records (those in `providence_cache`
+that landed STRICT and are still live) into the document corpus so
+retrieval can surface them as `self_reference_source` citations for
+new thoughts.
+
+See ``docs/self-reference-thought-chains-design.md`` for the full
+architecture. Trust model summary: STRICT live records older than
+the kindergarten window are emitted as documents. HYBRID / UNGROUNDED
+records and falsified records (state ∈ {failed, stale, quarantined})
+stay opaque to retrieval — the substrate trusts STRICT as fact unless
+a verifier falsifies it.
+"""
+
+from __future__ import annotations
+
+import sqlite3
+import time
+from typing import Iterator
+
+from aborist.document import Document
+from aborist.source import Source
+
+
+# Mirrors the mesh-sync kindergarten convention: records younger
+# than this many seconds are still "warm" and should not yet be
+# recyclable substrate. Default 1 hour gives mesh broadcast time
+# to settle and avoids tight echo-chamber loops.
+DEFAULT_KINDERGARTEN_SECONDS = 3600
+
+# URI scheme prefix for self-promoted providence documents. The
+# query-side `_classify_source_role` matches this prefix to assign
+# `self_reference_source`.
+PROVIDENCE_URI_PREFIX = "aborist://providence/"
+
+
+class ProvidenceSource(Source):
+    """Yield STRICT live providence_cache records as Documents.
+
+    Filters applied at iteration time:
+
+    1. ``audit_mode == 'STRICT'`` — fully grounded only. HYBRID and
+       UNGROUNDED records stay out of the substrate.
+    2. ``falsification_state == 'live'`` — falsified rows excluded.
+       The existing `aborist providence --falsify` machinery is the
+       verifier-falsification mechanism: when a record's state flips,
+       it stops being substrate on next ingest.
+    3. ``now - created_at >= kindergarten_seconds`` (default 1h) —
+       fresh thoughts cool before they're recyclable. Kills tight
+       echo-chamber loops.
+    4. **Anti-recursion** — records whose own answer text references a
+       prior self-reference URI are excluded. First-generation only.
+       Conservative; second-gen lands later if lazy-anchor /
+       semantic-frame work proves the error rate is low enough.
+
+    Each yielded Document:
+
+    - URI: ``aborist://providence/<cache_key>`` (content-addressed,
+      stable across runs, distinguishable from external URIs).
+    - title: the question text, truncated to ~120 chars.
+    - content: canonical layout ``Q: <question>\\n\\nA: <answer_text>``.
+      Standard chunker + Merkle apply via the ingest pipeline.
+    - source_type: ``"providence"``.
+    """
+
+    source_type: str = "providence"
+
+    def __init__(
+        self,
+        conn: sqlite3.Connection,
+        *,
+        kindergarten_seconds: int = DEFAULT_KINDERGARTEN_SECONDS,
+        now_seconds: float | None = None,
+    ):
+        self.conn = conn
+        self.kindergarten_seconds = max(0, kindergarten_seconds)
+        self._now_seconds = now_seconds  # injectable for tests
+
+    def _now(self) -> float:
+        return time.time() if self._now_seconds is None else self._now_seconds
+
+    def iter_documents(self) -> Iterator[Document]:
+        cutoff = self._now() - self.kindergarten_seconds
+        # Pull only rows that pass all three hard gates. The age check
+        # uses created_at (when the record first landed), not last_hit_at,
+        # so a recently-hit-but-old record still counts as cooled.
+        cursor = self.conn.execute(
+            """
+            SELECT cache_key, question_text, answer_text
+              FROM providence_cache
+             WHERE audit_mode = 'STRICT'
+               AND falsification_state = 'live'
+               AND created_at <= ?
+             ORDER BY cache_key ASC
+            """,
+            (cutoff,),
+        )
+        for row in cursor:
+            cache_key = row["cache_key"] if isinstance(row, sqlite3.Row) else row[0]
+            question = row["question_text"] if isinstance(row, sqlite3.Row) else row[1]
+            answer = row["answer_text"] if isinstance(row, sqlite3.Row) else row[2]
+            if not question or not answer:
+                continue
+            # Anti-recursion: skip records whose own answer cites a
+            # self-reference URI. First-generation only.
+            if PROVIDENCE_URI_PREFIX in (answer or ""):
+                continue
+            uri = f"{PROVIDENCE_URI_PREFIX}{cache_key}"
+            title = question[:120].strip()
+            content = f"Q: {question}\n\nA: {answer}"
+            yield Document(
+                uri=uri,
+                content=content,
+                source_type=self.source_type,
+                title=title,
+            )

docs/self-reference-thought-chains-design.md

@@ -0,0 +1,96 @@
+# Self-reference thought chains — recursive grounding on the providence ledger
+
+**Date:** 2026-05-01
+**Status:** design + MVP implementation. New module `aborist/sources/providence.py`, new Makefile target `ingest-self-providence`, allowlist update for `claim_lattice_allowed_source_roles`. Unit-tested. Live integration deferred (requires running ingest then bench).
+**Audience:** fox + future blackops shifts.
+**Hard constraint:** STRICT records are trusted as fact unless a verifier falsifies them. Other audit_modes stay opaque to retrieval until promoted.
+
+---
+
+## 1. Problem statement
+
+Aborist's namesake is "tends trees and forests of cross-linked information." Today the system tends Wikipedia trees but never grafts its own past Q&A records into the forest. Every query starts from cold corpus retrieval; prior providence records sit in `providence_cache` unread. The system answers a question, stores the answer in the cache, and never looks at that answer again unless someone re-asks the exact same question (cache_key match).
+
+The "kindergarten thought chains" framing names the gap: the system has a kindergarten of thoughts (early STRICT records) that should mature into citable substrate as they cool, then serve as anchors for new thoughts. Without that loop the substrate is a one-shot answerer, not a recursively-deepening reasoner.
+
+Concretely: ask "what is verify_claim_lattice?" today and Hermes guesses from training. The right primary source — the verify.py source code or the design docs in `docs/` — isn't in the corpus. Even if you `make ingest-self`, only the *code* gets ingested; the system's *answers about its own code* stay invisible to retrieval.
+
+## 2. Design
+
+### 2.1 Architecture
+
+A new `Source` subclass — `ProvidenceSource` — iterates `providence_cache` records and yields each as a `Document`:
+
+- **URI**: `aborist://providence/<cache_key>` — content-addressed, stable, distinguishable from Wikipedia/external URIs at retrieval time.
+- **Title**: the question text (truncated to ~120 chars).
+- **Content**: a canonical layout of `Q: <question>` then `A: <answer_text>` then per-claim line `[E#: cite]` if available. The content gets chunked + Merkle-rooted via the standard ingest pipeline.
+- **source_type**: `"providence"`.
+
+Records are filtered at iteration time by:
+
+1. **`audit_mode == "STRICT"`** — only fully-grounded records become substrate. HYBRID and UNGROUNDED stay opaque to retrieval (they're noisy or speculative).
+2. **`falsification_state == "live"`** — falsified records (state ∈ {failed, stale, quarantined}) are excluded. The existing falsification machinery is the verifier-falsification mechanism fox asked for: when `aborist providence --falsify` flips a record's state, it stops being substrate on next ingest.
+3. **`now - created_at >= kindergarten_seconds`** (default 3600s = 1h). Fresh thoughts cool before they're recyclable. Mirrors the mesh-sync kindergarten window. Without this, the system would self-cite a record from 30 seconds ago and you get echo-chamber loops.
+4. **Anti-recursion**: records whose own answer cited a `self_reference_source` are excluded — first-generation only. Otherwise a wrong-but-STRICT record gets repeatedly recompiled into deeper claims and the chain rots silently. (This is conservative; second-gen self-citation may land later if the lazy-anchor / NLI-sidecar work proves the error rate is low enough.)
+
+### 2.2 Retrieval integration
+
+The source-role classifier in `aborist/qa/query.py:_classify_source_role` recognizes the `aborist://providence/` URI scheme and tags those documents `self_reference_source`. This role gets added to the default `claim_lattice_allowed_source_roles` allowlist so claims can verify against self-reference spans.
+
+`SOURCE_ROLE_BUDGET_WEIGHTS` for `self_reference_source` = 1.0 (same as background; deliberately not boosted — Wikipedia is still the canonical primary).
+
+### 2.3 Recursive Merkle proof
+
+When Q2 cites Q1's answer span, the audit chain becomes "Q2 → Q1 → Wikipedia chunk." The original `chunk_root` remains the leaf; Q1's `run_dag_root` becomes an intermediate node. v9.8's admissibility ledger already supports this layering — that's the recursive-cores insight ("planet toward center compression" in CLAUDE.md). No schema change needed; the providence record's `merkle_proof` blob carries the parent chain.
+
+### 2.4 Falsification trust model
+
+Per fox: "we trust strict statements as fact unless a verifier falsifies it."
+
+- STRICT live records → trusted, ingested as substrate
+- A falsifier (manual via `aborist providence --falsify`, drift-detection in `evict_to_cold` rehydrate, mesh-sync mismatch) flips state to `failed` / `stale` / `quarantined`
+- On next `ingest-self-providence` run, the falsified record's document gets removed from FTS (idempotent re-ingest with `supersedes` edge — the documents table already supports this)
+- Until that re-ingest, the stale record stays in retrieval but the verifier on a Q2 citing it sees `falsification_state != live` for the underlying `source_root` and rejects the citation. **Fail-closed: a falsified record CANNOT serve as STRICT substrate even if it's still in the documents table.**
+
+### 2.5 Operational model
+
+- `make ingest-self-providence` — promote STRICT-live providence records older than the kindergarten window into the document corpus
+- Run on a cron (every hour, mirroring the mesh kindergarten window)
+- Idempotent: same record → same document_root → no-op insert. Replaced records get a `supersedes` edge linking new → old.
+
+### 2.6 What's NOT in this design
+
+- **Aggregation** of multiple Q&A records into a synthesized "summary" record. Possible follow-on but not MVP.
+- **Cross-shard self-reference**: each shard can self-promote within itself; cross-shard cites work via the existing shards-dir UNION. No new code.
+- **Live retrieval from providence_cache** (Option B from the design discussion). The MVP uses snapshot ingestion (Option A) for simplicity. Live virtual sourcing would be an optimization if duplication becomes painful.
+- **Self-reference for HYBRID records**. STRICT only. HYBRID could be added as a soft-anchor source role with lower trust (own role: `self_reference_hybrid_source`, role weight 0.5). Holding for fox's call.
+
+## 3. Risks & mitigations
+
+| risk | mitigation |
+|------|-----------|
+| Lazy-anchor false-STRICT compounds — Q2 inherits Q1's bogus cite. | `lazy_anchor_demoted` records skipped on promotion. Once the NLI sidecar lands, `semantic_demoted` adds another gate. |
+| Echo-chamber: same fact recycled across many records. | Kindergarten window (1h) + first-gen-only anti-recursion check. |
+| Falsified record stays in retrieval until re-ingest. | Verifier checks `falsification_state` of the cited source_root at verify time; non-live cites get rejected. Fail-closed. |
+| Storage bloat: every Q&A becomes a document. | Same chunker + Merkle as everything else; per-record cost is small. Periodic burn (`burn-kindergarten`) trims old or low-quality rows. |
+| Schema/policy drift: a record promoted under v9.8.0 becomes invalid when chunking_version bumps. | Same schema-invariant rule as Wikipedia ingestion; `governance_policy_hash` migration applies. |
+
+## 4. Bench impact estimate
+
+Before any self-reference is in the corpus, no impact. Once `make ingest-self-providence` has been run for a while and a few hundred STRICT records are substrate:
+
+- Questions about aborist itself (mostly UNGROUNDED today) start grounding
+- Questions tangentially related to past STRICT answers gain new anchors
+- Strict-rate slowly climbs as the substrate accumulates trusted self-knowledge
+- New failure modes: badly-anchored citations to providence chunks (the Wall + JP-dinos pattern repeats inside the self-reference layer). The lazy-anchor smell sidecar already covers this.
+
+## 5. Implementation summary
+
+Files added / modified:
+- `aborist/sources/providence.py` — new module, `ProvidenceSource(Source)` with kindergarten / state filters
+- `aborist/qa/query.py` — `_classify_source_role` recognizes `aborist://providence/` URIs; `claim_lattice_allowed_source_roles` adds `self_reference_source`
+- `aborist/qa/runner.py` — same allowlist update for the per-document `ask` path
+- `Makefile` — new `ingest-self-providence` target
+- `tests/test_providence_source.py` — unit tests for the source filter + classifier
+
+CLI wire-up + bench validation deferred to follow-up commits once we've ingested-and-bench-validated on real data.

tests/test_providence_source.py

@@ -0,0 +1,228 @@
+"""Tests for ProvidenceSource — self-reference thought chains.
+
+Covers the four iteration-time gates documented in
+docs/self-reference-thought-chains-design.md:
+
+  1. audit_mode == 'STRICT' (HYBRID/UNGROUNDED excluded)
+  2. falsification_state == 'live' (failed/stale/quarantined excluded)
+  3. now - created_at >= kindergarten_seconds (fresh records cool first)
+  4. anti-recursion: records whose answer cites a self-reference URI
+     are excluded (first-generation only)
+
+Plus the classifier-side: documents with `aborist://providence/`
+URIs classify as `self_reference_source`.
+"""
+
+from __future__ import annotations
+
+import sqlite3
+import time
+
+import pytest
+
+from aborist.qa.query import _classify_source_role
+from aborist.sources.providence import (
+    PROVIDENCE_URI_PREFIX,
+    ProvidenceSource,
+)
+from aborist.store import connect
+
+
+def _seed(conn: sqlite3.Connection, **fields) -> str:
+    """Insert one minimal providence_cache row. Returns the cache_key."""
+    defaults = {
+        "cache_key": fields.get("cache_key", "ck_" + str(int(time.time() * 1e6))),
+        "source_root": "00" * 32,
+        "document_uri": "test://doc",
+        "question_hash": "11" * 32,
+        "question_text": "stub question",
+        "answer_text": "stub answer",
+        "merkle_proof": '{"proofs": []}',
+        "model_profile_hash": "22" * 32,
+        "conversation_hash": "33" * 32,
+        "governance_policy_hash": "44" * 32,
+        "schema_version": "v9.8.0",
+        "canonicalization_version": "norm-v1",
+        "chunking_version": "tok-512-v1",
+        "falsification_state": "live",
+        "chain": "private",
+        "audit_event_hash": "55" * 32,
+        "created_at": time.time() - 7200,  # 2h old by default — past kindergarten
+        "last_hit_at": None,
+        "hit_count": 0,
+        "audit_mode": "STRICT",
+        "n_quotes": 1,
+        "n_verified": 1,
+        "unverified_quotes": None,
+        "verifier_method": "claim_lattice",
+        "run_dag_root": "66" * 32,
+        "run_dag_blob": "{}",
+    }
+    defaults.update(fields)
+    cols = ", ".join(defaults.keys())
+    placeholders = ", ".join("?" * len(defaults))
+    conn.execute(
+        f"INSERT INTO providence_cache ({cols}) VALUES ({placeholders})",
+        tuple(defaults.values()),
+    )
+    conn.commit()
+    return defaults["cache_key"]
+
+
+def test_yields_strict_live_record_past_kindergarten(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        ck = _seed(conn, cache_key="strict-cooled", question_text="who is X?",
+                   answer_text="X is Y.")
+        # Source created with kindergarten=3600. Record is 2h old → eligible.
+        src = ProvidenceSource(conn, kindergarten_seconds=3600)
+        docs = list(src.iter_documents())
+        assert len(docs) == 1
+        d = docs[0]
+        assert d.uri == f"{PROVIDENCE_URI_PREFIX}{ck}"
+        assert d.source_type == "providence"
+        assert d.title == "who is X?"
+        assert "Q: who is X?" in d.content
+        assert "A: X is Y." in d.content
+    finally:
+        conn.close()
+
+
+def test_excludes_hybrid_and_ungrounded(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        _seed(conn, cache_key="strict-1", audit_mode="STRICT")
+        _seed(conn, cache_key="hybrid-1", audit_mode="HYBRID")
+        _seed(conn, cache_key="ungrounded-1", audit_mode="UNGROUNDED")
+        docs = list(ProvidenceSource(conn).iter_documents())
+        uris = {d.uri for d in docs}
+        assert f"{PROVIDENCE_URI_PREFIX}strict-1" in uris
+        assert f"{PROVIDENCE_URI_PREFIX}hybrid-1" not in uris
+        assert f"{PROVIDENCE_URI_PREFIX}ungrounded-1" not in uris
+    finally:
+        conn.close()
+
+
+def test_excludes_falsified_records(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        _seed(conn, cache_key="live-strict", audit_mode="STRICT", falsification_state="live")
+        _seed(conn, cache_key="failed-strict", audit_mode="STRICT", falsification_state="failed")
+        _seed(conn, cache_key="stale-strict", audit_mode="STRICT", falsification_state="stale")
+        _seed(conn, cache_key="quarantined-strict", audit_mode="STRICT",
+              falsification_state="quarantined")
+        docs = list(ProvidenceSource(conn).iter_documents())
+        uris = {d.uri for d in docs}
+        assert f"{PROVIDENCE_URI_PREFIX}live-strict" in uris
+        # Falsified rows excluded — verifier-falsification mechanism
+        # works exactly because state=live is the gate.
+        assert f"{PROVIDENCE_URI_PREFIX}failed-strict" not in uris
+        assert f"{PROVIDENCE_URI_PREFIX}stale-strict" not in uris
+        assert f"{PROVIDENCE_URI_PREFIX}quarantined-strict" not in uris
+    finally:
+        conn.close()
+
+
+def test_kindergarten_window_excludes_fresh_records(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        now = time.time()
+        # 30s old — fresh, inside the 1h window.
+        _seed(conn, cache_key="fresh", created_at=now - 30)
+        # 2h old — past the window.
+        _seed(conn, cache_key="cooled", created_at=now - 7200)
+        src = ProvidenceSource(conn, kindergarten_seconds=3600, now_seconds=now)
+        docs = list(src.iter_documents())
+        uris = {d.uri for d in docs}
+        assert f"{PROVIDENCE_URI_PREFIX}cooled" in uris
+        assert f"{PROVIDENCE_URI_PREFIX}fresh" not in uris
+    finally:
+        conn.close()
+
+
+def test_kindergarten_zero_admits_all_strict_live(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        now = time.time()
+        _seed(conn, cache_key="just-now", created_at=now - 1)
+        _seed(conn, cache_key="cooled", created_at=now - 86400)
+        src = ProvidenceSource(conn, kindergarten_seconds=0, now_seconds=now)
+        docs = list(src.iter_documents())
+        assert len(docs) == 2
+    finally:
+        conn.close()
+
+
+def test_anti_recursion_excludes_self_referencing_records(tmp_path):
+    """A STRICT record whose own answer text contains
+    `aborist://providence/...` is excluded — first-generation only.
+    Prevents echo-chamber chains where a wrong-but-STRICT record
+    keeps getting recompiled into deeper claims."""
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        _seed(conn, cache_key="first-gen",
+              answer_text="X is Y per primary source.")
+        _seed(conn, cache_key="second-gen",
+              answer_text=f"X is Y per {PROVIDENCE_URI_PREFIX}other-key cited record.")
+        docs = list(ProvidenceSource(conn).iter_documents())
+        uris = {d.uri for d in docs}
+        assert f"{PROVIDENCE_URI_PREFIX}first-gen" in uris
+        assert f"{PROVIDENCE_URI_PREFIX}second-gen" not in uris
+    finally:
+        conn.close()
+
+
+def test_skips_records_with_empty_question_or_answer(tmp_path):
+    db = tmp_path / "p.db"
+    conn = connect(db)
+    try:
+        _seed(conn, cache_key="no-q", question_text="", answer_text="A")
+        _seed(conn, cache_key="no-a", question_text="Q", answer_text="")
+        _seed(conn, cache_key="both", question_text="Q", answer_text="A")
+        docs = list(ProvidenceSource(conn).iter_documents())
+        uris = {d.uri for d in docs}
+        assert f"{PROVIDENCE_URI_PREFIX}both" in uris
+        assert f"{PROVIDENCE_URI_PREFIX}no-q" not in uris
+        assert f"{PROVIDENCE_URI_PREFIX}no-a" not in uris
+    finally:
+        conn.close()
+
+
+def test_classify_source_role_recognizes_providence_uri():
+    """`_classify_source_role` short-circuits on the URI scheme
+    regardless of title shape. Trust model is URI-based, not
+    title-heuristic-based."""
+    qstem = {"foo", "bar"}
+    role = _classify_source_role(
+        "Some Title (film)",
+        qstem,
+        document_uri=f"{PROVIDENCE_URI_PREFIX}abc123",
+    )
+    assert role == "self_reference_source"
+
+
+def test_classify_source_role_falls_through_for_non_providence_uri():
+    """External URIs get the existing title-based classification."""
+    qstem = {"jurassic", "park", "film"}
+    role = _classify_source_role(
+        "Jurassic Park (film)",
+        qstem,
+        document_uri="https://en.wikipedia.org/wiki/Jurassic_Park_(film)",
+    )
+    # Title has 3 stems matching the 3-stem query → primary.
+    assert role == "primary_answer_source"
+
+
+def test_classify_source_role_handles_missing_uri():
+    """document_uri is optional; without it the function falls back
+    to the existing title-based classification (backward compat)."""
+    qstem = {"jurassic", "park"}
+    role = _classify_source_role("Jurassic Park (film)", qstem)
+    # Stems match, primary classification.
+    assert role in ("primary_answer_source", "background_source")