arborist/docs
russell@unturf.com 178cc422e2
ticket #000037 §3.1: unconscious sweep also drains ingest backlog
Original ticket spec (drafted earlier today by a parallel-shift
session, untracked in working tree until this commit) covers the
unconscious sweep walking providence_cache rows during ingest
lulls. Fox flagged a second backlog the controller has to drain:
ingested DOCUMENTS that bypassed meta-cognition at ingest time.

Today's ingest pipeline is intentionally fast: fetch →
canonicalize → chunk → Merkle commit → audit row → done. No
witness fan-out, no canonical-projection cross-check, no warrant
resolution. That speed is the right trade-off for getting bytes
into the queryable Merkle + FTS5 tree, but it leaves a
meta-cognition gap: the substrate has content it never *probed*.

New §3.1 spells out two sweep targets, both processed during
ingest lulls:

Target A — providence_cache rows older than τ_qa
  Re-runs multi-modality witness fan-out (#000028) on cached
  Q&A answers. Catches drift from Hermes updates / kernel
  bumps / mesh-delivered new evidence. Default τ_qa: 7 days.

Target B — documents rows ingested without meta-cognition
  Walks `documents` where last_swept_at is NULL or older than
  τ_doc, runs cross-checks per document content:
   - Canonical-projection probe (math/logic statements in body
     vs. matching π* canonical answer)
   - Warrant resolver re-run (newly-ingested textbook can
     resolve previously-unresolved claim-pack records)
   - Source-URL freshness probe (HEAD-only via existing
     recrawl-check; mark stale/gone)
   - Witness fan-out on document content (kernel+LLM+cache as
     if doc were a Q&A answer citing itself)

State tracking: new column documents.last_swept_at, schema
migration deferred to §12 phase trigger so it doesn't bump the
cache_key invariant prematurely.

Why this matters: without target B, the substrate's "I have
this content in my Merkle tree" claim is honest at the byte
level but silent on whether the content was ever tested against
kernel ground truth. Sleep is when you do the tests you
deferred to keep the day moving.
2026-05-09 18:34:09 -04:00
..
_source docs: clear all 39 cold-build Sphinx warnings (truly green) 2026-05-09 18:00:15 -04:00
diagrams docs/diagrams: rename aborist-modules → arborist-modules (post-rename cleanup) 2026-05-08 07:50:28 -04:00
tickets ticket #000037 §3.1: unconscious sweep also drains ingest backlog 2026-05-09 18:34:09 -04:00
bench-maxing.md aborist/arborist 2026-05-07 09:31:49 -04:00
benchmarks.md aborist/arborist 2026-05-07 09:31:49 -04:00
cti-architecture.md aborist/arborist 2026-05-07 09:31:49 -04:00
dav1dprometheus-update-2026-05-09.md docs: propagate kernel/sub-battery counts to canonical surfaces 2026-05-09 17:27:02 -04:00
mesh.md aborist/arborist 2026-05-07 09:31:49 -04:00
pi-star-composition.md pi_star: land ticket #000015 (π* domain library + composition algebra) 2026-05-07 16:51:33 -04:00
qa-modes-bench.md aborist/arborist 2026-05-07 09:31:49 -04:00
seven-point-program.md aborist/arborist 2026-05-07 09:31:49 -04:00
soft-hash-channel-analysis.md fan-out: witness automation · #000018 analysis · #000033 pillar VII (Claude) 2026-05-09 14:36:39 -04:00
spec-methodology.md docs: land ticket #000019 (spec methodology for π*, V, policy fields) 2026-05-07 16:53:28 -04:00
TICKETS.md docs/tickets: open #000041 — citation-aliases table 2026-05-09 18:27:10 -04:00
tool-action-dag-design.md docs: add tool-action-dag-design.md research path (pre-ticket) 2026-05-07 19:47:50 -04:00
v7w-frontier-catalog.md #000013 closed: v7-W spatial-temporal substrate paper + namespace 2026-05-09 15:00:05 -04:00
v8-fork-score.md v8: ticket #000012 Phase 1a — ForkScore consumes the new bench substrate 2026-05-08 07:18:22 -04:00
zk-frontier-bench.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00
zk-wire-protocol.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00