The Substrate group is Dav1d's formal Merkle-AGI research that drove the
design — valuable, but it sat third in the nav and read like required
reading. IA-only fix (no content touched):
- re-caption 'Substrate' -> 'Substrate theory (advanced)' + a preamble
marking it optional (nothing in Getting started / API depends on it);
- reorder it BELOW the practical sections (now: Summary -> Getting started
-> API -> Substrate theory -> Project);
- pull 'bench' out of Substrate into Getting started (it's the practical
benchmark surface, not theory).
Practitioner path comes first; theory stays intact + credited, just last.
Graphviz DOT diagram of the arborist pipeline as the answer to the Google
L5 system-design prompt 'design a RAG pipeline for 10M docs with zero
hallucination'. Maps our components onto the canonical 10-box RAG design
and shows the three extensions that buy zero-hallucination + near-zero
cost: deterministic verifier (not a model confidence score) -> honest
UNGROUNDED; Merkle-bound cache that skips the GPU; measured energy COGS
(~$0.07-0.16/1k answers, non-reasoning). Includes the mapping table +
the 3.47M->10M scaling math (+77GB, sourcing/storage not redesign).
Enables sphinx.ext.graphviz (SVG output) + graphviz apt package on the
RTD build; adds the page to the Summary-pages toctree. Builds clean
(page renders, DOT validates via dot -Tsvg).
1-pager (docs/_source/arborist-one-pager.rst, 1 page) for AI-literate readers: the trichotomy, the 8-dim cache key, CTI synthetic-elision-impossible, soft-channel separation, real-traffic bench numbers (mis-cite 100% @ 0% FP, warrant 92/92, quote 0.54 STRICT-rate).
2-pager (docs/_source/arborist-two-pager.rst, 3 pages = 2 body + 1 appendix) for technical reviewers: letterhead, Permacomputer Preamble license box, six numbered sections, plus appendix figures (pager-arch-stack 3-layer architecture, pager-verifier-flow question→pointer→verifier→trichotomy).
Both pages live under docs/_source/ so the same RST renders into the Sphinx readthedocs site (toctree caption "Summary pages" added to docs/_source/index.rst) AND into standalone PDFs via rst2pdf (docs/pager.style, lazy install into .venv).
Makefile targets: docs-one-pager, docs-two-pager, docs-pagers, docs-pagers-clean. Diagrams render through the existing DOT pipeline.
Brought the Merkle-AGI v7 formal substrate spec into the repo as
docs/_source/merkle-agi-dag-v7.rst (previously referenced only as the
un-version-controlled ~/Downloads/merkle-agi-dag_v7.txt). Section
structure converted to reStructuredText; inline math kept in the
source's literal notation; added to the docs/_source/index.rst
"Substrate" toctree (also added the pre-existing merkle-agi-v8-consensus
entry that was missing from it).
Folded ticket #000035's § 9.10 + § 9.10.1 (anchor PRG map φ_PRG;
dav1d-reviewed-final, little-endian, HMAC-SHA-512 / 32-byte seed) in at
their numbered positions, after § 9.9, with a .. note:: citing the
reference implementation (arborist/substrate/anchor_prg.py). #000035 ->
closed (Phase 1 + Phase 2 both landed); #000018 §9.2 ("which PRG?")
resolves to HMAC-SHA-512 with a 32-byte committed seed. Full upstream
v7 spec revision stays exogenous; this lands the amendment into the
tracked in-repo copy where future amendments also go.
(docs/TICKETS.md also carries the in-flight #000048 index-row update
from a concurrent session.)
A `make docs-api-clean && make docs-api` cold rebuild now succeeds
with zero WARNING/ERROR lines (was 39).
Docstring fixes (RST hygiene — no semantic change):
- arborist/qa/{keys,runner,query,verify,quantifier,metacognition,dag,
evidence}.py — add blank lines around indented blocks, convert
ad-hoc indented sections to literal blocks (`::`), avoid line-broken
inline literals (e.g. UNKNOWN_EVIDENCE_ID), and replace nested
bracket/quote literals with cleaner wording.
- arborist/concepts/__init__.py — wrap function-signature listing in
a literal block so bare `*` (kwarg marker) doesn't trip docutils.
- arborist/store.py — blank line before bullet lists in module +
connect docstrings.
- arborist/evict.py — replace ad-hoc `{ ... }` enum block with prose.
Surface fixes:
- docs/_source/_ext/makefile_targets.py — escape `*` in auto-
generated Makefile target descriptions (covers `*-parallel`,
`*.dot`, `*.db`, `π*`, etc.) so the generator emits clean RST.
- docs/_source/index.rst, concepts.rst — extend title underlines
to match title length.
- docs/_source/concepts.rst, v8-fork-score.rst — widen first column
of grid tables so cells no longer overflow into the column margin.
- docs/_source/merkle-agi-v7w-spatial-temporal.rst — switch
pseudocode JSON block from `code-block:: json` to `text` (the
`<int32 x 3>` placeholders aren't valid JSON tokens).
Verification:
- make docs-api-clean && make docs-api → build succeeded, 0 warnings
- make test → 1588 passed, 28 skipped
- make chain-check-shards → 0 breaks across all 7 shards
- import-time SyntaxWarning escalation on edited modules → clean
Two small Sphinx hygiene fixes surfaced while fox was building the
docs locally:
1. docs/_source/merkle-agi-v7w-spatial-temporal.rst was authored
earlier this session (#000013) but never added to a toctree
so Sphinx flagged it as orphan. Slotted under "Substrate"
alongside pi-star / bench / v8-fork-score where it belongs.
2. The same file had |translation_max| as raw text in an
ε-bound expression; RST parsed the pipes as a substitution
reference and errored. Wrapped the expression in double
backticks so it renders as literal math.
Build now drops from 40 warnings (counting docstring noise) to
5; the 5 remaining are pre-existing module-docstring formatting
in arborist/qa/*.py that pre-date this work.
Makefile cuts (64 → 58 documented targets):
- ingest-cur-parallel, ingest-old-parallel: parallel-shared mode
superseded by attached (no WAL contention)
- distill-shards: sequential never preferred over parallel variant
- bench-qa-quick: bench-qa-smoke covers same use case (~30s vs ~10s)
- ingest-grok, ingest-grok-media: single-DB grok rare; -attached is
canonical path
All cuts land in code that the underlying CLI still exposes — operators
who need the dropped variant call '.venv/bin/aborist ingest --shard ...'
directly. No behavior loss, just shortcut removal.
Docs improvements:
- New Concepts page (docs/_source/concepts.rst): orientation on what
aborist is, three layers (surface/core/providence), Merkle commitment,
8-dim cache key, audit chain, trichotomy + four-rung ladder, layered
verifier, falsification state, sidecars. Embeds module-graph and
verifier-ladder SVG diagrams.
- New Cookbook page (docs/_source/cookbook.rst): 8 recipes — recrawl,
falsify, ingest-self-providence, mixed-corpus query, LLM endpoint
override, integrity after bulk ops, bench, retrieval tuning.
- Quickstart embeds query-pipeline SVG diagram.
- docs/_source/diagrams symlinks to docs/diagrams so Sphinx can include
the SVGs (was orphaned, only README referenced them).
Better Makefile RTD page (docs/_source/_ext/makefile_targets.py):
- Group by workflow phase (Setup → Fetch → Ingest → Distill → Query →
Verify → Operations → Tests → Docs → Clean) instead of alphabetical
prefix. Tells a new operator the order they'd actually run things.
- Phase descriptions added; targets prefixed with 'make ' for copy-paste.
- Uncategorized leftover surfaces missing entries in PHASES list.
New docs/_source/quickstart.rst — install, two end-to-end paths
(Wikipedia 2003 + crawler), after-the-answer commands, query pipeline
overview, and links to deeper reference pages. Mirrors the README's
quickstart but adapts cross-references to the Sphinx structure.
Added to index.rst as a 'Getting started' toctree section above the
API modules — RTD users land on it first.
- docs/_source/license.rst — new RTD page that literalincludes the
repo's LICENSE file (single source of truth, no duplication)
- conf.py rst_epilog — appended to every RST source so every doc page
carries the Permacomputer Preamble + AGPL-3.0-only notice + link to
the full license
- index.rst — adds 'Project / License' section to the toctree
The LICENSE file already had the full Permacomputer Preamble + complete
GNU Affero GPL v3 text (matches the whitepaper version).
Adds a Sphinx extension at docs/_source/_ext/makefile_targets.py that
parses the project Makefile's '## description' annotations and writes
docs/_source/api/makefile.rst at build time. Same convention 'make help'
uses, so the reference stays in sync with the source.
Generated page is grouped by target prefix (fetch-, ingest-, distill-,
docs-, etc.) and rendered as a list-table. Shows on RTD alongside the
autodoc API modules.
Generated file is gitignored — RTD regenerates on every build.
Implements Read the Docs infrastructure to generate API documentation
directly from code docstrings. Replaces static modules.md (1200+ lines).
New structure:
- docs/_source/conf.py — Sphinx configuration (furo theme)
- docs/_source/index.rst — Main TOC
- docs/_source/api/*.rst — Module groups (substrate, storage, retrieval,
qa, distill, mesh, cli)
- docs/_source/Makefile — Local build targets
- docs/_source/README.md — Documentation on building and extending
Makefile integration:
- make docs-api — generate HTML (output: docs/_source/_build/html/)
- make docs-api-clean — remove build artifacts
Build output (40 HTML files):
- API module reference with docstrings
- Source code links (:viewcode: extension)
- Full-text search index
- Module index (genindex, py-modindex)
Sphinx installed in venv as dev dependency. HTML is browseable at
docs/_source/_build/html/index.html (open in browser after build).
This justifies the deletion of modules.md: code docstrings + Sphinx
autodoc = automatically-generated, always-current API reference.
