arborist/docs
russell@unturf.com 508fec6975
session: Merkle-rooted multi-turn Q&A REPL with Bates ledger
`arborist session` is an interactive multi-turn Q&A REPL where every
turn (or fork) mints one node in a per-session SQLite-backed tree.
Each node carries a stable Bates id (`<sid>-<6-digit>`) and folds into
a Merkle subtree-hash chain; the root node's subtree_hash is the
session_root.

Tree shape lets:

- **Forks** happen implicitly: `/cd <bates>` to a prior node, ask
  again → sibling under that parent. Branch points (≥2 children)
  surfaced by `/branches`.
- **Page-refresh caching** stay cheap: a client tracking
  (bates → subtree_hash, body) only refetches subtrees whose hash
  changed. Sibling subtrees that didn't change are byte-identical
  → cache-equivalent. Same property git pack-protocol and IPFS MFS
  use.
- **Audit-chain verification** be per-session and independent: each
  session db has its own session_audit_events with event_hash =
  sha256(prev_hash || canonical_body). `make session-chain-check`
  walks all sessions; 0 breaks each = intact.

Wire:

- arborist/qa/session.py — Session class, Bates minting, Merkle
  recompute on O(depth) insert, audit chain, helpers (list, render,
  resolve <bates|seq|label>).
- arborist/cli.py — `session` subcommand: REPL + --list / --tree
  / --chain-check / --gc / --json flags. Ancestor-titles → retrieval
  keywords (parsed from cited-pointer lines in answer_text) flow down
  the branch via policy["retrieval_keywords"].
- arborist/qa/providence_query.py — honor policy["retrieval_keywords"]:
  augment FTS5 retrieval query without touching cache_key (mirrors
  legacy --retrieval-keywords discipline, #000001).
- Makefile — `make session [SID=...]`, `make session-list`,
  `make session-tree SID=...`, `make session-chain-check`,
  `make session-gc SESSION_KEEP=N`.
- docs/sessions.md — schema, Merkle conventions (portability for
  non-Python consumers), REPL command reference.
- tests/test_session.py — 15 tests: create, resume, add_node, fork
  via cd, branches, root determinism, audit chain (intact + tampered),
  resolve, list, render, sibling-invariance of subtree_hash.

Storage: ~/.arborist/sessions/<sid>.db (self-contained — no FK into
main store). Answers live in providence_cache keyed by cache_key;
session only carries conversation shape. Cache hits stay live across
sessions. Bounded growth via --gc.

Phase 1 scope: tree + Merkle + Bates + retrieval-keyword flow.
NOT in Phase 1: LLM-side conversation_history (threading prior Q&A
into the LLM prompt + conversation_hash). A bare-pronoun follow-up
("who created him?") gets the right retrieval today but the LLM may
still UNGROUNDED because it sees only the new question as user
message. Folding conversation_history into the prompt + cache_key's
conversation_hash dimension is the natural Phase 2.

197 tests pass.
2026-06-01 16:58:08 -04:00
..
_source docs: 3 concepts diagrams + Python-library cookbook recipes 2026-05-24 14:12:26 -04:00
dav1d-reviews #000071: dav1d review 2026-06-01 — REJECT-AS-WRITTEN, GO-with-rewrite 2026-06-01 07:13:23 -04:00
diagrams docs: 3 concepts diagrams + Python-library cookbook recipes 2026-05-24 14:12:26 -04:00
tickets #000070: rewrite ticket as directive forward-spec (Joseph6 as worked example) 2026-06-01 07:24:17 -04:00
bench-maxing.md aborist/arborist 2026-05-07 09:31:49 -04:00
benchmark-matrix.md docs(#000057): capture known-good serving invocations for the salt buildout 2026-05-20 15:20:34 -04:00
benchmarks.md docs+code: ground §12 judge pipeline in the actual judge_code.py 2026-05-22 19:17:28 -04:00
calculator-test-patterns.md ticket #000036 Tier-2: dav1d Option B (conservative B1 envelope) applied in v1 2026-05-11 07:06:50 -04:00
cold-object-store.md cold pack: --jit-blobs mode for online JIT consumer flow 2026-05-30 07:19:01 -04:00
corpus-history.md #000065 closed: production reshard landed; record in corpus-history 2026-05-26 15:51:20 -04:00
crawler.md crawl central-db + query auto-include + read-seam provenance 2026-05-29 13:45:47 -04:00
cti-architecture.md aborist/arborist 2026-05-07 09:31:49 -04:00
embedding.md feat: arborist.embed — supported library-embedding surface 2026-05-22 13:03:15 -04:00
energy-cogs-benchmark.md docs(#000057): correct cost claim — <$0.10/1k-q is hermes-8B only, not qwen 2026-05-21 13:50:01 -04:00
lexical-first-rationale.md docs: lexical-first-rationale.md — why the cheap retrieval path is the default 2026-05-12 09:23:50 -04:00
mesh.md aborist/arborist 2026-05-07 09:31:49 -04:00
onnx-vendor-capture-immunity.md feat(#000049 §7 #28): tinygrad NLI backend + deterministic engine-agreement A/B; ONNX-immunity rationale 2026-05-19 12:34:04 -04:00
pager.style docs: arborist-one-pager + arborist-two-pager — Dav1d/fox-signoff summaries with letterhead, license, and 2 strategic appendix diagrams 2026-05-14 09:48:50 -04:00
pi-star-composition.md pi_star: land ticket #000015 (π* domain library + composition algebra) 2026-05-07 16:51:33 -04:00
qa-modes-bench.md docs(#000057): Addendum 8 — control sweep retrieval × model × framing × reasoning 2026-05-20 06:52:42 -04:00
relevance-and-veto-synthesis-for-dav1d.md docs: relevance-and-veto-synthesis-for-dav1d.md — single decision brief synthesizing #000049 + #000052 §3.1 + §3.2 for forward review 2026-05-13 15:34:21 -04:00
sessions.md session: Merkle-rooted multi-turn Q&A REPL with Bates ledger 2026-06-01 16:58:08 -04:00
seven-point-program.md tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix) 2026-05-10 16:15:52 -04:00
soft-hash-channel-analysis.md docs/#000018 §9.2: mark resolved — φ_PRG = HMAC-SHA-512 (#000035 closed) 2026-05-11 17:20:48 -04:00
soft-hash-channel-t3-bound.md ticket #000036: add KAT-regen tooling + close 2026-05-11 08:02:25 -04:00
spec-methodology.md docs: land ticket #000019 (spec methodology for π*, V, policy fields) 2026-05-07 16:53:28 -04:00
stock-v1-config.md fix(#000057): GPU COGS = generation only — retrieval/verify don't touch the card 2026-05-21 11:07:37 -04:00
TICKETS.md #000070: rewrite ticket as directive forward-spec (Joseph6 as worked example) 2026-06-01 07:24:17 -04:00
tool-action-dag-design.md docs: add tool-action-dag-design.md research path (pre-ticket) 2026-05-07 19:47:50 -04:00
user-payload-layout.md docs/user-payload-layout: reflect #000068 Phase 1+2+3 shipped state 2026-05-27 10:55:38 -04:00
v7w-frontier-catalog.md #000013 closed: v7-W spatial-temporal substrate paper + namespace 2026-05-09 15:00:05 -04:00
v8-fork-score.md CLI: arborist v8 score → arborist substrate score 2026-05-10 09:12:34 -04:00
warrant-substrate-cookbook.md test_doc_counts: AUTOCOUNT db-where supports *: glob for corpus-wide claims 2026-05-31 11:37:39 -04:00
zk-frontier-bench.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00
zk-wire-protocol.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00