session: single-shard forest + FTS5 search + cross-session forks

Refactor from per-session sqlite files to one shared shard at
~/.arborist/sessions.db. Three things that didn't work before now do:

1. Queries are first-class members of the tree.
   nodes_fts (FTS5 over question + answer_text + cited_titles) lets
   /find <query> walk every prior turn across every session. Cached
   answers and threads become findable, surface in the REPL as
   `[<bates>] <audit> <question>` lines.

2. Forking from history works the same as forking from a sibling.
   parent_bates can cross sids. After /find returns a hit from
   last week's session, /cd <bates> + ask = your next question
   lands as a child under that historical turn. The cross-session
   parent's subtree_hash ripples up its session_root.

3. One global audit chain instead of per-file.
   audit_events.event_hash = sha256(prev || canonical body), one
   chain over every state change in the shard. `make
   session-chain-check` is now a single pass; tampering anywhere
   in the operator's history breaks the chain.

Wire:
- arborist/qa/session.py — drop file-per-session SessionStore class;
  Session becomes a viewport on SessionStore. cited_titles_json +
  n_cited_sources materialized at insert time so FTS5 doesn't need
  a join into providence_cache.
- arborist/cli.py — `session` subcommand swaps --gc for --find;
  REPL adds /find. Ancestor-keyword extraction now reads
  nodes.cited_titles_json directly (no qa.db roundtrip).
- Makefile — `make session-find Q="..." [LIMIT=N JSON=1]`; `make
  session-gc` retired (no per-session files to GC).
- docs/sessions.md — rewritten for the single-shard shape.
- tests/test_session.py — 17 tests: create, add, fork (incl.
  cross-session), find (FTS5 + by_cache_key), path_to_root crossing
  sessions, audit chain (intact + tampered), cited-title extraction,
  subtree_hash ripple across sessions.

Migration: pre-existing per-session dbs at ~/.arborist/sessions/*.db
become orphaned. None lost data — test sessions only. Operator can
rm -rf ~/.arborist/sessions/ (or rename to sessions-old/) at leisure.

126 session+providence+verify+inspect tests pass.
This commit is contained in:
russell@unturf.com 2026-06-01 17:32:44 -04:00
parent 5f07d9ece2
commit 45a348b4f0
No known key found for this signature in database
5 changed files with 868 additions and 781 deletions

View file

@ -224,7 +224,7 @@ session: bootstrap ## interactive multi-turn Q&A REPL [SID=... LLM=qwen|hermes A
@echo "# llm: $(SESSION_LLM_ENDPOINT) / $(SESSION_LLM_MODEL)" >&2
$(ARBORIST) --shards-dir $(SHARDS_DIR) session $(if $(SID),--sid $(SID),) --top-k $(QUERY_TOP_K) --endpoint $(SESSION_LLM_ENDPOINT) --model $(SESSION_LLM_MODEL) $(if $(ANSWER_MODE),--answer-mode $(ANSWER_MODE),) $(if $(BURN),--burn,)
session-list: bootstrap ## list all session dbs in ~/.arborist/sessions/
session-list: bootstrap ## list all sessions in the single shard
$(ARBORIST) session --list $(if $(JSON),--json,)
session-tree: bootstrap ## print one session's tree [SID=...]
@ -233,12 +233,14 @@ session-tree: bootstrap ## print one session's tree [SID=...]
fi
$(ARBORIST) session --sid $(SID) --tree $(if $(JSON),--json,)
session-chain-check: bootstrap ## verify session audit-chain integrity [SID=... (default: all)]
$(ARBORIST) session --chain-check $(if $(SID),--sid $(SID),)
session-chain-check: bootstrap ## verify the single shard's global audit chain
$(ARBORIST) session --chain-check
SESSION_KEEP ?= 100
session-gc: bootstrap ## delete old session dbs, keep $(SESSION_KEEP) most recent [SESSION_KEEP=N]
$(ARBORIST) session --gc $(SESSION_KEEP)
session-find: bootstrap ## FTS5 search over all session turns [Q="..." LIMIT=N JSON=1]
@if [ -z "$$Q" ] && [ -z "$(Q)" ]; then \
echo "usage: make session-find Q=\"query text\" [LIMIT=10] [JSON=1]"; exit 2; \
fi
$(ARBORIST) session --find "$(Q)" $(if $(LIMIT),--limit $(LIMIT),) $(if $(JSON),--json,)
query-dry: bootstrap ## like 'make query' but skip the LLM call (dry-run) [JSON=1 BURN=1 ANSWER_MODE=... BROAD=1 REJECT_BROAD=1 ALLOW_BROAD=1 XLANG=1 XLANG_MT=1]
@if [ -z "$$Q" ] && [ -z "$(Q)" ]; then \