session: single-shard forest + FTS5 search + cross-session forks
Refactor from per-session sqlite files to one shared shard at ~/.arborist/sessions.db. Three things that didn't work before now do: 1. Queries are first-class members of the tree. nodes_fts (FTS5 over question + answer_text + cited_titles) lets /find <query> walk every prior turn across every session. Cached answers and threads become findable, surface in the REPL as `[<bates>] <audit> <question>` lines. 2. Forking from history works the same as forking from a sibling. parent_bates can cross sids. After /find returns a hit from last week's session, /cd <bates> + ask = your next question lands as a child under that historical turn. The cross-session parent's subtree_hash ripples up its session_root. 3. One global audit chain instead of per-file. audit_events.event_hash = sha256(prev || canonical body), one chain over every state change in the shard. `make session-chain-check` is now a single pass; tampering anywhere in the operator's history breaks the chain. Wire: - arborist/qa/session.py — drop file-per-session SessionStore class; Session becomes a viewport on SessionStore. cited_titles_json + n_cited_sources materialized at insert time so FTS5 doesn't need a join into providence_cache. - arborist/cli.py — `session` subcommand swaps --gc for --find; REPL adds /find. Ancestor-keyword extraction now reads nodes.cited_titles_json directly (no qa.db roundtrip). - Makefile — `make session-find Q="..." [LIMIT=N JSON=1]`; `make session-gc` retired (no per-session files to GC). - docs/sessions.md — rewritten for the single-shard shape. - tests/test_session.py — 17 tests: create, add, fork (incl. cross-session), find (FTS5 + by_cache_key), path_to_root crossing sessions, audit chain (intact + tampered), cited-title extraction, subtree_hash ripple across sessions. Migration: pre-existing per-session dbs at ~/.arborist/sessions/*.db become orphaned. None lost data — test sessions only. Operator can rm -rf ~/.arborist/sessions/ (or rename to sessions-old/) at leisure. 126 session+providence+verify+inspect tests pass.
This commit is contained in:
parent
5f07d9ece2
commit
45a348b4f0
5 changed files with 868 additions and 781 deletions
14
Makefile
14
Makefile
|
|
@ -224,7 +224,7 @@ session: bootstrap ## interactive multi-turn Q&A REPL [SID=... LLM=qwen|hermes A
|
|||
@echo "# llm: $(SESSION_LLM_ENDPOINT) / $(SESSION_LLM_MODEL)" >&2
|
||||
$(ARBORIST) --shards-dir $(SHARDS_DIR) session $(if $(SID),--sid $(SID),) --top-k $(QUERY_TOP_K) --endpoint $(SESSION_LLM_ENDPOINT) --model $(SESSION_LLM_MODEL) $(if $(ANSWER_MODE),--answer-mode $(ANSWER_MODE),) $(if $(BURN),--burn,)
|
||||
|
||||
session-list: bootstrap ## list all session dbs in ~/.arborist/sessions/
|
||||
session-list: bootstrap ## list all sessions in the single shard
|
||||
$(ARBORIST) session --list $(if $(JSON),--json,)
|
||||
|
||||
session-tree: bootstrap ## print one session's tree [SID=...]
|
||||
|
|
@ -233,12 +233,14 @@ session-tree: bootstrap ## print one session's tree [SID=...]
|
|||
fi
|
||||
$(ARBORIST) session --sid $(SID) --tree $(if $(JSON),--json,)
|
||||
|
||||
session-chain-check: bootstrap ## verify session audit-chain integrity [SID=... (default: all)]
|
||||
$(ARBORIST) session --chain-check $(if $(SID),--sid $(SID),)
|
||||
session-chain-check: bootstrap ## verify the single shard's global audit chain
|
||||
$(ARBORIST) session --chain-check
|
||||
|
||||
SESSION_KEEP ?= 100
|
||||
session-gc: bootstrap ## delete old session dbs, keep $(SESSION_KEEP) most recent [SESSION_KEEP=N]
|
||||
$(ARBORIST) session --gc $(SESSION_KEEP)
|
||||
session-find: bootstrap ## FTS5 search over all session turns [Q="..." LIMIT=N JSON=1]
|
||||
@if [ -z "$$Q" ] && [ -z "$(Q)" ]; then \
|
||||
echo "usage: make session-find Q=\"query text\" [LIMIT=10] [JSON=1]"; exit 2; \
|
||||
fi
|
||||
$(ARBORIST) session --find "$(Q)" $(if $(LIMIT),--limit $(LIMIT),) $(if $(JSON),--json,)
|
||||
|
||||
query-dry: bootstrap ## like 'make query' but skip the LLM call (dry-run) [JSON=1 BURN=1 ANSWER_MODE=... BROAD=1 REJECT_BROAD=1 ALLOW_BROAD=1 XLANG=1 XLANG_MT=1]
|
||||
@if [ -z "$$Q" ] && [ -z "$(Q)" ]; then \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue