# `aborist.store`

The v9.8 SQLite schema, the audit chain, and the cross-shard
read-only view. Every table that holds runtime state lives here.

## Schema overview (per shard)

```
documents              – one row per source document, keyed on document_root
chunks                 – per-document chunk content + tier (hot/cold)
chunks_fts             – FTS5 contentless index, rowid = chunks.chunk_id
merkle_nodes           – internal-node hashes for proof reconstruction
edges                  – src_root → dst_root link graph (wikilink, href, …)
derivations            – core_root ← src_root with proof_blob (Merkle)
providence_cache       – Q&A records keyed on the v9.8 8-dim cache_key
audit_events           – linear chain; event_hash = sha256(prev || canonical(body))
falsifications         – record_id → state transition + reason + actor
snapshots              – named corpus roots (one hash names a forest)
document_http_meta     – ETag + Last-Modified for crawler conditional fetches
concept_relations      – per-shard synonym/rivalry/category/antonym overlay
mesh_*                 – federation tables (off by default)
```

## v9.8 invariants (do not break)

- **Aborist is a v9.8 store.** Every providence record carries the
  full **8-dim cache_key**: `source_root | question_hash |
  model_profile_hash | conversation_hash | governance_policy_hash |
  schema_version | canonicalization_version | chunking_version`.
  Bumping any one invalidates prior records on lookup.
- **`falsification_state ∈ {live, failed, stale, quarantined}`.**
  Cache lookups must filter on `state='live'`. Drift detection
  flips to `stale`.
- **Audit chain.** Every state-changing op writes one row in
  `audit_events` with `event_hash = sha256(prev_event_hash ||
  canonical(body))`. Chain integrity is verified in
  `make analyze-shards`. **Never insert into `audit_events`
  directly — use `aborist.store.append_audit`.**
- **Cores never evict.** `evict_to_cold` only touches `kind='surface'`.
- **Idempotent re-ingest.** Same content → same `document_root` →
  no-op insert. Same URI + different content → new doc + `supersedes`
  edge linking new → old (lossless history).

## API surface

```python
from aborist.store import (
    connect,           # writable connection to a single shard
    connect_query,     # read-only UNION view across all shards
    discover_shards,   # list *.db files in a shards_dir
    transaction,       # BEGIN IMMEDIATE / COMMIT / ROLLBACK context manager
    get_meta, set_meta,
    append_audit,      # the ONLY way to write audit_events
)
```

Cross-shard reads use `connect_query(shards_dir=...)` which ATTACHes
every `*.db` and creates UNION views over the shardable tables
(`documents`, `chunks`, `merkle_nodes`, `edges`, `derivations`,
`providence_cache`, `audit_events`, `falsifications`,
`concept_relations`).

## Performance pragmas

`connect()` applies these per-connection:
- `journal_mode=WAL` (set in SCHEMA_SQL once at first ingest)
- `synchronous=NORMAL` (skip per-commit fsync; safe under WAL)
- `cache_size=-65536` (64 MB page cache)
- `temp_store=MEMORY` (no /tmp churn for temp tables)

Don't downgrade to `synchronous=FULL` without a measured reason —
costs ~5× throughput.

## Diagrams

![module graph](../diagrams/aborist-modules.svg)
![ingest pipeline](../diagrams/ingest-pipeline.svg)

## Source

[`aborist/store.py`](../../aborist/store.py)