# Ticket #000017 — Memory-root: lifelong learning audit chain

**Status:** closed · landed 2026-05-07
**Opened:** 2026-05-07
**Closed:** 2026-05-07
**Scope:** Spec + initial wire-up of a `memory_root` commitment that
binds an arborist-hosted agent's evolving cross-query memory into the
audit chain. Distinct from per-query providence cache (which is keyed
on cache_key dimensions); memory_root captures lifelong, cross-query
state — accepted/rejected belief updates, recurring failure motifs,
domain-specific corrections.
**Audience:** fox + future blackops shifts.
**Hard constraint:** memory_root is hard-hash committed; no soft
state in its preimage. Memory falsification cascades MUST NOT
invalidate the entire prior providence cache; cascade bounded by
explicit policy.

---

## 1. Problem statement

DNA analogy thread (Asia/Kuala_Lumpur, 2026-05-04) lists the
heritable units a Merkle-AGI organism passes to its child:

> what worked, what failed, why it failed, which benchmark proved
> the fix, which policy allowed the fix, which verifier accepted
> the fix, which SelfModel update resulted.

The genome schema includes `memory_root: ...`. arborist has no such
object. Today's memory model:

| Memory layer | Today |
|---|---|
| Providence cache | Per-cache_key answers; no cross-query continuity. |
| Audit events | Per-shard chain; not summarized. |
| Concepts table | Per-shard rivalry/synonym graph; static once built. |
| SelfModel (ticket #000014) | Identity, not behavior history. |
| Long-term failure recall | None. Same failure shape repeats across queries with no surfacing. |

Without `memory_root`, an agent that hallucinates about spin glass
on Monday has no audit-bound memory of that on Tuesday. The
falsification record exists in `audit_events` but is not summarized,
not surfaced to the verifier on the next related query, and not
inheritable when a child shard forks.

### 1.1 Concrete failure scenarios

**Scenario A — Repeating a known motif.** Query 1 hallucinates
"spin glass cited from Quantum chromodynamics article" → audited as
TITLE_MISMATCH → falsification recorded. Query 2 (related topic)
runs through the same retrieval path, hits the same chunk, makes a
similar mistake. Without memory_root, the verifier has no way to
say "this retrieval path produced a falsified claim before; up-
weight skepticism."

**Scenario B — Forked agent loses context.** A new shard splits off
with `parent_shard_root` pointing to the existing chain. The new
shard has no record of recurring failure motifs unless the audit
chain is replayed. memory_root would let the new shard inherit a
compact summary without replaying every event.

**Scenario C — Operator query "what does this agent struggle with?"**
Today: no answer. With memory_root + a small CLI surface: a
queryable summary backed by audit history.

---

## 2. Design choices

### 2.1 What goes in memory_root

**A. Just a digest of the audit-event chain.** Trivial; redundant
with audit chain.

**B. Curated summaries: failure-motif counts, recurring-error topic
clusters, accepted-correction list.** Useful, but how is the
curation deterministic?

**C. Hierarchical: chain-digest at root, multiple branch-digests for
domain-specific summaries (e.g., "physics-domain-failures",
"language-domain-corrections").** Compositional. Selectively
inheritable.

**Recommendation:** C. Each summary is itself a deterministic
projection over a slice of audit_events; the slice rule is
committed in the memory-root manifest so verifiers can recompute.

### 2.2 Update cadence

**A. Every audit_event triggers re-summary.** High churn; expensive.

**B. Periodic snapshot (every N events, or every N seconds).**
Bounded cost; staleness window.

**C. On-demand.** Operator triggers re-snapshot. Lowest churn,
risk of stale state at critical moments.

**Recommendation:** B at default cadence (every 1000 audit_events,
configurable). Operators can also force-trigger via CLI.

### 2.3 Falsification cascade scope

If a memory-summary entry is invalidated (e.g., a previously-
accepted correction turns out to be wrong), what gets re-evaluated?

**A. Nothing. Memory is purely advisory; falsification recorded but
no cache impact.** Safest. Loses the "memory shapes future
decisions" benefit.

**B. Only future queries. Existing providence cache stays valid.**
Practical. Cache continues to reflect what was known at the time.

**C. Cascade re-evaluation. Falsification triggers re-evaluation
of every providence record that cites the falsified memory entry.**
Strongest semantic guarantee. Expensive at scale.

**Recommendation:** B. Memory falsification flips state on
upstream-citing records but doesn't auto-rerun them. Operator can
trigger cascade re-evaluation explicitly when they want correctness
> efficiency.

### 2.4 Cache_key folding

Same options as ticket #000014:

**A. Fold into governance_policy_hash.** Memory changes invalidate
prior cache.

**B. Sibling field on providence record, not in cache_key.** Cache
hits across memory shifts; memory changes are advisory.

**Recommendation:** B. Memory is advisory by default. Operator can
opt into A via `policy["memory_binding"]=True` in deployments
where memory changes should invalidate prior answers.

---

## 3. Recommendation

Hierarchical memory-root with periodic snapshot at every-1000-events
cadence, advisory-by-default cache impact (sibling field, not in
cache_key by default), explicit cascade-re-evaluation only on
operator request.

---

## 4. Implementation sketch

### 4.1 Schema

```sql
CREATE TABLE memory_records (
  memory_root TEXT PRIMARY KEY,
  schema_version TEXT NOT NULL,
  parent_memory_root TEXT,
  audit_events_high_water TEXT NOT NULL,    -- last event_hash included
  branch_summaries_blob BLOB NOT NULL,      -- canonical-JSON of branch digests
  state TEXT NOT NULL DEFAULT 'live'
        CHECK(state IN ('live','stale','falsified')),
  created_at TEXT NOT NULL,
  falsified_at TEXT,
  falsified_reason TEXT
);

CREATE TABLE memory_branch_summaries (
  branch_id TEXT NOT NULL,        -- e.g., "failure-motif:title-mismatch"
  memory_root TEXT NOT NULL,
  summary_digest TEXT NOT NULL,
  summary_blob BLOB NOT NULL,
  count INTEGER NOT NULL,
  PRIMARY KEY(branch_id, memory_root),
  FOREIGN KEY(memory_root) REFERENCES memory_records(memory_root)
);
```

### 4.2 Module layout

```
arborist/memory/
├── __init__.py
├── snapshot.py         # build memory_root from audit_events slice
├── projections.py      # branch projection rules (deterministic)
├── falsify.py          # mark memory entries falsified
└── store.py            # CRUD
```

### 4.3 Initial branch projections (curated set, expandable)

- `failure-motif:title-mismatch` — count of TITLE_MISMATCH
  violations grouped by source title.
- `failure-motif:warrant-missing` — count grouped by claim shape.
- `failure-motif:deflection` — deflection signals grouped by topic.
- `correction:operator-applied` — corrections explicitly logged via
  CLI.

Each projection rule lives in `projections.py` and is canonicalized
+ hashed; the hash is part of the branch summary's preimage so
projection-rule changes produce a different memory_root.

### 4.4 CLI surface

```
arborist memory snapshot --shards-dir DIR
arborist memory show --root ROOT
arborist memory branches --root ROOT
arborist memory falsify --branch BR --reason TXT
arborist memory cascade --memory-root ROOT  # re-evaluate dependent records
```

### 4.5 SelfModel relationship

memory_root and selfmodel_root are sibling commitments. SelfModel
is identity (capability claims, profile fingerprint). memory_root
is *behavior history* (recurring motifs, accepted corrections). A
SelfModel snapshot may *cite* a memory_root in its body (via
`memory_root` field) so the audit reader can trace from "agent
claims X" to "agent's behavior history when claim was made."

### 4.6 Tests

- Unit: snapshot deterministic given fixed audit_events high-water.
- Unit: branch projection rule changes produce different memory_root.
- Unit: parent_memory_root chain traverses cleanly.
- Integration: memory falsification flips state; cascade is no-op
  unless explicitly invoked.
- Audit chain: chain-check stays clean after memory_root landing.

### 4.7 Bumps

- New schema (`memory_records`, `memory_branch_summaries`). Additive.
- Optional `governance_policy.memory_binding` flag. Default off.
- New CLI subcommand `arborist memory`.

---

## 5. Out of scope

- Cross-shard memory reconciliation. Each shard maintains its own
  memory_records.
- Soft memory (e.g., recent-failure cache, sliding-window
  popularity counts). Sidecar territory; not in this ticket.
- Memory-driven retrieval ("avoid this source — it produced a
  falsified claim recently"). Future capability layer.
- Fork inheritance (child shard inherits memory_root from parent).
  Mesh-level concern; depends on v8 consensus (ticket #000012).

---

## 6. Status

**Closed 2026-05-07.** Scope delivered:

- Schema migration `_migrate_memory_root` adds `memory_records` +
  `memory_branch_summaries` tables. Sibling state — does NOT enter
  cache_key by default (per §2.4 advisory mode).
- Module `arborist.memory`:
  - `canonical.py` — `MemorySnapshot` + `BranchSummary` dataclasses,
    `branch_digest`, `memory_root` SHA-256 derivation. Branch order
    invariant (sorted by branch_id before hashing).
  - `projections.py` — three deterministic projections at v1:
    `failure-motif:violations`, `audit-mode-distribution`,
    `falsification-state`. Projection rule version pinned via
    `PROJECTION_VERSION = "projections-v1"`.
  - `snapshot.py` — `snapshot(conn)` runs default branch set against
    current state, returns a `MemorySnapshot`. Side-effect-free.
  - `store.py` — `store_snapshot` persists + emits
    `memory_snapshot_landed` audit event. Idempotent on same root.
  - `falsify.py` — `falsify` and `mark_stale` flip state and emit
    `memory_falsified` / `memory_marked_stale` audit events.
- CLI: `arborist memory snapshot|show|branches|falsify`.
- SelfModel integration (ticket #000014 already shipped): SelfModel
  snapshot reads the latest live memory_root and folds it into the
  SelfModel canonical body. SelfModel root therefore changes when
  memory_root changes.
- Tests: `tests/test_memory_root.py` — 15 cases covering canonical
  body stability, root invariance under branch order, snapshot
  determinism + high-water progression, store idempotency,
  falsify/mark_stale, projection determinism, audit-chain integrity.
  Full suite: 1040 passed, 36 skipped.

Out-of-scope items (deferred to follow-ups):

- Cascade re-evaluation (`arborist memory cascade`). Memory
  falsification stays advisory (option B in §2.3); explicit
  cascade is a follow-up.
- Cross-shard memory reconciliation. Per-shard for now.
- Memory-driven retrieval routing.
- Fork inheritance (mesh / v8 concern).
