arborist/docs/tickets/ticket-000014-self-model.md

Ticket #000014 — SelfModel: schema, falsification, integration

Status: closed · landed 2026-05-07 Opened: 2026-05-07 Closed: 2026-05-07 Scope: Spec a first-class SelfModel object that an arborist-hosted agent maintains across queries: capability claims, known failure modes, verifier identity, model-profile fingerprint, accepted/rejected patch roots. Wire SelfModel falsification into the audit chain so a stale SelfModel triggers re-evaluation rather than silent drift. Audience: fox + future blackops shifts. Hard constraint: SelfModel is hard-hash committed (enters proof path). Soft self-impressions (heuristic confidence, last-N-success counters) live in sidecars, never in the SelfModel preimage. The 8-dim cache_key gains no new dimension; SelfModel folds into governance_policy_hash (or stays parallel — see §2.4).

---

1. Problem statement

Cited verbatim from the v9.8 design notes (2026-05-04):

SelfModel → still missing; the next logical addition (as proposed in tickets #000011).

The referenced ticket (#000011) ended up being SOFT_PREFLIGHT_HINT, not SelfModel. No SelfModel ticket exists in arborist's index. This ticket opens it.

Without a SelfModel:

Failure	What happens today
Capability drift	Model upgrade silently changes which questions return STRICT vs HYBRID; cache stays warm with stale verdicts.
Failure-mode amnesia	The agent has no memory of "I previously hallucinated about spin glass" → repeats the failure on related queries.
Verifier identity slippage	Verifier v1 → v2 changes admissibility; existing records don't auto-revalidate.
Patch lineage gaps	Accepted policy patches are committed in audit_events but the agent has no compiled view of "what's currently active."
Self-introspection	arborist inspect can report sidecars, but cannot answer "what does this agent claim to be capable of?" — there's no committed answer.

1.1 What SelfModel covers

A frozen-at-checkpoint snapshot of the agent's identity:

{
  schema_version:           "selfmodel-v1",
  model_profile_hash:       "...",   # which LLM, which quantization
  verifier_method_root:     "...",   # union of verifier strategies
  governance_policy_hash:   "...",   # current policy fingerprint
  canonicalization_version: "norm-v1",
  chunking_version:         "tok-512-v1",
  capability_claims:        [...],   # signed claims with eval digests
  known_failure_modes:      [...],   # signed by past audit findings
  accepted_patch_root:      "...",   # Merkle root of accepted patches
  rejected_patch_root:      "...",   # Merkle root of rejected patches
  parent_selfmodel_root:    "...",   # lineage
  timestamp:                "...",
  selfmodel_root:           "..."    # SHA-256 over canonical(self)
}

This is the v7-style "model genome" applied to an arborist-hosted agent. It binds what the agent says it is to bytes verifiers can recompute.

1.2 Failure modes the SelfModel must surface

• Capability claim falsification. A claim like "STRICT-rate ≥ 0.50 on broad-quantifier questions" must be re-checkable from the cited eval_digest. If subsequent bench drops below 0.50, the claim is falsified → SelfModel becomes stale.
• Verifier-method drift. If a new verifier strategy lands, the SelfModel's verifier_method_root changes → all claims must be re-evaluated against the new verifier.
• Patch-set drift. A new accepted patch shifts accepted_patch_root → SelfModel becomes stale until claims are re-evaluated under the new patch set.

---

2. Design choices

2.1 Storage: row vs file vs Merkle blob

A. SQLite row in selfmodel_records table. Indexable, queryable. Mixes hard-bound state with row-level fields.

B. Merkle blob (canonical-JSON) in shard, with selfmodel_root as column. Matches the existing run_dag_blob pattern. Storage- efficient.

C. Hybrid: row with extracted hot fields + blob for canonical body. Like providence_cache today.

Recommendation: C. Matches arborist's existing pattern.

2.2 Lifecycle: when does SelfModel evolve?

A. On every accepted patch. Frequent; high audit-chain churn.

B. On checkpoint cadence (e.g., per release). Aligns with v8 selection cadence; lower churn.

C. Lazy — recomputed on read when audit-chain shows changes. Avoids periodic re-snapshot but breaks the "frozen at checkpoint" discipline.

Recommendation: B. Matches v8 selection cadence. SelfModel checkpoint coincides with Proof-of-Upgrade.

2.3 Falsification semantics

A SelfModel becomes stale when:

• Any cited capability_claim.eval_digest re-runs and shows regression below the claim's stated threshold (with bench-floor tolerance — 5pp per bench-maxing.md).
• verifier_method_root changes upstream of this SelfModel without a new SelfModel snapshot landing.
• accepted_patch_root shifts.

stale is a soft signal — it does not invalidate the SelfModel's existing audit chain entries. It signals that new queries should prefer a fresher SelfModel if available.

2.4 Cache_key folding

Two options:

A. Fold into governance_policy_hash. Adds one input to the already-existing hash. No 8-dim invariant break. SelfModel changes invalidate prior cache.

B. Sibling parallel field. SelfModel reference stored on providence record but not in cache_key. Cache hits across SelfModel changes (interpretable as "answer is the same regardless of self-claims").

Recommendation: A. SelfModel changes are real policy changes; they should invalidate prior records on lookup. The 8-dim invariant holds because we're folding into existing dimensions, not adding a 9th.

2.5 Capability claim format

Each claim is a signed assertion:

{
  "claim_id": "...",
  "claim_text": "STRICT-rate ≥ 0.50 on broad-quantifier questions",
  "metric": "strict_rate",
  "threshold": 0.50,
  "eval_digest": "...",          // canonical bench fixture digest
  "measured_value": 0.54,
  "measured_at": "2026-05-02T15:07Z",
  "validity_horizon": "next-checkpoint",
  "claim_hash": "..."
}

Falsification = re-running eval_digest and finding measured below threshold. The audit-chain entry that surfaces falsification cites the claim_hash and includes the new measurement.

---

3. Recommendation

Land SelfModel as a Merkle-blob + extracted-hot-fields table (selfmodel_records), re-snapshotted at checkpoint cadence (matching v8 selection), folded into governance_policy_hash, with capability-claim falsification triggering stale state.

---

4. Implementation sketch

4.1 Schema (SQLite, per-shard)

CREATE TABLE selfmodel_records (
  selfmodel_root TEXT PRIMARY KEY,
  schema_version TEXT NOT NULL,
  parent_selfmodel_root TEXT,
  model_profile_hash TEXT NOT NULL,
  verifier_method_root TEXT NOT NULL,
  governance_policy_hash TEXT NOT NULL,
  canonicalization_version TEXT NOT NULL,
  chunking_version TEXT NOT NULL,
  accepted_patch_root TEXT,
  rejected_patch_root TEXT,
  state TEXT NOT NULL DEFAULT 'live'
        CHECK(state IN ('live','stale','falsified')),
  body_blob BLOB NOT NULL,         -- canonical-JSON
  created_at TEXT NOT NULL,
  falsified_at TEXT,
  falsified_reason TEXT
);

CREATE TABLE selfmodel_capability_claims (
  claim_hash TEXT PRIMARY KEY,
  selfmodel_root TEXT NOT NULL,
  metric TEXT NOT NULL,
  threshold REAL NOT NULL,
  eval_digest TEXT NOT NULL,
  measured_value REAL,
  measured_at TEXT,
  validity_horizon TEXT,
  body_blob BLOB NOT NULL,
  FOREIGN KEY(selfmodel_root) REFERENCES selfmodel_records(selfmodel_root)
);

CREATE INDEX idx_claim_metric ON selfmodel_capability_claims(metric);
CREATE INDEX idx_selfmodel_state ON selfmodel_records(state);

4.2 Module layout

arborist/selfmodel/
├── __init__.py
├── canonical.py         # canonical-JSON serialization
├── falsify.py           # capability-claim re-evaluation
├── snapshot.py          # build a SelfModel from current store state
└── store.py             # CRUD on selfmodel_records / claims

4.3 governance_policy_hash extension

# arborist/qa/keys.py — extend the hash input list
governance_inputs = [
    ...existing fields...,
    selfmodel_root or "none",   # opt-in via policy
]

When policy["selfmodel_binding"] is True, fold; default False for greenfield landing → existing tests stay green. Operators graduating to SelfModel-bound flip the flag and accept cache invalidation.

4.4 CLI surface

arborist selfmodel snapshot --shards-dir DIR
arborist selfmodel show --root ROOT
arborist selfmodel falsify --re-evaluate-claims
arborist selfmodel diff PARENT CHILD

4.5 Audit-event emission

New event types in audit_events:

• selfmodel_snapshot_landed
• selfmodel_capability_claim_added
• selfmodel_falsified
• selfmodel_marked_stale

Each carries the SelfModel root and the relevant claim hash.

4.6 Tests

• Unit: canonical-JSON round-trip; SelfModel root stable across field reorderings.
• Unit: capability-claim falsification flips state from live to falsified and records reason.
• Unit: parent_selfmodel_root chain traverses cleanly.
• Integration: SelfModel-bound governance hash invalidates prior cache; SelfModel-unbound governance hash stays compatible.
• Audit: chain-check still passes after SelfModel landing.

4.7 Bumps

• New schema (selfmodel_records, selfmodel_capability_claims). Additive; no schema_version bump (new tables ignored by older readers).
• governance_policy_hash opt-in extension. Default off → existing cache stays valid.
• New CLI subcommand arborist selfmodel.

---

5. Out of scope

• Cross-shard SelfModel reconciliation. Each shard maintains its own selfmodel_records. Mesh-level SelfModel agreement is part of v8 consensus (ticket #000012).
• Soft self-impressions (running success counters, recent-failure cache). Belong in sidecars; not in this ticket.
• SelfModel-driven retrieval routing ("agent claims it knows about X → prefer X-shard"). Future capability layer; this ticket is identity, not behavior.
• Automatic capability-claim mining. SelfModel is hand-authored or bench-derived in this ticket; auto-mining is a follow-up.

---

6. Status

Closed 2026-05-07. Landed in same commit as ticket batch. Scope delivered:

• Schema migration _migrate_selfmodel_tables adds selfmodel_records
  • selfmodel_capability_claims to existing shards. Idempotent; greenfield-compatible (no impact on prior cache).
• Module arborist.selfmodel:
  • canonical.py — SelfModel + CapabilityClaim dataclasses, canonical-JSON serialization, selfmodel_root SHA-256 derivation, claim_hash derivation, with_claims helper. Order-invariant on capability-claim hashes.
  • snapshot.py — snapshot(conn) derives a SelfModel from current store state (latest providence row + verifier methods + memory root if present). Side-effect-free.
  • store.py — store_snapshot(conn, model, claims) persists + emits selfmodel_snapshot_landed and selfmodel_capability_claim_added audit events. Idempotent on same root.
  • falsify.py — falsify(...) and mark_stale(...) flip state and emit selfmodel_falsified / selfmodel_marked_stale. Both are idempotent on terminal states.
• CLI: arborist selfmodel snapshot|show|falsify|list.
• Tests: tests/test_selfmodel.py — 14 cases covering canonical-JSON stability, root invariance under claim order, snapshot determinism, store idempotency, audit-event emission, falsify/mark_stale semantics, audit-chain integrity. All pass; full suite 1012 pass.

Out-of-scope items (deferred to follow-ups):

• Cache_key folding (policy["selfmodel_binding"]). Default off → no cache invalidation. Operator opt-in lands when bench evidence shows the binding is wanted.
• Capability-claim re-evaluation harness (arborist selfmodel falsify --re-evaluate-claims). Manual falsify works today; auto- re-evaluation depends on ticket #000021 fixture set landing.
• arborist selfmodel diff PARENT CHILD CLI subcommand.
• Cross-shard SelfModel reconciliation (mesh-level, depends on v8).