docs/tickets: open #000012-#000021 design batch (v7/v9.8 gap coverage)
Ten design-only tickets covering the architectural gaps surfaced in the DNA↔Merkle-DAG / Merkle-AGI v7 / SQD whitepaper review: - #000012 Selection & consensus protocol (Merkle-AGI v8) - #000013 Spatial-temporal substrate (Merkle-AGI v7-W) - #000014 SelfModel: schema, falsification, integration - #000015 π* domain library + cross-domain composition - #000016 ZK Phase-2 frontier proof (concretize the hand-wave) - #000017 Memory-root: lifelong learning audit chain - #000018 Adversarial soft-hash covert-channel analysis - #000019 Specification methodology for π* and V - #000020 Capital-cost ledger (8-capital queues) - #000021 5S/5T/5R benchmark fixtures + harness All open · awaiting go/no-go. Next ID bumped 000012 → 000022. No code in this commit. Implementation per ticket lands in follow-ups once fox picks priority.
This commit is contained in:
parent
b9eb0fc176
commit
8fe0144d81
11 changed files with 2654 additions and 1 deletions
|
|
@ -57,6 +57,16 @@ Newest first. Update on every open/close.
|
|||
|
||||
| ID | Title | Status | Opened | Directive |
|
||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||
| #000021 | 5S/5T/5R benchmark fixtures + harness | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000020 | Capital-cost ledger (8-capital queues) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000019 | Specification methodology for π* and V | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000018 | Adversarial soft-hash covert-channel analysis | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000017 | Memory-root: lifelong learning audit chain | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000016 | ZK Phase-2 frontier proof (concretize) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000015 | π* domain library + cross-domain composition | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000014 | SelfModel: schema, falsification, integration | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000013 | Spatial-temporal substrate (Merkle-AGI v7-W) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000012 | Selection & consensus protocol (Merkle-AGI v8) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000011 | SOFT_PREFLIGHT_HINT model-assisted sidecar | closed · landed 2026-05-04 (zero-shot full impl) | 2026-05-04 | D1 (preserves) |
|
||||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
||||
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses) | 2026-05-03 | D3, D4 |
|
||||
|
|
@ -71,4 +81,4 @@ Newest first. Update on every open/close.
|
|||
|
||||
## Next ID
|
||||
|
||||
`000012`
|
||||
`000022`
|
||||
|
|
|
|||
264
docs/tickets/ticket-000012-selection-consensus-protocol.md
Normal file
264
docs/tickets/ticket-000012-selection-consensus-protocol.md
Normal file
|
|
@ -0,0 +1,264 @@
|
|||
# Ticket #000012 — Selection & consensus protocol (Merkle-AGI v8)
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Spec for the loop-closing consensus protocol that turns the v7
|
||||
substrate + v9.8 runtime into actual Darwinian selection across multiple
|
||||
validators. Covers fitness scoring, mutation acceptance, fork-choice on
|
||||
disagreement, Sybil resistance, validator economics. Doc-only — no code
|
||||
in this ticket; this is the design substrate paper that follow-up
|
||||
implementation tickets will reference.
|
||||
**Audience:** fox + future blackops shifts + downstream Merkle-AGI v8
|
||||
authors.
|
||||
**Hard constraint:** no per-query consensus. Selection runs at
|
||||
checkpoint cadence (Proof-of-Upgrade scope), not at inference time.
|
||||
v9.8 cache_key invariant stays at 8 dims; consensus-state lives in a
|
||||
sibling table, not folded into the answer-cache.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
The DNA ↔ Merkle-DAG analogy promises:
|
||||
|
||||
```text
|
||||
copy → vary → express → test → select → preserve → repeat
|
||||
```
|
||||
|
||||
Merkle-AGI v7 ships everything except `select`. Section 13.4
|
||||
("Proof-of-Upgrade") sketches a procedure where a candidate model
|
||||
(M', C(M')) is admitted if it shows non-regression on a published
|
||||
eval set + non-regression of ε-coverage at sentinel frontiers. That
|
||||
procedure is **single-validator regression testing with a Merkle
|
||||
receipt**. It is not consensus.
|
||||
|
||||
Specifically v7 § 13.4 leaves these gaps:
|
||||
|
||||
| Gap | Concrete failure under current spec |
|
||||
|---|---|
|
||||
| Validator set discovery | No protocol. Two labs reach different verdicts on (M'); no fork-choice. |
|
||||
| Byzantine fault tolerance | One dishonest validator can sign acceptance for a poisoned (M'). |
|
||||
| Sybil resistance | A single actor spinning up 50 validators wins every quorum. |
|
||||
| Fork-choice on disagreement | If validators V1, V2 publish conflicting acceptance receipts, downstream nodes have no rule to pick. |
|
||||
| Liveness vs safety trade | No bound on how long acceptance can stall when validators are offline. |
|
||||
| Validator incentives | Why would anyone run a validator? What's the slashing condition for cheating? |
|
||||
| Stake / membership semantics | Permissionless? Permissioned? Hybrid? v7 silent. |
|
||||
|
||||
Without a protocol that closes these, "digital evolution" is
|
||||
metaphor — a single lab signing its own upgrades, no different in
|
||||
trust model from current model-card releases.
|
||||
|
||||
### 1.1 Concrete failure scenario
|
||||
|
||||
Lab A trains M_t → M_{t+1} with a backdoor that triggers on a rare
|
||||
input pattern. Lab A signs Proof-of-Upgrade: ε-coverage non-
|
||||
regressed at every published frontier (because the backdoor lives at
|
||||
a non-published frontier). Lab A publishes C(M_{t+1}). Anyone pulling
|
||||
the registry sees an "accepted" upgrade. v7 has no way to surface
|
||||
that no independent validator audited the upgrade.
|
||||
|
||||
The selection protocol must make "accepted under v8" mean
|
||||
"accepted by N independent validators meeting policy P," verifiable
|
||||
to anyone, not "Lab A signed it."
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Validator set: permissionless vs permissioned
|
||||
|
||||
**A. Permissionless (Bitcoin-style).** Anyone with stake (or proof-of-work
|
||||
or proof-of-storage) can validate. Maximum censorship resistance.
|
||||
Costs: economic incentive design, possible centralization through
|
||||
mining/staking concentration, latency.
|
||||
|
||||
**B. Permissioned (consortium).** Validator set is curated by a
|
||||
governance body. Easier to bootstrap, easier to slash, lower
|
||||
latency. Costs: who curates? captures regulatory risk; "patch the
|
||||
planet" mission frowns on gatekeepers.
|
||||
|
||||
**C. Hybrid (delegated proof-of-stake-like).** Permissionless
|
||||
participation but stake required; misbehavior slashable. Common
|
||||
middle-ground (Tezos / Cosmos). Bootstrappable and Sybil-resistant.
|
||||
|
||||
**Recommendation:** Hybrid. Permissionless joining with stake is
|
||||
compatible with permacomputer values (no gatekeepers) and Sybil-
|
||||
resistant in practice. Bootstrap from a small honest set with
|
||||
explicit graduation criteria.
|
||||
|
||||
### 2.2 Fitness function: who defines it?
|
||||
|
||||
**A. Lab-defined (per upgrade).** Submitter declares the metric set
|
||||
and eval digests; validators check non-regression. Flexible, gameable.
|
||||
|
||||
**B. Registry-defined (canonical bench suite).** A single canonical
|
||||
suite gates every upgrade. Simple, brittle, hard to evolve.
|
||||
|
||||
**C. Layered (canonical floor + lab-declared ceiling).** Every upgrade
|
||||
must non-regress on the canonical floor; lab can additionally
|
||||
declare metrics they want validated. Default-on safety; allows
|
||||
specialization.
|
||||
|
||||
**Recommendation:** C (Layered). Canonical floor is the safety
|
||||
substrate every release passes through. Layered specialization
|
||||
keeps domain models from being stuck behind irrelevant gates.
|
||||
|
||||
### 2.3 Quorum rule
|
||||
|
||||
**A. Simple majority (51%).** Liveness-friendly. Vulnerable to slim
|
||||
majorities and hostile takeovers.
|
||||
|
||||
**B. Supermajority (2/3 +).** Standard BFT bound. Tolerates 1/3
|
||||
Byzantine. Tighter than majority, slower under partition.
|
||||
|
||||
**C. Threshold signature (k-of-n).** Cryptographic accumulator;
|
||||
single signature represents quorum. Cheap verification downstream;
|
||||
needs ceremony to mint signing key.
|
||||
|
||||
**Recommendation:** B for safety floor, with threshold signature
|
||||
(C) as a downstream optimization once the protocol stabilizes.
|
||||
Tolerates the canonical 1/3 Byzantine fraction without giving up
|
||||
liveness on small disagreements.
|
||||
|
||||
### 2.4 Slashing condition
|
||||
|
||||
A validator that signs acceptance for a candidate that subsequently
|
||||
fails the canonical floor's audit replay loses stake. A validator
|
||||
that signs conflicting acceptances (forks) loses stake. A validator
|
||||
that double-signs (signs both accept and reject for same C(M'))
|
||||
loses stake.
|
||||
|
||||
This requires:
|
||||
- An audit-replay protocol that re-runs the canonical floor and
|
||||
publishes a Merkle-bound result.
|
||||
- A challenge window during which any party can submit a
|
||||
counter-receipt invalidating an earlier acceptance.
|
||||
- Time-locked stake unbonding so a validator can't sign and exit
|
||||
before challenges land.
|
||||
|
||||
### 2.5 Fork choice
|
||||
|
||||
When two valid acceptance chains diverge, downstream nodes must
|
||||
pick one. Options:
|
||||
|
||||
**A. Longest valid chain.** Bitcoin-style. Vulnerable to deep reorgs.
|
||||
|
||||
**B. Highest-stake-weighted acceptance.** Eth-style finality.
|
||||
Resistant to short-range reorgs.
|
||||
|
||||
**C. First-finalized wins.** GRANDPA-style. Once 2/3+ stake signs,
|
||||
no reorg.
|
||||
|
||||
**Recommendation:** C. Once 2/3+ validators finalize an upgrade,
|
||||
it's permanent. Latency is acceptable for checkpoint-cadence
|
||||
selection (not inference).
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Hybrid permissionless validator set with stake, layered fitness
|
||||
floor + per-upgrade ceiling, supermajority quorum (2/3+), slashing
|
||||
on audit-replay disagreement and equivocation, GRANDPA-style fork
|
||||
choice. Bootstrap from a small honest set with explicit slashing
|
||||
window before opening to permissionless joining.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
This ticket commissions Merkle-AGI v8 as a sister paper to v7. v8
|
||||
must specify:
|
||||
|
||||
1. **Validator state machine.**
|
||||
- States: `bonding`, `active`, `challenged`, `slashed`,
|
||||
`unbonding`.
|
||||
- Transitions: stake deposit, signature, challenge, slash, exit.
|
||||
2. **Acceptance protocol.**
|
||||
- Proposer submits `(C(M'), eval_digest, frontier_coverage_diff,
|
||||
metric_delta_signed)`.
|
||||
- Validators run audit-replay, sign accept/reject within window.
|
||||
- Aggregate signature minted at quorum.
|
||||
3. **Challenge protocol.**
|
||||
- Anyone submits `(C(M'), counter_evidence)` within challenge
|
||||
window.
|
||||
- If counter-evidence verifies (re-runs canonical floor and
|
||||
finds regression), all signing validators are slashed.
|
||||
4. **Fork choice rule.**
|
||||
- Validators only sign on candidates whose parent C(M_t) is
|
||||
finalized.
|
||||
- Once 2/3+ stake signs C(M_{t+1}), it's finalized.
|
||||
5. **Stake mechanics.**
|
||||
- Bond / unbond windows.
|
||||
- Slashing fraction per offense class.
|
||||
- Reward distribution per honest signature.
|
||||
6. **Mesh wire format.**
|
||||
- Extension to arborist `mesh/wire.py` for validator gossip.
|
||||
- Aggregate signature canonicalization (so verification is
|
||||
stake-weight-independent).
|
||||
|
||||
The paper itself is the ticket-#000012 deliverable. Code lives in
|
||||
follow-up tickets that cite this one.
|
||||
|
||||
### 4.1 Concrete artifacts this ticket produces
|
||||
|
||||
- `docs/merkle-agi-v8-consensus.rst` — sister to the v7 substrate
|
||||
paper. Sections: validator state machine, acceptance, challenge,
|
||||
fork choice, slashing, mesh wire format, BFT analysis.
|
||||
- `docs/v8-policy-fields.md` — the policy fields v8 introduces and
|
||||
how they fold into `governance_policy_hash` (or whether they
|
||||
live in a sibling `consensus_policy_hash`).
|
||||
- A worked-example Merkle-AGI v8 acceptance ledger reflecting one
|
||||
imaginary upgrade cycle (paper's appendix; does not require
|
||||
running validators).
|
||||
|
||||
### 4.2 What does not change
|
||||
|
||||
- v9.8 8-dim cache_key. Consensus state lives in `consensus_events`
|
||||
(sibling table), not in `cache_key`.
|
||||
- arborist's per-shard audit chain. v8 is checkpoint-cadence,
|
||||
cross-validator; per-shard audit chain stays per-node.
|
||||
- Existing falsification-state semantics (`live`, `failed`, `stale`,
|
||||
`quarantined`).
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Implementation of the v8 protocol in code. That is at minimum 3-4
|
||||
follow-up tickets (validator state machine, mesh wire extension,
|
||||
audit-replay harness, slashing accountant).
|
||||
- Economic parameter calibration (stake amounts, slashing fractions,
|
||||
reward rates). v8 paper specifies the *form*; calibration is a
|
||||
governance decision.
|
||||
- Cross-chain anchoring (publishing v8 finalizations to Bitcoin /
|
||||
Ethereum / etc.). Optional bolt-on.
|
||||
- Selection of frontier benchmark fixtures (covered by ticket
|
||||
#000021).
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Liveness vs censorship.** A validator set that requires 2/3+ to
|
||||
finalize stalls under 1/3 hostile partition. Acceptable for
|
||||
checkpoint cadence; needs explicit liveness floor in the spec.
|
||||
- **Bootstrap honesty.** The initial validator set must be honest
|
||||
for the protocol to converge. Solution: explicit bootstrap window
|
||||
with permissioned set + scheduled transition to permissionless.
|
||||
- **Stake captures.** Large staker can dominate. Mitigation: cap on
|
||||
individual stake weight, or convex weighting (sqrt-stake).
|
||||
- **Re-staking attacks.** Validators staking the same capital across
|
||||
multiple v8 instances. Out of scope here; addressed by
|
||||
cross-instance slashing accumulator if/when v8 multiplies.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** This is a research-paper-scope ticket.
|
||||
Implementation is gated on fox approving the paper's design choices
|
||||
and committing v8 substrate work.
|
||||
|
||||
Closure criterion: `docs/merkle-agi-v8-consensus.rst` lands and is
|
||||
reviewed.
|
||||
250
docs/tickets/ticket-000013-spatial-temporal-substrate.md
Normal file
250
docs/tickets/ticket-000013-spatial-temporal-substrate.md
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
# Ticket #000013 — Spatial-temporal substrate (Merkle-AGI v7-W)
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Spec for a third commitment substrate — spatial-temporal
|
||||
world-state — sister to Merkle-AGI v7 (logic/math substrate) and
|
||||
arborist v9.8 (language/claim-lattice substrate). Doc-only — this
|
||||
ticket commissions the substrate paper. Implementation tickets cite it.
|
||||
**Audience:** fox + future blackops shifts + downstream world-model
|
||||
authors.
|
||||
**Hard constraint:** stays inside A1–A3 (canonical encoding, public
|
||||
quantization, collision-resistant hash). No new axiom. World-state
|
||||
canonical projection π*_w must be definable on quantized integer state,
|
||||
not on continuous tensors.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
Fox's own framing identifies three substrates needed for ASI:
|
||||
|
||||
```text
|
||||
3 > (language, logic, substrate)
|
||||
1 = recursive-falsification merkle-agi (logic / math)
|
||||
2 = language / claim-lattice (arborist v9.8)
|
||||
3 = spatial-temporal vision / world-models ← MISSING
|
||||
```
|
||||
|
||||
Without a world-state substrate, an agent cannot:
|
||||
|
||||
| Capability | Failure under language+logic only |
|
||||
|---|---|
|
||||
| Persistent identity across time | Cache is keyed on text; physical "where am I" has no commitment. |
|
||||
| Causal physics reasoning | Logic substrate verifies proof steps; world substrate verifies "what happened next." |
|
||||
| Multi-modal grounding | A claim about an object's location can't be falsified without a state commitment. |
|
||||
| Robotic / embodied use | No way to commit "I observed X at (t, x, y, z) with confidence c." |
|
||||
| Video / time-series audit | No π* for temporal signals; no equivalent of `wikitext-base-v1`. |
|
||||
|
||||
v7 § 11 multimodal composition handles vision + language *structurally*
|
||||
(commit the conv kernel, the bridge, the projection). It does not
|
||||
commit *world-state*: the abstract scene representation, the spatial
|
||||
relations, the temporal predicates. Those are derived from the model's
|
||||
forward pass and never committed as first-class objects.
|
||||
|
||||
### 1.1 What world-state means here
|
||||
|
||||
Not raw pixels. Not raw audio. The **derived state** an agent maintains:
|
||||
|
||||
```text
|
||||
- objects: { id, class, bbox, pose, confidence }
|
||||
- relations: { subject_id, predicate, object_id, time_window }
|
||||
- events: { type, t_start, t_end, participants, place }
|
||||
- places: { id, frame_of_reference, geometry, parent_place }
|
||||
- agents: { id, position_trace, pose_trace, attention_trace }
|
||||
```
|
||||
|
||||
This is the substrate of a world-model. It's what a SLAM system
|
||||
exports, what a Genie/SORA-style world-model dreams about, what a
|
||||
robotics planner consumes. None of it has a canonical commitment
|
||||
form today.
|
||||
|
||||
### 1.2 The verifiability challenge
|
||||
|
||||
Unlike text (where `to_base()` canonicalizes prose) and unlike model
|
||||
weights (where Q(Θ) gives integer commitment), world-state is:
|
||||
|
||||
- **Continuous in space and time** — needs canonical discretization
|
||||
(a π*_w with explicit grid choice).
|
||||
- **Probabilistic** — every observation has confidence; commitments
|
||||
must bind the confidence, not just the claim.
|
||||
- **Frame-dependent** — same scene, different observers, different
|
||||
coordinates. π*_w must canonicalize frame.
|
||||
- **Temporally extended** — events span intervals. Canonical
|
||||
interval representation needed.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Canonical discretization
|
||||
|
||||
**A. Fixed grid (per-substrate constant).** Δ_x, Δ_t fixed in the
|
||||
substrate manifest. Simple. Brittle for multi-scale tasks.
|
||||
|
||||
**B. Per-claim grid.** Each commitment declares its grid in attrs.
|
||||
Flexible. Spec-brittleness explosion (every claim becomes its own
|
||||
canonicalization).
|
||||
|
||||
**C. Hierarchical grid (octree / quadtree levels).** Substrate
|
||||
declares a finite set of grid levels; each commitment names which
|
||||
level it's anchored at. Compositional.
|
||||
|
||||
**Recommendation:** C. Hierarchical octree-style grids match how
|
||||
spatial reasoning actually works (coarse-to-fine), match
|
||||
established standards (e.g., S2 / H3 for geographic indexing), and
|
||||
keep π*_w finitely specified.
|
||||
|
||||
### 2.2 Frame canonicalization
|
||||
|
||||
**A. Single canonical frame (e.g., ECEF, object-fixed).** Forces
|
||||
every observation into a global frame. Loses local-frame ergonomics.
|
||||
|
||||
**B. Frame as part of commitment.** Each observation declares its
|
||||
frame; cross-frame reasoning needs explicit transforms (also
|
||||
committed).
|
||||
|
||||
**C. Equivalence classes (like arborist's question_hash strict /
|
||||
equivalence_class).** π*_w accepts multiple frames; equivalence
|
||||
class membership is committed.
|
||||
|
||||
**Recommendation:** B. Forces explicit frame at every observation,
|
||||
and frame transforms become first-class committed objects (matching
|
||||
v7's "every causally relevant transformation must be committed"
|
||||
axiom).
|
||||
|
||||
### 2.3 Temporal canonicalization
|
||||
|
||||
**A. Discrete tick (e.g., milliseconds since epoch).** Trivial. Locks
|
||||
substrate into one clock domain.
|
||||
|
||||
**B. Per-substrate clock (declared in manifest).** Substrate manifest
|
||||
declares Δ_t and reference epoch. Cross-substrate joins need
|
||||
explicit clock transforms.
|
||||
|
||||
**C. Logical time (Lamport / vector clocks).** Decouples from
|
||||
wall-clock. Required for distributed observation merging without
|
||||
trusting clocks.
|
||||
|
||||
**Recommendation:** B for single-agent, C for multi-agent. Substrate
|
||||
manifest declares which mode. Mixed deployments use B locally + C
|
||||
across agents.
|
||||
|
||||
### 2.4 Probabilistic commitment
|
||||
|
||||
Two viable approaches:
|
||||
|
||||
**A. Quantize confidence to integer (e.g., centi-confidence 0-100).**
|
||||
Commits the discretized confidence as part of claim. Simple. Coarse.
|
||||
|
||||
**B. Range commitment.** Each claim commits a confidence interval
|
||||
[lo, hi] as integers. More expressive. More disclosure surface.
|
||||
|
||||
**Recommendation:** A by default; B opt-in for safety-critical
|
||||
deployments where confidence intervals matter (medical, robotics).
|
||||
|
||||
### 2.5 ε-frontiers in v7-W
|
||||
|
||||
What's the equivalent of "affine preactivation" for world-models?
|
||||
|
||||
**Candidates:**
|
||||
- **Pose update step** (ego-motion / kinematic integration). Linear
|
||||
in state under small-time-step assumption. ε-frontier candidate.
|
||||
- **Observation-to-state update** (Kalman update step). Affine.
|
||||
ε-frontier candidate.
|
||||
- **Object detection logits** (already covered by v7).
|
||||
- **Scene-graph edge prediction logits** (affine projections on
|
||||
pair embeddings). ε-frontier candidate.
|
||||
|
||||
**Recommendation:** declare four canonical ε-frontiers in v7-W:
|
||||
`pose_integration`, `observation_update`, `object_logits`,
|
||||
`relation_logits`. Each is affine after appropriate canonical
|
||||
projection.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
A v7-W substrate paper specifying:
|
||||
|
||||
1. Hierarchical-grid spatial discretization (S2/H3-like).
|
||||
2. Frame as committed object with explicit transforms.
|
||||
3. Substrate-declared clock (single-agent) + Lamport (multi-agent).
|
||||
4. Quantized confidence (centi-confidence) default; range opt-in.
|
||||
5. Four canonical ε-frontiers: pose, observation, object, relation.
|
||||
6. Standard π*_w on the (object, relation, event, place, agent)
|
||||
tuple-space defined above.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
This ticket commissions the substrate paper. v7-W structure follows
|
||||
v7:
|
||||
|
||||
- **Part 1 — Introduction & motivation.** Why world-state needs its
|
||||
own substrate; why v7 multimodal composition isn't enough.
|
||||
- **Part 2 — Substrate definition (π*_w).** Hierarchical grid, frame,
|
||||
clock, confidence. Canonical encoding.
|
||||
- **Part 3 — Theorems.** T1-W (state binding), T2-W (causal
|
||||
completeness), T3-W (frame-transform soundness), T4-W (ε at
|
||||
affine frontiers).
|
||||
- **Part 4 — Verifier kernels.** Pose integration, Kalman update,
|
||||
scene-graph edge predictor (each as deterministic integer kernel).
|
||||
- **Part 5 — Multimodal composition with v7.** When does a
|
||||
v7 vision encoder feed v7-W state, and where do ε-proofs anchor.
|
||||
- **Part 6 — Adversarial corners.** Frame spoofing, time-skew,
|
||||
observation injection.
|
||||
- **Appendix — Worked example.** A toy SLAM scenario with full
|
||||
commitment trail.
|
||||
|
||||
### 4.1 Concrete artifacts this ticket produces
|
||||
|
||||
- `docs/merkle-agi-v7w-spatial-temporal.rst` — substrate paper.
|
||||
- `docs/v7w-frontier-catalog.md` — canonical ε-frontier catalog
|
||||
for world-models.
|
||||
- Stub `arborist/world/__init__.py` reserving the namespace; no
|
||||
implementation yet.
|
||||
|
||||
### 4.2 What does not change
|
||||
|
||||
- arborist's text/claim-lattice substrate. v7-W is parallel, not
|
||||
replacement.
|
||||
- v7's affine-frontier ε-proof discipline. v7-W's frontiers follow
|
||||
the same pattern at different op types.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Actually running a SLAM stack inside arborist. v7-W defines the
|
||||
commitment substrate; world-model engines (SLAM, Gaussian
|
||||
splatting, predictive video) plug into it via adapters that are
|
||||
separate tickets.
|
||||
- Cross-modal joint reasoning (text claim + spatial state). That
|
||||
needs the cross-domain π* composition theorem (ticket #000015).
|
||||
- Specific sensor adapters (LIDAR, RGB-D, IMU). Each is a separate
|
||||
source-adapter ticket once v7-W lands.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Discretization tax.** Every spatial claim incurs grid-rounding
|
||||
cost. Need empirical bench on whether ε at world-model frontiers
|
||||
remains tight enough to be useful.
|
||||
- **Standards adoption.** S2/H3 vs custom octree vs Cesium 3D Tiles.
|
||||
Picking a standard reduces interop friction; picking custom keeps
|
||||
permacomputer values intact. Recommend S2 + Lamport as default,
|
||||
custom escape hatch.
|
||||
- **Privacy.** A world-state commitment substrate is also a
|
||||
surveillance substrate. Frame discipline + Phase-2 ZK (ticket
|
||||
#000016) more important here than for text/logic.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Research-paper-scope ticket. Closure
|
||||
criterion: `docs/merkle-agi-v7w-spatial-temporal.rst` lands and is
|
||||
reviewed.
|
||||
310
docs/tickets/ticket-000014-self-model.md
Normal file
310
docs/tickets/ticket-000014-self-model.md
Normal file
|
|
@ -0,0 +1,310 @@
|
|||
# Ticket #000014 — SelfModel: schema, falsification, integration
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Spec a first-class SelfModel object that an arborist-hosted
|
||||
agent maintains across queries: capability claims, known failure modes,
|
||||
verifier identity, model-profile fingerprint, accepted/rejected patch
|
||||
roots. Wire SelfModel falsification into the audit chain so a stale
|
||||
SelfModel triggers re-evaluation rather than silent drift.
|
||||
**Audience:** fox + future blackops shifts.
|
||||
**Hard constraint:** SelfModel is hard-hash committed (enters proof
|
||||
path). Soft self-impressions (heuristic confidence, last-N-success
|
||||
counters) live in sidecars, never in the SelfModel preimage. The 8-dim
|
||||
cache_key gains no new dimension; SelfModel folds into
|
||||
`governance_policy_hash` (or stays parallel — see §2.4).
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
Cited verbatim from the v9.8 design notes (2026-05-04):
|
||||
|
||||
> SelfModel → still missing; the next logical addition (as proposed in
|
||||
> tickets #000011).
|
||||
|
||||
The referenced ticket (#000011) ended up being SOFT_PREFLIGHT_HINT, not
|
||||
SelfModel. No SelfModel ticket exists in arborist's index. This ticket
|
||||
opens it.
|
||||
|
||||
Without a SelfModel:
|
||||
|
||||
| Failure | What happens today |
|
||||
|---|---|
|
||||
| Capability drift | Model upgrade silently changes which questions return STRICT vs HYBRID; cache stays warm with stale verdicts. |
|
||||
| Failure-mode amnesia | The agent has no memory of "I previously hallucinated about spin glass" → repeats the failure on related queries. |
|
||||
| Verifier identity slippage | Verifier v1 → v2 changes admissibility; existing records don't auto-revalidate. |
|
||||
| Patch lineage gaps | Accepted policy patches are committed in audit_events but the agent has no compiled view of "what's currently active." |
|
||||
| Self-introspection | `arborist inspect` can report sidecars, but cannot answer "what does this agent claim to be capable of?" — there's no committed answer. |
|
||||
|
||||
### 1.1 What SelfModel covers
|
||||
|
||||
A frozen-at-checkpoint snapshot of the agent's identity:
|
||||
|
||||
```
|
||||
{
|
||||
schema_version: "selfmodel-v1",
|
||||
model_profile_hash: "...", # which LLM, which quantization
|
||||
verifier_method_root: "...", # union of verifier strategies
|
||||
governance_policy_hash: "...", # current policy fingerprint
|
||||
canonicalization_version: "norm-v1",
|
||||
chunking_version: "tok-512-v1",
|
||||
capability_claims: [...], # signed claims with eval digests
|
||||
known_failure_modes: [...], # signed by past audit findings
|
||||
accepted_patch_root: "...", # Merkle root of accepted patches
|
||||
rejected_patch_root: "...", # Merkle root of rejected patches
|
||||
parent_selfmodel_root: "...", # lineage
|
||||
timestamp: "...",
|
||||
selfmodel_root: "..." # SHA-256 over canonical(self)
|
||||
}
|
||||
```
|
||||
|
||||
This is the v7-style "model genome" applied to an arborist-hosted
|
||||
agent. It binds *what the agent says it is* to *bytes verifiers can
|
||||
recompute*.
|
||||
|
||||
### 1.2 Failure modes the SelfModel must surface
|
||||
|
||||
- **Capability claim falsification.** A claim like "STRICT-rate ≥
|
||||
0.50 on broad-quantifier questions" must be re-checkable from the
|
||||
cited eval_digest. If subsequent bench drops below 0.50, the
|
||||
claim is falsified → SelfModel becomes `stale`.
|
||||
- **Verifier-method drift.** If a new verifier strategy lands, the
|
||||
SelfModel's `verifier_method_root` changes → all claims must be
|
||||
re-evaluated against the new verifier.
|
||||
- **Patch-set drift.** A new accepted patch shifts
|
||||
`accepted_patch_root` → SelfModel becomes stale until claims are
|
||||
re-evaluated under the new patch set.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Storage: row vs file vs Merkle blob
|
||||
|
||||
**A. SQLite row in `selfmodel_records` table.** Indexable, queryable.
|
||||
Mixes hard-bound state with row-level fields.
|
||||
|
||||
**B. Merkle blob (canonical-JSON) in shard, with `selfmodel_root` as
|
||||
column.** Matches the existing `run_dag_blob` pattern. Storage-
|
||||
efficient.
|
||||
|
||||
**C. Hybrid: row with extracted hot fields + blob for canonical body.**
|
||||
Like `providence_cache` today.
|
||||
|
||||
**Recommendation:** C. Matches arborist's existing pattern.
|
||||
|
||||
### 2.2 Lifecycle: when does SelfModel evolve?
|
||||
|
||||
**A. On every accepted patch.** Frequent; high audit-chain churn.
|
||||
|
||||
**B. On checkpoint cadence (e.g., per release).** Aligns with v8
|
||||
selection cadence; lower churn.
|
||||
|
||||
**C. Lazy — recomputed on read when audit-chain shows changes.**
|
||||
Avoids periodic re-snapshot but breaks the "frozen at checkpoint"
|
||||
discipline.
|
||||
|
||||
**Recommendation:** B. Matches v8 selection cadence. SelfModel
|
||||
checkpoint coincides with Proof-of-Upgrade.
|
||||
|
||||
### 2.3 Falsification semantics
|
||||
|
||||
A SelfModel becomes `stale` when:
|
||||
|
||||
- Any cited `capability_claim.eval_digest` re-runs and shows
|
||||
regression below the claim's stated threshold (with bench-floor
|
||||
tolerance — 5pp per `bench-maxing.md`).
|
||||
- `verifier_method_root` changes upstream of this SelfModel without
|
||||
a new SelfModel snapshot landing.
|
||||
- `accepted_patch_root` shifts.
|
||||
|
||||
`stale` is a soft signal — it does *not* invalidate the SelfModel's
|
||||
existing audit chain entries. It signals that *new* queries should
|
||||
prefer a fresher SelfModel if available.
|
||||
|
||||
### 2.4 Cache_key folding
|
||||
|
||||
Two options:
|
||||
|
||||
**A. Fold into governance_policy_hash.** Adds one input to the
|
||||
already-existing hash. No 8-dim invariant break. SelfModel changes
|
||||
invalidate prior cache.
|
||||
|
||||
**B. Sibling parallel field.** SelfModel reference stored on
|
||||
providence record but not in cache_key. Cache hits across
|
||||
SelfModel changes (interpretable as "answer is the same regardless
|
||||
of self-claims").
|
||||
|
||||
**Recommendation:** A. SelfModel changes are real policy changes;
|
||||
they should invalidate prior records on lookup. The 8-dim invariant
|
||||
holds because we're folding into existing dimensions, not adding a
|
||||
9th.
|
||||
|
||||
### 2.5 Capability claim format
|
||||
|
||||
Each claim is a signed assertion:
|
||||
|
||||
```json
|
||||
{
|
||||
"claim_id": "...",
|
||||
"claim_text": "STRICT-rate ≥ 0.50 on broad-quantifier questions",
|
||||
"metric": "strict_rate",
|
||||
"threshold": 0.50,
|
||||
"eval_digest": "...", // canonical bench fixture digest
|
||||
"measured_value": 0.54,
|
||||
"measured_at": "2026-05-02T15:07Z",
|
||||
"validity_horizon": "next-checkpoint",
|
||||
"claim_hash": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Falsification = re-running `eval_digest` and finding measured below
|
||||
threshold. The audit-chain entry that surfaces falsification cites
|
||||
the claim_hash and includes the new measurement.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Land SelfModel as a Merkle-blob + extracted-hot-fields table
|
||||
(`selfmodel_records`), re-snapshotted at checkpoint cadence
|
||||
(matching v8 selection), folded into `governance_policy_hash`,
|
||||
with capability-claim falsification triggering `stale` state.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Schema (SQLite, per-shard)
|
||||
|
||||
```sql
|
||||
CREATE TABLE selfmodel_records (
|
||||
selfmodel_root TEXT PRIMARY KEY,
|
||||
schema_version TEXT NOT NULL,
|
||||
parent_selfmodel_root TEXT,
|
||||
model_profile_hash TEXT NOT NULL,
|
||||
verifier_method_root TEXT NOT NULL,
|
||||
governance_policy_hash TEXT NOT NULL,
|
||||
canonicalization_version TEXT NOT NULL,
|
||||
chunking_version TEXT NOT NULL,
|
||||
accepted_patch_root TEXT,
|
||||
rejected_patch_root TEXT,
|
||||
state TEXT NOT NULL DEFAULT 'live'
|
||||
CHECK(state IN ('live','stale','falsified')),
|
||||
body_blob BLOB NOT NULL, -- canonical-JSON
|
||||
created_at TEXT NOT NULL,
|
||||
falsified_at TEXT,
|
||||
falsified_reason TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE selfmodel_capability_claims (
|
||||
claim_hash TEXT PRIMARY KEY,
|
||||
selfmodel_root TEXT NOT NULL,
|
||||
metric TEXT NOT NULL,
|
||||
threshold REAL NOT NULL,
|
||||
eval_digest TEXT NOT NULL,
|
||||
measured_value REAL,
|
||||
measured_at TEXT,
|
||||
validity_horizon TEXT,
|
||||
body_blob BLOB NOT NULL,
|
||||
FOREIGN KEY(selfmodel_root) REFERENCES selfmodel_records(selfmodel_root)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_claim_metric ON selfmodel_capability_claims(metric);
|
||||
CREATE INDEX idx_selfmodel_state ON selfmodel_records(state);
|
||||
```
|
||||
|
||||
### 4.2 Module layout
|
||||
|
||||
```
|
||||
arborist/selfmodel/
|
||||
├── __init__.py
|
||||
├── canonical.py # canonical-JSON serialization
|
||||
├── falsify.py # capability-claim re-evaluation
|
||||
├── snapshot.py # build a SelfModel from current store state
|
||||
└── store.py # CRUD on selfmodel_records / claims
|
||||
```
|
||||
|
||||
### 4.3 governance_policy_hash extension
|
||||
|
||||
```python
|
||||
# arborist/qa/keys.py — extend the hash input list
|
||||
governance_inputs = [
|
||||
...existing fields...,
|
||||
selfmodel_root or "none", # opt-in via policy
|
||||
]
|
||||
```
|
||||
|
||||
When `policy["selfmodel_binding"]` is `True`, fold; default
|
||||
`False` for greenfield landing → existing tests stay green.
|
||||
Operators graduating to SelfModel-bound flip the flag and accept
|
||||
cache invalidation.
|
||||
|
||||
### 4.4 CLI surface
|
||||
|
||||
```
|
||||
arborist selfmodel snapshot --shards-dir DIR
|
||||
arborist selfmodel show --root ROOT
|
||||
arborist selfmodel falsify --re-evaluate-claims
|
||||
arborist selfmodel diff PARENT CHILD
|
||||
```
|
||||
|
||||
### 4.5 Audit-event emission
|
||||
|
||||
New event types in `audit_events`:
|
||||
|
||||
- `selfmodel_snapshot_landed`
|
||||
- `selfmodel_capability_claim_added`
|
||||
- `selfmodel_falsified`
|
||||
- `selfmodel_marked_stale`
|
||||
|
||||
Each carries the SelfModel root and the relevant claim hash.
|
||||
|
||||
### 4.6 Tests
|
||||
|
||||
- Unit: canonical-JSON round-trip; SelfModel root stable across
|
||||
field reorderings.
|
||||
- Unit: capability-claim falsification flips state from `live` to
|
||||
`falsified` and records reason.
|
||||
- Unit: parent_selfmodel_root chain traverses cleanly.
|
||||
- Integration: SelfModel-bound governance hash invalidates prior
|
||||
cache; SelfModel-unbound governance hash stays compatible.
|
||||
- Audit: chain-check still passes after SelfModel landing.
|
||||
|
||||
### 4.7 Bumps
|
||||
|
||||
- New schema (`selfmodel_records`, `selfmodel_capability_claims`).
|
||||
Additive; no `schema_version` bump (new tables ignored by older
|
||||
readers).
|
||||
- `governance_policy_hash` opt-in extension. Default off → existing
|
||||
cache stays valid.
|
||||
- New CLI subcommand `arborist selfmodel`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Cross-shard SelfModel reconciliation. Each shard maintains its own
|
||||
`selfmodel_records`. Mesh-level SelfModel agreement is part of v8
|
||||
consensus (ticket #000012).
|
||||
- Soft self-impressions (running success counters, recent-failure
|
||||
cache). Belong in sidecars; not in this ticket.
|
||||
- SelfModel-driven retrieval routing ("agent claims it knows about
|
||||
X → prefer X-shard"). Future capability layer; this ticket is
|
||||
identity, not behavior.
|
||||
- Automatic capability-claim mining. SelfModel is hand-authored or
|
||||
bench-derived in this ticket; auto-mining is a follow-up.
|
||||
|
||||
---
|
||||
|
||||
## 6. Status
|
||||
|
||||
**Open · awaiting go/no-go.** This ticket has the smallest
|
||||
implementation surface among the v8-related tickets — schema +
|
||||
serialization + opt-in policy fold. Recommended as first
|
||||
implementation candidate after design batch lands.
|
||||
|
||||
Closure criterion: schema lands, `arborist selfmodel snapshot`
|
||||
runs against existing shards, capability-claim falsification flips
|
||||
state correctly under bench re-evaluation.
|
||||
275
docs/tickets/ticket-000015-pi-star-domain-library.md
Normal file
275
docs/tickets/ticket-000015-pi-star-domain-library.md
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
# Ticket #000015 — π* domain library + cross-domain composition
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Stand up a unified library of canonical projections (π*) for
|
||||
the modalities arborist's QA layer touches, plus the composition theory
|
||||
needed to chain them. Doc + code: docs covers the algebra of π*
|
||||
composition; code adds a stub `arborist/pi_star/` namespace with
|
||||
existing canonicalizers (text, claim-lattice) re-homed and a clear
|
||||
interface for new modalities (code/AST, time-series, image-graph, etc.).
|
||||
**Audience:** fox + future authors of new source adapters.
|
||||
**Hard constraint:** every π* must be deterministic and version-pinned.
|
||||
Adding a new π* never silently invalidates prior cache; new π* land as
|
||||
new `name@version` rather than mutating existing canonicalizers.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
The SQD whitepaper (§13.3) names the bottleneck:
|
||||
|
||||
> Specification brittleness is the real bottleneck. Even with perfect
|
||||
> optimization, if π* or V encodes the wrong target, you get verified
|
||||
> nonsense.
|
||||
|
||||
arborist has two π*'s in flight today:
|
||||
|
||||
| π* | Where | Versioned? |
|
||||
|---|---|---|
|
||||
| Text (wikitext → prose) | `arborist/wikitext.py:to_base()` | yes (`wikitext-base-v1`) |
|
||||
| Claim-lattice | `arborist/qa/parse_claims.py` | yes (folded into `governance_policy_hash`) |
|
||||
|
||||
That's it. Real AGI use needs π* for at minimum:
|
||||
|
||||
- **Code** (parse → AST → canonical S-expression). Already partially
|
||||
approximated by ad-hoc tokenization; no canonical projection.
|
||||
- **Math/logic** (parse → kernel proof object). v7 § 14.3 sketches
|
||||
this; arborist has no implementation.
|
||||
- **Time-series** (resample → quantize → committed integer vector).
|
||||
No π* exists.
|
||||
- **Image graph** (object/relation extraction → committed scene
|
||||
graph). Depends on ticket #000013 (v7-W).
|
||||
- **Tabular data** (canonical column ordering, type-pinned cells).
|
||||
No π*.
|
||||
|
||||
Beyond per-domain π*, the **cross-domain composition** problem is
|
||||
unsolved:
|
||||
|
||||
```
|
||||
π*_text → claim about a code change (need π*_code chained)
|
||||
π*_logic → proof citing a tabular dataset (need π*_tabular chained)
|
||||
π*_world → object's path described in prose (need π*_text chained)
|
||||
```
|
||||
|
||||
v7 § 11 multimodal composition handles **structural** composition.
|
||||
Cross-domain π* composition needs an **algebraic** treatment:
|
||||
|
||||
```
|
||||
when does π*_a ∘ π*_b preserve canonicality?
|
||||
when does it preserve transitivity of equivalence?
|
||||
what type-theoretic constraint must hold for the chain to commit?
|
||||
```
|
||||
|
||||
No paper or doc in the corpus answers this.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Library shape: one module vs many
|
||||
|
||||
**A. One module `arborist/pi_star/` with submodules per domain.**
|
||||
Centralized; easy to enumerate.
|
||||
|
||||
**B. Distributed: each source/domain owns its π*.** Decentralized;
|
||||
mirrors `sources/` layout. Harder to enforce versioning.
|
||||
|
||||
**C. Hybrid: `arborist/pi_star/` defines the protocol + base class;
|
||||
implementations live alongside the data they canonicalize.**
|
||||
|
||||
**Recommendation:** C. Protocol + ABC in `arborist/pi_star/`,
|
||||
implementations stay near their data. Matches arborist's existing
|
||||
`SearchBackend` / `Source` protocol pattern.
|
||||
|
||||
### 2.2 π* protocol
|
||||
|
||||
```python
|
||||
class PiStar(Protocol):
|
||||
name: str # e.g., "code-py-ast-v1"
|
||||
version: str # always vN; never reused
|
||||
def canonicalize(self, raw: bytes) -> bytes:
|
||||
"""raw → canonical bytes, or raise PiStarError if undefined."""
|
||||
def equivalence_class_id(self, raw: bytes) -> str:
|
||||
"""SHA-256 over canonicalize(raw); the equivalence-class label."""
|
||||
def domain(self) -> str:
|
||||
"""e.g., 'text', 'code', 'logic', 'tabular', 'time-series', 'world'."""
|
||||
```
|
||||
|
||||
Every π* registers itself in a `pi_star.registry` dict keyed by
|
||||
`f"{name}@{version}"`. Lookups in code use the same key, so changing
|
||||
π* means changing the key, not mutating in place.
|
||||
|
||||
### 2.3 Composition algebra
|
||||
|
||||
The core question: **when is π*_a ∘ π*_b a valid canonical
|
||||
projection?**
|
||||
|
||||
A composition is canonical iff:
|
||||
|
||||
1. **Type-compatible.** The codomain of π*_b is in the domain of π*_a.
|
||||
2. **Determinism preserved.** Both individual π*'s are deterministic
|
||||
⇒ composition is deterministic.
|
||||
3. **Equivalence-class preservation.** If `π*_b(x) = π*_b(y)` then
|
||||
`(π*_a ∘ π*_b)(x) = (π*_a ∘ π*_b)(y)`. Trivially true if π*_a is a
|
||||
function. Required when π*_a may distinguish things π*_b
|
||||
conflated.
|
||||
4. **Lossy composition is allowed but flagged.** If `(π*_a ∘ π*_b)(x)`
|
||||
does not round-trip through `(π*_a ∘ π*_b)`'s codomain, the
|
||||
composition is *projective*, not *invertible*. Flagged in the
|
||||
composition manifest.
|
||||
|
||||
The composition itself is registered as its own π*:
|
||||
```
|
||||
"text-then-claim-lattice@v1" = π*_claim ∘ π*_text
|
||||
```
|
||||
This keeps the registry flat — every composition is a first-class
|
||||
π* with its own version.
|
||||
|
||||
### 2.4 Cross-domain anchor projection
|
||||
|
||||
For genuinely cross-domain claims (e.g., "this prose paragraph
|
||||
describes that code function"), we need an **anchor projection**
|
||||
π*_anchor : Domain_a × Domain_b → CanonicalAnchor. The anchor is a
|
||||
joint commitment to both projections' canonical forms:
|
||||
|
||||
```
|
||||
anchor = SHA-256(
|
||||
canon_a_id || canon_b_id || relation_kind || relation_payload_canonical
|
||||
)
|
||||
```
|
||||
|
||||
This is the v7-style "every causally relevant transformation must be
|
||||
committed" axiom applied to cross-domain joins. Anchors are first-
|
||||
class committed objects; they fold into `governance_policy_hash` if
|
||||
operator policy says so.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
**A. Code library:** `arborist/pi_star/` namespace with protocol +
|
||||
ABC + registry. Re-home `wikitext.to_base()` and `qa.parse_claims`
|
||||
under registry keys (`wikitext-base@v1`, `claim-lattice@v1`).
|
||||
|
||||
**B. Stub modalities:** add stub modules for `code-py-ast`,
|
||||
`logic-kernel`, `time-series-quantized`, `tabular-pinned`. Each
|
||||
raises `NotImplementedError` with a pointer to its implementation
|
||||
ticket. This makes the gap visible and registers the namespace.
|
||||
|
||||
**C. Composition theory doc:** `docs/pi-star-composition.md` covering
|
||||
the algebra above with worked examples. This doc + at least one
|
||||
worked composition (text-then-claim-lattice) is the design
|
||||
deliverable.
|
||||
|
||||
**D. Out-of-scope here:** actually implementing the new modalities.
|
||||
Each modality's implementation is its own follow-up ticket.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Module layout (this ticket)
|
||||
|
||||
```
|
||||
arborist/pi_star/
|
||||
├── __init__.py # registry + re-exports
|
||||
├── protocol.py # PiStar Protocol, PiStarError
|
||||
├── registry.py # name@version dict + lookup helpers
|
||||
├── compose.py # composition algebra + worked example
|
||||
├── text.py # re-homes wikitext.to_base() → "wikitext-base@v1"
|
||||
├── claim_lattice.py # re-homes parse_claims → "claim-lattice@v1"
|
||||
├── code.py # stub for "code-py-ast@v1"
|
||||
├── logic.py # stub for "logic-kernel@v1"
|
||||
├── time_series.py # stub for "time-series-quantized@v1"
|
||||
└── tabular.py # stub for "tabular-pinned@v1"
|
||||
```
|
||||
|
||||
### 4.2 Re-home rules
|
||||
|
||||
Existing imports (`from arborist.wikitext import to_base`) keep
|
||||
working — `arborist/wikitext.py` re-exports
|
||||
`arborist.pi_star.text.canonicalize`. This is the only soft-rename;
|
||||
the registry key replaces ad-hoc version mention sites over time.
|
||||
|
||||
### 4.3 Composition manifest
|
||||
|
||||
Each composition gets a manifest:
|
||||
|
||||
```yaml
|
||||
# arborist/pi_star/compositions/text-then-claim-lattice-v1.yaml
|
||||
name: text-then-claim-lattice
|
||||
version: v1
|
||||
domain_in: html_or_wikitext
|
||||
domain_out: claim_lattice
|
||||
chain:
|
||||
- wikitext-base@v1
|
||||
- claim-lattice@v1
|
||||
properties:
|
||||
deterministic: true
|
||||
invertible: false
|
||||
equivalence-class-preserving: true
|
||||
```
|
||||
|
||||
Manifest is canonicalized + hashed; the hash becomes the
|
||||
composition's identity in the registry.
|
||||
|
||||
### 4.4 Tests
|
||||
|
||||
- Round-trip: canonicalize twice → byte-equal.
|
||||
- Registry: lookup by name@version is stable; missing key raises.
|
||||
- Composition: re-running `(π*_a ∘ π*_b)(x)` and `π*_a(π*_b(x))`
|
||||
must produce identical bytes.
|
||||
- Existing `wikitext-base-v1` round-trip tests still pass (under
|
||||
re-homed module).
|
||||
|
||||
### 4.5 Bumps
|
||||
|
||||
- No schema bumps. Pure code reorganization + protocol introduction.
|
||||
- No `governance_policy_hash` impact — π* version names are already
|
||||
pinned in policy fields; this ticket just centralizes them.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Implementing `code-py-ast` / `logic-kernel` / `time-series-quantized`
|
||||
/ `tabular-pinned`. Each gets its own ticket once #000015 lands.
|
||||
- Cross-domain anchor projections in code (the algebra is in this
|
||||
ticket's doc; first concrete implementation is its own ticket).
|
||||
- Migration of existing call sites to use `pi_star.registry.lookup(...)`
|
||||
instead of direct imports. Soft migration over time.
|
||||
- Versioning policy for π* deprecation. v1 of this library does not
|
||||
retire any π*; deprecation policy needs its own ticket once v2 of
|
||||
any modality lands.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Registry as global mutable state.** Same risk as Python's
|
||||
dynamic imports. Mitigation: registry is populated at import time
|
||||
only; no runtime mutation. Tests assert registry is read-only
|
||||
after import.
|
||||
- **Composition explosion.** N modalities have N² composition pairs.
|
||||
We commit *named* compositions only — the algebra is general but
|
||||
the registry is curated.
|
||||
- **Cross-domain anchor authentication.** The relation_kind in an
|
||||
anchor is itself canonical content; if relation_kind is just a
|
||||
string, two valid anchors with different intent collide. Mitigate
|
||||
by pinning relation_kind to a controlled vocabulary committed in
|
||||
a sibling registry.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Mid-sized implementation surface:
|
||||
~600 lines of code (registry + protocol + 2 re-homes + stubs +
|
||||
composition module) + ~200 lines of doc.
|
||||
|
||||
Closure criterion: `arborist/pi_star/` lands with re-homed
|
||||
`wikitext-base@v1` + `claim-lattice@v1`, stub modalities raising
|
||||
`NotImplementedError`, composition algebra documented in
|
||||
`docs/pi-star-composition.md` with the text-then-claim-lattice
|
||||
worked example tested.
|
||||
265
docs/tickets/ticket-000016-zk-frontier-proof.md
Normal file
265
docs/tickets/ticket-000016-zk-frontier-proof.md
Normal file
|
|
@ -0,0 +1,265 @@
|
|||
# Ticket #000016 — ZK Phase-2 frontier proof (concretize the hand-wave)
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Build one zero-knowledge proof for one v7-Local affine
|
||||
preactivation frontier on a small model, measure prover time + proof
|
||||
size, and use that data to decide whether ZK is viable for arborist's
|
||||
QA layer or stays out of scope. Replaces v7's hand-waved "swap
|
||||
SHA-256 → Poseidon" with a measured artifact.
|
||||
**Audience:** fox + future ZK adopters.
|
||||
**Hard constraint:** ZK is opt-in. The v7-Local sound path with full
|
||||
local revelation stays the default. If ZK lands, it lives behind a
|
||||
policy flag `governance_policy.frontier_proof_mode ∈ {reveal, zk}`.
|
||||
Cache invariants stay at 8 dims.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
Merkle-AGI v7 § 16.1:
|
||||
|
||||
> Phase-2 ZK Track: replace local integer checks at affine frontiers
|
||||
> with a ZK circuit that proves ‖z_full − z_S‖_p ≤ ε‖z_full‖_p
|
||||
> without revealing a_u(x) or w. Swap SHA-256 for Poseidon/Rescue in
|
||||
> proof-only paths; keep SHA-256 in public commitments.
|
||||
|
||||
That is **not a swap.** Specifically:
|
||||
|
||||
- Poseidon is a different trust model than SHA-256 — newer, less
|
||||
vetted, fewer cryptanalysis-years, higher prover cost per byte
|
||||
hashed than SHA-256 but lower per-constraint inside a SNARK.
|
||||
- Building the actual frontier circuit is non-trivial. An affine
|
||||
preactivation = matrix-vector multiply + bias + norm computation.
|
||||
Inside a ZK circuit, every multiplication and addition costs
|
||||
constraints; circuit size scales with operator size.
|
||||
- Prover time on a frontier-scale layer (e.g., 4096 × 4096 GEMM) on
|
||||
consumer hardware is unmeasured. Could be seconds, could be hours.
|
||||
- Proof size and verification cost downstream are unmeasured.
|
||||
|
||||
Without numbers, "Phase-2 ZK" is a roadmap stub, not a viable path.
|
||||
This ticket commissions the measurement.
|
||||
|
||||
### 1.1 Why this matters for arborist specifically
|
||||
|
||||
arborist's QA layer doesn't currently run model inference — it
|
||||
queries `hermes.ai.unturf.com`. So v7-Local proofs (full local
|
||||
revelation) require that endpoint to expose its activations, which
|
||||
is privacy-hostile. ZK is the *only* path to an answer→model-weights
|
||||
binding in arborist's deployment shape.
|
||||
|
||||
Concretely: if a user asks a question via `arborist ask`, today the
|
||||
audit chain binds the question, the retrieval plan, the context, and
|
||||
the answer. It does *not* bind which model weights produced the
|
||||
answer. v7-Local would require Hermes to publish its activations on
|
||||
every query — leak channel of unbounded size. ZK is the only sound
|
||||
way to bind without leaking.
|
||||
|
||||
If ZK at frontier scale is unaffordable (e.g., minutes-per-frontier
|
||||
prover time), the answer→model binding stays out of scope and v7-Local
|
||||
remains v7's terminal contribution. If ZK is affordable
|
||||
(sub-second prover at typical frontier widths), arborist can fold
|
||||
`model_weights_zk_root` into the providence record.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Proof system
|
||||
|
||||
**A. Halo2 (PLONKish, KZG commitments).** Mature, used by Zcash. No
|
||||
trusted setup beyond ceremony. Decent prover time on commodity
|
||||
hardware.
|
||||
|
||||
**B. Plonky3 (FRI-based, recursive-friendly).** Newer. Quantum-
|
||||
resistant (FRI). Faster prover on small circuits. Recursion supports
|
||||
proof aggregation.
|
||||
|
||||
**C. Groth16.** Smallest proofs (~200 bytes), trusted setup per-
|
||||
circuit. Battle-tested. Expensive to redo per circuit revision.
|
||||
|
||||
**D. STARK (FRI-based, no trusted setup).** Largest proofs but no
|
||||
trusted setup, post-quantum. Plonky3 is in this family.
|
||||
|
||||
**Recommendation:** B (Plonky3). Fast prover on commodity hardware,
|
||||
recursion-friendly for batching across multiple frontiers, no
|
||||
trusted setup, post-quantum hedge. Halo2 is the second choice if
|
||||
Plonky3 maturity proves blocking.
|
||||
|
||||
### 2.2 Hash function inside the circuit
|
||||
|
||||
**A. Poseidon.** SNARK-friendly. ~10-20× cheaper than SHA-256 inside
|
||||
a circuit. Less vetted.
|
||||
|
||||
**B. Rescue / Rescue-Prime.** Similar to Poseidon, slightly newer.
|
||||
|
||||
**C. Native SHA-256 inside circuit.** Most expensive but maximum
|
||||
trust transfer from existing cryptanalysis.
|
||||
|
||||
**Recommendation:** A (Poseidon) inside the circuit, SHA-256 for
|
||||
all public commitments. Document the trust delta explicitly; if
|
||||
Poseidon is later broken, only the proof's binding is affected, not
|
||||
the model-root commitment itself.
|
||||
|
||||
### 2.3 Quantization choice
|
||||
|
||||
The v7-Local kernels run on int64 with 128-bit accumulators. Inside
|
||||
a SNARK, native field arithmetic (~256 bits) gives more headroom
|
||||
than int64. Choices:
|
||||
|
||||
**A. Match v7-Local int64 + bigint accumulators.** Maximum equivalence
|
||||
with non-ZK path; circuit must encode bigint arithmetic.
|
||||
|
||||
**B. Native field arithmetic, with a wider quantization (Δ_zk).**
|
||||
Smaller circuit, but the ZK-mode commitments differ from v7-Local
|
||||
commitments at the same node. Two parallel commitments per node.
|
||||
|
||||
**C. Native field with int64 sub-encoding.** Compromise: int64 values
|
||||
encoded as field elements, arithmetic done in the field, range
|
||||
checks proved.
|
||||
|
||||
**Recommendation:** C. Standard zkML practice. Keeps Δ uniform with
|
||||
v7-Local non-ZK path, manageable circuit size, range-check-friendly.
|
||||
|
||||
### 2.4 Measurement scope (this ticket)
|
||||
|
||||
To produce a usable verdict we need:
|
||||
|
||||
- One affine layer at three sizes: 256×256, 1024×1024, 4096×4096.
|
||||
- Wall-clock prover time on commodity hardware (e.g., Apple M3 Max
|
||||
+ a generic Linux x86 box for cross-platform).
|
||||
- Proof size in bytes.
|
||||
- Verification time in ms.
|
||||
- Memory peak during prover.
|
||||
|
||||
Three sizes give enough data to project to frontier-scale (e.g.,
|
||||
LLaMA-class hidden width). Two platforms catch Apple-Silicon-only
|
||||
optimizations.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Build one Plonky3 circuit for one affine preactivation node
|
||||
(256×256 GEMM + bias + L1 norm + ε-check). Measure prover time +
|
||||
proof size + verify time at sizes {256, 1024, 4096} on two
|
||||
platforms. Report results in `docs/zk-frontier-bench.md`. Use those
|
||||
numbers to either commission ZK as a v8 capability or definitively
|
||||
park it as unaffordable.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Repo layout
|
||||
|
||||
```
|
||||
arborist/zk/
|
||||
├── __init__.py
|
||||
├── circuits/
|
||||
│ └── affine_preact.rs # Plonky3 circuit
|
||||
├── prover.py # Python wrapper (calls Rust binary)
|
||||
├── verifier.py # Python wrapper
|
||||
└── README.md
|
||||
bench/
|
||||
└── zk_frontier_bench.py # the measurement harness
|
||||
```
|
||||
|
||||
Rust crate is required because Plonky3 is Rust-native. Python
|
||||
calls a compiled binary via subprocess. arborist proper stays Python;
|
||||
ZK lives behind a Rust boundary.
|
||||
|
||||
### 4.2 Circuit shape
|
||||
|
||||
For a single affine frontier:
|
||||
|
||||
- Inputs: int64 weights W, int64 activations A, int64 bias B, int64
|
||||
subset-mask S, integer epsilon (numerator/denominator).
|
||||
- Witness (private): full A, full W.
|
||||
- Public outputs: SHA-256 commitments of W and A (re-bound to the
|
||||
v9.8 audit chain), commitment to (S, ε), boolean "ε-inequality
|
||||
holds."
|
||||
- Constraints: range checks on int64 values; field-level GEMM;
|
||||
L1-norm constraint via sum of absolute values; final inequality
|
||||
check.
|
||||
|
||||
### 4.3 Bench harness
|
||||
|
||||
```python
|
||||
# bench/zk_frontier_bench.py
|
||||
sizes = [256, 1024, 4096]
|
||||
platforms = [host_id()] # caller runs on each platform
|
||||
for size in sizes:
|
||||
A = random_int64(size)
|
||||
W = random_int64((size, size))
|
||||
B = random_int64(size)
|
||||
t0 = time.time()
|
||||
proof = prove_affine(W, A, B, S, eps)
|
||||
prove_ms = (time.time() - t0) * 1000
|
||||
t1 = time.time()
|
||||
ok = verify_affine(proof, public_inputs)
|
||||
verify_ms = (time.time() - t1) * 1000
|
||||
record({
|
||||
"size": size,
|
||||
"platform": platforms[-1],
|
||||
"prove_ms": prove_ms,
|
||||
"verify_ms": verify_ms,
|
||||
"proof_bytes": len(proof),
|
||||
"ok": ok,
|
||||
})
|
||||
```
|
||||
|
||||
Results land in `docs/zk-frontier-bench.md` as a table.
|
||||
|
||||
### 4.4 Acceptance thresholds
|
||||
|
||||
For ZK to be commissioned as a v8 capability:
|
||||
|
||||
- Prover ≤ 30 seconds at size 4096 on commodity hardware.
|
||||
- Proof size ≤ 100 KB.
|
||||
- Verify ≤ 100 ms.
|
||||
|
||||
If any threshold fails, ZK stays parked.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Productionization of the ZK path. This ticket measures viability;
|
||||
shipping ZK in `arborist ask` is its own ticket.
|
||||
- Multi-frontier proof aggregation (recursive proofs). If the size-4096
|
||||
number is acceptable, aggregation is a follow-up.
|
||||
- Cross-validator ZK proof exchange. That's part of v8 consensus
|
||||
(ticket #000012).
|
||||
- Integration with Hermes-side activations. If ZK is commissioned,
|
||||
Hermes must expose a circuit-friendly activation export — separate
|
||||
ticket on the Hermes side.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Plonky3 maturity.** Library is young; API stability not
|
||||
guaranteed. Mitigation: pin commit, document.
|
||||
- **Rust toolchain dependency** newly introduced to arborist.
|
||||
Mitigation: ZK lives in `arborist/zk/`, optional dep flagged in
|
||||
`pyproject.toml` extras, default install path stays pure-Python.
|
||||
- **Bench reproducibility.** Wall-clock varies across machines.
|
||||
Mitigation: run twice on each platform; report median + range.
|
||||
- **Negative result.** Most likely outcome at first attempt is
|
||||
"size 4096 takes minutes, ZK is parked." That's a useful result,
|
||||
not a failure — closes the hand-wave.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Implementation requires Rust toolchain
|
||||
introduction. Recommend deferring until a v8 / v7-W ticket creates
|
||||
demand, OR landing as a small standalone repo
|
||||
(`arborist-zk-bench`) so arborist proper stays toolchain-clean
|
||||
unless results justify integration.
|
||||
|
||||
Closure criterion: `docs/zk-frontier-bench.md` exists with measured
|
||||
numbers at three sizes on at least one platform. Verdict line at
|
||||
top: "viable" or "parked" with the threshold check shown.
|
||||
258
docs/tickets/ticket-000017-memory-root-lifelong-learning.md
Normal file
258
docs/tickets/ticket-000017-memory-root-lifelong-learning.md
Normal file
|
|
@ -0,0 +1,258 @@
|
|||
# Ticket #000017 — Memory-root: lifelong learning audit chain
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Spec + initial wire-up of a `memory_root` commitment that
|
||||
binds an arborist-hosted agent's evolving cross-query memory into the
|
||||
audit chain. Distinct from per-query providence cache (which is keyed
|
||||
on cache_key dimensions); memory_root captures lifelong, cross-query
|
||||
state — accepted/rejected belief updates, recurring failure motifs,
|
||||
domain-specific corrections.
|
||||
**Audience:** fox + future blackops shifts.
|
||||
**Hard constraint:** memory_root is hard-hash committed; no soft
|
||||
state in its preimage. Memory falsification cascades MUST NOT
|
||||
invalidate the entire prior providence cache; cascade bounded by
|
||||
explicit policy.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
DNA analogy thread (Asia/Kuala_Lumpur, 2026-05-04) lists the
|
||||
heritable units a Merkle-AGI organism passes to its child:
|
||||
|
||||
> what worked, what failed, why it failed, which benchmark proved
|
||||
> the fix, which policy allowed the fix, which verifier accepted
|
||||
> the fix, which SelfModel update resulted.
|
||||
|
||||
The genome schema includes `memory_root: ...`. arborist has no such
|
||||
object. Today's memory model:
|
||||
|
||||
| Memory layer | Today |
|
||||
|---|---|
|
||||
| Providence cache | Per-cache_key answers; no cross-query continuity. |
|
||||
| Audit events | Per-shard chain; not summarized. |
|
||||
| Concepts table | Per-shard rivalry/synonym graph; static once built. |
|
||||
| SelfModel (ticket #000014) | Identity, not behavior history. |
|
||||
| Long-term failure recall | None. Same failure shape repeats across queries with no surfacing. |
|
||||
|
||||
Without `memory_root`, an agent that hallucinates about spin glass
|
||||
on Monday has no audit-bound memory of that on Tuesday. The
|
||||
falsification record exists in `audit_events` but is not summarized,
|
||||
not surfaced to the verifier on the next related query, and not
|
||||
inheritable when a child shard forks.
|
||||
|
||||
### 1.1 Concrete failure scenarios
|
||||
|
||||
**Scenario A — Repeating a known motif.** Query 1 hallucinates
|
||||
"spin glass cited from Quantum chromodynamics article" → audited as
|
||||
TITLE_MISMATCH → falsification recorded. Query 2 (related topic)
|
||||
runs through the same retrieval path, hits the same chunk, makes a
|
||||
similar mistake. Without memory_root, the verifier has no way to
|
||||
say "this retrieval path produced a falsified claim before; up-
|
||||
weight skepticism."
|
||||
|
||||
**Scenario B — Forked agent loses context.** A new shard splits off
|
||||
with `parent_shard_root` pointing to the existing chain. The new
|
||||
shard has no record of recurring failure motifs unless the audit
|
||||
chain is replayed. memory_root would let the new shard inherit a
|
||||
compact summary without replaying every event.
|
||||
|
||||
**Scenario C — Operator query "what does this agent struggle with?"**
|
||||
Today: no answer. With memory_root + a small CLI surface: a
|
||||
queryable summary backed by audit history.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 What goes in memory_root
|
||||
|
||||
**A. Just a digest of the audit-event chain.** Trivial; redundant
|
||||
with audit chain.
|
||||
|
||||
**B. Curated summaries: failure-motif counts, recurring-error topic
|
||||
clusters, accepted-correction list.** Useful, but how is the
|
||||
curation deterministic?
|
||||
|
||||
**C. Hierarchical: chain-digest at root, multiple branch-digests for
|
||||
domain-specific summaries (e.g., "physics-domain-failures",
|
||||
"language-domain-corrections").** Compositional. Selectively
|
||||
inheritable.
|
||||
|
||||
**Recommendation:** C. Each summary is itself a deterministic
|
||||
projection over a slice of audit_events; the slice rule is
|
||||
committed in the memory-root manifest so verifiers can recompute.
|
||||
|
||||
### 2.2 Update cadence
|
||||
|
||||
**A. Every audit_event triggers re-summary.** High churn; expensive.
|
||||
|
||||
**B. Periodic snapshot (every N events, or every N seconds).**
|
||||
Bounded cost; staleness window.
|
||||
|
||||
**C. On-demand.** Operator triggers re-snapshot. Lowest churn,
|
||||
risk of stale state at critical moments.
|
||||
|
||||
**Recommendation:** B at default cadence (every 1000 audit_events,
|
||||
configurable). Operators can also force-trigger via CLI.
|
||||
|
||||
### 2.3 Falsification cascade scope
|
||||
|
||||
If a memory-summary entry is invalidated (e.g., a previously-
|
||||
accepted correction turns out to be wrong), what gets re-evaluated?
|
||||
|
||||
**A. Nothing. Memory is purely advisory; falsification recorded but
|
||||
no cache impact.** Safest. Loses the "memory shapes future
|
||||
decisions" benefit.
|
||||
|
||||
**B. Only future queries. Existing providence cache stays valid.**
|
||||
Practical. Cache continues to reflect what was known at the time.
|
||||
|
||||
**C. Cascade re-evaluation. Falsification triggers re-evaluation
|
||||
of every providence record that cites the falsified memory entry.**
|
||||
Strongest semantic guarantee. Expensive at scale.
|
||||
|
||||
**Recommendation:** B. Memory falsification flips state on
|
||||
upstream-citing records but doesn't auto-rerun them. Operator can
|
||||
trigger cascade re-evaluation explicitly when they want correctness
|
||||
> efficiency.
|
||||
|
||||
### 2.4 Cache_key folding
|
||||
|
||||
Same options as ticket #000014:
|
||||
|
||||
**A. Fold into governance_policy_hash.** Memory changes invalidate
|
||||
prior cache.
|
||||
|
||||
**B. Sibling field on providence record, not in cache_key.** Cache
|
||||
hits across memory shifts; memory changes are advisory.
|
||||
|
||||
**Recommendation:** B. Memory is advisory by default. Operator can
|
||||
opt into A via `policy["memory_binding"]=True` in deployments
|
||||
where memory changes should invalidate prior answers.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Hierarchical memory-root with periodic snapshot at every-1000-events
|
||||
cadence, advisory-by-default cache impact (sibling field, not in
|
||||
cache_key by default), explicit cascade-re-evaluation only on
|
||||
operator request.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_records (
|
||||
memory_root TEXT PRIMARY KEY,
|
||||
schema_version TEXT NOT NULL,
|
||||
parent_memory_root TEXT,
|
||||
audit_events_high_water TEXT NOT NULL, -- last event_hash included
|
||||
branch_summaries_blob BLOB NOT NULL, -- canonical-JSON of branch digests
|
||||
state TEXT NOT NULL DEFAULT 'live'
|
||||
CHECK(state IN ('live','stale','falsified')),
|
||||
created_at TEXT NOT NULL,
|
||||
falsified_at TEXT,
|
||||
falsified_reason TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE memory_branch_summaries (
|
||||
branch_id TEXT NOT NULL, -- e.g., "failure-motif:title-mismatch"
|
||||
memory_root TEXT NOT NULL,
|
||||
summary_digest TEXT NOT NULL,
|
||||
summary_blob BLOB NOT NULL,
|
||||
count INTEGER NOT NULL,
|
||||
PRIMARY KEY(branch_id, memory_root),
|
||||
FOREIGN KEY(memory_root) REFERENCES memory_records(memory_root)
|
||||
);
|
||||
```
|
||||
|
||||
### 4.2 Module layout
|
||||
|
||||
```
|
||||
arborist/memory/
|
||||
├── __init__.py
|
||||
├── snapshot.py # build memory_root from audit_events slice
|
||||
├── projections.py # branch projection rules (deterministic)
|
||||
├── falsify.py # mark memory entries falsified
|
||||
└── store.py # CRUD
|
||||
```
|
||||
|
||||
### 4.3 Initial branch projections (curated set, expandable)
|
||||
|
||||
- `failure-motif:title-mismatch` — count of TITLE_MISMATCH
|
||||
violations grouped by source title.
|
||||
- `failure-motif:warrant-missing` — count grouped by claim shape.
|
||||
- `failure-motif:deflection` — deflection signals grouped by topic.
|
||||
- `correction:operator-applied` — corrections explicitly logged via
|
||||
CLI.
|
||||
|
||||
Each projection rule lives in `projections.py` and is canonicalized
|
||||
+ hashed; the hash is part of the branch summary's preimage so
|
||||
projection-rule changes produce a different memory_root.
|
||||
|
||||
### 4.4 CLI surface
|
||||
|
||||
```
|
||||
arborist memory snapshot --shards-dir DIR
|
||||
arborist memory show --root ROOT
|
||||
arborist memory branches --root ROOT
|
||||
arborist memory falsify --branch BR --reason TXT
|
||||
arborist memory cascade --memory-root ROOT # re-evaluate dependent records
|
||||
```
|
||||
|
||||
### 4.5 SelfModel relationship
|
||||
|
||||
memory_root and selfmodel_root are sibling commitments. SelfModel
|
||||
is identity (capability claims, profile fingerprint). memory_root
|
||||
is *behavior history* (recurring motifs, accepted corrections). A
|
||||
SelfModel snapshot may *cite* a memory_root in its body (via
|
||||
`memory_root` field) so the audit reader can trace from "agent
|
||||
claims X" to "agent's behavior history when claim was made."
|
||||
|
||||
### 4.6 Tests
|
||||
|
||||
- Unit: snapshot deterministic given fixed audit_events high-water.
|
||||
- Unit: branch projection rule changes produce different memory_root.
|
||||
- Unit: parent_memory_root chain traverses cleanly.
|
||||
- Integration: memory falsification flips state; cascade is no-op
|
||||
unless explicitly invoked.
|
||||
- Audit chain: chain-check stays clean after memory_root landing.
|
||||
|
||||
### 4.7 Bumps
|
||||
|
||||
- New schema (`memory_records`, `memory_branch_summaries`). Additive.
|
||||
- Optional `governance_policy.memory_binding` flag. Default off.
|
||||
- New CLI subcommand `arborist memory`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Cross-shard memory reconciliation. Each shard maintains its own
|
||||
memory_records.
|
||||
- Soft memory (e.g., recent-failure cache, sliding-window
|
||||
popularity counts). Sidecar territory; not in this ticket.
|
||||
- Memory-driven retrieval ("avoid this source — it produced a
|
||||
falsified claim recently"). Future capability layer.
|
||||
- Fork inheritance (child shard inherits memory_root from parent).
|
||||
Mesh-level concern; depends on v8 consensus (ticket #000012).
|
||||
|
||||
---
|
||||
|
||||
## 6. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Smaller surface than SelfModel
|
||||
(ticket #000014); could land standalone or batched with #000014
|
||||
since they reference each other. Recommended sequence: #000014
|
||||
first (identity), then #000017 (history) since SelfModel optionally
|
||||
cites memory_root.
|
||||
|
||||
Closure criterion: schema lands, `arborist memory snapshot`
|
||||
produces a deterministic memory_root, branch projections covered
|
||||
by tests, audit chain stays clean.
|
||||
235
docs/tickets/ticket-000018-soft-hash-covert-channel.md
Normal file
235
docs/tickets/ticket-000018-soft-hash-covert-channel.md
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
# Ticket #000018 — Adversarial soft-hash covert channel analysis
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Formal threat model + falsifier set for whether training
|
||||
gradients can route information from training data through soft hashes
|
||||
to influence hard hashes (committed bytes). Doc-only ticket. Either
|
||||
proves no covert channel exists under stated assumptions, or specifies
|
||||
mitigation. Outcome feeds back into v7 § 9 plastic-training spec.
|
||||
**Audience:** fox + future blackops shifts + downstream Merkle-AGI v7+
|
||||
authors.
|
||||
**Hard constraint:** the v7 separation theorem (T6) MUST hold. If this
|
||||
analysis surfaces a channel, the channel either gets closed by
|
||||
construction (preferred) or gets a measurable bound.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
v7 § 9.9 dismisses "backdoor via soft hash" in two sentences:
|
||||
|
||||
> Backdoor via soft hash: impossible to affect published commitments;
|
||||
> any effective backdoor must appear in Q(Θ) and is thus bound by
|
||||
> C(M).
|
||||
|
||||
This is correct *to first order*: the hard hash preimage doesn't
|
||||
include soft-hash bytes. But training is the **inverse direction** of
|
||||
the hard→soft anchor map:
|
||||
|
||||
```
|
||||
hard hash (frozen at checkpoint t)
|
||||
↓ φ (fixed deterministic embedding)
|
||||
hard-anchor vector
|
||||
↑ L2 loss pulls soft hash toward anchor
|
||||
soft hash (h_v at checkpoint t+1, before re-snapshot)
|
||||
↑
|
||||
plastic loss + task loss
|
||||
↑
|
||||
gradients
|
||||
↑
|
||||
training data
|
||||
```
|
||||
|
||||
The training loop drives **Q(Θ) updates** based on training data. At
|
||||
the next checkpoint, **C(M_{t+1}) is determined by Q(Θ_{t+1})**. The
|
||||
question: can an adversary controlling training data systematically
|
||||
push Q(Θ) toward a target subset such that **C(M_{t+1}) lands on a
|
||||
prearranged value or value-class**?
|
||||
|
||||
If yes: a covert channel from training data → committed model bytes
|
||||
exists, even though soft hashes never enter the hard preimage.
|
||||
|
||||
### 1.1 Threat model
|
||||
|
||||
Three adversary capabilities of increasing strength:
|
||||
|
||||
**T1 — Data poisoner.** Adversary controls a fraction f of training
|
||||
data. Training loop is honest. Cannot directly modify Q(Θ).
|
||||
|
||||
**T2 — Gradient adversary.** Adversary additionally controls a
|
||||
fraction of gradient computations (e.g., compromised data-parallel
|
||||
worker). Honest aggregator.
|
||||
|
||||
**T3 — Hyperparameter adversary.** Adversary controls some learning
|
||||
rate / scheduler decisions in addition to T2.
|
||||
|
||||
For each, the question: under v7 § 9 plastic dual-hash training,
|
||||
can the adversary engineer Q(Θ_{t+1}) to land in a prearranged
|
||||
SHA-256-class (e.g., `C(M_{t+1})` first byte equals 0xAA)?
|
||||
|
||||
### 1.2 Why this isn't trivial
|
||||
|
||||
The naive answer is "Q(Θ) is a function of training data, so trivially
|
||||
training data influences Q(Θ), so trivially influences C(M)." Not
|
||||
useful — that's the whole point of training.
|
||||
|
||||
The interesting question: can the influence be **steered** to a
|
||||
*specific* hash-space target? Hash-space is uniform under SHA-256
|
||||
assumptions; steering to a target requires either:
|
||||
|
||||
- Solving the inverse SHA-256 problem (infeasible under A3).
|
||||
- Iterating through many candidate Q(Θ) values until one lands in
|
||||
the target class (feasible only with grid search; bounded by
|
||||
computational budget).
|
||||
- Exploiting structure in the hard→soft anchor map φ that lets the
|
||||
soft loss feed back into Q(Θ) in a hash-aligned way.
|
||||
|
||||
Option 3 is the live concern. φ in v7 reference (`embed_hard_to_vec`)
|
||||
is "interpret the 32-byte digest as 64×uint32, normalize to [-1,1],
|
||||
random-project to dim_h." The random projection has a fixed seed → it's a
|
||||
fixed deterministic linear map. If that map has any exploitable
|
||||
structure relative to the parameter geometry, gradient updates that
|
||||
push the soft hash toward a target anchor *do* push Q(Θ) in a
|
||||
correlated direction.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Analysis approach
|
||||
|
||||
**A. Information-theoretic.** Bound the channel capacity from
|
||||
training-data → C(M_{t+1}) under each threat model. Likely yields
|
||||
"adversary can move ≤ k bits of C(M) per N training steps with
|
||||
probability p."
|
||||
|
||||
**B. Cryptographic reduction.** Reduce "covert channel exploitable"
|
||||
to "SHA-256 partial-preimage attack" under explicit assumptions
|
||||
about φ. If reduction holds, channel is closed under SHA-256.
|
||||
|
||||
**C. Empirical.** Construct an actual data-poisoning experiment;
|
||||
measure whether a poisoner achieves any control over hash bytes.
|
||||
|
||||
**Recommendation:** B for the formal result (this ticket's
|
||||
deliverable), with C as a follow-up sanity check experiment if B
|
||||
leaves any gap.
|
||||
|
||||
### 2.2 Mitigation candidates (if a channel exists)
|
||||
|
||||
**M1 — Make φ unfriendly.** Replace `embed_hard_to_vec`'s linear
|
||||
projection with a non-linear map (e.g., PRG-based) so gradient feedback
|
||||
through soft hash is uncorrelated with parameter-space updates that
|
||||
would shift hash bytes.
|
||||
|
||||
**M2 — Re-randomize anchors per checkpoint.** φ's seed becomes a
|
||||
per-checkpoint nonce committed in the manifest. Adversary can't pre-
|
||||
plan against the next anchor.
|
||||
|
||||
**M3 — Decouple plastic loss from anchor.** Drop the L2 anchor term
|
||||
entirely and only use task loss + diversity. Loses some training
|
||||
ergonomics; closes the channel by construction.
|
||||
|
||||
**M4 — Plastic-loss budget cap.** Bound the magnitude of soft-hash
|
||||
gradients per step relative to task-loss gradients. Limits the
|
||||
adversary's per-step manipulation budget.
|
||||
|
||||
**Recommendation:** M2 first (cheapest), with M1 as a fallback if
|
||||
the analysis shows residual channel even with re-randomized
|
||||
anchors.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Produce a formal-analysis doc (`docs/soft-hash-channel-analysis.md`)
|
||||
that:
|
||||
|
||||
1. States the threat model precisely.
|
||||
2. Defines the channel capacity formally as I(training-data;
|
||||
C(M_{t+1})) conditional on the public training history.
|
||||
3. Reduces "channel exploitable" to "SHA-256 partial-preimage" under
|
||||
explicit assumptions about φ.
|
||||
4. Lists mitigations and the analysis residual after each.
|
||||
5. Recommends one specific mitigation to fold into v7 § 9 for the
|
||||
next revision.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Doc structure
|
||||
|
||||
```
|
||||
docs/soft-hash-channel-analysis.md
|
||||
├── 1. Threat model (T1, T2, T3)
|
||||
├── 2. v7 § 9 plastic training, recap
|
||||
├── 3. Channel capacity definition
|
||||
├── 4. Reduction to SHA-256 partial-preimage
|
||||
├── 5. Assumptions on φ (the embed_hard_to_vec map)
|
||||
├── 6. Mitigations M1-M4 with residual analysis
|
||||
├── 7. Recommendation
|
||||
└── Appendix — worked example with concrete φ choice
|
||||
```
|
||||
|
||||
### 4.2 Key questions to answer
|
||||
|
||||
- Is `embed_hard_to_vec` (frozen-seed linear projection) friendly
|
||||
to gradient-aligned attacks? Yes/no with proof.
|
||||
- Does periodic re-snapshotting of C(M) every K steps reduce per-
|
||||
attack budget linearly with K? Yes/no with proof.
|
||||
- Does the bigint-accumulator integer-kernel discipline (v7 § 5)
|
||||
reduce the channel's analog leakage to zero, or only bound it?
|
||||
- What's the residual channel if φ is replaced with a PRG?
|
||||
|
||||
### 4.3 Optional follow-up: empirical sanity check
|
||||
|
||||
If the analysis result is "channel closed under SHA-256," a
|
||||
small experiment running v7 § 9 plastic training with deliberate
|
||||
data poisoning targeting a specific hash byte should show:
|
||||
|
||||
- Adversary's success rate is indistinguishable from random (1/256
|
||||
per byte).
|
||||
- Mitigation-applied training shows the same baseline.
|
||||
|
||||
If the empirical result diverges from the analysis prediction, the
|
||||
analysis is wrong. Empirical run is its own follow-up ticket; not
|
||||
in scope here.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Implementing any mitigation in code. This ticket recommends; a
|
||||
follow-up implementation ticket lands the chosen mitigation.
|
||||
- Empirical validation experiment. Sanity-check follow-up ticket.
|
||||
- Attacks against the hard hash itself (SHA-256). Out of scope by
|
||||
A3 assumption.
|
||||
- Side-channel attacks on training infrastructure (compromised CPU,
|
||||
Spectre-class). Different threat model.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Result might be "channel exists but bounded."** Then v7 needs a
|
||||
budget cap (M4) and the bound becomes a published artifact.
|
||||
Acceptable but adds operational complexity.
|
||||
- **Result might be "needs M3 (drop anchor entirely)."** Then v7's
|
||||
plastic-training ergonomics regress; the dual-hash advantage
|
||||
shrinks. Document the trade clearly.
|
||||
- **Time horizon.** This is several weeks of analysis work, not a
|
||||
one-session ticket. Recommendation: park unless v7+ deployment
|
||||
is imminent.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Doc-only deliverable. Recommended
|
||||
deferral until v7 plastic training has a concrete deployment
|
||||
target — analysis without deployment is theoretical hedging.
|
||||
|
||||
Closure criterion: `docs/soft-hash-channel-analysis.md` lands with
|
||||
a stated reduction (proof or counterexample), recommended mitigation,
|
||||
and residual-bound table.
|
||||
199
docs/tickets/ticket-000019-specification-methodology.md
Normal file
199
docs/tickets/ticket-000019-specification-methodology.md
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
# Ticket #000019 — Specification methodology for π* and V
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Doc-only methodology guide for authoring canonical
|
||||
projections (π*) and verifier predicates (V) such that the
|
||||
spec-brittleness bottleneck SQD §13.3 names doesn't surface every
|
||||
time a new domain or corpus lands. Outcome: a checklist + worked
|
||||
template every new π*/V author follows.
|
||||
**Audience:** future authors of new source adapters, verifiers,
|
||||
distillers; future blackops shifts onboarding.
|
||||
**Hard constraint:** methodology must not assume Merkle-AGI v8 or
|
||||
any unbuilt substrate. Must work today against arborist v9.8 + v7
|
||||
substrate as-is.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
SQD §13.3 (Master Internal Whitepaper) names the real bottleneck:
|
||||
|
||||
> Even with perfect optimization, if π* or V encodes the wrong
|
||||
> target, you get verified nonsense. Spec writing/testing must be
|
||||
> first-class.
|
||||
|
||||
arborist already lives this. Examples:
|
||||
|
||||
- The `wikitext-base-v1` canonicalizer evolved through several
|
||||
iterations; each prior iteration was committed and stale records
|
||||
remain. No methodology document explains *why* v1 won over the
|
||||
alternatives.
|
||||
- The verifier's quote/span/entity/paraphrase strategy ordering
|
||||
(`arborist/qa/verify.py`) is the result of bench-driven iteration
|
||||
with no explicit "how to author a new strategy" guide.
|
||||
- Trailing-citation strip (`_strip_trailing_citation`) is gated on
|
||||
citation-cue heuristics. New authors don't know whether to extend
|
||||
this in place or create a sibling strategy.
|
||||
- `governance_policy_hash` folds in 7+ policy fields. Adding an
|
||||
8th has no guidance.
|
||||
|
||||
Without a methodology:
|
||||
|
||||
| Problem | Consequence |
|
||||
|---|---|
|
||||
| Each new π* re-derives the discipline | Inconsistent versioning, accidental cache invalidation. |
|
||||
| V soundness/completeness not measured | Authors ship V without knowing which axis they're on. |
|
||||
| New policy fields land ad-hoc | Some fold into governance_policy_hash, some don't; rule unclear. |
|
||||
| Spec versioning policy unclear | When does v2 retire v1? When does v1 stay forever? |
|
||||
|
||||
---
|
||||
|
||||
## 2. What the methodology must answer
|
||||
|
||||
### 2.1 For π* authors
|
||||
|
||||
1. **Versioning rule.** When does a change require a new version
|
||||
number vs an in-place fix? (Default: any byte-affecting change is
|
||||
a new version.)
|
||||
2. **Round-trip test.** Every π* must satisfy
|
||||
`canonicalize(canonicalize(x)) == canonicalize(x)`. How is this
|
||||
tested?
|
||||
3. **Equivalence class declaration.** What strings does this π*
|
||||
collapse? What does it preserve? Author must declare both
|
||||
explicitly.
|
||||
4. **Policy-fold rule.** When does the π*'s version land in
|
||||
`governance_policy_hash` vs `canonicalization_version` vs
|
||||
`chunking_version`?
|
||||
5. **Deprecation policy.** v1 stays valid until when? How are v1
|
||||
records re-canonicalized to v2 (if they are)?
|
||||
|
||||
### 2.2 For V authors
|
||||
|
||||
1. **Soundness statement.** "V(a)=1 implies <stated semantic
|
||||
property>." Must be explicit.
|
||||
2. **Completeness statement.** What does V *fail* to accept that
|
||||
should be true under the intended semantics? Honest gap
|
||||
declaration.
|
||||
3. **Strategy ordering.** When V has multiple internal strategies
|
||||
(quote, span, entity, paraphrase), why this order? What does
|
||||
reordering invalidate?
|
||||
4. **Falsification surface.** What signal would falsify V?
|
||||
(Bench-suite regression, falsified counter-examples, etc.)
|
||||
5. **Sidecar discipline.** Soft signals never enter V's hard
|
||||
output. Where do they go? Pre-existing rule in CLAUDE.md;
|
||||
methodology codifies the test.
|
||||
|
||||
### 2.3 For policy field authors
|
||||
|
||||
1. **Cache_key impact.** Does this field invalidate prior cache?
|
||||
(If yes, fold into `governance_policy_hash`. If no, sibling
|
||||
field on providence record.)
|
||||
2. **Audit-event emission.** When does setting this field generate
|
||||
an audit event?
|
||||
3. **Default value rule.** Greenfield-acceptable means default
|
||||
matches prior implicit value.
|
||||
4. **Test coverage requirement.** Hash-divergence test (different
|
||||
field values produce different governance_policy_hash) +
|
||||
default-stability test (default value preserves prior cache).
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
A single doc `docs/spec-methodology.md` with:
|
||||
|
||||
1. **The 5 π* questions** above with worked examples drawn from
|
||||
`wikitext-base-v1` and `claim-lattice@v1`.
|
||||
2. **The 5 V questions** with worked examples from
|
||||
`arborist/qa/verify.py`.
|
||||
3. **The 4 policy-field questions** with worked examples from
|
||||
`quantifier_guard_apply_caps`, `quantifier_reminder_enabled`, etc.
|
||||
4. **A new-π* author checklist** (one page; printable).
|
||||
5. **A new-V author checklist** (one page; printable).
|
||||
6. **A new-policy-field checklist** (one page; printable).
|
||||
7. **Bench-discipline cross-reference** to `docs/bench-maxing.md`
|
||||
(since methodology and benching intersect).
|
||||
|
||||
The methodology doc lives in `docs/`, not `docs/tickets/`. It's a
|
||||
reference, not a proposal. This ticket commissions writing it.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Doc outline
|
||||
|
||||
```
|
||||
docs/spec-methodology.md
|
||||
├── 0. Why this doc exists
|
||||
├── 1. Authoring a new π*
|
||||
│ 1.1 Versioning rule
|
||||
│ 1.2 Round-trip test
|
||||
│ 1.3 Equivalence class declaration
|
||||
│ 1.4 Policy-fold rule
|
||||
│ 1.5 Deprecation
|
||||
│ 1.6 Worked example: wikitext-base-v1
|
||||
│ 1.7 Checklist
|
||||
├── 2. Authoring a new V (verifier strategy)
|
||||
│ 2.1 Soundness statement
|
||||
│ 2.2 Completeness statement
|
||||
│ 2.3 Strategy ordering
|
||||
│ 2.4 Falsification surface
|
||||
│ 2.5 Sidecar discipline
|
||||
│ 2.6 Worked example: paraphrase strategy
|
||||
│ 2.7 Checklist
|
||||
├── 3. Authoring a new policy field
|
||||
│ 3.1 Cache_key impact
|
||||
│ 3.2 Audit-event emission
|
||||
│ 3.3 Default value rule
|
||||
│ 3.4 Test coverage requirement
|
||||
│ 3.5 Worked example: quantifier_guard_apply_caps
|
||||
│ 3.6 Checklist
|
||||
├── 4. Cross-references
|
||||
│ 4.1 docs/bench-maxing.md (5pp signal floor)
|
||||
│ 4.2 docs/seven-point-program.md (architectural directives)
|
||||
│ 4.3 CLAUDE.md (hard rules — verifier stays binary, no soft signals)
|
||||
└── 5. Updating this doc itself
|
||||
5.1 When new π*/V/policy categories emerge, add a section.
|
||||
5.2 Worked examples drift; refresh with each major version.
|
||||
```
|
||||
|
||||
### 4.2 Tests
|
||||
|
||||
This is a doc, not code. Tests are:
|
||||
|
||||
- Lint check: every checklist item is grep-able from
|
||||
the respective π*/V/policy-field implementation.
|
||||
- Bench reference: every "5pp signal floor" mention links to
|
||||
`docs/bench-maxing.md`.
|
||||
|
||||
### 4.3 Companion CI hook (optional, follow-up)
|
||||
|
||||
A pre-commit hook that flags new π*/V/policy fields that don't have
|
||||
the methodology checklist filled in. Not in this ticket; future
|
||||
once methodology is stable.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Tooling to auto-generate canonicalizers. Manual authoring stays.
|
||||
- Cross-language methodology (e.g., for π* implemented in Rust for
|
||||
ZK circuits). When ticket #000016 lands, methodology gets a Rust
|
||||
sub-section.
|
||||
- Methodology for source adapters in general. Adapters use π*'s but
|
||||
also have their own concerns (rate limits, caching, etc.).
|
||||
Source-adapter methodology is its own potential ticket.
|
||||
|
||||
---
|
||||
|
||||
## 6. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Pure doc work. Could land
|
||||
opportunistically alongside any new π*/V landing — a "if you're
|
||||
adding one, also document why" companion.
|
||||
|
||||
Closure criterion: `docs/spec-methodology.md` lands with worked
|
||||
examples for all three author classes (π*, V, policy field) and
|
||||
checklists pinned.
|
||||
286
docs/tickets/ticket-000020-capital-cost-ledger.md
Normal file
286
docs/tickets/ticket-000020-capital-cost-ledger.md
Normal file
|
|
@ -0,0 +1,286 @@
|
|||
# Ticket #000020 — Capital-cost ledger: tying verifiable AGI to 8-capital queues
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Schema + audit-event tag for a capital-cost ledger that
|
||||
attributes every state-changing arborist op to one or more of the 8
|
||||
forms of capital (Living, Material, Financial, Intellectual,
|
||||
Experiential, Social, Cultural, Spiritual). Connects "patch the
|
||||
planet" mission to mechanism. Doc + minimal code.
|
||||
**Audience:** fox + future blackops shifts + downstream operators
|
||||
running arborist as part of permacomputer infrastructure.
|
||||
**Hard constraint:** ledger is advisory and additive. Capital-cost
|
||||
records do NOT enter cache_key, do NOT enter audit_events.event_hash
|
||||
preimage. Ledger exists in a sibling table; integrity is per-row, not
|
||||
chained. (Reasoning below.)
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
BLACKOPS shard codifies the 8 forms of capital (Roland & Landua via
|
||||
unturf.com/eight-forms-of-capital):
|
||||
|
||||
| # | Form | Destroyed by |
|
||||
|---|---|---|
|
||||
| 1 | Living | Food deserts, exhausted drivers, poisoned supply chains |
|
||||
| 2 | Material | Planned obsolescence, deferred maintenance |
|
||||
| 3 | Financial | Platform rent extraction, O(N²) transaction tax |
|
||||
| 4 | Intellectual | Paywalls, patent moats, closed algorithms |
|
||||
| 5 | Experiential | Gig economy churn, no apprenticeship |
|
||||
| 6 | Social | Algorithmic isolation, surveillance capitalism |
|
||||
| 7 | Cultural | Monoculture, platform homogenization |
|
||||
| 8 | Spiritual | Infinite scroll, manufactured urgency |
|
||||
|
||||
The shard's stewardship rule:
|
||||
|
||||
> Before shipping any feature, ask: which of our 8 capital queues
|
||||
> does this affect? Does it drain a workaholic to feed a glutton?
|
||||
> Does it route throughput away from a food desert? Does it grow
|
||||
> financial capital at the expense of living capital? If yes —
|
||||
> stop.
|
||||
|
||||
arborist's audit chain commits *what happened* (state changes,
|
||||
falsifications, ε-coverage). It does not commit *what it cost*.
|
||||
Every cache_key, audit_event, ε-proof, ZK proof, distillation pass,
|
||||
re-evaluation has a real cost in compute (financial + material +
|
||||
living capital), storage (material), attention (experiential), and
|
||||
trust (social). None of those are tracked.
|
||||
|
||||
### 1.1 Concrete failure modes today
|
||||
|
||||
- **Disclosure cost invisible.** A v7-Local ε-proof at a frontier-
|
||||
scale layer reveals all activations. The cost-to-the-querier is
|
||||
borne by the operator (storage, network, possibly privacy of the
|
||||
user whose query produced those activations). Not measured.
|
||||
- **Recompute amplification.** A schema bump that invalidates prior
|
||||
cache forces re-running every prior query. Compute cost
|
||||
(financial + material + living) is real; ledger has no record.
|
||||
- **Verifier latency tax.** Adding a verifier strategy slows every
|
||||
query. Slowed query = experiential capital cost (operator wait
|
||||
time). Not surfaced.
|
||||
- **Capital-blind scheduling.** When two ops both need to run, no
|
||||
signal ranks them by total capital cost.
|
||||
|
||||
### 1.2 Why an additive sibling, not a fold
|
||||
|
||||
If capital-cost records entered audit_events.event_hash, then any
|
||||
re-estimation of cost (which is empirical and changes with hardware,
|
||||
energy prices, etc.) would invalidate the audit chain. That's wrong:
|
||||
the audit chain commits what happened factually. Costs are
|
||||
empirical attributions, retroactively re-estimable. Sibling table.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Cost units
|
||||
|
||||
**A. Single normalized unit (e.g., "compute-second-equivalent").**
|
||||
Simple. Forces every capital form into one scale. Loses
|
||||
multi-dimensional truth.
|
||||
|
||||
**B. Per-capital-form units, with conversion table.** Each capital
|
||||
form has its own native unit (kWh for material, USD for financial,
|
||||
person-minutes for living, etc.). Conversion table is published
|
||||
alongside the ledger.
|
||||
|
||||
**C. Per-capital-form units, no conversion.** Eight separate
|
||||
columns. Operators view each independently.
|
||||
|
||||
**Recommendation:** C. Conversion is opinionated and politically
|
||||
charged (USD-per-kWh, person-minute-per-USD). The ledger should
|
||||
preserve dimensional truth; downstream tools convert if needed.
|
||||
|
||||
### 2.2 Cost estimation source
|
||||
|
||||
**A. Hardcoded constants per op type.** Cheap, brittle.
|
||||
|
||||
**B. Measured in real time during the op.** Accurate, expensive
|
||||
(adds instrumentation everywhere).
|
||||
|
||||
**C. Estimated post-hoc from op signature + system telemetry.**
|
||||
Cheaper than B, more accurate than A.
|
||||
|
||||
**Recommendation:** Start with A for rough attribution, evolve to C
|
||||
as instrumentation maturity allows. Document the estimator
|
||||
function so retroactive re-estimation is possible.
|
||||
|
||||
### 2.3 Op categorization
|
||||
|
||||
Every state-changing op gets a `capital_profile` declaration:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class CapitalProfile:
|
||||
living: float = 0.0 # person-minutes equivalent
|
||||
material: float = 0.0 # kWh equivalent
|
||||
financial: float = 0.0 # operator USD equivalent
|
||||
intellectual: float = 0.0 # +/- net contribution
|
||||
experiential: float = 0.0 # operator wait time minutes
|
||||
social: float = 0.0 # +/- trust units (named scale)
|
||||
cultural: float = 0.0 # +/- (named scale)
|
||||
spiritual: float = 0.0 # +/- (named scale)
|
||||
```
|
||||
|
||||
Initial declarations live alongside the op (e.g., `ingest.py` declares
|
||||
its profile, `evict.py` declares its profile). Defaults to all-zero
|
||||
until measured.
|
||||
|
||||
### 2.4 Stewardship-rule check
|
||||
|
||||
The shard's "if yes — stop" rule could be enforced at policy time:
|
||||
|
||||
```yaml
|
||||
governance_policy:
|
||||
stewardship_check:
|
||||
enabled: false # default off; opt-in
|
||||
halt_on:
|
||||
- "drains_workaholic_feeds_glutton"
|
||||
- "routes_away_from_food_desert"
|
||||
- "grows_financial_at_expense_of_living"
|
||||
```
|
||||
|
||||
When enabled, ops that hit any halt condition return a
|
||||
`StewardshipBlocked` error and emit a `stewardship_halt` audit event.
|
||||
The check runs against the predicted capital_profile of the op
|
||||
before execution.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
Land an additive `capital_ledger` table with 8 columns (one per
|
||||
capital form), a `capital_profile` decoration on state-changing op
|
||||
types, a per-op estimator function with hardcoded initial
|
||||
constants, and a `stewardship_halt` audit-event tag (no event hash
|
||||
impact — sibling table). Defer the runtime stewardship-check policy
|
||||
flag to a follow-up; this ticket lays the substrate.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE capital_ledger (
|
||||
ledger_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
audit_event_hash TEXT NOT NULL, -- foreign key to audit_events
|
||||
op_type TEXT NOT NULL,
|
||||
living REAL NOT NULL DEFAULT 0,
|
||||
material REAL NOT NULL DEFAULT 0,
|
||||
financial REAL NOT NULL DEFAULT 0,
|
||||
intellectual REAL NOT NULL DEFAULT 0,
|
||||
experiential REAL NOT NULL DEFAULT 0,
|
||||
social REAL NOT NULL DEFAULT 0,
|
||||
cultural REAL NOT NULL DEFAULT 0,
|
||||
spiritual REAL NOT NULL DEFAULT 0,
|
||||
estimator_version TEXT NOT NULL,
|
||||
estimator_inputs_blob BLOB, -- canonical-JSON of inputs to estimator
|
||||
recorded_at TEXT NOT NULL,
|
||||
FOREIGN KEY(audit_event_hash) REFERENCES audit_events(event_hash)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_capital_ledger_audit ON capital_ledger(audit_event_hash);
|
||||
CREATE INDEX idx_capital_ledger_op ON capital_ledger(op_type);
|
||||
```
|
||||
|
||||
### 4.2 Module layout
|
||||
|
||||
```
|
||||
arborist/capital/
|
||||
├── __init__.py
|
||||
├── profile.py # CapitalProfile dataclass + estimator interface
|
||||
├── estimators/
|
||||
│ ├── ingest.py
|
||||
│ ├── distill.py
|
||||
│ ├── evict.py
|
||||
│ ├── qa.py
|
||||
│ └── ... (one per state-changing op type)
|
||||
└── store.py # ledger CRUD
|
||||
```
|
||||
|
||||
### 4.3 Wire-up
|
||||
|
||||
Each existing state-changing op (`store.append_audit`, `ingest.ingest`,
|
||||
`distill.runner.distill`, `evict.evict_to_cold`, `qa.runner.ask`)
|
||||
gets a one-line addition: after writing the audit_event, call
|
||||
`capital.record(audit_event_hash, op_type, profile)`.
|
||||
|
||||
```python
|
||||
# arborist/store.py — sketch
|
||||
def append_audit(conn, body, capital_profile=None):
|
||||
event = ... # existing append logic
|
||||
if capital_profile is not None:
|
||||
capital.record(conn, event.event_hash, event.op_type, capital_profile)
|
||||
return event
|
||||
```
|
||||
|
||||
`capital_profile` is None-default → existing call sites unchanged
|
||||
unless they pass it. New call sites pass it. Greenfield landing.
|
||||
|
||||
### 4.4 CLI surface
|
||||
|
||||
```
|
||||
arborist capital summary --shards-dir DIR [--since DATE]
|
||||
arborist capital op-cost --op-type ingest
|
||||
arborist capital top --form living --limit 10
|
||||
```
|
||||
|
||||
### 4.5 Tests
|
||||
|
||||
- Unit: ledger row created when capital_profile passed.
|
||||
- Unit: ledger row absent when capital_profile None (backward compat).
|
||||
- Integration: ledger query returns expected per-op-type aggregates
|
||||
on a fresh shard.
|
||||
- Audit chain: `make chain-check-shards` still reports 0 breaks.
|
||||
|
||||
### 4.6 Bumps
|
||||
|
||||
- New schema (`capital_ledger`). Additive.
|
||||
- New CLI subcommand `arborist capital`.
|
||||
- No `cache_key`, `governance_policy_hash`, or audit-chain bumps.
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- Stewardship-halt runtime enforcement (`halt_on` policy
|
||||
evaluation). Follow-up ticket once ledger is populated and op
|
||||
profiles are tuned.
|
||||
- Conversion tables between capital forms. Politically charged;
|
||||
separate discussion.
|
||||
- Real-time instrumentation (option 2.2 B). Hardcoded constants
|
||||
(option 2.2 A) ship first.
|
||||
- Cross-shard / cross-node ledger aggregation. Per-shard for now.
|
||||
- Connection to undefect.com factory model DAG. Separate
|
||||
visualization concern.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Estimator drift.** Hardcoded constants will be wrong in detail.
|
||||
Mitigation: `estimator_version` field; retroactive re-estimation
|
||||
rebuilds ledger rows but keeps audit chain untouched.
|
||||
- **Capital-form units aren't comparable.** Operators may want
|
||||
rolled-up reports. Conversion table is opinionated; defer.
|
||||
- **Capital-blind ops opt out.** A new op author can pass
|
||||
`capital_profile=None` and the ledger stays empty for that op.
|
||||
Acceptable as default; v8 governance can promote
|
||||
`capital_profile` to required if/when stewardship-halt enforcement
|
||||
lands.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Small implementation surface
|
||||
(~300 lines code + ~150 lines doc + schema). One of the more
|
||||
shippable tickets in this batch.
|
||||
|
||||
Closure criterion: `capital_ledger` schema lands, at least three op
|
||||
types (`ingest`, `qa.runner.ask`, `distill.runner.distill`) emit
|
||||
ledger rows on execution, `arborist capital summary` returns
|
||||
non-empty totals after a workload, audit chain stays clean.
|
||||
301
docs/tickets/ticket-000021-bench-fixtures-5s-5t-5r.md
Normal file
301
docs/tickets/ticket-000021-bench-fixtures-5s-5t-5r.md
Normal file
|
|
@ -0,0 +1,301 @@
|
|||
# Ticket #000021 — 5S/5T/5R benchmark fixtures + harness extension
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Implement the SQD whitepaper's named bench batteries
|
||||
(5S Syntax/Semantics/Synthesis/Syllogism/Semiotics, 5T Transfer/
|
||||
Triangulate/Timing/Transitivity/Truth, 5R React/Rearrange/Restore/
|
||||
Replicate/Resonate) as concrete fixtures + harness targets so future
|
||||
tickets can cite measured numbers instead of theoretical bench shapes.
|
||||
Code + fixtures + Makefile target.
|
||||
**Audience:** fox + future bench authors + future v8 / v7-W /
|
||||
SelfModel work that needs concrete fitness targets.
|
||||
**Hard constraint:** fixtures land deterministically (committed JSONL,
|
||||
versioned). Bench-maxing discipline (5pp signal floor, n≥3 runs)
|
||||
applies. Each battery is independently runnable.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement
|
||||
|
||||
SQD §9 names the bench shapes:
|
||||
|
||||
```text
|
||||
5S = Syntax · Semantics · Synthesis · Syllogism · Semiotics
|
||||
5T = Transfer · Triangulate · Timing · Transitivity · Truth
|
||||
5R = React · Rearrange · Restore · Replicate · Resonate
|
||||
```
|
||||
|
||||
These are abstract. arborist has bench harness targets
|
||||
(`make bench-qa`) that test arborist's *own* QA stack on
|
||||
question-answering, not the abstract 5S/5T/5R shapes.
|
||||
|
||||
Without concrete fixtures:
|
||||
|
||||
| Failure | What happens |
|
||||
|---|---|
|
||||
| v8 selection protocol can't reference a fitness target | Proof-of-Upgrade has no canonical floor to non-regress on. |
|
||||
| Cross-validator fitness comparison has no schema | No common bench → no cross-validator agreement. |
|
||||
| New verifier-strategy work has no transfer test | Strategy A wins on training corpus, fails silently on novel domain. |
|
||||
| Self-improvement loop has no objective | Recursive update can't be scored. |
|
||||
|
||||
This ticket converts the SQD names into actual fixtures + a runner.
|
||||
|
||||
### 1.1 What a fixture looks like
|
||||
|
||||
A fixture is a JSONL file with one task per line:
|
||||
|
||||
```jsonl
|
||||
{"id": "5s-syntax-001", "battery": "syntax", "kind": "parse-pass",
|
||||
"input": "...", "expected": "pass" }
|
||||
{"id": "5s-semantics-014", "battery": "semantics", "kind": "paraphrase-stability",
|
||||
"input_a": "...", "input_b": "...", "expected_invariant": true }
|
||||
{"id": "5t-transfer-002", "battery": "transfer", "kind": "domain-shift",
|
||||
"train_corpus": "...", "eval_corpus": "...", "metric": "strict_rate" }
|
||||
```
|
||||
|
||||
The harness reads a fixture file, runs the prescribed test, computes
|
||||
the metric, compares to threshold.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design choices
|
||||
|
||||
### 2.1 Fixture authority: where does each fixture come from?
|
||||
|
||||
**A. Hand-curated from existing arborist bench data.** Cheapest;
|
||||
preserves continuity with `bench/qa_sweep.py`.
|
||||
|
||||
**B. Generated synthetically (e.g., template-based for syntax tests).**
|
||||
Cheap; reproducibility tax (template + seed must be committed).
|
||||
|
||||
**C. External corpora (e.g., Wikipedia, code corpora) downscoped to
|
||||
canonical subsets.** Most realistic; license/distribution complexity.
|
||||
|
||||
**Recommendation:** A for first round (re-use what's there), with
|
||||
B for syntax/semantics where templates are natural, and C as a
|
||||
downstream expansion when interop with external benchmarks
|
||||
matters.
|
||||
|
||||
### 2.2 Fixture versioning
|
||||
|
||||
Each fixture file commits to a version pin:
|
||||
|
||||
```jsonl
|
||||
{"_meta": {"battery": "syntax", "version": "v1",
|
||||
"fixture_digest": "...", "task_count": 142}}
|
||||
```
|
||||
|
||||
Version pin enters `governance_policy_hash` if the bench is gated
|
||||
on policy. Otherwise sibling reference.
|
||||
|
||||
### 2.3 Battery-runner shape
|
||||
|
||||
Each battery runs through a common interface:
|
||||
|
||||
```python
|
||||
class Battery(Protocol):
|
||||
name: str
|
||||
def run(self, fixtures_path: Path, runtime_config) -> BatteryResult:
|
||||
...
|
||||
|
||||
class BatteryResult:
|
||||
pass_count: int
|
||||
fail_count: int
|
||||
metrics: dict[str, float] # per-battery metrics (e.g., strict_rate)
|
||||
per_task: list[TaskResult]
|
||||
fixture_digest: str
|
||||
runtime_digest: str
|
||||
timestamp: str
|
||||
```
|
||||
|
||||
The runner emits `BatteryResult` as JSON on stdout; CI ingests it.
|
||||
|
||||
### 2.4 5R running on workspace state
|
||||
|
||||
The 5R battery is structurally different — it tests workspace
|
||||
operators (React/Rearrange/Restore/Replicate/Resonate) over a
|
||||
running agent's state. Implementation needs:
|
||||
|
||||
- A workspace abstraction the battery can poke (probably mocking
|
||||
arborist's existing providence + audit chain as the workspace).
|
||||
- Each operator (React, etc.) is a verb the battery applies to
|
||||
workspace state and observes the result.
|
||||
|
||||
Recommend implementing 5S and 5T first (they're per-task fixtures);
|
||||
defer 5R until SelfModel (#000014) and memory_root (#000017) land,
|
||||
since 5R tests workspace operators those tickets define.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommendation
|
||||
|
||||
**Phase 1 (this ticket):** 5S + 5T fixture sets + runner.
|
||||
~50-200 fixtures per battery, hand-curated from existing arborist
|
||||
bench corpora plus a small number of synthetic templates.
|
||||
Makefile target `make bench-5s5t`.
|
||||
|
||||
**Phase 2 (follow-up ticket):** 5R battery, gated on #000014 +
|
||||
#000017.
|
||||
|
||||
**Phase 3 (follow-up ticket):** External-corpus expansion (option
|
||||
C above) for cross-substrate transfer testing.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation sketch
|
||||
|
||||
### 4.1 Repo layout
|
||||
|
||||
```
|
||||
bench/fixtures/
|
||||
├── 5s/
|
||||
│ ├── syntax-v1.jsonl
|
||||
│ ├── semantics-v1.jsonl
|
||||
│ ├── synthesis-v1.jsonl
|
||||
│ ├── syllogism-v1.jsonl
|
||||
│ └── semiotics-v1.jsonl
|
||||
├── 5t/
|
||||
│ ├── transfer-v1.jsonl
|
||||
│ ├── triangulate-v1.jsonl
|
||||
│ ├── timing-v1.jsonl
|
||||
│ ├── transitivity-v1.jsonl
|
||||
│ └── truth-v1.jsonl
|
||||
└── README.md # fixture authoring + version pinning policy
|
||||
|
||||
bench/batteries/
|
||||
├── __init__.py
|
||||
├── base.py # Battery protocol + BatteryResult
|
||||
├── b_5s.py # 5S battery runner
|
||||
├── b_5t.py # 5T battery runner
|
||||
└── runner.py # CLI entry point
|
||||
```
|
||||
|
||||
### 4.2 Battery semantics (initial cuts)
|
||||
|
||||
**5S — Syntax.** Each task: a string + a parser ID + expected
|
||||
pass/fail. Metric: parse-pass-rate. Tests arborist's
|
||||
`pi_star.text.canonicalize` (and other π*'s) at the parser layer.
|
||||
|
||||
**5S — Semantics.** Each task: two strings + an expected
|
||||
equivalence-class flag (same/different). Metric: equivalence-
|
||||
recovery-rate. Tests claim-lattice equivalence.
|
||||
|
||||
**5S — Synthesis.** Each task: a goal description + a fact set +
|
||||
expected derivation pass/fail. Metric: derivation-pass-rate. Tests
|
||||
end-to-end `arborist ask` with a constrained source set.
|
||||
|
||||
**5S — Syllogism.** Each task: a proof chain + a step index +
|
||||
expected step-valid pass/fail. Metric: step-validity-rate.
|
||||
|
||||
**5S — Semiotics.** Each task: a controlled label-swap + invariant
|
||||
expected. Metric: invariance-under-swap.
|
||||
|
||||
**5T — Transfer.** Each task: train-on-A, eval-on-B; metric is
|
||||
domain-shift drop. Cross-corpus.
|
||||
|
||||
**5T — Triangulate.** Each task: a set of independent verifiers;
|
||||
metric is conjunction-failure-rate vs single-verifier rate.
|
||||
|
||||
**5T — Timing.** Each task: a chronological sequence with a
|
||||
reorder; metric is reorder-detection-rate.
|
||||
|
||||
**5T — Transitivity.** Each task: a multi-step entailment; metric
|
||||
is full-chain-pass-rate.
|
||||
|
||||
**5T — Truth.** Each task: a verifier-acceptance recorded against
|
||||
disagreement log; metric is verifier-truth-rate.
|
||||
|
||||
### 4.3 Fixture sourcing (Phase 1)
|
||||
|
||||
- 5S Syntax: synthetic templates via `bench/fixtures/_templates/`
|
||||
with committed seed.
|
||||
- 5S Semantics: hand-curated from arborist's existing claim-lattice
|
||||
test fixtures.
|
||||
- 5S Synthesis: hand-curated from `bench/qa_sweep.py` corpora.
|
||||
- 5S Syllogism: hand-curated from `arborist/qa/parse_claims.py`
|
||||
test cases.
|
||||
- 5S Semiotics: hand-curated label-swap pairs.
|
||||
- 5T Transfer: cross-shard splits using existing wiki shards.
|
||||
- 5T Triangulate: existing verifier-strategy outputs replayed.
|
||||
- 5T Timing: small hand-curated chronology set.
|
||||
- 5T Transitivity: hand-curated multi-step claims.
|
||||
- 5T Truth: replays of existing audit_events with declared
|
||||
verdict.
|
||||
|
||||
### 4.4 Makefile
|
||||
|
||||
```makefile
|
||||
bench-5s: ## Run 5S battery
|
||||
$(VENV)/bin/python -m bench.batteries.runner \
|
||||
--battery 5s \
|
||||
--fixtures bench/fixtures/5s \
|
||||
--shards-dir $(SHARDS_DIR) \
|
||||
--out bench/results/5s-$(shell date -u +%Y%m%dT%H%M%SZ).json
|
||||
|
||||
bench-5t: ## Run 5T battery
|
||||
$(VENV)/bin/python -m bench.batteries.runner \
|
||||
--battery 5t \
|
||||
--fixtures bench/fixtures/5t \
|
||||
--shards-dir $(SHARDS_DIR) \
|
||||
--out bench/results/5t-$(shell date -u +%Y%m%dT%H%M%SZ).json
|
||||
|
||||
bench-5s5t: bench-5s bench-5t
|
||||
```
|
||||
|
||||
### 4.5 Tests
|
||||
|
||||
- Unit: each battery runner accepts a tiny fixture set and produces
|
||||
`BatteryResult`.
|
||||
- Integration: `make bench-5s` against checked-in fixtures runs end
|
||||
to end and writes a result file.
|
||||
- Determinism: same fixture set + same shard set + same seed →
|
||||
identical metric values across runs.
|
||||
- Bench discipline: 5pp signal floor honored; n≥3 runs averaged
|
||||
before declaring a metric.
|
||||
|
||||
### 4.6 Bumps
|
||||
|
||||
- New repo subtree (`bench/fixtures/`, `bench/batteries/`).
|
||||
- New Makefile targets.
|
||||
- No schema changes.
|
||||
- No `governance_policy_hash` changes (battery results are sibling
|
||||
artifacts; folding fixture digests into governance is a v8
|
||||
concern — ticket #000012).
|
||||
|
||||
---
|
||||
|
||||
## 5. Out of scope
|
||||
|
||||
- 5R battery. Deferred to Phase 2 (depends on SelfModel + memory_root).
|
||||
- External-corpus expansion. Deferred to Phase 3.
|
||||
- Continuous-integration scheduling. CI cron is its own concern.
|
||||
- Cross-validator fitness comparison protocol. Part of v8 (ticket
|
||||
#000012).
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks & open questions
|
||||
|
||||
- **Fixture quality.** Hand-curated fixtures bias toward existing
|
||||
bench corpora. Mitigation: explicit version pinning; v2 of each
|
||||
battery can land later with broader sourcing.
|
||||
- **Coverage gaps.** 5S/5T as named in SQD aren't exhaustive of
|
||||
what AGI evaluation needs. Treat as foundation, not ceiling.
|
||||
- **Determinism vs realism.** Hand-curated fixtures may overfit to
|
||||
arborist's current behavior. Cross-validator regression on novel
|
||||
fixtures (Phase 3) is the real test.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Mid-sized implementation surface
|
||||
(~600 lines code + 200-1000 fixtures). Recommended Phase 1 scope
|
||||
keeps it shippable in a single session for a focused operator;
|
||||
Phase 2/3 are future tickets.
|
||||
|
||||
Closure criterion: `bench/fixtures/5s/`, `bench/fixtures/5t/`,
|
||||
`bench/batteries/{b_5s,b_5t,runner}.py` land; `make bench-5s5t`
|
||||
runs end-to-end against the existing shards and produces a JSON
|
||||
result file with all 10 battery slots populated.
|
||||
Loading…
Add table
Add a link
Reference in a new issue