arborist/docs/zk-wire-protocol.md

# ZK wire protocol — arborist consumer schema

**Companion to:** `docs/zk-frontier-bench.md` (the bench plan +
parked verdict for #000016).
**Status:** draft v0, 2026-05-09.

This document specifies the **wire format** arborist consumes
when ZK proofs become available. The toolchain producing those
proofs lives in a sibling repo (`arborist-zk-bench`); arborist
itself stays pure-Python and never gains a Rust dependency. The
language constraint from #000016 §1.21 is hard — this protocol
is the bridge.

---

## 1. Scope

**What this protocol covers:**

- Canonical-byte JSON schema for one frontier-proof artifact.
- The minimum binding info arborist needs to fold a proof into
  the v9.8 audit chain.
- Versioning + forward-compatibility rules.
- The trigger condition (`governance_policy.frontier_proof_mode`)
  for when arborist asks for a proof.

**What this protocol does NOT cover:**

- The prover or verifier itself (sibling repo).
- Plonky3 / Halo2 / Groth16 internals.
- Circuit construction.
- ZK-side privacy analysis.

## 2. Trigger condition

ZK proofs are opt-in via a governance-policy flag:

```python
governance_policy["frontier_proof_mode"] = "zk"   # opt-in
governance_policy["frontier_proof_mode"] = "reveal"  # default
```

The flag folds into `governance_policy_hash` so the v9.8
8-dim cache_key reflects which mode produced any given record.
Switching modes invalidates prior records on lookup.

When `mode == "zk"`, arborist's QA path (or any caller producing
a frontier-proof binding) calls out to the sibling repo, gets
back a wire-protocol artifact, validates it, and folds the proof
root into the providence record.

When `mode == "reveal"` (default), no ZK call happens; activations
are revealed locally per v7-Local.

## 3. Artifact shape (`proof_v1.json`)

```json
{
  "schema_version": "arborist-zk-proof-v1",
  "proof_system": "plonky3 | halo2 | groth16 | stark",
  "proof_system_version": "<vendor-specific git sha or release tag>",
  "circuit_id": "<sha256 of canonical circuit definition>",
  "circuit_kind": "affine_preact" ,
  "frontier_node": {
    "node_kind": "affine_preact",
    "size": 4096,
    "epsilon_num": 1,
    "epsilon_den": 100
  },
  "public_inputs": {
    "weights_commitment_sha256": "<64-hex>",
    "activations_commitment_sha256": "<64-hex>",
    "subset_mask_sha256": "<64-hex>",
    "epsilon_pair": [1, 100]
  },
  "public_outputs": {
    "epsilon_inequality_holds": true
  },
  "proof_bytes_b64": "<standard-base64 of the raw proof bytes>",
  "verifier_setup_id": "<sha256 of the verifier's KZG/CRS or FRI setup>",
  "issued_at": "<ISO-8601 UTC>",
  "issuer_pubkey_ed25519": "<32-byte hex>",
  "issuer_signature_ed25519": "<64-byte hex over canonical-bytes(this object)>"
}
```

### 3.1 Field-by-field

- **`schema_version`** — fixed string `"arborist-zk-proof-v1"`. Arborist
  rejects any artifact with an unknown schema_version; v2 onward bumps
  the validator.
- **`proof_system`** — vendor name. Determines which deserializer
  (in the sibling repo) parses `proof_bytes_b64`.
- **`circuit_id`** — sha256 of the canonical circuit definition. Two
  artifacts produced by the same circuit have the same `circuit_id`;
  this is what arborist binds to in the audit chain rather than the
  raw proof bytes (which vary per witness).
- **`frontier_node`** — describes WHICH node was proved. arborist
  matches this against the model snapshot's frontier-node enumeration
  to confirm the proof's scope.
- **`public_inputs`** — the inputs the verifier checks. arborist
  recomputes `weights_commitment_sha256` from its own copy of the
  weight bytes, `activations_commitment_sha256` from the published
  activation chain, and confirms equality before consuming the
  proof.
- **`public_outputs.epsilon_inequality_holds`** — the boolean the
  prover claims is true, which the verifier checks.
- **`proof_bytes_b64`** — the actual proof. Arborist does NOT parse
  this; it forwards to the sibling-repo verifier (or a separate
  trusted binary).
- **`verifier_setup_id`** — sha256 of the verifier's setup (KZG
  CRS / FRI parameters). Pinned per-circuit; rotation is a circuit
  bump, which changes `circuit_id`.
- **`issued_at`** — wall-clock UTC, sanity-check only.
- **`issuer_pubkey_ed25519` + `issuer_signature_ed25519`** — the
  prover signs the canonical-byte form of this artifact (with the
  signature field omitted) so arborist can verify provenance of
  the proof artifact independently of the ZK proof itself.

### 3.2 Canonical-byte form

For SHA-256 commitment + signature purposes, the canonical bytes
of this artifact are: JSON with sorted keys, separators=(",",":"),
no whitespace, UTF-8 encoded, EXCLUDING the
`issuer_signature_ed25519` field. Standard JSON-canonicalization
the rest of arborist uses (`_canonical_json` in `arborist.qa.dag`).

## 4. Binding into the v9.8 audit chain

When arborist consumes a valid proof, it folds the proof's
identity into the providence record:

```sql
ALTER TABLE providence_cache ADD COLUMN model_weights_zk_root TEXT;
ALTER TABLE providence_cache ADD COLUMN frontier_proof_circuit_id TEXT;
```

(Schema migration would land at the integration ticket, not
this doc. Sketched here so the binding shape is explicit.)

The new fields:

- `model_weights_zk_root` — SHA-256 of the artifact's canonical
  bytes (excluding signature). Becomes part of the audit-event body
  for `providence_query` rows produced under `frontier_proof_mode
  = "zk"`.
- `frontier_proof_circuit_id` — the artifact's `circuit_id`, exposed
  for cross-record queries ("show me all rows that used circuit X").

`governance_policy_hash` folds in the `frontier_proof_mode` field
so cache_key changes when the mode flips. Records produced under
`reveal` and `zk` for the same question land at distinct cache_keys.

## 5. Validation pipeline (arborist side)

When arborist receives an artifact:

```python
def validate_zk_proof(artifact: dict) -> None:
    """Pure-Python validation. NEVER runs the ZK verifier;
    delegates that to the sibling-repo verifier binary."""
    # 1. Schema check.
    if artifact.get("schema_version") != "arborist-zk-proof-v1":
        raise PiStarError("unknown schema_version")
    for required in (
        "proof_system", "circuit_id", "circuit_kind",
        "frontier_node", "public_inputs", "public_outputs",
        "proof_bytes_b64", "verifier_setup_id", "issued_at",
        "issuer_pubkey_ed25519", "issuer_signature_ed25519",
    ):
        if required not in artifact:
            raise PiStarError(f"missing field: {required!r}")

    # 2. Recompute commitment sanity.
    pub = artifact["public_inputs"]
    if not _is_hex_64(pub["weights_commitment_sha256"]):
        raise PiStarError("bad weights commitment")
    # … similar for activations + subset_mask

    # 3. Issuer signature check (Ed25519).
    canonical = _canonical_json_no_signature(artifact)
    if not _ed25519_verify(
        artifact["issuer_pubkey_ed25519"],
        canonical,
        artifact["issuer_signature_ed25519"],
    ):
        raise PiStarError("issuer signature failed")

    # 4. ZK verification — DELEGATED.
    # This is where arborist hands off to the sibling-repo verifier
    # binary or a trusted external process. Arborist itself does
    # NOT run the ZK verifier.
    if not _delegate_zk_verify(artifact):
        raise PiStarError("zk verifier rejected proof")

    # 5. Bind into audit chain.
    # Caller adds model_weights_zk_root + frontier_proof_circuit_id
    # to the providence row.
```

`_delegate_zk_verify` is the wire-protocol's other side: arborist
either spawns a subprocess running the sibling verifier OR posts
to a known endpoint. Either way, arborist sees only the boolean
verdict.

## 6. Versioning + forward compatibility

- Schema bumps require a new `schema_version` string. v1 → v2
  drops a field, adds one, or changes semantics.
- arborist supports a fixed set of schema versions at any time;
  unknown versions are rejected loudly.
- The sibling repo's git sha pins which prover/verifier produced
  the proof. arborist surfaces this in the audit body so a
  future replay knows which sibling-repo version to invoke.
- `circuit_id` is content-addressed; circuit changes always
  produce a new `circuit_id`, never reuse.

## 7. Threat model

- **Compromised sibling-repo prover.** A compromised prover
  can forge `epsilon_inequality_holds = true` only if it can
  forge the proof bytes themselves AND the issuer signature.
  Arborist's defenses: Ed25519 signature check on the artifact
  (step 3), delegated verifier invocation (step 4). The verifier
  is the trust anchor; if it's compromised, the chain breaks
  upstream of arborist.
- **Wire-format injection.** Adversary submits malformed JSON
  hoping to bypass schema validation. Mitigation: every required
  field checked; type checks on commitment hex; signature check
  before delegating.
- **Proof replay.** Adversary replays a valid proof for a
  different question. Mitigation: `circuit_id` and `frontier_node`
  are tied to the model snapshot's frontier enumeration;
  arborist binds the proof to a specific (cache_key,
  frontier_node) pair before storage. Replay against a different
  cache_key fails the binding check.
- **Issuer-key compromise.** Adversary obtains the prover's
  signing key. Mitigation: keys rotate per major-version;
  arborist maintains a published list of trusted issuer keys,
  rejects artifacts signed under revoked keys.

## 8. What arborist gains over time

If ZK turns out VIABLE per the bench (`docs/zk-frontier-bench.md`),
arborist gains:

- An `[zk]` extra in `pyproject.toml` pulling JSON-schema
  validator + Ed25519 verification (NOT the prover/verifier
  binary).
- An `arborist/zk/wire.py` module implementing
  `validate_zk_proof()` plus `_delegate_zk_verify()` calling
  the sibling-repo verifier.
- A schema migration adding `model_weights_zk_root` and
  `frontier_proof_circuit_id` columns to `providence_cache`,
  plus a CHECK constraint update.
- A new policy field `frontier_proof_mode` folded into
  `governance_policy_hash`.
- New audit-event type `providence_zk_proof` for the binding
  event.

If ZK stays UNAFFORDABLE, this protocol stays as a documented
artifact — when ZK matures (hardware, libraries, theory), the
arborist side is ready.

## 9. Cross-references

- `docs/zk-frontier-bench.md` — bench plan + parked verdict.
- #000016 — the ticket this closes the arborist-side
  hand-wave for.
- #000018 — soft-hash covert-channel analysis; the M3 mitigation
  there is the only existing defense if ZK proves UNAFFORDABLE.
- #000013 — v7-W spatial-temporal substrate; world-state
  observations have the same privacy concerns and inherit
  this protocol.
- v7 § 16.1 — the original "swap SHA-256 → Poseidon" hand-wave.

## 10. Status

Draft v0. Becomes the consumer-side schema spec when the sibling
repo lands measurements + the validator. Until then: this is the
contract arborist commits to honor on the receive side.