arborist/docs/zk-wire-protocol.md

ZK wire protocol — arborist consumer schema

Companion to: docs/zk-frontier-bench.md (the bench plan + parked verdict for #000016). Status: draft v0, 2026-05-09.

This document specifies the wire format arborist consumes when ZK proofs become available. The toolchain producing those proofs lives in a sibling repo (arborist-zk-bench); arborist itself stays pure-Python and never gains a Rust dependency. The language constraint from #000016 §1.21 is hard — this protocol is the bridge.

---

1. Scope

What this protocol covers:

• Canonical-byte JSON schema for one frontier-proof artifact.
• The minimum binding info arborist needs to fold a proof into the v9.8 audit chain.
• Versioning + forward-compatibility rules.
• The trigger condition (governance_policy.frontier_proof_mode) for when arborist asks for a proof.

What this protocol does NOT cover:

• The prover or verifier itself (sibling repo).
• Plonky3 / Halo2 / Groth16 internals.
• Circuit construction.
• ZK-side privacy analysis.

2. Trigger condition

ZK proofs are opt-in via a governance-policy flag:

governance_policy["frontier_proof_mode"] = "zk"   # opt-in
governance_policy["frontier_proof_mode"] = "reveal"  # default

The flag folds into governance_policy_hash so the v9.8 8-dim cache_key reflects which mode produced any given record. Switching modes invalidates prior records on lookup.

When mode == "zk", arborist's QA path (or any caller producing a frontier-proof binding) calls out to the sibling repo, gets back a wire-protocol artifact, validates it, and folds the proof root into the providence record.

When mode == "reveal" (default), no ZK call happens; activations are revealed locally per v7-Local.

3. Artifact shape (proof_v1.json)

{
  "schema_version": "arborist-zk-proof-v1",
  "proof_system": "plonky3 | halo2 | groth16 | stark",
  "proof_system_version": "<vendor-specific git sha or release tag>",
  "circuit_id": "<sha256 of canonical circuit definition>",
  "circuit_kind": "affine_preact" ,
  "frontier_node": {
    "node_kind": "affine_preact",
    "size": 4096,
    "epsilon_num": 1,
    "epsilon_den": 100
  },
  "public_inputs": {
    "weights_commitment_sha256": "<64-hex>",
    "activations_commitment_sha256": "<64-hex>",
    "subset_mask_sha256": "<64-hex>",
    "epsilon_pair": [1, 100]
  },
  "public_outputs": {
    "epsilon_inequality_holds": true
  },
  "proof_bytes_b64": "<standard-base64 of the raw proof bytes>",
  "verifier_setup_id": "<sha256 of the verifier's KZG/CRS or FRI setup>",
  "issued_at": "<ISO-8601 UTC>",
  "issuer_pubkey_ed25519": "<32-byte hex>",
  "issuer_signature_ed25519": "<64-byte hex over canonical-bytes(this object)>"
}

3.1 Field-by-field

• schema_version — fixed string "arborist-zk-proof-v1". Arborist rejects any artifact with an unknown schema_version; v2 onward bumps the validator.
• proof_system — vendor name. Determines which deserializer (in the sibling repo) parses proof_bytes_b64.
• circuit_id — sha256 of the canonical circuit definition. Two artifacts produced by the same circuit have the same circuit_id; this is what arborist binds to in the audit chain rather than the raw proof bytes (which vary per witness).
• frontier_node — describes WHICH node was proved. arborist matches this against the model snapshot's frontier-node enumeration to confirm the proof's scope.
• public_inputs — the inputs the verifier checks. arborist recomputes weights_commitment_sha256 from its own copy of the weight bytes, activations_commitment_sha256 from the published activation chain, and confirms equality before consuming the proof.
• public_outputs.epsilon_inequality_holds — the boolean the prover claims is true, which the verifier checks.
• proof_bytes_b64 — the actual proof. Arborist does NOT parse this; it forwards to the sibling-repo verifier (or a separate trusted binary).
• verifier_setup_id — sha256 of the verifier's setup (KZG CRS / FRI parameters). Pinned per-circuit; rotation is a circuit bump, which changes circuit_id.
• issued_at — wall-clock UTC, sanity-check only.
• issuer_pubkey_ed25519 + issuer_signature_ed25519 — the prover signs the canonical-byte form of this artifact (with the signature field omitted) so arborist can verify provenance of the proof artifact independently of the ZK proof itself.

3.2 Canonical-byte form

For SHA-256 commitment + signature purposes, the canonical bytes of this artifact are: JSON with sorted keys, separators=(",",":"), no whitespace, UTF-8 encoded, EXCLUDING the issuer_signature_ed25519 field. Standard JSON-canonicalization the rest of arborist uses (_canonical_json in arborist.qa.dag).

4. Binding into the v9.8 audit chain

When arborist consumes a valid proof, it folds the proof's identity into the providence record:

ALTER TABLE providence_cache ADD COLUMN model_weights_zk_root TEXT;
ALTER TABLE providence_cache ADD COLUMN frontier_proof_circuit_id TEXT;

(Schema migration would land at the integration ticket, not this doc. Sketched here so the binding shape is explicit.)

The new fields:

• model_weights_zk_root — SHA-256 of the artifact's canonical bytes (excluding signature). Becomes part of the audit-event body for providence_query rows produced under frontier_proof_mode = "zk".
• frontier_proof_circuit_id — the artifact's circuit_id, exposed for cross-record queries ("show me all rows that used circuit X").

governance_policy_hash folds in the frontier_proof_mode field so cache_key changes when the mode flips. Records produced under reveal and zk for the same question land at distinct cache_keys.

5. Validation pipeline (arborist side)

When arborist receives an artifact:

def validate_zk_proof(artifact: dict) -> None:
    """Pure-Python validation. NEVER runs the ZK verifier;
    delegates that to the sibling-repo verifier binary."""
    # 1. Schema check.
    if artifact.get("schema_version") != "arborist-zk-proof-v1":
        raise PiStarError("unknown schema_version")
    for required in (
        "proof_system", "circuit_id", "circuit_kind",
        "frontier_node", "public_inputs", "public_outputs",
        "proof_bytes_b64", "verifier_setup_id", "issued_at",
        "issuer_pubkey_ed25519", "issuer_signature_ed25519",
    ):
        if required not in artifact:
            raise PiStarError(f"missing field: {required!r}")

    # 2. Recompute commitment sanity.
    pub = artifact["public_inputs"]
    if not _is_hex_64(pub["weights_commitment_sha256"]):
        raise PiStarError("bad weights commitment")
    # … similar for activations + subset_mask

    # 3. Issuer signature check (Ed25519).
    canonical = _canonical_json_no_signature(artifact)
    if not _ed25519_verify(
        artifact["issuer_pubkey_ed25519"],
        canonical,
        artifact["issuer_signature_ed25519"],
    ):
        raise PiStarError("issuer signature failed")

    # 4. ZK verification — DELEGATED.
    # This is where arborist hands off to the sibling-repo verifier
    # binary or a trusted external process. Arborist itself does
    # NOT run the ZK verifier.
    if not _delegate_zk_verify(artifact):
        raise PiStarError("zk verifier rejected proof")

    # 5. Bind into audit chain.
    # Caller adds model_weights_zk_root + frontier_proof_circuit_id
    # to the providence row.

_delegate_zk_verify is the wire-protocol's other side: arborist either spawns a subprocess running the sibling verifier OR posts to a known endpoint. Either way, arborist sees only the boolean verdict.

6. Versioning + forward compatibility

• Schema bumps require a new schema_version string. v1 → v2 drops a field, adds one, or changes semantics.
• arborist supports a fixed set of schema versions at any time; unknown versions are rejected loudly.
• The sibling repo's git sha pins which prover/verifier produced the proof. arborist surfaces this in the audit body so a future replay knows which sibling-repo version to invoke.
• circuit_id is content-addressed; circuit changes always produce a new circuit_id, never reuse.

7. Threat model

• Compromised sibling-repo prover. A compromised prover can forge epsilon_inequality_holds = true only if it can forge the proof bytes themselves AND the issuer signature. Arborist's defenses: Ed25519 signature check on the artifact (step 3), delegated verifier invocation (step 4). The verifier is the trust anchor; if it's compromised, the chain breaks upstream of arborist.
• Wire-format injection. Adversary submits malformed JSON hoping to bypass schema validation. Mitigation: every required field checked; type checks on commitment hex; signature check before delegating.
• Proof replay. Adversary replays a valid proof for a different question. Mitigation: circuit_id and frontier_node are tied to the model snapshot's frontier enumeration; arborist binds the proof to a specific (cache_key, frontier_node) pair before storage. Replay against a different cache_key fails the binding check.
• Issuer-key compromise. Adversary obtains the prover's signing key. Mitigation: keys rotate per major-version; arborist maintains a published list of trusted issuer keys, rejects artifacts signed under revoked keys.

8. What arborist gains over time

If ZK turns out VIABLE per the bench (docs/zk-frontier-bench.md), arborist gains:

• An [zk] extra in pyproject.toml pulling JSON-schema validator + Ed25519 verification (NOT the prover/verifier binary).
• An arborist/zk/wire.py module implementing validate_zk_proof() plus _delegate_zk_verify() calling the sibling-repo verifier.
• A schema migration adding model_weights_zk_root and frontier_proof_circuit_id columns to providence_cache, plus a CHECK constraint update.
• A new policy field frontier_proof_mode folded into governance_policy_hash.
• New audit-event type providence_zk_proof for the binding event.

If ZK stays UNAFFORDABLE, this protocol stays as a documented artifact — when ZK matures (hardware, libraries, theory), the arborist side is ready.

9. Cross-references

• docs/zk-frontier-bench.md — bench plan + parked verdict.
• #000016 — the ticket this closes the arborist-side hand-wave for.
• #000018 — soft-hash covert-channel analysis; the M3 mitigation there is the only existing defense if ZK proves UNAFFORDABLE.
• #000013 — v7-W spatial-temporal substrate; world-state observations have the same privacy concerns and inherit this protocol.
• v7 § 16.1 — the original "swap SHA-256 → Poseidon" hand-wave.

10. Status

Draft v0. Becomes the consumer-side schema spec when the sibling repo lands measurements + the validator. Until then: this is the contract arborist commits to honor on the receive side.