# ZK wire protocol — arborist consumer schema **Companion to:** `docs/zk-frontier-bench.md` (the bench plan + parked verdict for #000016). **Status:** draft v0, 2026-05-09. This document specifies the **wire format** arborist consumes when ZK proofs become available. The toolchain producing those proofs lives in a sibling repo (`arborist-zk-bench`); arborist itself stays pure-Python and never gains a Rust dependency. The language constraint from #000016 §1.21 is hard — this protocol is the bridge. --- ## 1. Scope **What this protocol covers:** - Canonical-byte JSON schema for one frontier-proof artifact. - The minimum binding info arborist needs to fold a proof into the v9.8 audit chain. - Versioning + forward-compatibility rules. - The trigger condition (`governance_policy.frontier_proof_mode`) for when arborist asks for a proof. **What this protocol does NOT cover:** - The prover or verifier itself (sibling repo). - Plonky3 / Halo2 / Groth16 internals. - Circuit construction. - ZK-side privacy analysis. ## 2. Trigger condition ZK proofs are opt-in via a governance-policy flag: ```python governance_policy["frontier_proof_mode"] = "zk" # opt-in governance_policy["frontier_proof_mode"] = "reveal" # default ``` The flag folds into `governance_policy_hash` so the v9.8 8-dim cache_key reflects which mode produced any given record. Switching modes invalidates prior records on lookup. When `mode == "zk"`, arborist's QA path (or any caller producing a frontier-proof binding) calls out to the sibling repo, gets back a wire-protocol artifact, validates it, and folds the proof root into the providence record. When `mode == "reveal"` (default), no ZK call happens; activations are revealed locally per v7-Local. ## 3. Artifact shape (`proof_v1.json`) ```json { "schema_version": "arborist-zk-proof-v1", "proof_system": "plonky3 | halo2 | groth16 | stark", "proof_system_version": "", "circuit_id": "", "circuit_kind": "affine_preact" , "frontier_node": { "node_kind": "affine_preact", "size": 4096, "epsilon_num": 1, "epsilon_den": 100 }, "public_inputs": { "weights_commitment_sha256": "<64-hex>", "activations_commitment_sha256": "<64-hex>", "subset_mask_sha256": "<64-hex>", "epsilon_pair": [1, 100] }, "public_outputs": { "epsilon_inequality_holds": true }, "proof_bytes_b64": "", "verifier_setup_id": "", "issued_at": "", "issuer_pubkey_ed25519": "<32-byte hex>", "issuer_signature_ed25519": "<64-byte hex over canonical-bytes(this object)>" } ``` ### 3.1 Field-by-field - **`schema_version`** — fixed string `"arborist-zk-proof-v1"`. Arborist rejects any artifact with an unknown schema_version; v2 onward bumps the validator. - **`proof_system`** — vendor name. Determines which deserializer (in the sibling repo) parses `proof_bytes_b64`. - **`circuit_id`** — sha256 of the canonical circuit definition. Two artifacts produced by the same circuit have the same `circuit_id`; this is what arborist binds to in the audit chain rather than the raw proof bytes (which vary per witness). - **`frontier_node`** — describes WHICH node was proved. arborist matches this against the model snapshot's frontier-node enumeration to confirm the proof's scope. - **`public_inputs`** — the inputs the verifier checks. arborist recomputes `weights_commitment_sha256` from its own copy of the weight bytes, `activations_commitment_sha256` from the published activation chain, and confirms equality before consuming the proof. - **`public_outputs.epsilon_inequality_holds`** — the boolean the prover claims is true, which the verifier checks. - **`proof_bytes_b64`** — the actual proof. Arborist does NOT parse this; it forwards to the sibling-repo verifier (or a separate trusted binary). - **`verifier_setup_id`** — sha256 of the verifier's setup (KZG CRS / FRI parameters). Pinned per-circuit; rotation is a circuit bump, which changes `circuit_id`. - **`issued_at`** — wall-clock UTC, sanity-check only. - **`issuer_pubkey_ed25519` + `issuer_signature_ed25519`** — the prover signs the canonical-byte form of this artifact (with the signature field omitted) so arborist can verify provenance of the proof artifact independently of the ZK proof itself. ### 3.2 Canonical-byte form For SHA-256 commitment + signature purposes, the canonical bytes of this artifact are: JSON with sorted keys, separators=(",",":"), no whitespace, UTF-8 encoded, EXCLUDING the `issuer_signature_ed25519` field. Standard JSON-canonicalization the rest of arborist uses (`_canonical_json` in `arborist.qa.dag`). ## 4. Binding into the v9.8 audit chain When arborist consumes a valid proof, it folds the proof's identity into the providence record: ```sql ALTER TABLE providence_cache ADD COLUMN model_weights_zk_root TEXT; ALTER TABLE providence_cache ADD COLUMN frontier_proof_circuit_id TEXT; ``` (Schema migration would land at the integration ticket, not this doc. Sketched here so the binding shape is explicit.) The new fields: - `model_weights_zk_root` — SHA-256 of the artifact's canonical bytes (excluding signature). Becomes part of the audit-event body for `providence_query` rows produced under `frontier_proof_mode = "zk"`. - `frontier_proof_circuit_id` — the artifact's `circuit_id`, exposed for cross-record queries ("show me all rows that used circuit X"). `governance_policy_hash` folds in the `frontier_proof_mode` field so cache_key changes when the mode flips. Records produced under `reveal` and `zk` for the same question land at distinct cache_keys. ## 5. Validation pipeline (arborist side) When arborist receives an artifact: ```python def validate_zk_proof(artifact: dict) -> None: """Pure-Python validation. NEVER runs the ZK verifier; delegates that to the sibling-repo verifier binary.""" # 1. Schema check. if artifact.get("schema_version") != "arborist-zk-proof-v1": raise PiStarError("unknown schema_version") for required in ( "proof_system", "circuit_id", "circuit_kind", "frontier_node", "public_inputs", "public_outputs", "proof_bytes_b64", "verifier_setup_id", "issued_at", "issuer_pubkey_ed25519", "issuer_signature_ed25519", ): if required not in artifact: raise PiStarError(f"missing field: {required!r}") # 2. Recompute commitment sanity. pub = artifact["public_inputs"] if not _is_hex_64(pub["weights_commitment_sha256"]): raise PiStarError("bad weights commitment") # … similar for activations + subset_mask # 3. Issuer signature check (Ed25519). canonical = _canonical_json_no_signature(artifact) if not _ed25519_verify( artifact["issuer_pubkey_ed25519"], canonical, artifact["issuer_signature_ed25519"], ): raise PiStarError("issuer signature failed") # 4. ZK verification — DELEGATED. # This is where arborist hands off to the sibling-repo verifier # binary or a trusted external process. Arborist itself does # NOT run the ZK verifier. if not _delegate_zk_verify(artifact): raise PiStarError("zk verifier rejected proof") # 5. Bind into audit chain. # Caller adds model_weights_zk_root + frontier_proof_circuit_id # to the providence row. ``` `_delegate_zk_verify` is the wire-protocol's other side: arborist either spawns a subprocess running the sibling verifier OR posts to a known endpoint. Either way, arborist sees only the boolean verdict. ## 6. Versioning + forward compatibility - Schema bumps require a new `schema_version` string. v1 → v2 drops a field, adds one, or changes semantics. - arborist supports a fixed set of schema versions at any time; unknown versions are rejected loudly. - The sibling repo's git sha pins which prover/verifier produced the proof. arborist surfaces this in the audit body so a future replay knows which sibling-repo version to invoke. - `circuit_id` is content-addressed; circuit changes always produce a new `circuit_id`, never reuse. ## 7. Threat model - **Compromised sibling-repo prover.** A compromised prover can forge `epsilon_inequality_holds = true` only if it can forge the proof bytes themselves AND the issuer signature. Arborist's defenses: Ed25519 signature check on the artifact (step 3), delegated verifier invocation (step 4). The verifier is the trust anchor; if it's compromised, the chain breaks upstream of arborist. - **Wire-format injection.** Adversary submits malformed JSON hoping to bypass schema validation. Mitigation: every required field checked; type checks on commitment hex; signature check before delegating. - **Proof replay.** Adversary replays a valid proof for a different question. Mitigation: `circuit_id` and `frontier_node` are tied to the model snapshot's frontier enumeration; arborist binds the proof to a specific (cache_key, frontier_node) pair before storage. Replay against a different cache_key fails the binding check. - **Issuer-key compromise.** Adversary obtains the prover's signing key. Mitigation: keys rotate per major-version; arborist maintains a published list of trusted issuer keys, rejects artifacts signed under revoked keys. ## 8. What arborist gains over time If ZK turns out VIABLE per the bench (`docs/zk-frontier-bench.md`), arborist gains: - An `[zk]` extra in `pyproject.toml` pulling JSON-schema validator + Ed25519 verification (NOT the prover/verifier binary). - An `arborist/zk/wire.py` module implementing `validate_zk_proof()` plus `_delegate_zk_verify()` calling the sibling-repo verifier. - A schema migration adding `model_weights_zk_root` and `frontier_proof_circuit_id` columns to `providence_cache`, plus a CHECK constraint update. - A new policy field `frontier_proof_mode` folded into `governance_policy_hash`. - New audit-event type `providence_zk_proof` for the binding event. If ZK stays UNAFFORDABLE, this protocol stays as a documented artifact — when ZK matures (hardware, libraries, theory), the arborist side is ready. ## 9. Cross-references - `docs/zk-frontier-bench.md` — bench plan + parked verdict. - #000016 — the ticket this closes the arborist-side hand-wave for. - #000018 — soft-hash covert-channel analysis; the M3 mitigation there is the only existing defense if ZK proves UNAFFORDABLE. - #000013 — v7-W spatial-temporal substrate; world-state observations have the same privacy concerns and inherit this protocol. - v7 § 16.1 — the original "swap SHA-256 → Poseidon" hand-wave. ## 10. Status Draft v0. Becomes the consumer-side schema spec when the sibling repo lands measurements + the validator. Until then: this is the contract arborist commits to honor on the receive side.