"""HTTP gossip wire — minimum viable. Implements the protocol contract pinned in ``docs/mesh.md``: ANNOUNCE_ROOT peer says "I have this document_root" ANNOUNCE_DERIVATION peer says "core C derives from surfaces S1..Sn" ANNOUNCE_PROVIDENCE peer says "I cached an answer at this 8-dim key" ANNOUNCE_FALSIFICATION peer says "this cache_key is wrong" REQUEST_BODY peer asks for a document body on local miss DELIVER_BODY peer responds with bytes + Merkle proof Every envelope is signed (Ed25519) by the sender's `sign_priv`; receiver verifies against the sender's `sign_pub` looked up in `mesh_roster` at the envelope's `epoch_id`. A non-member of that epoch cannot produce a valid signature, so non-members are silently rejected. Each receive writes a `mesh_received` event into the *local* audit chain whose body is the full signed envelope. The receiver's chain stays internally consistent because we append in receive-order. **Per-peer chain-of-claims tracking.** Every envelope built via the sender helper (``MeshWireClient`` outbound traffic, plus ``WireEnvelope.with_chain``) carries two extra fields: - ``prev_event_hash`` — the sender's previous chain-of-claims hash, or ``None`` for the first envelope a peer ever broadcasts. - ``event_hash`` — ``sha256(prev_event_hash || canonical_envelope_bytes)``. The receiver: 1. Verifies the Ed25519 signature (existing). 2. Recomputes ``event_hash`` from ``prev_event_hash`` + the envelope's canonical bytes (excluding ``event_hash`` itself, which is circular). Mismatch → 400. 3. Looks up the sender's last-seen ``event_hash`` in ``mesh_peer_chains``. If a row exists, the envelope's ``prev_event_hash`` MUST equal it; otherwise the chain has forked and we return 409 Conflict. If no row exists, this is the first envelope we've seen from this peer; any ``prev_event_hash`` is accepted (we have no prior to compare against). 4. On accept, ``mesh_peer_chains`` is updated to the new ``event_hash`` + bumped ``last_seq`` + ``now``. Backward compatibility: envelopes that arrive WITHOUT ``prev_event_hash`` and ``event_hash`` (legacy, hand-built test fixtures from before the chain-tracking rollout) skip per-peer chain enforcement. The signature check still runs. New code paths always carry chain fields, so this fallback only matters for legacy test envelopes. v1 limitations (deferred to a later commit): - **No multi-event catchup / replay.** If a receiver missed event N (peer sent N+1 with prev=N's hash but receiver still has prev=N-1), we reject with 409 and rely on an operator to retry or run a catchup pass. v2 will fetch the missing range from the peer and replay. - **No cross-peer reconciliation.** Each peer's chain is tracked independently — alice's broadcasts to bob and bob's broadcasts to carol are unrelated streams. """ from __future__ import annotations import base64 import hashlib import json import os import threading import time from dataclasses import asdict, dataclass from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path from typing import Any import httpx from aborist.mesh.crypto import aead_decrypt, aead_encrypt, sign, verify from aborist.mesh.state import ( current_epoch, load_identity, recover_epoch_secret, roster_at, ) from aborist.store import append_audit, connect, latest_event_hash # --------------------------------------------------------------------------- # Wire constants # --------------------------------------------------------------------------- WIRE_VERSION = 1 TYPE_ANNOUNCE_ROOT = "ANNOUNCE_ROOT" TYPE_ANNOUNCE_DERIVATION = "ANNOUNCE_DERIVATION" TYPE_ANNOUNCE_PROVIDENCE = "ANNOUNCE_PROVIDENCE" TYPE_ANNOUNCE_FALSIFICATION = "ANNOUNCE_FALSIFICATION" TYPE_REQUEST_BODY = "REQUEST_BODY" TYPE_DELIVER_BODY = "DELIVER_BODY" ALL_TYPES = frozenset({ TYPE_ANNOUNCE_ROOT, TYPE_ANNOUNCE_DERIVATION, TYPE_ANNOUNCE_PROVIDENCE, TYPE_ANNOUNCE_FALSIFICATION, TYPE_REQUEST_BODY, TYPE_DELIVER_BODY, }) # HTTP paths PATH_ANNOUNCE = "/mesh/announce" PATH_REQUEST = "/mesh/request" PATH_INFO = "/mesh/info" DEFAULT_PORT = 8400 # --------------------------------------------------------------------------- # Envelope: the signed unit of gossip # --------------------------------------------------------------------------- @dataclass(frozen=True) class WireEnvelope: """One signed gossip message. `body` is type-specific; canonicalization is via JSON sorted-keys with no whitespace, identical to `aborist.store._canonical_json`. Confidentiality is opt-in. When `encrypted_body` is set, `body` is a placeholder (typically `{}`) and the real body lives AEAD-encrypted inside `encrypted_body = {"nonce_b64": ..., "ct_b64": ...}`. Both fields participate in `canonical_bytes()`, so the Ed25519 signature binds the ciphertext: a MITM cannot swap `encrypted_body` without invalidating the signature. Receivers detect encryption by the presence of `encrypted_body` and decrypt before handler dispatch. `prev_event_hash` and `event_hash` carry the sender's chain-of-claims state. Both default to ``None`` for backward compatibility with hand- built test envelopes. Production traffic is built via :meth:`with_chain` (or `MeshWireClient`) which always populates them. `event_hash` is intentionally NOT included in the signed canonical bytes — it is computed FROM those bytes, so embedding it would be circular. The receiver re-derives ``event_hash`` from ``sha256(prev_event_hash || canonical_bytes_without_event_hash)`` and asserts equality before accepting. """ type: str sender_id: str epoch_id: int ts: int body: dict[str, Any] v: int = WIRE_VERSION encrypted_body: dict[str, str] | None = None prev_event_hash: str | None = None event_hash: str | None = None def canonical_bytes(self) -> bytes: """Bytes that get signed. Stable, reproducible. Excludes ``event_hash`` (circular: it's the hash OF these bytes) but includes ``prev_event_hash`` (linkage proof — the sender is committing to a specific predecessor) and ``encrypted_body`` (so signatures bind the ciphertext too). The ``encrypted_body`` field is dropped from the canonical form when ``None`` so cleartext-only envelopes stay byte-identical to the v1 wire format — forward-compat for receivers that don't speak AEAD yet. """ d = asdict(self) d.pop("event_hash", None) if d.get("encrypted_body") is None: d.pop("encrypted_body", None) return _canonical_json(d).encode("utf-8") def compute_event_hash(self) -> str: """Deterministic hash of this envelope as a chain-of-claims node. ``sha256(prev_event_hash_bytes || canonical_bytes)`` where ``prev_event_hash_bytes`` is the raw 32-byte digest if ``prev_event_hash`` is set, else empty. """ h = hashlib.sha256() if self.prev_event_hash is not None: h.update(bytes.fromhex(self.prev_event_hash)) h.update(self.canonical_bytes()) return h.hexdigest() def with_chain(self, *, prev_event_hash: str | None) -> "WireEnvelope": """Return a copy with chain fields populated. ``event_hash`` is computed deterministically from ``prev_event_hash`` + this envelope's canonical bytes. The returned envelope is the one the sender signs and broadcasts. """ # Construct the linked envelope first so canonical_bytes() sees # the new prev_event_hash. event_hash itself is excluded from the # canonical bytes, so we set it last. linked = WireEnvelope( type=self.type, sender_id=self.sender_id, epoch_id=self.epoch_id, ts=self.ts, body=self.body, v=self.v, encrypted_body=self.encrypted_body, prev_event_hash=prev_event_hash, event_hash=None, ) eh = linked.compute_event_hash() return WireEnvelope( type=linked.type, sender_id=linked.sender_id, epoch_id=linked.epoch_id, ts=linked.ts, body=linked.body, v=linked.v, encrypted_body=linked.encrypted_body, prev_event_hash=linked.prev_event_hash, event_hash=eh, ) def sign_with(self, sign_priv: bytes) -> dict[str, Any]: """Wrap envelope + Ed25519 signature in a dict ready for HTTP body.""" sig = sign(sign_priv, self.canonical_bytes()) env_d = asdict(self) if env_d.get("encrypted_body") is None: env_d.pop("encrypted_body", None) return { "envelope": env_d, "sig_b64": base64.b64encode(sig).decode("ascii"), } @classmethod def from_signed(cls, signed: dict[str, Any]) -> tuple["WireEnvelope", bytes]: """Reverse of `sign_with`. Returns (envelope, sig_bytes). Does NOT verify — caller verifies via `verify_envelope_sig` and (when chain fields are present) via the receiver's chain check. ``prev_event_hash`` and ``event_hash`` are optional in the wire payload — legacy envelopes that pre-date chain tracking decode with both as ``None``. """ if not isinstance(signed, dict) or "envelope" not in signed or "sig_b64" not in signed: raise ValueError("signed payload missing envelope/sig_b64") env_d = signed["envelope"] if not isinstance(env_d, dict): raise ValueError("envelope is not a dict") for required in ("type", "sender_id", "epoch_id", "ts", "body", "v"): if required not in env_d: raise ValueError(f"envelope missing field: {required}") if env_d["type"] not in ALL_TYPES: raise ValueError(f"unknown envelope type: {env_d['type']!r}") if env_d["v"] != WIRE_VERSION: raise ValueError(f"unsupported wire version: {env_d['v']!r}") encrypted_body = env_d.get("encrypted_body") if encrypted_body is not None: if not isinstance(encrypted_body, dict): raise ValueError("encrypted_body must be a dict if present") for k in ("nonce_b64", "ct_b64"): if k not in encrypted_body: raise ValueError(f"encrypted_body missing field: {k}") prev = env_d.get("prev_event_hash") eh = env_d.get("event_hash") if prev is not None and not isinstance(prev, str): raise ValueError("prev_event_hash must be str or null") if eh is not None and not isinstance(eh, str): raise ValueError("event_hash must be str or null") env = cls( type=env_d["type"], sender_id=env_d["sender_id"], epoch_id=int(env_d["epoch_id"]), ts=int(env_d["ts"]), body=env_d["body"], v=int(env_d["v"]), encrypted_body=encrypted_body, prev_event_hash=prev, event_hash=eh, ) try: sig = base64.b64decode(signed["sig_b64"], validate=True) except Exception as e: raise ValueError(f"sig_b64 is not valid base64: {e!r}") from e return env, sig def _envelope_aad( *, type_: str, sender_id: str, epoch_id: int, ts: int, ) -> bytes: """AEAD additional-authenticated-data for body encryption. Binds the ciphertext to the envelope metadata so a captured (nonce, ct) cannot be replayed under a different `(type, sender, epoch, ts)` tuple. The signature already binds these fields — AAD is belt-and-suspenders against the case where a peer accepts an envelope with a fresh sig but recycled ciphertext (e.g. cross-type confusion). """ return f"{type_}|{sender_id}|{epoch_id}|{ts}".encode("utf-8") def encrypt_body_for_envelope( *, type_: str, sender_id: str, epoch_id: int, ts: int, body: dict[str, Any], epoch_secret: bytes, ) -> dict[str, str]: """Serialize `body` canonically, AEAD-encrypt under `epoch_secret`. Returns a dict of base64-encoded `nonce_b64` + `ct_b64` suitable to drop into `WireEnvelope.encrypted_body`. The plaintext is the same canonical JSON that would have appeared in `body`, so a receiver decrypts and JSON-parses to recover the original dict. """ plaintext = _canonical_json(body).encode("utf-8") nonce = os.urandom(12) aad = _envelope_aad( type_=type_, sender_id=sender_id, epoch_id=epoch_id, ts=ts ) ct = aead_encrypt(epoch_secret, nonce, plaintext, aad=aad) return { "nonce_b64": base64.b64encode(nonce).decode("ascii"), "ct_b64": base64.b64encode(ct).decode("ascii"), } def decrypt_envelope_body( envelope: WireEnvelope, epoch_secret: bytes, ) -> dict[str, Any]: """AEAD-decrypt `envelope.encrypted_body` and return the recovered dict. Raises ValueError on missing encrypted_body, malformed base64, AEAD tag mismatch (wrong key, tampering, AAD drift), or non-dict plaintext. """ if envelope.encrypted_body is None: raise ValueError("envelope has no encrypted_body to decrypt") try: nonce = base64.b64decode(envelope.encrypted_body["nonce_b64"]) ct = base64.b64decode(envelope.encrypted_body["ct_b64"]) except Exception as e: raise ValueError(f"encrypted_body base64 decode failed: {e!r}") from e aad = _envelope_aad( type_=envelope.type, sender_id=envelope.sender_id, epoch_id=envelope.epoch_id, ts=envelope.ts, ) plaintext = aead_decrypt(epoch_secret, nonce, ct, aad=aad) decoded = json.loads(plaintext.decode("utf-8")) if not isinstance(decoded, dict): raise ValueError("decrypted body must be a JSON object") return decoded def _peer_chain_row(conn, peer_member_id: str) -> tuple[str, int] | None: """Return (last_event_hash, last_seq) for a peer, or None if unseen.""" row = conn.execute( "SELECT last_event_hash, last_seq FROM mesh_peer_chains " "WHERE peer_member_id = ?", (peer_member_id,), ).fetchone() if row is None: return None return row["last_event_hash"], int(row["last_seq"]) def _update_peer_chain( conn, peer_member_id: str, new_event_hash: str, *, now: int, ) -> None: """Upsert mesh_peer_chains[peer] = new_event_hash, last_seq+=1, now.""" existing = _peer_chain_row(conn, peer_member_id) if existing is None: conn.execute( "INSERT INTO mesh_peer_chains " "(peer_member_id, last_event_hash, last_seq, last_seen_at) " "VALUES (?, ?, ?, ?)", (peer_member_id, new_event_hash, 1, now), ) else: _, last_seq = existing conn.execute( "UPDATE mesh_peer_chains " "SET last_event_hash = ?, last_seq = ?, last_seen_at = ? " "WHERE peer_member_id = ?", (new_event_hash, last_seq + 1, now, peer_member_id), ) def verify_envelope_sig( conn, envelope: WireEnvelope, sig: bytes, ) -> bool: """True iff `sig` is a valid Ed25519 signature over the envelope's canonical bytes by `envelope.sender_id`'s `sign_pub` at `envelope.epoch_id` in the local roster. Returns False (never raises) on missing roster entry, bad signature, or any structural issue. The caller treats False as "drop the message." """ try: roster = roster_at(conn, envelope.epoch_id) except Exception: return False sender = next((m for m in roster if m.member_id == envelope.sender_id), None) if sender is None: return False return verify(sender.sign_pub, sig, envelope.canonical_bytes()) # --------------------------------------------------------------------------- # Server # --------------------------------------------------------------------------- def _make_handler_class(mesh_server: "MeshWireServer") -> type[BaseHTTPRequestHandler]: """Build a request-handler class bound to this MeshWireServer. Done as a factory so the handler can reach back to the mesh server without colliding with BaseHTTPRequestHandler's own ``self.server`` attribute (which is the underlying ThreadingHTTPServer). """ class Handler(BaseHTTPRequestHandler): # Bound at class level by the factory closure. mesh: MeshWireServer = mesh_server def log_message(self, fmt, *args): # silence default stderr spam return def _read_json(self) -> dict[str, Any]: n = int(self.headers.get("content-length", "0")) raw = self.rfile.read(n) if n > 0 else b"" return json.loads(raw or b"{}") def _respond(self, status: int, body: dict[str, Any]) -> None: data = json.dumps(body).encode("utf-8") self.send_response(status) self.send_header("content-type", "application/json") self.send_header("content-length", str(len(data))) self.end_headers() self.wfile.write(data) def do_GET(self) -> None: if self.path == PATH_INFO: self._respond(200, self.mesh.info()) return self._respond(404, {"error": "not found"}) def do_POST(self) -> None: try: payload = self._read_json() except Exception as e: self._respond(400, {"error": f"bad json: {e!s}"}) return if self.path == PATH_ANNOUNCE: self._respond(*self.mesh.handle_announce(payload)) return if self.path == PATH_REQUEST: self._respond(*self.mesh.handle_request(payload)) return self._respond(404, {"error": "not found"}) return Handler class MeshWireServer: """Threaded HTTP gossip server. Use as ``with MeshWireServer(db_path, host, port) as srv: srv.serve()`` for one-shot scripts, or call ``start_in_thread()`` for tests / long-lived daemons. ``stop()`` is idempotent. """ def __init__(self, db_path: Path | str, host: str = "127.0.0.1", port: int = DEFAULT_PORT): self.db_path = str(db_path) handler_cls = _make_handler_class(self) self._http = ThreadingHTTPServer((host, port), handler_cls) self._thread: threading.Thread | None = None # ------------------------------------------------------------------ context def __enter__(self) -> "MeshWireServer": return self def __exit__(self, *_a) -> None: self.stop() # ------------------------------------------------------------------ lifecycle @property def address(self) -> tuple[str, int]: return self._http.server_address # type: ignore[return-value] @property def url(self) -> str: host, port = self.address return f"http://{host}:{port}" def start_in_thread(self) -> None: if self._thread is not None: return t = threading.Thread(target=self._http.serve_forever, daemon=True) t.start() self._thread = t def serve(self) -> None: self._http.serve_forever() def stop(self) -> None: # shutdown() blocks until serve_forever() acknowledges, so only call # it when we actually started a serving thread. server_close() is # safe in either case — it just releases the socket. if self._thread is not None: try: self._http.shutdown() except Exception: pass try: self._http.server_close() except Exception: pass self._thread = None # ------------------------------------------------------------------ handlers def info(self) -> dict[str, Any]: conn = connect(self.db_path) try: me = load_identity(conn) epoch = current_epoch(conn) finally: conn.close() if me is None: return {"error": "mesh not initialized"} return { "v": WIRE_VERSION, "member_id": me.member_id, "group_name": me.group_name, "current_epoch": epoch, "sign_pub_hex": me.sign_pub.hex(), } def handle_announce(self, payload: dict[str, Any]) -> tuple[int, dict[str, Any]]: """Verify sig, dedup, persist as a `mesh_received` audit event. If the envelope carries `encrypted_body`, decrypt it under this peer's recovered epoch secret before audit-logging. A peer with no slot in the epoch envelope (evicted or never enrolled) returns 401: confidentiality failure is indistinguishable from auth failure at this layer. """ try: env, sig = WireEnvelope.from_signed(payload) except ValueError as e: return 400, {"error": str(e)} if env.type not in ( TYPE_ANNOUNCE_ROOT, TYPE_ANNOUNCE_DERIVATION, TYPE_ANNOUNCE_PROVIDENCE, TYPE_ANNOUNCE_FALSIFICATION, ): return 400, {"error": f"announce endpoint rejects {env.type!r}"} conn = connect(self.db_path) try: if not verify_envelope_sig(conn, env, sig): return 401, {"error": "signature verify failed"} # Per-peer chain-of-claims tracking. Only enforced when the # sender attached chain fields. Hand-built legacy envelopes # (env.event_hash is None) skip enforcement — signature alone. if env.event_hash is not None: expected_eh = env.compute_event_hash() if expected_eh != env.event_hash: return 400, { "error": "envelope event_hash mismatch: " f"computed {expected_eh}, claimed {env.event_hash}" } existing = _peer_chain_row(conn, env.sender_id) if existing is not None: last_eh, _ = existing if env.prev_event_hash != last_eh: return 409, { "error": ( f"fork detected: peer {env.sender_id} " f"expected prev_event_hash {last_eh}, " f"got {env.prev_event_hash}" ) } # else: first envelope from this peer; any prev_event_hash # (None or arbitrary string) is accepted — we have no # prior to compare against. _update_peer_chain( conn, env.sender_id, env.event_hash, now=int(time.time()), ) # AEAD decrypt happens after sig + chain checks so we don't # waste a secret recovery on rejected traffic. effective_body = env.body if env.encrypted_body is not None: try: epoch_secret = recover_epoch_secret(conn, epoch_id=env.epoch_id) effective_body = decrypt_envelope_body(env, epoch_secret) except (ValueError, RuntimeError) as e: return 401, {"error": f"AEAD decrypt failed: {e!s}"} event_hash = append_audit( conn, event_type="mesh_received", body={ "wire_type": env.type, "sender_id": env.sender_id, "epoch_id": env.epoch_id, "remote_ts": env.ts, "envelope_body": effective_body, "encrypted": env.encrypted_body is not None, # Sender's chain-of-claims hash (if present). Records # what we accepted so future audits can reconstruct # the sender's claimed history. "sender_event_hash": env.event_hash, "sender_prev_event_hash": env.prev_event_hash, }, subject_root=str( effective_body.get("document_root") or effective_body.get("cache_key") or "" ), ) finally: conn.close() return 200, { "ok": True, "wire_type": env.type, "audit_event_hash": event_hash, } def handle_request(self, payload: dict[str, Any]) -> tuple[int, dict[str, Any]]: """REQUEST_BODY -> DELIVER_BODY (or 404 if root not present locally). v1: returns the document text concatenated from hot chunks. The Merkle proof is the document_root itself plus per-chunk leaf hashes — receiver re-derives the root and checks. (Full per-chunk inclusion proofs are a v2 refinement; root + leaves is enough to verify what we deliver since the leaves are the deterministic SHA-256 over canonicalized chunk content.) """ try: env, sig = WireEnvelope.from_signed(payload) except ValueError as e: return 400, {"error": str(e)} if env.type != TYPE_REQUEST_BODY: return 400, {"error": f"request endpoint rejects {env.type!r}"} conn = connect(self.db_path) try: if not verify_envelope_sig(conn, env, sig): return 401, {"error": "signature verify failed"} root = env.body.get("root") if not isinstance(root, str) or len(root) != 64: return 400, {"error": "body.root must be a 64-char hex sha256"} row = conn.execute( "SELECT document_uri FROM documents WHERE document_root=?", (root,) ).fetchone() if row is None: return 404, {"error": f"no such document_root: {root}"} chunks = conn.execute( "SELECT idx, leaf_hash, content FROM chunks " "WHERE document_root=? AND tier='hot' ORDER BY idx", (root,), ).fetchall() from aborist.compress import unpack_chunk leaves_hex = [r["leaf_hash"] for r in chunks] text = "\n\n".join(unpack_chunk(r["content"]) or "" for r in chunks) me = load_identity(conn) epoch = current_epoch(conn) if me is None or epoch is None: return 503, {"error": "mesh not initialized on responder"} prev = latest_event_hash(conn) deliver = WireEnvelope( type=TYPE_DELIVER_BODY, sender_id=me.member_id, epoch_id=epoch, ts=int(time.time()), body={ "root": root, "document_uri": row["document_uri"], "leaves_hex": leaves_hex, "text": text, }, ).with_chain(prev_event_hash=prev) # Mirror the client's audit-log of every outbound chain-linked # envelope so DELIVER_BODY also extends the responder's chain. from dataclasses import asdict as _asdict audit_body = _asdict(deliver) audit_body.pop("event_hash", None) # Mirror canonical_bytes(): drop None encrypted_body so the # audit hash matches event_hash for cleartext envelopes. if audit_body.get("encrypted_body") is None: audit_body.pop("encrypted_body", None) audit_eh = append_audit( conn, event_type="mesh_sent", body=audit_body, subject_root=root, ts=deliver.ts, ) assert audit_eh == deliver.event_hash return 200, deliver.sign_with(me.sign_priv) finally: conn.close() # --------------------------------------------------------------------------- # Client # --------------------------------------------------------------------------- class MeshWireClient: """Thin client for outbound gossip + body pulls. Holds no long-lived state beyond the db_path; every call opens its own connection so we never share sqlite handles across threads. """ def __init__(self, db_path: Path | str, peer_url: str, *, timeout: float = 10.0): self.db_path = str(db_path) self.peer_url = peer_url.rstrip("/") self._client = httpx.Client(timeout=timeout) def close(self) -> None: self._client.close() def __enter__(self) -> "MeshWireClient": return self def __exit__(self, *_a) -> None: self.close() # ------------------------------------------------------------------ helpers def _signed_envelope( self, type_: str, body: dict[str, Any], *, encrypt: bool = False, ) -> dict[str, Any]: """Build, chain-link, optionally encrypt, sign, and audit-log one outbound envelope. Three responsibilities, in order: 1. **Confidentiality (optional).** When ``encrypt=True``, the body is AEAD-encrypted under this peer's recovered epoch secret. Sender-side recovery uses ``recover_epoch_secret`` which re-unwraps our own slot — no sender-side cache. 2. **Chain-of-claims.** Populate ``prev_event_hash`` from our own ``latest_event_hash`` and compute ``event_hash`` so the receiver can enforce per-peer chain consistency. 3. **Local audit log.** Append a ``mesh_sent`` event whose body is the envelope's canonical (event-hash-excluded) dict. By construction its ``audit_event.event_hash`` equals the envelope's ``event_hash``, unifying the wire chain-of-claims with the local audit chain. ``latest_event_hash`` then advances to this envelope's ``event_hash``, so back-to-back sends extend a single linear chain. Drop the same dict shape into both wire and audit so a future forensic replay only needs the audit body to reproduce the envelope bytes (modulo signature). """ from dataclasses import asdict as _asdict conn = connect(self.db_path) try: me = load_identity(conn) epoch = current_epoch(conn) if me is None or epoch is None: raise RuntimeError("mesh not initialized; call init_identity first") ts = int(time.time()) if encrypt: secret = recover_epoch_secret(conn, epoch_id=epoch) encrypted = encrypt_body_for_envelope( type_=type_, sender_id=me.member_id, epoch_id=epoch, ts=ts, body=body, epoch_secret=secret, ) base_env = WireEnvelope( type=type_, sender_id=me.member_id, epoch_id=epoch, ts=ts, body={}, # placeholder; real body lives in encrypted_body encrypted_body=encrypted, ) else: base_env = WireEnvelope( type=type_, sender_id=me.member_id, epoch_id=epoch, ts=ts, body=body, ) prev = latest_event_hash(conn) env = base_env.with_chain(prev_event_hash=prev) audit_body = _asdict(env) audit_body.pop("event_hash", None) # Drop None-valued encrypted_body so cleartext audit bodies # stay byte-identical to the v1 audit shape (mirrors # canonical_bytes' behavior — keeps audit hash == event_hash # for cleartext envelopes built by older callers). if audit_body.get("encrypted_body") is None: audit_body.pop("encrypted_body", None) audit_event_hash = append_audit( conn, event_type="mesh_sent", body=audit_body, subject_root=str(body.get("document_root") or body.get("cache_key") or ""), ts=env.ts, ) assert audit_event_hash == env.event_hash, ( f"chain unification broke: audit hash {audit_event_hash} " f"!= envelope event_hash {env.event_hash}" ) signed = env.sign_with(me.sign_priv) finally: conn.close() return signed # ------------------------------------------------------------------ ops def info(self) -> dict[str, Any]: r = self._client.get(self.peer_url + PATH_INFO) r.raise_for_status() return r.json() def announce_root( self, *, document_root: str, source_uri: str, chunking_version: str, canonicalization_version: str, schema_version: str, encrypt: bool = False, ) -> dict[str, Any]: """Announce a `document_root`. Default cleartext; pass `encrypt=True` to AEAD-encrypt the body under this peer's epoch secret. The signature still binds the ciphertext, so a MITM cannot swap payloads without invalidating the envelope. """ signed = self._signed_envelope( TYPE_ANNOUNCE_ROOT, { "document_root": document_root, "source_uri": source_uri, "chunking_version": chunking_version, "canonicalization_version": canonicalization_version, "schema_version": schema_version, }, encrypt=encrypt, ) r = self._client.post(self.peer_url + PATH_ANNOUNCE, json=signed) r.raise_for_status() return r.json() def announce_falsification( self, *, cache_key: str, reason: str, encrypt: bool = False, ) -> dict[str, Any]: """Announce a falsification. `encrypt` mirrors `announce_root`.""" signed = self._signed_envelope( TYPE_ANNOUNCE_FALSIFICATION, {"cache_key": cache_key, "reason": reason}, encrypt=encrypt, ) r = self._client.post(self.peer_url + PATH_ANNOUNCE, json=signed) r.raise_for_status() return r.json() def request_body(self, *, root: str) -> dict[str, Any]: """Pull `root` from the peer. Verifies the returned signature and re-derives the Merkle root from delivered leaf hashes. On any verification failure, raises — the caller never sees an unverified body. """ signed = self._signed_envelope(TYPE_REQUEST_BODY, {"root": root}) r = self._client.post(self.peer_url + PATH_REQUEST, json=signed) r.raise_for_status() deliver_payload = r.json() env, sig = WireEnvelope.from_signed(deliver_payload) if env.type != TYPE_DELIVER_BODY: raise ValueError(f"expected DELIVER_BODY, got {env.type!r}") conn = connect(self.db_path) try: if not verify_envelope_sig(conn, env, sig): raise ValueError("delivered envelope signature failed verification") finally: conn.close() delivered_root = env.body.get("root") leaves_hex = env.body.get("leaves_hex") or [] if delivered_root != root: raise ValueError( f"delivered root {delivered_root!r} does not match requested {root!r}" ) if not _merkle_root_matches(leaves_hex, root): raise ValueError( "delivered leaves do not Merkle-derive to the claimed root" ) return env.body # --------------------------------------------------------------------------- # Helpers # --------------------------------------------------------------------------- def _canonical_json(obj: Any) -> str: """Stable JSON: sorted keys, no whitespace, ensure_ascii=False so unicode bytes survive a round-trip without \\uXXXX expansion. Same convention as `aborist.store._canonical_json` — kept local to avoid a private import.""" return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False) def _merkle_root_matches(leaves_hex: list[str], expected_root_hex: str) -> bool: """Re-derive the Merkle root from `leaves_hex` (already-hashed chunk leaves) and compare to `expected_root_hex`. Uses aborist.merkle's convention so single-leaf documents and odd-element trees behave identically to the local ingest path. """ from aborist.merkle import MerkleTree if not leaves_hex: return False try: leaves = [bytes.fromhex(h) for h in leaves_hex] except ValueError: return False tree = MerkleTree.build(leaves) return tree.root.hex() == expected_root_hex