From e868b95530ce8740f92f83393e2e2fbde652792b Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Tue, 28 Apr 2026 17:24:37 -0400 Subject: [PATCH] docs: mesh-deploy runbook for two-host gossip setup Step-by-step operator guide for standing up an aborist mesh between two real hosts: pre-flight, init/enable on both peers, out-of-band pubkey exchange, mutual enrollment, mesh serve, mesh sync, audit chain integrity checks, eviction protocol, reset/teardown. Calls out what's not yet wired (mesh pull, per-peer chain merge, AEAD body encryption) so operators don't expect features that ship later. Pairs with docs/mesh.md (protocol contract) and the e2e tests in tests/test_mesh_wire_e2e.py. --- docs/mesh-deploy.md | 257 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 docs/mesh-deploy.md diff --git a/docs/mesh-deploy.md b/docs/mesh-deploy.md new file mode 100644 index 0000000..43d22ec --- /dev/null +++ b/docs/mesh-deploy.md @@ -0,0 +1,257 @@ +# Mesh deploy — two-host runbook + +This is the operator runbook for standing up an aborist mesh between two +real hosts. The protocol is documented in `docs/mesh.md`; this file +covers what to type, in what order, with what to verify at each step. + +> Every step assumes both peers have aborist installed and have already +> ingested some local corpus. If you only want to test the wire on +> localhost-loopback, see `tests/test_mesh_wire_e2e.py` instead. + +## Naming + +Two hosts in this runbook: + +- **`alice`** — admin, founder of group `myteam`. Runs `mesh serve`. +- **`bob`** — joining member. Runs `mesh sync` outbound to alice. + +Substitute hostnames / IPs / ports per your environment. + +## 0. Pre-flight on each host + +```sh +# Confirm aborist + extras + tests: +make bootstrap +make test +.venv/bin/aborist --version +``` + +Both hosts should be on the same git commit. Mismatched +`schema_version` / `chunking_version` / `canonicalization_version` +will be rejected by the v9.8 admissibility check; aborist refuses +silent corruption. + +## 1. Initialize mesh on each peer + +```sh +# alice — first peer becomes founder of epoch 0: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh init --group myteam --member-id alice +.venv/bin/aborist --db ~/.aborist/aborist.db mesh enable +.venv/bin/aborist --db ~/.aborist/aborist.db mesh status + +# bob — initializes a separate identity in the same group name: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh init --group myteam --member-id bob +.venv/bin/aborist --db ~/.aborist/aborist.db mesh enable +.venv/bin/aborist --db ~/.aborist/aborist.db mesh status +``` + +At this point alice and bob both think they're the sole member of +`myteam`. The next step adds bob to alice's roster — and vice versa — +so both rosters carry both pubkeys at the same epoch. + +## 2. Exchange pubkeys (out-of-band) + +bob's public keys must reach alice via a trusted channel. Mesh has no +built-in introduction protocol; signal, in-person hand-off, or a +signed file all work. + +```sh +# On bob: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh status | jq '.identity' +# -> { "member_id": "bob", "sign_pub_hex": "...", "dh_pub_hex": "..." } +``` + +Send the two hex strings to alice. Verify the channel out of band. + +Same direction in reverse so bob can enroll alice: + +```sh +# On alice: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh status | jq '.identity' +``` + +## 3. Mutual enrollment + +```sh +# On alice (admin) — enroll bob at alice's epoch: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh add \ + --member-id bob \ + --sign-pub \ + --dh-pub + +# On bob — enroll alice at bob's epoch (so bob's local roster knows +# alice's pubkey for signature verification when alice's announces +# arrive): +.venv/bin/aborist --db ~/.aborist/aborist.db mesh add \ + --member-id alice \ + --sign-pub \ + --dh-pub +``` + +Each `mesh add` bumps the local epoch and writes a `mesh_epoch_rotate` +audit event. Verify: + +```sh +.venv/bin/aborist --db ~/.aborist/aborist.db mesh status +.venv/bin/aborist --db ~/.aborist/aborist.db mesh members +``` + +Both peers should now show 2 members at epoch 1. + +## 4. Start alice's gossip server + +```sh +# On alice — bind on a routable interface: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh serve \ + --host 0.0.0.0 --port 8400 +# Stdout: {"status": "serving", "url": "http://0.0.0.0:8400", ...} +``` + +Leave this running. Open the firewall to allow bob to reach +`alice:8400`. Production deployments should put a TLS terminator +(Caddy / nginx) in front; the wire speaks plain HTTP — TLS is the +operator's choice. + +Verify reachability from bob: + +```sh +# On bob: +curl http://alice.example.com:8400/mesh/info | jq +# -> { "v": 1, "member_id": "alice", "group_name": "myteam", +# "current_epoch": 1, "sign_pub_hex": "..." } +``` + +If `member_id` matches what bob has in his roster for "alice", and the +`sign_pub_hex` matches what bob enrolled, you're good. If they don't +match, bob is talking to the wrong peer or alice's enrollment was wrong; +do not push gossip until the discrepancy is resolved. + +## 5. Bob announces his local roots to alice + +```sh +# On bob: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh sync \ + --peer http://alice.example.com:8400 \ + --limit 100 +``` + +Output: JSON status, count of acked vs errored announces. + +Each accepted announce writes one `mesh_received` event into alice's +audit chain. Verify on alice: + +```sh +sqlite3 ~/.aborist/aborist.db \ + "SELECT COUNT(*) FROM audit_events WHERE event_type='mesh_received'" +``` + +Cross-check chain integrity: + +```sh +make chain-check +``` + +Should return `0` chain breaks. + +## 6. Make sync bidirectional + +The current `mesh sync` only pushes outbound. To get alice's roots +onto bob, run `mesh sync` from alice pointed at bob: + +```sh +# On alice (in a new terminal — keep mesh serve running): +.venv/bin/aborist --db ~/.aborist/aborist.db mesh sync \ + --peer http://bob.example.com:8400 --limit 100 +``` + +This requires bob to also be running `mesh serve`. Symmetric setup is +the standard mode. + +## 7. Pulling missing bodies (when shipped) + +Today only ANNOUNCE_ROOT is exercised by `mesh sync`. The wire layer +already implements REQUEST_BODY/DELIVER_BODY with Merkle verification +on the client side; the CLI hookup (`mesh pull --root --peer +`) is on the queue. Once shipped, the pull-on-miss workflow is: + +```sh +# bob discovers via announces that alice has document X he doesn't: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh pull \ + --root \ + --peer http://alice.example.com:8400 +# bob's local store now has X (Merkle-verified at receive). +``` + +## 8. Eviction protocol + +If bob leaves the team, alice (admin) kicks. This bumps alice's epoch +and rewraps the next epoch secret to the post-bob roster: + +```sh +# On alice: +.venv/bin/aborist --db ~/.aborist/aborist.db mesh kick \ + --member-id bob \ + --reason "left team 2026-04-28" +``` + +Bob's prior signatures stay verifiable forever (his roster row at +older epochs is preserved on disk). Any AEAD-protected gossip from +epoch+1 onward is opaque to him — that's the eviction guarantee. + +Important: kicking only mutates alice's local state. If bob's host is +still running `mesh serve`, alice should also stop sending to him (or +he'll keep accepting alice's announces under the OLD epoch she's no +longer broadcasting from). Operationally, kicks should pair with +either: (a) dropping bob's URL from alice's sync targets, OR (b) +announcing the kick to remaining members so they update their +rosters. + +## 9. Operational checks + +Every chain-write op (`mesh init`, `mesh add`, `mesh kick`, `mesh +rotate`, every received announce) extends the local audit chain. The +fast probe is: + +```sh +make chain-check # single db +make chain-check-shards # every db in $(SHARDS_DIR) +``` + +Both should always print `0`. Non-zero = operator must investigate +before further operations; data has diverged from the audit trail. + +## 10. Reset / teardown + +To take a peer fully out of the mesh (irreversible — fresh keys +needed to rejoin): + +```sh +.venv/bin/aborist --db ~/.aborist/aborist.db mesh disable +# Identity + roster history stay on disk for forensics. Re-enable +# requires another peer to re-add this member at a fresh epoch. +``` + +For full local nuke (developers only — destroys identity + chain): + +```sh +sqlite3 ~/.aborist/aborist.db <<'SQL' +DELETE FROM mesh_identity; +DELETE FROM mesh_roster; +DELETE FROM mesh_epochs; +SQL +``` + +## What's not yet shipped + +These features are documented in the protocol but not wired into the +CLI yet: + +- **`mesh pull `** — fetch a body on cache miss. +- **Per-peer audit-chain merge** — receiver tracks each sender's last + known event_hash and rejects fork events. Today the receiver verifies + signatures only; chain-of-claims tracking is deferred. +- **AEAD body encryption** — wire signs envelope bytes for integrity; + body confidentiality (encrypting against the per-epoch shared + secret) is optional in the contract and not yet wired. + +When these land, this runbook will gain steps for them.