russell@unturf.com
39f8aa1fb4
- '.. class:: center' parsed as a Python class named 'center' (duplicate
across one-pager + two-pager) -> '.. rst-class:: center' (styling, no
object). Clears the duplicate-object-description warnings.
- two v8-consensus section underlines were shorter than their titles
('underline too short') -> extended to title length.
Pager + v8 pages now build with zero warnings.
841 lines
33 KiB
ReStructuredText
841 lines
33 KiB
ReStructuredText
Merkle-AGI v8 — Selection & Consensus Protocol
|
||
===============================================
|
||
|
||
:Author: fox + agent blackops, on the unsandbox / unturf / permacomputer platform
|
||
:Date: 2026-05-10 (draft v0)
|
||
:Status: substrate paper for ticket #000012 Phase 1b; closure draft.
|
||
|
||
This document specifies the **selection layer** of the Merkle-AGI
|
||
lineage — sister to v7 (logic/math substrate; plastic training) and
|
||
v7-W (spatial-temporal substrate). v8 commits the loop-closing step
|
||
that turns single-validator Proof-of-Upgrade into Darwinian
|
||
selection across an open validator set.
|
||
|
||
Read alongside:
|
||
|
||
- ``docs/v8-fork-score.md`` — canonical formula reference for
|
||
``arborist.substrate.fork_score`` (v8 Phase 1a; landed 2026-05-08).
|
||
The ScoredFork function this paper invokes for fitness scoring.
|
||
- ``docs/_source/merkle-agi-v7w-spatial-temporal.rst`` — sister
|
||
substrate paper; same RST conventions; same audience.
|
||
- The v7 substrate spec
|
||
(``~/Downloads/merkle-agi-dag_v7.txt`` § 13.4) — the Proof-of-
|
||
Upgrade procedure v8 extends from single-validator to multi-
|
||
validator consensus.
|
||
|
||
|
||
Part 1 — Introduction & motivation
|
||
----------------------------------
|
||
|
||
§1.1 The DNA ↔ Merkle-DAG loop
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The substrate-lineage analogy is:
|
||
|
||
.. code-block:: text
|
||
|
||
copy → vary → express → test → select → preserve → repeat
|
||
|
||
Merkle-AGI v7 ships everything except ``select``. Section 13.4
|
||
("Proof-of-Upgrade") sketches a procedure where a candidate model
|
||
``(M', C(M'))`` is admitted if it shows non-regression on a
|
||
published eval set + non-regression of ε-coverage at sentinel
|
||
frontiers. That procedure is **single-validator regression testing
|
||
with a Merkle receipt**. It is not consensus.
|
||
|
||
§1.2 The selection gap under v7 § 13.4
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The unaddressed gaps from v7 § 13.4, each with a concrete failure
|
||
under the current spec:
|
||
|
||
================================ ============================================
|
||
Gap Concrete failure
|
||
================================ ============================================
|
||
Validator set discovery Two labs reach different verdicts on M'; no
|
||
fork-choice rule for downstream consumers.
|
||
Byzantine fault tolerance One dishonest validator can sign acceptance
|
||
for a poisoned (M').
|
||
Sybil resistance A single actor spinning up 50 validators
|
||
wins every quorum.
|
||
Fork-choice on disagreement If V1, V2 publish conflicting acceptance
|
||
receipts, downstream nodes have no rule.
|
||
Liveness vs safety trade No bound on how long acceptance can stall
|
||
when validators are offline.
|
||
Validator incentives Why would anyone run a validator? What's
|
||
the slashing condition for cheating?
|
||
Stake / membership semantics Permissionless? Permissioned? Hybrid?
|
||
v7 silent.
|
||
================================ ============================================
|
||
|
||
Without a protocol that closes these, "digital evolution" is
|
||
metaphor — a single lab signing its own upgrades, no different in
|
||
trust model from current model-card releases.
|
||
|
||
§1.3 Concrete attack scenario (motivating example)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Lab A trains ``M_t → M_{t+1}`` with a backdoor that triggers on a
|
||
rare input pattern. Lab A signs Proof-of-Upgrade: ε-coverage
|
||
non-regressed at every published frontier (because the backdoor
|
||
lives at a non-published frontier). Lab A publishes ``C(M_{t+1})``.
|
||
Anyone pulling the registry sees an "accepted" upgrade. v7 has no
|
||
way to surface that no independent validator audited the upgrade.
|
||
|
||
The v8 selection protocol must make "accepted under v8" mean
|
||
"accepted by N independent validators meeting policy P", verifiable
|
||
to anyone, not "Lab A signed it."
|
||
|
||
§1.4 What this paper IS and IS NOT
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
**IS**:
|
||
|
||
- A protocol specification: validator state machine, acceptance
|
||
ledger, challenge window, fork-choice rule, slashing schedule,
|
||
mesh wire format extension.
|
||
- A BFT-analysis sketch with explicit safety/liveness bounds.
|
||
- A worked-example acceptance cycle (appendix) reflecting one
|
||
imaginary upgrade.
|
||
|
||
**IS NOT**:
|
||
|
||
- An implementation. Code lands in follow-up tickets that cite this
|
||
paper.
|
||
- An economic calibration. Stake amounts, slashing fractions, and
|
||
reward rates are governance decisions; the paper specifies their
|
||
shape, not their magnitude.
|
||
- A cross-chain anchoring spec. Publishing v8 finalizations to
|
||
Bitcoin / Ethereum / etc. is an optional bolt-on; not in this
|
||
paper.
|
||
|
||
|
||
Part 2 — Substrate definition
|
||
-----------------------------
|
||
|
||
§2.1 Hard constraints (carried from #000012)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
- Stays inside SQD A1 (canonical encoding), A2 (public
|
||
quantization), A3 (collision-resistant hash). v8 introduces no
|
||
new cryptographic axiom.
|
||
- No per-query consensus. Selection runs at checkpoint cadence
|
||
(Proof-of-Upgrade scope), not at inference time.
|
||
- v9.8 ``cache_key`` invariant stays at 8 dimensions. Consensus
|
||
state lives in a sibling ``consensus_events`` table, not folded
|
||
into the answer-cache.
|
||
- arborist's per-shard audit chain stays per-node. v8 is
|
||
checkpoint-cadence, cross-validator; nothing in v8 modifies
|
||
``audit_events.event_hash`` preimage on individual shards.
|
||
- Existing falsification-state semantics
|
||
(``live``, ``failed``, ``stale``, ``quarantined``) stay
|
||
unchanged. v8 reuses them for validator state, not for cache
|
||
state.
|
||
|
||
§2.2 What v8 commits
|
||
~~~~~~~~~~~~~~~~~~~~
|
||
|
||
v8 commits the **acceptance ledger**: a Merkle-chained sequence of
|
||
finalized ``(C(M'), proposer_id, validator_signatures, audit_replay
|
||
_digest, fork_score_breakdown)`` rows. Each row's
|
||
``event_hash`` is the SHA-256 of the canonical encoding of the row
|
||
body chained against the previous row's ``event_hash``.
|
||
|
||
.. code-block:: text
|
||
|
||
consensus_events.event_hash =
|
||
SHA-256( prev_event_hash || canonical_encoding(body) )
|
||
|
||
body = {
|
||
"candidate_root": "<C(M') SHA-256>",
|
||
"parent_root": "<C(M_t) SHA-256>",
|
||
"proposer_id": "<validator pubkey>",
|
||
"eval_digest": "<canonical eval-suite digest>",
|
||
"frontier_coverage_diff": "<ε-coverage delta blob>",
|
||
"fork_score": {<ScoredFork breakdown>},
|
||
"validator_signatures": [<aggregate or individual sigs>],
|
||
"audit_replay_digest": "<replay result digest>",
|
||
"verdict": "ACCEPTED" | "REJECTED" | "CHALLENGED",
|
||
"finalized_at": <unix-ts>
|
||
}
|
||
|
||
The canonical encoding follows the SQD A1 convention (sorted JSON
|
||
keys, UTF-8, no whitespace) so two independent serializers produce
|
||
byte-identical bytes.
|
||
|
||
§2.3 Sibling-table semantics (no `cache_key` pollution)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Per ticket #000012 §4.2 hard constraint: consensus state never
|
||
enters ``providence_cache.cache_key``. The 8-dim invariant
|
||
(``source_root | question_hash | model_profile_hash |
|
||
conversation_hash | governance_policy_hash | schema_version |
|
||
canonicalization_version | chunking_version``) is unchanged.
|
||
|
||
v8 introduces a **ninth-conceptual** invariant — the
|
||
**consensus_policy_hash** — but folds it into a sibling table, not
|
||
``cache_key``. The motivation: per-query cache invalidation should
|
||
not fire every time a validator joins or leaves the set. Validator-
|
||
set churn is checkpoint-scale, not query-scale.
|
||
|
||
The two policy hashes are deliberately split:
|
||
|
||
- ``governance_policy_hash`` — folds answer-mode, verifier method,
|
||
claim caps, broad-quantifier guard config. Per-query; folds into
|
||
``cache_key``.
|
||
- ``consensus_policy_hash`` — folds validator set, quorum
|
||
threshold, slashing schedule, challenge window. Per-checkpoint;
|
||
folds into ``consensus_events`` row but NEVER ``cache_key``.
|
||
|
||
A consumer that wants to verify "this answer was produced under
|
||
finalized policy P at checkpoint T" walks the ``consensus_events``
|
||
chain to find P, then walks the per-shard ``audit_events`` chain to
|
||
find the answer.
|
||
|
||
|
||
Part 3 — Validator state machine
|
||
--------------------------------
|
||
|
||
§3.1 States
|
||
~~~~~~~~~~~
|
||
|
||
A validator transitions through five states across its lifetime:
|
||
|
||
================ ================================================
|
||
State Meaning
|
||
================ ================================================
|
||
``bonding`` Stake deposit pending; cannot sign yet. Bond
|
||
window must elapse before promotion.
|
||
``active`` Eligible to sign acceptance / rejection on
|
||
proposed candidates. Earns rewards on honest
|
||
signatures.
|
||
``challenged`` A counter-evidence submission claims one of this
|
||
validator's prior signatures was fraudulent. Stake
|
||
is frozen pending challenge resolution.
|
||
``slashed`` Challenge resolved against the validator. Stake
|
||
partially or fully forfeit per offense-class
|
||
schedule (§6.2). Validator cannot rejoin under the
|
||
same pubkey.
|
||
``unbonding`` Validator initiated voluntary exit. Stake is
|
||
time-locked for an unbond window so outstanding
|
||
challenges can land before withdrawal.
|
||
================ ================================================
|
||
|
||
§3.2 Transitions
|
||
~~~~~~~~~~~~~~~~
|
||
|
||
.. code-block:: text
|
||
|
||
stake_deposit
|
||
(start) ─────────────────────────► bonding
|
||
│
|
||
bond_window elapsed
|
||
│
|
||
▼
|
||
┌──────────────────────────────► active ───────────────────────┐
|
||
│ │ │
|
||
│ stake withdraw │
|
||
│ initiated │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ unbonding │
|
||
│ │ │
|
||
│ unbond_window elapsed │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ (exit) │
|
||
│ │
|
||
│ challenge_submitted │
|
||
│ │
|
||
▼ │
|
||
challenged │
|
||
│ │
|
||
│ challenge_resolved_in_favor │
|
||
└──────────────────────────────────────────────────────────┘
|
||
│
|
||
│ challenge_resolved_against
|
||
▼
|
||
slashed
|
||
│
|
||
│ (terminal — same pubkey cannot rejoin)
|
||
│
|
||
▼
|
||
|
||
§3.3 Invariants
|
||
~~~~~~~~~~~~~~~
|
||
|
||
- A validator in ``slashed`` state remains in the chain forever
|
||
under its original pubkey; rejoin requires a fresh keypair and
|
||
fresh stake.
|
||
- ``challenged`` is a **transient** state. The challenge window
|
||
bounds resolution time (§4.4); ambiguous challenges resolve
|
||
in favor of the validator by default (presumption of innocence).
|
||
- ``unbonding`` blocks signing. A validator that has initiated exit
|
||
cannot sign new acceptances; this prevents the
|
||
sign-then-exit-then-let-challenge-bounce attack.
|
||
- Bond and unbond windows are governance parameters in
|
||
``consensus_policy_hash``; changing them invalidates the policy
|
||
hash and therefore which ``consensus_events`` rows verify under
|
||
it.
|
||
|
||
|
||
Part 4 — Acceptance protocol
|
||
----------------------------
|
||
|
||
§4.1 Proposer submission
|
||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
A proposer (any actor — does not need to be a validator) submits:
|
||
|
||
.. code-block:: text
|
||
|
||
{
|
||
"candidate_root": "<C(M') SHA-256>",
|
||
"parent_root": "<C(M_t) SHA-256>",
|
||
"eval_digest": "<canonical eval-suite digest>",
|
||
"frontier_coverage_diff": "<ε-coverage delta blob>",
|
||
"fork_score_self_reported": {<ScoredFork breakdown>},
|
||
"proposer_signature": "<sig over above fields>"
|
||
}
|
||
|
||
The proposer is bound to the submission by their signature; this
|
||
prevents anonymous spam. The self-reported ``fork_score`` is
|
||
*advisory* — validators re-compute it independently during audit-
|
||
replay (§4.3).
|
||
|
||
§4.2 Layered fitness floor
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Per ticket #000012 §2.2 (Recommendation C), every candidate must
|
||
clear a **canonical floor** before validators are obligated to
|
||
audit. The canonical floor is published in the v8 substrate paper
|
||
and pinned in ``consensus_policy_hash``:
|
||
|
||
.. list-table::
|
||
:header-rows: 1
|
||
:widths: 42 58
|
||
|
||
* - Floor metric
|
||
- Threshold
|
||
* - ``Δ5S`` (5-syndrome battery)
|
||
- ≥ 0 (no regression)
|
||
* - ``Δ5T`` (5-transfer battery)
|
||
- ≥ 0 (no regression)
|
||
* - ``Δ5F`` (5-falsification battery)
|
||
- ≥ 0 (no regression)
|
||
* - ε-coverage at published frontiers
|
||
- ≥ floor specified per-frontier
|
||
* - ``audit_completeness``
|
||
- = 1.0 (every state-change row hashes through ``audit_events``)
|
||
* - ``hard_regression_flag``
|
||
- = false (no ScoredFork hard regress)
|
||
|
||
A proposer can additionally declare lab-specific ceiling metrics
|
||
(e.g., "this upgrade is for combinatorial reasoning; please verify
|
||
combinatorics@v1 round-trip rate"). Validators that audit the
|
||
ceiling metrics earn bonus rewards (§6.3); ceiling failures do not
|
||
trigger slashing.
|
||
|
||
§4.3 Audit replay
|
||
~~~~~~~~~~~~~~~~~
|
||
|
||
Each ``active`` validator independently:
|
||
|
||
1. Pulls ``C(M')`` and verifies its content-address matches the
|
||
submitted root.
|
||
2. Runs the canonical floor benchmarks against ``M'``. This is a
|
||
deterministic procedure pinned by ``eval_digest`` — two honest
|
||
validators MUST produce byte-identical replay digests.
|
||
3. Computes ``ScoredFork(parent=M_t, child=M')`` using the
|
||
``arborist.substrate.fork_score`` reference implementation.
|
||
4. Compares its replay against the proposer's self-reported
|
||
``fork_score``. Discrepancy ≥ 5pp (per the SIGNAL_FLOOR convention
|
||
from ``docs/bench-maxing.md``) flags a soft mismatch; ≥ 20pp or
|
||
any hard-regression-flag flip triggers automatic rejection.
|
||
5. Signs ``accept`` or ``reject`` with its validator key, attaching
|
||
its replay digest.
|
||
|
||
§4.4 Quorum & finalization
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The signing window is bounded (governance parameter; recommend
|
||
24h). Within the window, signatures are gossiped via the mesh wire
|
||
format (§7). When ≥ 2/3 of active stake signs ``accept`` AND no
|
||
``reject`` exceeds 1/3, the candidate finalizes:
|
||
|
||
.. code-block:: text
|
||
|
||
accept_stake ≥ (2/3) · total_active_stake
|
||
AND reject_stake < (1/3) · total_active_stake
|
||
⇒ FINALIZE(C(M')) at current consensus_events tip
|
||
|
||
Finalization mints an aggregate signature (BLS-style if a threshold
|
||
key was bootstrapped, otherwise concatenated individual sigs) and
|
||
writes one row to ``consensus_events`` chained against the previous
|
||
tip. Per §2.3 the row body is canonical-encoded; ``event_hash`` is
|
||
SHA-256 of ``prev_event_hash || canonical(body)``.
|
||
|
||
§4.5 Liveness floor
|
||
~~~~~~~~~~~~~~~~~~~
|
||
|
||
If the signing window elapses without quorum, the candidate is
|
||
marked ``CHALLENGED-OR-STALLED`` and re-proposed at the next
|
||
checkpoint. Repeated failures (governance parameter; recommend 3)
|
||
trigger automatic rejection without slashing — partition-tolerance
|
||
floor.
|
||
|
||
|
||
Part 5 — Challenge protocol
|
||
---------------------------
|
||
|
||
§5.1 Challenge window
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
After finalization, a **challenge window** opens (governance
|
||
parameter; recommend 7 days). Anyone — validator or not — may
|
||
submit counter-evidence claiming one of the signing validators
|
||
misreported its audit-replay digest.
|
||
|
||
§5.2 Counter-evidence shape
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
.. code-block:: json
|
||
|
||
{
|
||
"challenged_candidate_root": "<C(M') SHA-256>",
|
||
"challenged_validator_pubkey": "<validator pubkey>",
|
||
"challenger_replay_digest": "<independent replay result>",
|
||
"discrepancy_evidence": "<diff between challenger and
|
||
challenged digests>",
|
||
"challenger_signature": "<sig over above fields>"
|
||
}
|
||
|
||
§5.3 Resolution
|
||
~~~~~~~~~~~~~~~
|
||
|
||
The challenge is resolved by the **next active validator set**
|
||
(excluding the challenged validator):
|
||
|
||
1. Each adjudicating validator runs its own audit-replay on
|
||
``C(M')`` and compares against both the challenged digest and
|
||
the challenger digest.
|
||
2. ≥ 2/3 of the adjudicating set agree on which side is honest →
|
||
challenge resolves in favor of that side.
|
||
3. If the challenged digest disagrees with the adjudicators by
|
||
≥ 5pp, the challenged validator is slashed per the offense-class
|
||
schedule (§6.2). All other validators that signed the same
|
||
``accept`` are also re-audited; if their replay disagrees too,
|
||
they too are slashed.
|
||
|
||
§5.4 Challenger reward
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
A successful challenger earns a slice of the slashed stake
|
||
(governance parameter; recommend 30%). This is a positive incentive
|
||
for independent audit; without it, the challenge protocol decays.
|
||
|
||
§5.5 Frivolous challenges
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
If the adjudicating set rules the challenge frivolous (challenger
|
||
digest matches the challenged digest within 5pp), the challenger
|
||
forfeits a small **challenge bond** (governance parameter; recommend
|
||
0.5% of the slashing-fraction-equivalent). Without this, an
|
||
adversary spams challenges to grief validators.
|
||
|
||
Acceptance row promotion to ``finalized`` requires the challenge
|
||
window to elapse without successful challenge. Until then the row
|
||
sits in ``finalized-pending`` state; consumers SHOULD wait for the
|
||
window to close before relying on the candidate.
|
||
|
||
|
||
Part 6 — Stake mechanics
|
||
------------------------
|
||
|
||
§6.1 Bond & unbond windows
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
================ =================================== ===========
|
||
Window Recommended duration Rationale
|
||
================ =================================== ===========
|
||
``bond`` 7 days Time for
|
||
fraud-on-
|
||
stake-deposit
|
||
challenges.
|
||
``unbond`` 2 × challenge_window Outstanding
|
||
challenges
|
||
must land
|
||
before exit.
|
||
``challenge`` 7 days Independent
|
||
audit
|
||
feasibility
|
||
per fox's
|
||
review note.
|
||
================ =================================== ===========
|
||
|
||
§6.2 Slashing schedule
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
================================ ==================================
|
||
Offense class Slashing fraction (recommended)
|
||
================================ ==================================
|
||
Audit-replay digest mismatch 30% of stake
|
||
Double-signing (accept + reject) 50% of stake
|
||
Conflicting acceptances (fork) 100% of stake
|
||
Equivocation under challenge 100% of stake
|
||
================================ ==================================
|
||
|
||
Fractions are *recommended*; calibration is a governance decision.
|
||
The paper pins the **shape** (single-offense → partial slash;
|
||
fork-equivocation → full slash) and the **monotonic ordering** (a
|
||
heavier offense always slashes ≥ a lighter offense's fraction).
|
||
|
||
§6.3 Reward distribution
|
||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
================================================ ====================
|
||
Action Reward (recommended)
|
||
================================================ ====================
|
||
Honest accept signature on finalized candidate proportional to stake
|
||
Honest reject signature on rejected candidate proportional to stake
|
||
Lab-ceiling-metric audit (bonus) flat per audit
|
||
Successful challenge (slash recipient) 30% of slashed stake
|
||
Frivolous challenge (forfeit bond) -0.5% slash-equiv
|
||
================================================ ====================
|
||
|
||
Rewards fund validator operation (compute for audit-replay,
|
||
bandwidth for mesh gossip). Without rewards, the validator set
|
||
decays to zero.
|
||
|
||
§6.4 Stake cap & sqrt-weighting (optional)
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Per ticket #000012 §6 risk note, raw stake-weighting risks
|
||
plutocratic capture. Two recommended mitigations, neither mandatory:
|
||
|
||
- **Stake cap**: cap any single validator's effective stake at
|
||
``X%`` of total (recommend 5%).
|
||
- **Sqrt-stake**: convex weighting (``effective_weight =
|
||
sqrt(raw_stake)``) so a single 100x staker counts as 10 unit
|
||
stakers, not 100.
|
||
|
||
These are governance-parameter knobs in ``consensus_policy_hash``;
|
||
the paper does not mandate which (or whether) to enable. Different
|
||
deployments may prefer different shapes.
|
||
|
||
|
||
Part 7 — Fork choice rule (GRANDPA-style)
|
||
-----------------------------------------
|
||
|
||
§7.1 Finalization is permanent
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Once ≥ 2/3 active stake signs ``accept`` AND the challenge window
|
||
elapses without successful challenge, the ``consensus_events`` row
|
||
is **finalized**. No reorg is possible past a finalized row.
|
||
|
||
This is the GRANDPA-style finality property (Polkadot / Kusama).
|
||
Trade-off: latency is higher than Bitcoin-style longest-chain
|
||
(at minimum: signing window + challenge window — recommend 1 + 7 =
|
||
8 days), but reorg risk past finality is *cryptographically zero*
|
||
under the 2/3-honest-stake assumption.
|
||
|
||
§7.2 Pre-finality fork choice
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
During the signing window, multiple candidates may be in flight
|
||
against the same parent. Validators MUST refuse to sign a candidate
|
||
whose parent is not finalized. This prevents:
|
||
|
||
- Long-range reorgs (validators cannot sign for a parent that
|
||
hasn't been finalized — no way to back-write history).
|
||
- Equivocation hazards (a validator signing on two parents at the
|
||
same checkpoint is automatically slashable, since both parents
|
||
cannot finalize).
|
||
|
||
§7.3 Liveness recovery
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
If finalization stalls on a parent (e.g., 1/3 of stake offline), no
|
||
child can finalize either. The protocol recovers when the offline
|
||
stake either rejoins or is slashed for non-participation
|
||
(governance parameter; recommend slashing after ``N`` missed
|
||
signing windows, ``N`` = 3).
|
||
|
||
This caps liveness loss at ``3 × signing_window = 3 days`` under
|
||
recommended parameters.
|
||
|
||
|
||
Part 8 — Mesh wire format extension
|
||
-----------------------------------
|
||
|
||
§8.1 New message kinds
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The arborist mesh wire format (``arborist/mesh/wire.py``) carries
|
||
per-shard sync today. v8 adds three new top-level message kinds:
|
||
|
||
================================ ==================================
|
||
Message kind Body
|
||
================================ ==================================
|
||
``v8_validator_announce`` Stake deposit announcement;
|
||
triggers ``bond`` window.
|
||
``v8_acceptance_signature`` Single validator's accept/reject
|
||
sig on a candidate during a
|
||
signing window.
|
||
``v8_challenge`` Counter-evidence submission
|
||
during a challenge window.
|
||
================================ ==================================
|
||
|
||
§8.2 Canonical encoding
|
||
~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Every message body is canonical-JSON (SQD A1) before signing. The
|
||
aggregate signature (when threshold-key is bootstrapped) is BLS-
|
||
flavored; otherwise the protocol falls back to a sorted-concat of
|
||
individual validator signatures, length-prefixed.
|
||
|
||
A consumer that wants to verify a finalized ``consensus_events``
|
||
row independently:
|
||
|
||
1. Parse the row's ``validator_signatures`` field.
|
||
2. For each signature, look up the signing validator's pubkey from
|
||
the ``consensus_events`` chain at the signing time.
|
||
3. Verify each signature against the canonical encoding of the row
|
||
body (minus the signature field).
|
||
4. Aggregate stake-weighted; confirm ≥ 2/3 active stake at
|
||
signing time.
|
||
|
||
Step 2 requires the validator set's stake snapshot at the row's
|
||
``finalized_at`` timestamp. v8 commits a **stake snapshot** in
|
||
every finalized row's body so step 2 is local-only — no separate
|
||
lookup table needed.
|
||
|
||
§8.3 Bandwidth profile
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
At recommended parameters (``signing_window`` = 1 day,
|
||
``challenge_window`` = 7 days, ``checkpoint_cadence`` = weekly),
|
||
the gossip load is dominated by ``v8_acceptance_signature`` traffic
|
||
during the signing window. For a 100-validator deployment with
|
||
1 KB per signature message, peak load is
|
||
|
||
.. code-block:: text
|
||
|
||
100 validators · 1 KB · gossip_fanout (8) = 800 KB per checkpoint
|
||
|
||
— well within arborist mesh's per-shard sync budget.
|
||
|
||
Challenge-window traffic is bursty but bounded (any single party
|
||
can submit O(1) challenges per finalized row).
|
||
|
||
|
||
Part 9 — BFT analysis
|
||
---------------------
|
||
|
||
§9.1 Safety
|
||
~~~~~~~~~~~
|
||
|
||
**Claim:** No two conflicting candidates can both finalize under
|
||
the 2/3 quorum rule + GRANDPA-style finality, assuming
|
||
≤ 1/3 Byzantine stake.
|
||
|
||
**Proof sketch:** Suppose ``M'_1`` and ``M'_2`` both finalize on
|
||
the same parent. Each requires ≥ 2/3 stake to accept. Total stake
|
||
signing accept across both candidates ≥ 4/3 — impossible without
|
||
overlap. Any validator signing both is automatically slashable per
|
||
§6.2 (conflicting acceptances → 100% slash). Therefore the
|
||
overlapping validators are Byzantine. By assumption Byzantine
|
||
stake ≤ 1/3 < 4/3 − 1 = 1/3 needed-overlap, contradiction.
|
||
Conclusion: at most one candidate finalizes per parent.
|
||
|
||
§9.2 Liveness
|
||
~~~~~~~~~~~~~
|
||
|
||
**Claim:** Under partition where < 1/3 stake is offline,
|
||
finalization progresses within bounded time.
|
||
|
||
**Proof sketch:** ≥ 2/3 stake is online and honest. Honest
|
||
validators sign on any candidate clearing the canonical floor.
|
||
Quorum is reached within the signing window. After the
|
||
challenge window closes without successful challenge, finalization
|
||
completes. Bound: ``signing_window + challenge_window``
|
||
(recommended: 8 days).
|
||
|
||
**Claim:** Under partition where ≥ 1/3 stake is offline,
|
||
finalization stalls but does not regress.
|
||
|
||
**Proof sketch:** Without 2/3 quorum, no candidate finalizes.
|
||
Pre-finality state remains accessible (downstream consumers see
|
||
"pending" status). Once the offline stake rejoins (or is slashed
|
||
per §7.3), the protocol resumes.
|
||
|
||
§9.3 Sybil resistance
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Stake bonding is the Sybil bound. An adversary controlling N
|
||
identities still controls only ``sum(N_stake_i)`` weight, not ``N``
|
||
votes. With optional sqrt-weighting (§6.4), even concentrated
|
||
stake is diluted.
|
||
|
||
§9.4 Bootstrap honesty
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The protocol cannot bootstrap from zero. v8 deployment must specify
|
||
an **initial validator set** with explicit honesty assumption. The
|
||
permacomputer-mission default is:
|
||
|
||
- Bootstrap from a small (5-7) permissioned validator set, all
|
||
publicly identified, with stake deposited to a multi-sig
|
||
bootstrap escrow.
|
||
- After ``M`` checkpoints (recommend ``M`` = 12, ~3 months at
|
||
weekly cadence) without successful challenges, the bootstrap
|
||
set's stake unlocks for normal unbond/slash semantics.
|
||
- New validators join permissionlessly during bootstrap; they don't
|
||
earn rewards until bootstrap closes (prevents bootstrap-stake
|
||
dilution attacks).
|
||
|
||
§9.5 Re-staking attacks
|
||
~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
A validator may stake the same capital across multiple v8 instances
|
||
(e.g., one per major Merkle-AGI lineage). Out of scope for this
|
||
paper — the cross-instance slashing accumulator is its own
|
||
follow-up if multi-instance v8 deployments emerge.
|
||
|
||
|
||
Part 10 — Worked example (appendix)
|
||
-----------------------------------
|
||
|
||
§10.1 Setup
|
||
~~~~~~~~~~~
|
||
|
||
Imaginary deployment: 7 active validators ``V1..V7``, each with 100
|
||
stake units (700 total). 2/3 threshold = 467 stake. Recommended
|
||
parameters: 1-day signing window, 7-day challenge window.
|
||
|
||
§10.2 Cycle
|
||
~~~~~~~~~~~
|
||
|
||
**Day 0**: Proposer P submits ``C(M'_1)`` against finalized parent
|
||
``C(M_t)``. Self-reported fork_score = +0.42.
|
||
|
||
**Day 0-1 (signing window)**: ``V1..V6`` independently audit-replay,
|
||
each producing fork_score in [+0.40, +0.43] (within signal floor).
|
||
All six sign ``accept`` with their replay digests. ``V7`` is
|
||
offline; doesn't sign. Stake signing accept = 600 ≥ 467.
|
||
|
||
**Day 1**: Quorum reached. Aggregate signature minted; new row
|
||
written to ``consensus_events`` chain. Row status:
|
||
``finalized-pending``.
|
||
|
||
**Day 1-8 (challenge window)**: External party C submits
|
||
counter-evidence claiming ``V3``'s replay digest disagrees with
|
||
canonical floor by 15pp.
|
||
|
||
**Day 8**: Adjudication set ``V1, V2, V4, V5, V6`` (excluding
|
||
challenged ``V3`` and offline ``V7``) re-audits. Each finds C's
|
||
replay matches the canonical result; ``V3``'s replay disagrees by
|
||
15pp.
|
||
|
||
**Day 8**: Adjudication majority (≥ 2/3 of 5 = 4) confirms
|
||
``V3``'s misreporting. ``V3`` is slashed 30% (audit-mismatch
|
||
class). C earns 30% of slashed stake = 9 units.
|
||
|
||
**Day 8**: The accept signature is still valid (V3's signature is
|
||
struck, remaining stake signing accept = 500 ≥ 467). Row
|
||
finalizes.
|
||
|
||
**Day 8+**: Downstream consumers verify the finalized row by
|
||
walking the ``consensus_events`` chain. They see ``M'_1`` is the
|
||
canonical successor of ``M_t``.
|
||
|
||
§10.3 What the audit ledger looks like
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
.. code-block:: text
|
||
|
||
consensus_events row at chain height N+1:
|
||
prev_event_hash: <sha256 of row N>
|
||
candidate_root: <C(M'_1)>
|
||
parent_root: <C(M_t)>
|
||
proposer_id: P
|
||
fork_score: {breakdown ...}
|
||
validator_signatures: [<V1>, <V2>, <V4>, <V5>, <V6>]
|
||
(V3 struck post-slash; V7 offline)
|
||
audit_replay_digest: <canonical replay sha256>
|
||
verdict: "FINALIZED"
|
||
finalized_at: <Day 8 ts>
|
||
consensus_policy_hash: <hash of v8 policy params>
|
||
|
||
event_hash = sha256( prev_event_hash || canonical(body) )
|
||
|
||
A future consumer pulling this row can verify:
|
||
|
||
1. Each validator signature against the canonical body.
|
||
2. Stake-weighted aggregate against the stake-snapshot inside the
|
||
body. ≥ 2/3.
|
||
3. Chain integrity against the previous row's ``event_hash``.
|
||
|
||
No external lookup required.
|
||
|
||
|
||
Part 11 — Out of scope
|
||
----------------------
|
||
|
||
- **Implementation**. Code lands in 3-4 follow-up tickets:
|
||
validator state-machine SQL + state-transition functions; mesh
|
||
wire format extension; audit-replay harness binding; slashing
|
||
accountant.
|
||
- **Economic parameter calibration**. Stake amounts, slashing
|
||
fractions, reward rates, challenge bonds — governance decisions,
|
||
not paper-pinned. The paper specifies the *form*; deployments
|
||
calibrate.
|
||
- **Cross-chain anchoring**. Publishing finalized
|
||
``consensus_events`` rows to Bitcoin / Ethereum / etc. is an
|
||
optional bolt-on. Out of scope here.
|
||
- **Frontier benchmark fixture selection**. Covered by ticket
|
||
#000021. v8 references the fixtures; doesn't curate them.
|
||
- **Bootstrap-set membership**. The actual identities of the
|
||
bootstrap validators are a governance decision for the deploying
|
||
consortium.
|
||
- **Cross-instance slashing accumulator**. Re-staking attacks
|
||
across multiple v8 instances. Future work; emerges when more than
|
||
one v8 instance exists.
|
||
- **Branch-set persistence**. Ticket #000012 Phase 1c (proposed,
|
||
not yet open). Multi-branch fork scoring at the same checkpoint.
|
||
Doc-only proposal in #000012; lands as its own follow-up if
|
||
operator pressure surfaces.
|
||
|
||
|
||
Closure
|
||
-------
|
||
|
||
Closure criterion (#000012 §4.1): this document plus the
|
||
``arborist.substrate.fork_score`` module (Phase 1a; landed 2026-05-08)
|
||
constitute the v8 substrate paper. The reference doc
|
||
``docs/v8-fork-score.md`` carries the formula details; this paper
|
||
carries the consensus / acceptance / slashing protocol.
|
||
|
||
Open questions tracked separately:
|
||
|
||
- **Initial validator set composition** — governance, not paper.
|
||
- **Threshold-key ceremony** — needed for BLS-aggregate signatures;
|
||
if not bootstrapped, fallback to sorted-concat signatures is
|
||
fully functional (just larger payloads).
|
||
- **ZK-replay** — verifiers proving canonical-floor replay without
|
||
re-running it locally. Captured in #000016 (ZK frontier proof);
|
||
optional optimization on top of v8.
|
||
- **Policy-hash transition mechanics** — when
|
||
``consensus_policy_hash`` changes (e.g., new floor metric added),
|
||
how do in-flight signing windows handle the transition? Recommend:
|
||
policy changes take effect at the next checkpoint after a
|
||
successful finalization under the new policy; in-flight
|
||
signatures honor the policy hash they were submitted under.
|
||
|
||
Implementation tickets that cite this paper land later — one per
|
||
validator-state-machine, mesh-wire-format extension, audit-replay
|
||
harness, and slashing accountant.
|
||
|
||
Status: closure-draft 2026-05-10. Ready for fox + downstream
|
||
review.
|