v8: ticket #000012 Phase 1a — ForkScore consumes the new bench substrate
Pure scoring function over (parent, child) BatteryResult bundles. Closes
the scoring half of fox's 2026-05-08 frontier note ("how does an organism
mutation become canonical?") — the canonicalization half (validator
state, acceptance, fork choice) stays under #000012 as the v8 paper.
Formula:
ForkScore = α·Δ5S + β·Δ5T + γ·Δ5F + δ·SelfModelCalibration
+ ε·AuditCompleteness + ζ·ValidatorDiversity
- η·RegressionPenalty - θ·CapitalCostPenalty
- ι·SecurityRisk - κ·Complexity - λ·MemoryInvalidation
Consumes every metric this session shipped:
- 5S/5T/5F sub-battery rates → Δ-rate per battery (mean over subs)
- adaptation_efficiency_mean_finite + adaptation_efficiency_infinite_count
→ 5F efficiency-aware bonus (damped + capped via INFINITE_BONUS_CAP)
- adaptation_efficiency_neg_infinite_count > parent → NEG_INF_REGRESSION
flag → automatic REJECT (free regression unsafe)
- capital_delta from #000020 ledger
- memory_invalidation_count from #000017
Verdict thresholds:
- ACCEPT when score >= SIGNAL_FLOOR (5pp; matches docs/bench-maxing.md)
- MARGINAL [0, SIGNAL_FLOOR)
- REJECT on negative score OR hard-regression OR neg-inf efficiency
Hard-regression flag fires if any single sub-battery rate drops by
>= 5pp parent→child, regardless of net score. CLI exits 1 on REJECT
so CI gates run `arborist v8 score` directly.
Surface:
- arborist/v8/{fork_score,weights}.py
- WeightSet dataclass with α…λ + DEFAULT_WEIGHTS (single-validator
tuned: ζ=0, ι=0, κ=0; η=2.0 weighted heavier than improvement
weights; θ=0.5 modest cost penalty)
- weights_from_dict accepts "lambda" key (Python reserved word)
- bench_result_to_metrics adapter from runner --all JSON
- CLI: arborist v8 score --parent P.json --child C.json [--weights W.json]
with override flags for SelfModelCalibrationGain, AuditCompleteness,
capital_delta, memory_invalidation_count, etc.
Reference: docs/v8-fork-score.md (formula + term semantics + verdict
matrix + Phase-1a vs Phase-1b boundary).
Tests: tests/test_v8_fork_score.py (25 cases)
- adapter from runner JSON
- all verdict paths (ACCEPT / MARGINAL / REJECT)
- hard-regression flag
- neg-inf efficiency rejection
- inf-bonus capping
- weight tuning (alpha scales 5s, eta scales penalty)
- breakdown completeness (11 terms) sums to score
- CLI smoke + explicit weights + REJECT exit code
- Determinism: same inputs → same output
Full suite: 1186 passed, 36 skipped.
#000012 status: in progress (Phase 1a landed, consensus paper still open).
This commit is contained in:
parent
fea761c577
commit
ea39455d75
8 changed files with 1286 additions and 7 deletions
|
|
@ -74,7 +74,7 @@ Newest first. Update on every open/close.
|
|||
| #000015 | π* domain library + cross-domain composition | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000014 | SelfModel: schema, falsification, integration | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000013 | Spatial-temporal substrate (Merkle-AGI v7-W) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000012 | Selection & consensus protocol (Merkle-AGI v8) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
| #000012 | Selection & consensus protocol (Merkle-AGI v8) | in progress · Phase 1a (ForkScore) landed 2026-05-08 | 2026-05-07 | — |
|
||||
| #000011 | SOFT_PREFLIGHT_HINT model-assisted sidecar | closed · landed 2026-05-04 (zero-shot full impl) | 2026-05-04 | D1 (preserves) |
|
||||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
||||
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses) | 2026-05-03 | D3, D4 |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000012 — Selection & consensus protocol (Merkle-AGI v8)
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Status:** in progress · Phase 1a (ForkScore) landed 2026-05-08; consensus paper still open
|
||||
**Opened:** 2026-05-07
|
||||
**Scope:** Spec for the loop-closing consensus protocol that turns the v7
|
||||
substrate + v9.8 runtime into actual Darwinian selection across multiple
|
||||
|
|
@ -256,9 +256,52 @@ follow-up tickets that cite this one.
|
|||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** This is a research-paper-scope ticket.
|
||||
Implementation is gated on fox approving the paper's design choices
|
||||
and committing v8 substrate work.
|
||||
**In progress · Phase 1a landed 2026-05-08.**
|
||||
|
||||
Closure criterion: `docs/merkle-agi-v8-consensus.rst` lands and is
|
||||
reviewed.
|
||||
### Phase 1a — ForkScore (landed)
|
||||
|
||||
Per fox's 2026-05-08 review note that the substrate is "good enough
|
||||
to support v8 selection/fork-choice design," shipping the scoring
|
||||
function ahead of the consensus paper:
|
||||
|
||||
- `arborist/v8/fork_score.py` — pure ScoredFork dataclass + scoring
|
||||
function over (parent, child) BatteryResult bundles. Consumes
|
||||
every metric this session shipped: 5S/5T/5F sub-battery rates,
|
||||
`adaptation_efficiency_*` and `feedback_efficiency_*` (incl.
|
||||
inf-aware aggregation per fbd99a8 review), capital-cost delta,
|
||||
memory-invalidation count.
|
||||
- `arborist/v8/weights.py` — `WeightSet` dataclass with α…λ +
|
||||
`DEFAULT_WEIGHTS` (single-validator-tuned) + `from_dict`
|
||||
adapter handling the `"lambda"`/`lambda_` Python-reserved-word
|
||||
issue.
|
||||
- CLI: `arborist v8 score --parent P.json --child C.json
|
||||
[--weights W.json]`. Exits 1 on REJECT (CI-gateable).
|
||||
- Verdict thresholds: ACCEPT (≥ SIGNAL_FLOOR=0.05), MARGINAL
|
||||
([0, SIGNAL_FLOOR)), REJECT (negative score OR hard-regression
|
||||
flag OR `NEG_INF_REGRESSION` flag).
|
||||
- Reference doc: `docs/v8-fork-score.md`.
|
||||
- Tests: `tests/test_v8_fork_score.py` — 25 cases covering the
|
||||
adapter, all verdict paths, hard-regression detection,
|
||||
inf-bonus capping, neg-inf rejection, weight-set tuning,
|
||||
breakdown completeness, CLI smoke. Full suite: 1186 passed.
|
||||
|
||||
### Phase 1b — Consensus paper (still open)
|
||||
|
||||
Closure criterion: `docs/merkle-agi-v8-consensus.rst` lands with
|
||||
validator state machine, acceptance protocol, challenge protocol,
|
||||
fork-choice rule (GRANDPA-style), slashing mechanics, mesh wire
|
||||
format extension. Per §1 of this ticket, `arborist/v8/fork_score.py`
|
||||
is the substrate the paper cites; the paper itself is still
|
||||
research-scope.
|
||||
|
||||
What's deliberately NOT in Phase 1a:
|
||||
|
||||
- Validator state machine (bonding / signing / slashing).
|
||||
- Acceptance protocol.
|
||||
- Challenge protocol.
|
||||
- Fork-choice rule.
|
||||
- Mesh wire format extensions.
|
||||
- Stake mechanics + economic incentives.
|
||||
- Cross-validator ZK proof exchange.
|
||||
|
||||
These belong to the v8 paper itself.
|
||||
|
|
|
|||
243
docs/v8-fork-score.md
Normal file
243
docs/v8-fork-score.md
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
# v8 ForkScore — Phase 1a reference
|
||||
|
||||
Reference for `arborist.v8.fork_score` (ticket #000012 Phase 1a).
|
||||
**Pure scoring function** over a (parent, child) BatteryResult pair.
|
||||
No validator state machine, no consensus protocol, no acceptance
|
||||
ledger — those are commissioned by the v8 paper itself, still open
|
||||
under #000012.
|
||||
|
||||
This document is the **canonical source** for the formula, term
|
||||
semantics, and verdict thresholds. Tests in
|
||||
`tests/test_v8_fork_score.py` enforce them.
|
||||
|
||||
## 1. Formula
|
||||
|
||||
```text
|
||||
ForkScore =
|
||||
α · Δ5S
|
||||
+ β · Δ5T
|
||||
+ γ · Δ5F # incl. efficiency-aware bonus
|
||||
+ δ · SelfModelCalibrationGain
|
||||
+ ε · AuditCompleteness
|
||||
+ ζ · ValidatorDiversity
|
||||
- η · RegressionPenalty
|
||||
- θ · CapitalCostPenalty
|
||||
- ι · SecurityRiskPenalty # reserved (Phase 1a = 0)
|
||||
- κ · ComplexityPenalty # reserved (Phase 1a = 0)
|
||||
- λ · MemoryInvalidationPenalty
|
||||
```
|
||||
|
||||
Sign convention: weights are non-negative; the sign of each term is
|
||||
encoded in the formula. `breakdown` keys in the output reflect the
|
||||
sign:
|
||||
|
||||
```json
|
||||
{
|
||||
"alpha_x_delta_5s": 0.025,
|
||||
"minus_eta_x_regression_penalty": -0.0,
|
||||
"minus_lambda_x_memory_invalidation": -0.5,
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
## 2. Term semantics
|
||||
|
||||
### 2.1 Δ5S, Δ5T, Δ5F (Δ-rate over each battery)
|
||||
|
||||
**Δ5S** = mean of per-sub-battery rate deltas across the 5S battery:
|
||||
|
||||
```
|
||||
syntax parse_pass_rate
|
||||
semantics equivalence_recovery_rate
|
||||
syllogism step_validity_rate
|
||||
synthesis derivation_pass_rate
|
||||
semiotics invariance_under_swap
|
||||
```
|
||||
|
||||
**Δ5T** = same shape, **canonical Dav1DPrometheus vocabulary** (the
|
||||
legacy `transfer` sub-battery is intentionally excluded so v2
|
||||
fixtures drive the score):
|
||||
|
||||
```
|
||||
transfer-learning transfer_learning_success_rate
|
||||
triangulation triangulation_agreement_rate
|
||||
truthtables truth_table_coverage_rate
|
||||
transitivity full_chain_pass_rate
|
||||
time temporal_context_preservation_rate
|
||||
```
|
||||
|
||||
**Δ5F** = mean of per-sub-battery rate deltas **plus** an
|
||||
efficiency-aware bonus:
|
||||
|
||||
```
|
||||
function function_pass_rate
|
||||
finetuning adaptation_improvement_rate
|
||||
falsification error_detection_rate
|
||||
formulate structural_match_rate
|
||||
feedback-loop integration_coverage_rate
|
||||
```
|
||||
|
||||
The efficiency bonus is computed from the metrics shipped under
|
||||
ticket #000025's zero-cost guards:
|
||||
|
||||
- `adaptation_efficiency_mean_finite` — Δ in mean efficiency,
|
||||
damped by 0.1 to keep efficiency from dominating rate-style
|
||||
improvements.
|
||||
- `adaptation_efficiency_infinite_count` — increase = bonus capped
|
||||
at `INFINITE_BONUS_CAP` (default 1.0). +∞ buckets must not pollute
|
||||
the ranking.
|
||||
- `adaptation_efficiency_neg_infinite_count` — increase = `NEG_INF_REGRESSION`
|
||||
flag. **Hard reject** (a free regression is unsafe).
|
||||
|
||||
Same shape applies to `feedback_efficiency_*` keys.
|
||||
|
||||
### 2.2 SelfModelCalibrationGain
|
||||
|
||||
Distance closed between SelfModel capability claim threshold and
|
||||
measured value. Phase 1a callers pass this as a scalar (default 0).
|
||||
Phase 1b will compute it from `selfmodel_capability_claims` history.
|
||||
|
||||
### 2.3 AuditCompleteness
|
||||
|
||||
Fraction of state-changing ops with an `audit_event` row. Phase 1a
|
||||
caller-supplied; Phase 1b reads from the chain directly. Range
|
||||
`[0, 1]`; missing rows visible in the audit chain bring this below 1.
|
||||
|
||||
### 2.4 ValidatorDiversity
|
||||
|
||||
Multi-validator agreement entropy. **Zero in single-validator
|
||||
mode** (Phase 1a default). v8 paper defines the multi-validator
|
||||
calibration. With ζ = 0 by default, this term has no effect on
|
||||
single-validator deployments.
|
||||
|
||||
### 2.5 RegressionPenalty
|
||||
|
||||
Sum over batteries of `max(0, -Δ_battery)`. A battery whose mean
|
||||
delta is negative contributes its absolute value to the penalty
|
||||
term (multiplied by η).
|
||||
|
||||
This is **softer** than the hard-regression flag (§3.2): if 4 of 5
|
||||
sub-batteries improve by 10pp and 1 drops by 8pp, the mean Δ is
|
||||
positive and `RegressionPenalty = 0` — but the hard flag still
|
||||
fires on that one sub-battery.
|
||||
|
||||
### 2.6 CapitalCostPenalty
|
||||
|
||||
Capital cost delta from the #000020 ledger. Phase 1a accepts a
|
||||
scalar `capital_delta`; positive means the child costs more. Phase
|
||||
1b will compute it from `capital_ledger.summary`.
|
||||
|
||||
### 2.7 SecurityRiskPenalty, ComplexityPenalty
|
||||
|
||||
**Reserved for Phase 1b+.** Defaults `ι = κ = 0`. No security-bench
|
||||
fixtures or LOC-delta surface exist in Phase 1a.
|
||||
|
||||
### 2.8 MemoryInvalidationPenalty
|
||||
|
||||
Count of `memory_records` rows the child fork would falsify. Phase
|
||||
1a accepts a scalar; Phase 1b will compute it by simulating the
|
||||
fork's verifier-method-root delta and counting memory_records that
|
||||
cite the changed verifier.
|
||||
|
||||
## 3. Verdicts
|
||||
|
||||
### 3.1 Verdict thresholds
|
||||
|
||||
| Score | Hard flags | Verdict |
|
||||
|---|---|---|
|
||||
| ≥ `SIGNAL_FLOOR` (0.05) | none | **ACCEPT** |
|
||||
| `[0, SIGNAL_FLOOR)` | none | **MARGINAL** (insufficient signal) |
|
||||
| `< 0` | any | **REJECT** |
|
||||
| any | hard-regression OR neg-inf efficiency | **REJECT** |
|
||||
|
||||
CLI exit code: `0` for ACCEPT/MARGINAL, `1` for REJECT — so CI
|
||||
gates can run `arborist v8 score …` directly.
|
||||
|
||||
### 3.2 Hard flags
|
||||
|
||||
- **REGRESSION_5S/5T/5F:** any single sub-battery dropping by ≥
|
||||
`HARD_REGRESSION_FLOOR` (default 5pp). Reported per sub-battery
|
||||
in `flags`.
|
||||
- **NEG_INF_REGRESSION:** `*_efficiency_neg_infinite_count`
|
||||
increased parent → child. Free regression is unsafe regardless
|
||||
of other gains.
|
||||
|
||||
A hard flag forces REJECT even when the net score is positive.
|
||||
Operators can disable hard-flag gating in Phase 1b by overriding
|
||||
the `_delta_*` helpers — but the soft `RegressionPenalty` term
|
||||
stays in the formula.
|
||||
|
||||
## 4. Default weights
|
||||
|
||||
```python
|
||||
WeightSet(
|
||||
alpha=1.0, # Δ5S
|
||||
beta=1.0, # Δ5T
|
||||
gamma=1.0, # Δ5F
|
||||
delta=0.5, # SelfModelCalibrationGain
|
||||
epsilon=0.3, # AuditCompleteness
|
||||
zeta=0.0, # ValidatorDiversity (off in single-validator)
|
||||
eta=2.0, # RegressionPenalty (heavy by design)
|
||||
theta=0.5, # CapitalCostPenalty
|
||||
iota=0.0, # SecurityRiskPenalty (reserved)
|
||||
kappa=0.0, # ComplexityPenalty (reserved)
|
||||
lambda_=0.5, # MemoryInvalidationPenalty
|
||||
)
|
||||
```
|
||||
|
||||
Override via JSON file passed to `--weights`:
|
||||
|
||||
```json
|
||||
{
|
||||
"alpha": 1.5,
|
||||
"eta": 5.0,
|
||||
"lambda": 1.0
|
||||
}
|
||||
```
|
||||
|
||||
(The JSON key `"lambda"` round-trips into `WeightSet.lambda_`
|
||||
because `lambda` is a Python reserved word.)
|
||||
|
||||
## 5. CLI
|
||||
|
||||
```bash
|
||||
arborist v8 score \
|
||||
--parent parent-bench.json \
|
||||
--child child-bench.json \
|
||||
[--weights weights.json] \
|
||||
[--capital-delta N] \
|
||||
[--selfmodel-calibration-gain N] \
|
||||
[--audit-completeness N] \
|
||||
[--memory-invalidation-count N]
|
||||
```
|
||||
|
||||
`parent-bench.json` and `child-bench.json` are the JSON output of
|
||||
`bench.batteries.runner --all` for each organism. The CLI prints
|
||||
the `ScoredFork` as JSON and exits 0 on ACCEPT/MARGINAL, 1 on
|
||||
REJECT.
|
||||
|
||||
## 6. What's NOT in Phase 1a
|
||||
|
||||
- Validator state machine (bonding / signing / slashing).
|
||||
- Acceptance protocol (proposal / quorum / finalization).
|
||||
- Challenge protocol (audit-replay disagreement).
|
||||
- Fork-choice rule (which of two competing finalizations wins).
|
||||
- Mesh wire format extensions for validator gossip.
|
||||
- Stake mechanics + economic incentives.
|
||||
- Cross-validator ZK proof exchange.
|
||||
|
||||
These are commissioned by the **v8 paper itself** — `docs/merkle-agi-v8-consensus.rst`, still open under ticket #000012. Phase 1a's
|
||||
scoring function is the substrate the v8 paper consumes; landing
|
||||
it now lets the paper cite measured values instead of stipulated
|
||||
ones.
|
||||
|
||||
## 7. Closure of the gap fox identified
|
||||
|
||||
The 2026-05-08 review of `fbd99a8` flagged: *"the immediate
|
||||
frontier has shifted from 'build the fitness surface' to 'define
|
||||
how an organism mutation becomes canonical under that fitness
|
||||
surface.'"*
|
||||
|
||||
Phase 1a closes the **scoring** half of that frontier. The
|
||||
**canonicalization** half (validator agreement, fork choice,
|
||||
slashing) remains under #000012 as the v8 paper deliverable.
|
||||
Loading…
Add table
Add a link
Reference in a new issue