refactor: arborist/v7+v8 → arborist/substrate (single topic dir)

fox's read: the version-prefixed namespace pattern (`arborist/v7/`,
`arborist/v8/`) coupled module location to the substrate-paper
version. That collided with the live SQLite schema version (v9.8)
and made readers ask "is this dir tracking schema or paper?" —
a real onboarding hazard surfaced when the v7 dir landed earlier
today (06c95a0) for #000035 Phase 1.

Resolution: collapse v7+v8 into one topic-named dir,
``arborist/substrate/``, which holds Merkle-AGI substrate primitives
that future paper specs require — decoupled from the paper version.

Moves
=====

  arborist/v7/anchor_prg.py   → arborist/substrate/anchor_prg.py
  arborist/v8/fork_score.py   → arborist/substrate/fork_score.py
  arborist/v8/weights.py      → arborist/substrate/weights.py

Empty v7/ + v8/ dirs deleted; their __init__.py docstrings folded
into the new arborist/substrate/__init__.py with an explanation of
why the version-prefixed pattern was retired.

Imports updated
===============

- arborist/cli.py:_cmd_v8_score — arborist.v8 → arborist.substrate
- arborist/substrate/fork_score.py — internal weights import
- tests/test_anchor_prg.py — module + module-docstring
- tests/test_v8_fork_score.py — three import lines

Docs updated
============

- docs/v8-fork-score.md — header note explaining the move
- docs/_source/v8-fork-score.rst — :class: ref updated
- docs/tickets/ticket-000012-selection-consensus-protocol.md — §7
  Phase 1a close-out paths refreshed (kept "Originally landed at
  arborist/v8/..." parenthetical so the historical record survives);
  §7 Phase 1b consensus-paper reference; §7 Phase 1c proposal §3
  read-API path
- docs/tickets/ticket-000035-prg-choice-phi-prg.md — §7 Phase 1
  close-out path refreshed (with full path-note explaining the
  move); §3.1 + §5 left as the original design log per CLAUDE.md
  "closed tickets stay in place as design log"

Left untouched
==============

- arborist/world/ — already topic-named; not version-prefixed; the
  v7-W reservation lives there with its own planned subdir layout.
- docs/tickets/ticket-000037-prometheus-sigma-...md §13 still refs
  ``arborist/v9/prometheus.py`` and ``arborist/v8/fork_score.py`` —
  fox has 792 lines of in-flight modifications on this file; those
  refs should refresh to ``arborist/substrate/`` when the in-flight
  edit lands. Avoiding interleaved edits.

Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1643; refactor preserved)
- make chain-check-shards → 0 across all 7 shards
- arborist.substrate namespace picked up by the existing
  pyproject.toml ``include = ["arborist*"]`` glob; no setup change.
This commit is contained in:
russell@unturf.com 2026-05-10 08:57:04 -04:00
parent 4d4e4d4249
commit 654d923da0
No known key found for this signature in database
13 changed files with 96 additions and 101 deletions

View file

@ -3041,11 +3041,11 @@ def _cmd_snapshot_diff(args: argparse.Namespace) -> int:
def _cmd_v8_score(args: argparse.Namespace) -> int: def _cmd_v8_score(args: argparse.Namespace) -> int:
"""Compute the v8 ForkScore over (parent, child) bench-result JSON files.""" """Compute the v8 ForkScore over (parent, child) bench-result JSON files."""
from arborist.v8 import ( from arborist.substrate import (
bench_result_to_metrics, bench_result_to_metrics,
fork_score, fork_score,
) )
from arborist.v8.weights import DEFAULT_WEIGHTS, from_dict as weights_from_dict from arborist.substrate.weights import DEFAULT_WEIGHTS, from_dict as weights_from_dict
with open(args.parent, "r", encoding="utf-8") as fh: with open(args.parent, "r", encoding="utf-8") as fh:
parent_payload = json.load(fh) parent_payload = json.load(fh)

View file

@ -0,0 +1,49 @@
"""Merkle-AGI substrate primitives — paper-spec'd cryptographic + decision
building blocks that future substrate papers (v7 plastic-training, v8
selection/consensus, v9 falsification controller, ) require.
Why one flat dir instead of paper-version subdirs: the substrate-paper
version (v7 / v8 / v9) and the live SQLite schema version (v9.8) are
two unrelated numbering schemes that share decimals. Version-prefixed
dirs (``arborist/v7/``, ``arborist/v8/``) made readers ask "is this
schema-v7 or paper-v7?" and made onboarding harder. Topic-named
``substrate/`` decouples the namespace from the paper version
entirely.
Each module names the paper section it implements + the ticket that
owns its design and acceptance criteria:
- :mod:`arborist.substrate.anchor_prg` HMAC-SHA-512 anchor map
φ_PRG (v7 plastic-training spec § 9.10; ticket #000035 + #000018
M1 mitigation).
- :mod:`arborist.substrate.fork_score` pure ScoredFork decision
function over (parent, child) BatteryResult bundles (v8 selection
/ consensus paper; ticket #000012 Phase 1a).
- :mod:`arborist.substrate.weights` :class:`WeightSet` dataclass +
:data:`DEFAULT_WEIGHTS` for the fork-score function above.
Future substrate primitives that arrive when their paper specs land
(v9 controller, v7-W frontier kernels, etc.) join this dir.
``arborist.world`` stays separate because its planned shape includes
its own subdirs (``world/pi_star/``, ``world/frontier/``,
``world/adapters/``) that's a substrate package, not a primitive.
"""
from __future__ import annotations
from arborist.substrate.fork_score import (
SIGNAL_FLOOR,
ScoredFork,
bench_result_to_metrics,
fork_score,
)
from arborist.substrate.weights import DEFAULT_WEIGHTS, WeightSet
__all__ = [
"DEFAULT_WEIGHTS",
"SIGNAL_FLOOR",
"ScoredFork",
"WeightSet",
"bench_result_to_metrics",
"fork_score",
]

View file

@ -1,14 +0,0 @@
"""v7 plastic-training primitives — Merkle-AGI v7 substrate building blocks.
Currently namespace stub. Public exports added as v7 lands:
- :mod:`arborist.v7.anchor_prg` HMAC-SHA-512 anchor-map for the M1
mitigation (ticket #000035 / #000018 §5.2 + §9.10).
The v7 plastic-training surface is still research-scope (see ticket
#000037 §17.2); this package houses the deterministic primitives the
spec depends on so they can land + be unit-tested ahead of the
deployment target.
"""
from __future__ import annotations

View file

@ -1,55 +0,0 @@
"""Merkle-AGI v8 fork-score (ticket #000012 Phase 1a).
Pure scoring function over (parent, child) BatteryResult bundles:
ForkScore =
α · Δ5S
+ β · Δ5T
+ γ · Δ5F
+ δ · SelfModelCalibrationGain
+ ε · AuditCompleteness
+ ζ · ValidatorDiversity
- η · RegressionPenalty
- θ · CapitalCostPenalty
- ι · SecurityRiskPenalty
- κ · ComplexityPenalty
- λ · MemoryInvalidationPenalty
This package ships ONLY the scoring function + CLI surface no
validator state machine, no acceptance protocol, no slashing, no
fork-choice rule. Those are commissioned by the v8 paper itself
(ticket #000012, still open).
What's here in Phase 1a:
- :class:`WeightSet` dataclass with αλ; defaults pinned in
:data:`DEFAULT_WEIGHTS`.
- :class:`ScoredFork` return value: scalar score + per-term
breakdown + verdict + flags.
- :func:`fork_score` pure function: (parent_metrics, child_metrics,
weights, capital_delta, ) ScoredFork.
- :func:`bench_result_to_metrics` adapter from
``bench.batteries.runner`` JSON to the metrics dict the scorer
consumes.
- CLI: ``arborist v8 score --parent P.json --child C.json
[--weights W.json]``.
"""
from __future__ import annotations
from arborist.v8.fork_score import (
SIGNAL_FLOOR,
ScoredFork,
bench_result_to_metrics,
fork_score,
)
from arborist.v8.weights import DEFAULT_WEIGHTS, WeightSet
__all__ = [
"DEFAULT_WEIGHTS",
"SIGNAL_FLOOR",
"ScoredFork",
"WeightSet",
"bench_result_to_metrics",
"fork_score",
]

View file

@ -116,7 +116,7 @@ of ``bench.batteries.runner --all``.
``--out`` mirrors stdout to a file; CI / mesh peers / downstream ``--out`` mirrors stdout to a file; CI / mesh peers / downstream
graders ingest the artifact without parsing pipe output. graders ingest the artifact without parsing pipe output.
Output: :class:`arborist.v8.fork_score.ScoredFork` with Output: :class:`arborist.substrate.fork_score.ScoredFork` with
``score``, ``verdict``, per-term ``breakdown``, ``flags`` list, ``score``, ``verdict``, per-term ``breakdown``, ``flags`` list,
and the active ``weights`` echoed back. and the active ``weights`` echoed back.

View file

@ -264,16 +264,18 @@ Per fox's 2026-05-08 review note that the substrate is "good enough
to support v8 selection/fork-choice design," shipping the scoring to support v8 selection/fork-choice design," shipping the scoring
function ahead of the consensus paper: function ahead of the consensus paper:
- `arborist/v8/fork_score.py` — pure ScoredFork dataclass + scoring - `arborist/substrate/fork_score.py` — pure ScoredFork dataclass +
function over (parent, child) BatteryResult bundles. Consumes scoring function over (parent, child) BatteryResult bundles.
every metric this session shipped: 5S/5T/5F sub-battery rates, Consumes every metric this session shipped: 5S/5T/5F sub-battery
`adaptation_efficiency_*` and `feedback_efficiency_*` (incl. rates, `adaptation_efficiency_*` and `feedback_efficiency_*`
inf-aware aggregation per fbd99a8 review), capital-cost delta, (incl. inf-aware aggregation per fbd99a8 review), capital-cost
memory-invalidation count. delta, memory-invalidation count. (Originally landed at
- `arborist/v8/weights.py``WeightSet` dataclass with α…λ + `arborist/v8/fork_score.py`; moved 2026-05-10 when the
`DEFAULT_WEIGHTS` (single-validator-tuned) + `from_dict` version-prefixed namespace pattern was retired.)
adapter handling the `"lambda"`/`lambda_` Python-reserved-word - `arborist/substrate/weights.py``WeightSet` dataclass with α…λ +
issue. `DEFAULT_WEIGHTS` (single-validator-tuned) + `from_dict` adapter
handling the `"lambda"`/`lambda_` Python-reserved-word issue.
(Originally `arborist/v8/weights.py`.)
- CLI: `arborist v8 score --parent P.json --child C.json - CLI: `arborist v8 score --parent P.json --child C.json
[--weights W.json]`. Exits 1 on REJECT (CI-gateable). [--weights W.json]`. Exits 1 on REJECT (CI-gateable).
- Verdict thresholds: ACCEPT (≥ SIGNAL_FLOOR=0.05), MARGINAL - Verdict thresholds: ACCEPT (≥ SIGNAL_FLOOR=0.05), MARGINAL
@ -290,9 +292,9 @@ function ahead of the consensus paper:
Closure criterion: `docs/merkle-agi-v8-consensus.rst` lands with Closure criterion: `docs/merkle-agi-v8-consensus.rst` lands with
validator state machine, acceptance protocol, challenge protocol, validator state machine, acceptance protocol, challenge protocol,
fork-choice rule (GRANDPA-style), slashing mechanics, mesh wire fork-choice rule (GRANDPA-style), slashing mechanics, mesh wire
format extension. Per §1 of this ticket, `arborist/v8/fork_score.py` format extension. Per §1 of this ticket,
is the substrate the paper cites; the paper itself is still `arborist/substrate/fork_score.py` is the substrate the paper
research-scope. cites; the paper itself is still research-scope.
What's deliberately NOT in Phase 1a: What's deliberately NOT in Phase 1a:
@ -357,7 +359,7 @@ flags:
- ``--persist-shard <PATH>`` — names the SQLite file to write to. - ``--persist-shard <PATH>`` — names the SQLite file to write to.
Defaults to ``$ARBORIST_QA_DB`` when set, else stays no-op. Defaults to ``$ARBORIST_QA_DB`` when set, else stays no-op.
**3. Read API.** One pure function in `arborist/v8/fork_score.py`: **3. Read API.** One pure function in `arborist/substrate/fork_score.py`:
```python ```python
def branch_set_density(conn, branch_set_id: str) -> int: def branch_set_density(conn, branch_set_id: str) -> int:

View file

@ -253,15 +253,24 @@ the moment v7 needs it.
### Phase 1 — reference implementation (landed 2026-05-10) ### Phase 1 — reference implementation (landed 2026-05-10)
- ``arborist/v7/__init__.py`` — namespace stub (v7 is currently - ``arborist/substrate/anchor_prg.py`` — ``phi_prg(hard_hash_32,
paper-stage per ticket #000037 §17.2; this is the first concrete dim_h, *, seed)`` per §3.1; HMAC-SHA-512 counter-mode KDF; pure
module landed under the namespace). stdlib (``hashlib`` + ``hmac``); no third-party dependency. Module
- ``arborist/v7/anchor_prg.py`` — ``phi_prg(hard_hash_32, dim_h, *, also exports ``PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"`` so
seed)`` per §3.1; HMAC-SHA-512 counter-mode KDF; pure stdlib future algorithm rotations can be detected at the call site
(``hashlib`` + ``hmac``); no third-party dependency. Module also without string-comparing module paths.
exports ``PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"`` so future
algorithm rotations can be detected at the call site without **Path note (2026-05-10):** §3.1's original sketch placed the
string-comparing module paths. module at ``arborist/v7/anchor_prg.py``. The version-prefixed
namespace pattern was retired the same day in favour of
``arborist/substrate/`` — the ``v`` in ``v7`` referred to the
substrate-paper version, which collided with the ``v9.8`` SQLite
schema version and confused readers about whether the dir tracked
paper version or schema version. ``arborist/substrate/`` is now
the topic dir for paper-spec'd Merkle-AGI primitives, decoupled
from any version number. Acceptance-criteria §5 item 1 reads
through to "or equivalent location" so the move doesn't
invalidate the original criterion.
- ``tests/test_anchor_prg.py`` — 20 tests covering determinism, - ``tests/test_anchor_prg.py`` — 20 tests covering determinism,
range invariants, chi² loose-uniformity sanity, dim_h boundary range invariants, chi² loose-uniformity sanity, dim_h boundary
(1, 16384), seed-bit-flip avalanche, hash-bit-flip avalanche, (1, 16384), seed-bit-flip avalanche, hash-bit-flip avalanche,

View file

@ -1,6 +1,10 @@
# v8 ForkScore — Phase 1a reference # v8 ForkScore — Phase 1a reference
Reference for `arborist.v8.fork_score` (ticket #000012 Phase 1a). Reference for `arborist.substrate.fork_score` (ticket #000012 Phase 1a;
module moved from `arborist.v8.fork_score` to `arborist.substrate.fork_score`
on 2026-05-10 when the version-prefixed namespace pattern was retired —
the `v` in `v8` referred to the substrate-paper version, which collided
with the `v9.8` SQLite schema version and confused readers).
**Pure scoring function** over a (parent, child) BatteryResult pair. **Pure scoring function** over a (parent, child) BatteryResult pair.
No validator state machine, no consensus protocol, no acceptance No validator state machine, no consensus protocol, no acceptance
ledger — those are commissioned by the v8 paper itself, still open ledger — those are commissioned by the v8 paper itself, still open

View file

@ -1,4 +1,4 @@
"""Tests for arborist.v7.anchor_prg per ticket #000035 §3.2. """Tests for arborist.substrate.anchor_prg per ticket #000035 §3.2.
Coverage matches the ticket's acceptance criteria: Coverage matches the ticket's acceptance criteria:
@ -24,7 +24,7 @@ from pathlib import Path
import pytest import pytest
from arborist.v7.anchor_prg import ( from arborist.substrate.anchor_prg import (
PHI_PRG_VERSION, PHI_PRG_VERSION,
PLACEHOLDER_SEED, PLACEHOLDER_SEED,
_bytes_to_floats, _bytes_to_floats,

View file

@ -18,7 +18,7 @@ from pathlib import Path
import pytest import pytest
from arborist.v8 import ( from arborist.substrate import (
DEFAULT_WEIGHTS, DEFAULT_WEIGHTS,
SIGNAL_FLOOR, SIGNAL_FLOOR,
ScoredFork, ScoredFork,
@ -26,11 +26,11 @@ from arborist.v8 import (
bench_result_to_metrics, bench_result_to_metrics,
fork_score, fork_score,
) )
from arborist.v8.fork_score import ( from arborist.substrate.fork_score import (
HARD_REGRESSION_FLOOR, HARD_REGRESSION_FLOOR,
INFINITE_BONUS_CAP, INFINITE_BONUS_CAP,
) )
from arborist.v8.weights import from_dict as weights_from_dict from arborist.substrate.weights import from_dict as weights_from_dict
# --------------------------------------------------------------------- # ---------------------------------------------------------------------