CLI version-bake sweep surfaced one real defect: the Phase 1 test scaffolding I shipped inc422216imported ``from arborist.v9 import prometheus``, baking in the version-prefixed namespace path that yesterday's substrate refactor (654d923) abolished. The skip mechanism (try/except ImportError → CONTROLLER_AVAILABLE = False) was masking the issue: when fox lands Phase 1 of #000037 at ``arborist/substrate/prometheus.py`` (per the topic-named convention), my tests would CONTINUE to skip with "controller module arborist/v9/prometheus.py absent" because the import target itself is wrong. The skip-stub becomes permanent dormancy instead of activating when the module ships. Two changes: - Import line: ``from arborist.v9 import prometheus`` → ``from arborist.substrate import prometheus``. - Skip-reason text + module-docstring: ``arborist/v9/prometheus.py`` → ``arborist/substrate/prometheus.py``, with a parenthetical noting the post-2026-05-10 topic-named convention and that §13's original sketch predated the v-dir retirement. The fix is a real one — when fox's Phase 1 of #000037 lands, my 17 skip-stubs now activate against the correct module path. Without this fix, they'd silently stay dormant. Sweep summary ============= Walked every CLI subcommand --help (arborist top-level + nested substrate / memory / capital / selfmodel / warrant-resolve / sweep / alias / mesh / crawl / providence) plus full-tree grep for ``arborist v[789]`` / ``arborist\\.v[789]`` / ``arborist/v[789]``. Remaining v-prefix mentions across the tree (all intentional): - arborist/cli.py:5166-5169 — historical-note comment for the v8→substrate rename - arborist/substrate/__init__.py:8 — same convention-explanation note - arborist/substrate/anchor_prg.py:59 — bytestring inside SHA-256 derivation of placeholder seed; can't change without breaking KAT - bench/fixtures/phi-prg/known-answer-tests.jsonl:1 — fixture header naming the v7 paper section §9.10 (paper version, correct) - docs/v8-fork-score.md:4 — historical-note ("module moved from") - docs/tickets/ticket-000019, ticket-000013 — "arborist v9.8" schema references (schema version, correct) - docs/_source/merkle-agi-v7w-spatial-temporal.rst — v7 paper + v9.8 schema references (both correct) Tests: 1915 passing, 45 skipped (was 1872; +43 from fox's parallel test additions during this commit's prep + the 17 prometheus skips activating against the correct path stub). CLI version-bake sweep complete. The substrate refactor is now surface-clean end-to-end.
217 lines
8.6 KiB
Python
217 lines
8.6 KiB
Python
"""Phase 1 test surface for ticket #000037 (Prometheus-Σ controller).
|
||
|
||
Phase 0 is doc-only — these tests are the **scaffolding** the §16.2
|
||
implementation contract names. All of them skip until Phase 1 lands
|
||
the controller module (``arborist/substrate/prometheus.py``,
|
||
per the post-2026-05-10 topic-named convention; the original
|
||
§13 sketch said ``arborist/v9/prometheus.py`` before the v-dir
|
||
namespace pattern was retired).
|
||
|
||
Why land scaffolding while Phase 0 is still doc-only:
|
||
|
||
- The 17 named tests are the §16.2 acceptance contract; pinning them
|
||
here means a future shift can't drift the contract by accident.
|
||
- ``pytest --collect-only`` lists them, so the test surface is
|
||
discoverable from the test runner today rather than buried in a
|
||
ticket.
|
||
- When Phase 1 lands, the implementer flips
|
||
``CONTROLLER_AVAILABLE = True``, drops the body of each test, and
|
||
the contract enforces itself.
|
||
|
||
Per CLAUDE.md "no half-finished implementations": these stubs are
|
||
*explicitly* documented as scaffolding. Each test contains a one-
|
||
sentence intent line tied to the controller invariant it pins; that
|
||
intent is what the eventual implementation must satisfy.
|
||
|
||
Phase 1 trigger gating (§12) is a separate concern measured by
|
||
``bench/prometheus_sigma_trigger_probe.py`` — these tests will run
|
||
regardless of trigger state once the module is in place.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import pytest
|
||
|
||
try:
|
||
from arborist.substrate import prometheus as _prom # noqa: F401
|
||
CONTROLLER_AVAILABLE = True
|
||
except ImportError:
|
||
CONTROLLER_AVAILABLE = False
|
||
|
||
skip_until_phase_1 = pytest.mark.skipif(
|
||
not CONTROLLER_AVAILABLE,
|
||
reason=(
|
||
"ticket #000037 Phase 1 not landed; controller module "
|
||
"arborist/substrate/prometheus.py absent. Skip is the contract — "
|
||
"implementer flips this when the module exists."
|
||
),
|
||
)
|
||
|
||
|
||
# ----------------------------------------------------------- core decision
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_all_vetoed_returns_reject_or_quarantine():
|
||
"""§14 row 2: all branches hard-vetoed → emit veto reasons; no
|
||
allocation; label ``REJECT`` or ``QUARANTINE`` (depending on the
|
||
veto class). Catches the "every branch is poisoned" case."""
|
||
pytest.fail("Phase 1 implementation pending; see §13 step 3.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_zero_budget_returns_deferred():
|
||
"""§14 row 3: ``B = 0`` → no LLM call; queue sleep if useful;
|
||
label ``DEFERRED``. Distinguishes "didn't evaluate due to budget"
|
||
from "evaluated but uncertain" (``MARGINAL``). Per David review
|
||
point 3."""
|
||
pytest.fail("Phase 1 implementation pending; see §4.3 + §14.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_stable_softmax_no_overflow():
|
||
"""§5: softmax must use ``exp(z_i − max z) / Σ exp(z_j − max z)``.
|
||
With large positive utilities (e.g. z = 1000) naïve ``exp(z)``
|
||
overflows; the normalized form must not. Per David review point 6."""
|
||
pytest.fail("Phase 1 implementation pending; see §5 + §13 step 5.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_negative_payoff_gets_zero_allocation():
|
||
"""§7 Kelly safety guard: ``b_i ≤ 0`` → zero allocation. A branch
|
||
with negative expected payoff must not consume budget. Per David
|
||
review point 8."""
|
||
pytest.fail("Phase 1 implementation pending; see §7.")
|
||
|
||
|
||
# ----------------------------------------------------------- vetoes
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_unsupported_carrier_quarantines():
|
||
"""§6 hard-veto class: claim references a carrier modality that
|
||
no live π* library supports → ``QUARANTINE``. Surfaces missing
|
||
domain coverage rather than papering over it."""
|
||
pytest.fail("Phase 1 implementation pending; see §6.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_cache_drift_quarantines():
|
||
"""§6 hard-veto class: cache row whose ``pi_star_ref`` no longer
|
||
matches a live kernel version → ``QUARANTINE``. Echoes the
|
||
CACHE-DRIFT outcome from #000028 §1.2; controller surfaces
|
||
rather than silently re-uses."""
|
||
pytest.fail("Phase 1 implementation pending; see §6 + #000028 §1.2.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_memory_invalidation_above_threshold_escalates():
|
||
"""§10 Gödel discipline: a branch whose acceptance would
|
||
invalidate too much committed memory → label ``ESCALATE``, not
|
||
``REJECT``. The controller explicitly steps aside per the "must
|
||
never infer" rule. Per David review point 12."""
|
||
pytest.fail("Phase 1 implementation pending; see §10.")
|
||
|
||
|
||
# ----------------------------------------------------------- difficulty / EMA
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_high_entropy_increases_difficulty():
|
||
"""§7.1 difficulty update law: high ``H_norm(p)`` → smoothed
|
||
increase in ``difficulty_ema``. The EMA smoothing keeps the
|
||
update from overshooting on a single noisy sample. Per David
|
||
review point 9."""
|
||
pytest.fail("Phase 1 implementation pending; see §7.1.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_low_entropy_decreases_or_preserves_difficulty():
|
||
"""§7.1 difficulty update law: low ``H_norm(p)`` (controller is
|
||
confident) → difficulty drops or holds. Symmetric to the
|
||
high-entropy test; together they pin the EMA's monotonicity."""
|
||
pytest.fail("Phase 1 implementation pending; see §7.1.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_divergence_increases_witness_sampling_recommendation():
|
||
"""§4.3 output / §17.1: high observed witness divergence → the
|
||
controller's recommendation field should bump ``canonical_witness_
|
||
sample_rate`` upward (an advisory; runtime decides whether to
|
||
apply). Pins the feedback loop into #000028's sample-rate field
|
||
we landed in 6d20aeb."""
|
||
pytest.fail("Phase 1 implementation pending; see §4.3.")
|
||
|
||
|
||
# ----------------------------------------------------------- discipline
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_no_llm_call_in_controller():
|
||
"""§10 Gödel + David review point 12: the controller is a pure
|
||
function. No ``ChatClient`` import in the call graph; no network
|
||
socket; no env-var that secretly enables one. The witness module
|
||
calls the LLM; the controller reads its results. Pins the
|
||
"LLM is witness, never authority" doctrine."""
|
||
pytest.fail("Phase 1 implementation pending; see §10 + §16.1.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_controller_does_not_modify_cache_key_inputs():
|
||
"""§4.4 update authority: the controller emits **proposals** for
|
||
MemoryRoot / SelfModel updates; it never mutates the cache_key
|
||
8-dim input itself. Pins the schema invariant from #000027:
|
||
cache_key is computed by the cache_key() function, not by any
|
||
advisory layer above it. Per David review point 14."""
|
||
pytest.fail("Phase 1 implementation pending; see §4.4 + §13 step 12.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_controller_outputs_advisory_event_only():
|
||
"""§13 step 10: the controller writes ``controller_decision`` /
|
||
``controller_difficulty`` / ``controller_budget_allocation`` as
|
||
sibling tags on ``audit_events`` — they do NOT enter
|
||
``event_hash`` preimage. Re-running the controller against the
|
||
same state cannot break the audit chain. Pins the same sibling-
|
||
table invariant the capital_ledger uses."""
|
||
pytest.fail("Phase 1 implementation pending; see §13 step 10.")
|
||
|
||
|
||
# ----------------------------------------------------------- §12 trigger guards
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_zero_mean_divergence_does_not_trigger_phase_1():
|
||
"""§12 Trigger 2: ``max(mean, ε)`` guard against div-by-zero
|
||
when divergence is uniformly low. Probe covers this; the
|
||
in-controller guard duplicates it so the controller can be run
|
||
on a fresh corpus without crashing on the first call."""
|
||
pytest.fail("Phase 1 implementation pending; see §12 Trigger 2.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_small_sample_does_not_trigger_phase_1():
|
||
"""§12 Trigger 2: ``N_min = 30`` floor below which the variance
|
||
trigger does not fire. Pins behavior on early-corpus deployments
|
||
where sample-count noise would dominate any signal."""
|
||
pytest.fail("Phase 1 implementation pending; see §12 Trigger 2.")
|
||
|
||
|
||
# ----------------------------------------------------------- proposals, not mutations
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_propose_not_mutate_memory_root():
|
||
"""§4.4 + §13 step 12: controller emits a MemoryRoot update
|
||
*proposal*; the existing memory-root write path validates and
|
||
commits. Direct mutation would let the controller poison hard-
|
||
hashed state without going through validation."""
|
||
pytest.fail("Phase 1 implementation pending; see §4.4.")
|
||
|
||
|
||
@skip_until_phase_1
|
||
def test_propose_not_mutate_self_model():
|
||
"""§4.4 + §13 step 12: controller emits a SelfModel update
|
||
*proposal*; the existing #000014 selfmodel write path validates
|
||
and commits. Same boundary as MemoryRoot."""
|
||
pytest.fail("Phase 1 implementation pending; see §4.4 + #000014.")
|