arborist/tests/test_prometheus_sigma.py
russell@unturf.com 35aeebbb24
tests/prometheus_sigma: refresh stale v9 import path → substrate
CLI version-bake sweep surfaced one real defect: the Phase 1 test
scaffolding I shipped in c422216 imported ``from arborist.v9
import prometheus``, baking in the version-prefixed namespace path
that yesterday's substrate refactor (654d923) abolished.

The skip mechanism (try/except ImportError → CONTROLLER_AVAILABLE
= False) was masking the issue: when fox lands Phase 1 of #000037
at ``arborist/substrate/prometheus.py`` (per the topic-named
convention), my tests would CONTINUE to skip with
"controller module arborist/v9/prometheus.py absent" because the
import target itself is wrong. The skip-stub becomes permanent
dormancy instead of activating when the module ships.

Two changes:

- Import line: ``from arborist.v9 import prometheus`` →
  ``from arborist.substrate import prometheus``.
- Skip-reason text + module-docstring: ``arborist/v9/prometheus.py``
  → ``arborist/substrate/prometheus.py``, with a parenthetical
  noting the post-2026-05-10 topic-named convention and that
  §13's original sketch predated the v-dir retirement.

The fix is a real one — when fox's Phase 1 of #000037 lands, my
17 skip-stubs now activate against the correct module path.
Without this fix, they'd silently stay dormant.

Sweep summary
=============

Walked every CLI subcommand --help (arborist top-level + nested
substrate / memory / capital / selfmodel / warrant-resolve / sweep
/ alias / mesh / crawl / providence) plus full-tree grep for
``arborist v[789]`` / ``arborist\\.v[789]`` / ``arborist/v[789]``.

Remaining v-prefix mentions across the tree (all intentional):

- arborist/cli.py:5166-5169 — historical-note comment for the
  v8→substrate rename
- arborist/substrate/__init__.py:8 — same convention-explanation note
- arborist/substrate/anchor_prg.py:59 — bytestring inside SHA-256
  derivation of placeholder seed; can't change without breaking KAT
- bench/fixtures/phi-prg/known-answer-tests.jsonl:1 — fixture header
  naming the v7 paper section §9.10 (paper version, correct)
- docs/v8-fork-score.md:4 — historical-note ("module moved from")
- docs/tickets/ticket-000019, ticket-000013 — "arborist v9.8" schema
  references (schema version, correct)
- docs/_source/merkle-agi-v7w-spatial-temporal.rst — v7 paper +
  v9.8 schema references (both correct)

Tests: 1915 passing, 45 skipped (was 1872; +43 from fox's parallel
test additions during this commit's prep + the 17 prometheus
skips activating against the correct path stub).

CLI version-bake sweep complete. The substrate refactor is now
surface-clean end-to-end.
2026-05-10 13:02:58 -04:00

217 lines
8.6 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

"""Phase 1 test surface for ticket #000037 (Prometheus-Σ controller).
Phase 0 is doc-only — these tests are the **scaffolding** the §16.2
implementation contract names. All of them skip until Phase 1 lands
the controller module (``arborist/substrate/prometheus.py``,
per the post-2026-05-10 topic-named convention; the original
§13 sketch said ``arborist/v9/prometheus.py`` before the v-dir
namespace pattern was retired).
Why land scaffolding while Phase 0 is still doc-only:
- The 17 named tests are the §16.2 acceptance contract; pinning them
here means a future shift can't drift the contract by accident.
- ``pytest --collect-only`` lists them, so the test surface is
discoverable from the test runner today rather than buried in a
ticket.
- When Phase 1 lands, the implementer flips
``CONTROLLER_AVAILABLE = True``, drops the body of each test, and
the contract enforces itself.
Per CLAUDE.md "no half-finished implementations": these stubs are
*explicitly* documented as scaffolding. Each test contains a one-
sentence intent line tied to the controller invariant it pins; that
intent is what the eventual implementation must satisfy.
Phase 1 trigger gating (§12) is a separate concern measured by
``bench/prometheus_sigma_trigger_probe.py`` — these tests will run
regardless of trigger state once the module is in place.
"""
from __future__ import annotations
import pytest
try:
from arborist.substrate import prometheus as _prom # noqa: F401
CONTROLLER_AVAILABLE = True
except ImportError:
CONTROLLER_AVAILABLE = False
skip_until_phase_1 = pytest.mark.skipif(
not CONTROLLER_AVAILABLE,
reason=(
"ticket #000037 Phase 1 not landed; controller module "
"arborist/substrate/prometheus.py absent. Skip is the contract — "
"implementer flips this when the module exists."
),
)
# ----------------------------------------------------------- core decision
@skip_until_phase_1
def test_all_vetoed_returns_reject_or_quarantine():
"""§14 row 2: all branches hard-vetoed → emit veto reasons; no
allocation; label ``REJECT`` or ``QUARANTINE`` (depending on the
veto class). Catches the "every branch is poisoned" case."""
pytest.fail("Phase 1 implementation pending; see §13 step 3.")
@skip_until_phase_1
def test_zero_budget_returns_deferred():
"""§14 row 3: ``B = 0`` → no LLM call; queue sleep if useful;
label ``DEFERRED``. Distinguishes "didn't evaluate due to budget"
from "evaluated but uncertain" (``MARGINAL``). Per David review
point 3."""
pytest.fail("Phase 1 implementation pending; see §4.3 + §14.")
@skip_until_phase_1
def test_stable_softmax_no_overflow():
"""§5: softmax must use ``exp(z_i max z) / Σ exp(z_j max z)``.
With large positive utilities (e.g. z = 1000) naïve ``exp(z)``
overflows; the normalized form must not. Per David review point 6."""
pytest.fail("Phase 1 implementation pending; see §5 + §13 step 5.")
@skip_until_phase_1
def test_negative_payoff_gets_zero_allocation():
"""§7 Kelly safety guard: ``b_i ≤ 0`` → zero allocation. A branch
with negative expected payoff must not consume budget. Per David
review point 8."""
pytest.fail("Phase 1 implementation pending; see §7.")
# ----------------------------------------------------------- vetoes
@skip_until_phase_1
def test_unsupported_carrier_quarantines():
"""§6 hard-veto class: claim references a carrier modality that
no live π* library supports → ``QUARANTINE``. Surfaces missing
domain coverage rather than papering over it."""
pytest.fail("Phase 1 implementation pending; see §6.")
@skip_until_phase_1
def test_cache_drift_quarantines():
"""§6 hard-veto class: cache row whose ``pi_star_ref`` no longer
matches a live kernel version → ``QUARANTINE``. Echoes the
CACHE-DRIFT outcome from #000028 §1.2; controller surfaces
rather than silently re-uses."""
pytest.fail("Phase 1 implementation pending; see §6 + #000028 §1.2.")
@skip_until_phase_1
def test_memory_invalidation_above_threshold_escalates():
"""§10 Gödel discipline: a branch whose acceptance would
invalidate too much committed memory → label ``ESCALATE``, not
``REJECT``. The controller explicitly steps aside per the "must
never infer" rule. Per David review point 12."""
pytest.fail("Phase 1 implementation pending; see §10.")
# ----------------------------------------------------------- difficulty / EMA
@skip_until_phase_1
def test_high_entropy_increases_difficulty():
"""§7.1 difficulty update law: high ``H_norm(p)`` → smoothed
increase in ``difficulty_ema``. The EMA smoothing keeps the
update from overshooting on a single noisy sample. Per David
review point 9."""
pytest.fail("Phase 1 implementation pending; see §7.1.")
@skip_until_phase_1
def test_low_entropy_decreases_or_preserves_difficulty():
"""§7.1 difficulty update law: low ``H_norm(p)`` (controller is
confident) → difficulty drops or holds. Symmetric to the
high-entropy test; together they pin the EMA's monotonicity."""
pytest.fail("Phase 1 implementation pending; see §7.1.")
@skip_until_phase_1
def test_divergence_increases_witness_sampling_recommendation():
"""§4.3 output / §17.1: high observed witness divergence → the
controller's recommendation field should bump ``canonical_witness_
sample_rate`` upward (an advisory; runtime decides whether to
apply). Pins the feedback loop into #000028's sample-rate field
we landed in 6d20aeb."""
pytest.fail("Phase 1 implementation pending; see §4.3.")
# ----------------------------------------------------------- discipline
@skip_until_phase_1
def test_no_llm_call_in_controller():
"""§10 Gödel + David review point 12: the controller is a pure
function. No ``ChatClient`` import in the call graph; no network
socket; no env-var that secretly enables one. The witness module
calls the LLM; the controller reads its results. Pins the
"LLM is witness, never authority" doctrine."""
pytest.fail("Phase 1 implementation pending; see §10 + §16.1.")
@skip_until_phase_1
def test_controller_does_not_modify_cache_key_inputs():
"""§4.4 update authority: the controller emits **proposals** for
MemoryRoot / SelfModel updates; it never mutates the cache_key
8-dim input itself. Pins the schema invariant from #000027:
cache_key is computed by the cache_key() function, not by any
advisory layer above it. Per David review point 14."""
pytest.fail("Phase 1 implementation pending; see §4.4 + §13 step 12.")
@skip_until_phase_1
def test_controller_outputs_advisory_event_only():
"""§13 step 10: the controller writes ``controller_decision`` /
``controller_difficulty`` / ``controller_budget_allocation`` as
sibling tags on ``audit_events`` — they do NOT enter
``event_hash`` preimage. Re-running the controller against the
same state cannot break the audit chain. Pins the same sibling-
table invariant the capital_ledger uses."""
pytest.fail("Phase 1 implementation pending; see §13 step 10.")
# ----------------------------------------------------------- §12 trigger guards
@skip_until_phase_1
def test_zero_mean_divergence_does_not_trigger_phase_1():
"""§12 Trigger 2: ``max(mean, ε)`` guard against div-by-zero
when divergence is uniformly low. Probe covers this; the
in-controller guard duplicates it so the controller can be run
on a fresh corpus without crashing on the first call."""
pytest.fail("Phase 1 implementation pending; see §12 Trigger 2.")
@skip_until_phase_1
def test_small_sample_does_not_trigger_phase_1():
"""§12 Trigger 2: ``N_min = 30`` floor below which the variance
trigger does not fire. Pins behavior on early-corpus deployments
where sample-count noise would dominate any signal."""
pytest.fail("Phase 1 implementation pending; see §12 Trigger 2.")
# ----------------------------------------------------------- proposals, not mutations
@skip_until_phase_1
def test_propose_not_mutate_memory_root():
"""§4.4 + §13 step 12: controller emits a MemoryRoot update
*proposal*; the existing memory-root write path validates and
commits. Direct mutation would let the controller poison hard-
hashed state without going through validation."""
pytest.fail("Phase 1 implementation pending; see §4.4.")
@skip_until_phase_1
def test_propose_not_mutate_self_model():
"""§4.4 + §13 step 12: controller emits a SelfModel update
*proposal*; the existing #000014 selfmodel write path validates
and commits. Same boundary as MemoryRoot."""
pytest.fail("Phase 1 implementation pending; see §4.4 + #000014.")