From a4058a43bc10844a03bca515ce127ec38201cc3d Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Sun, 10 May 2026 13:35:52 -0400 Subject: [PATCH] =?UTF-8?q?tests:=20rename=20v8=20=E2=86=92=20substrate=20?= =?UTF-8?q?+=20close=209-item=20checklist=20gaps=20across=203=20files?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three threads bundled, all surfaced by today's calculator-test- patterns.md audit + fox's directive to remove v-prefix from test filenames: THREAD 1 — rename test_v8_fork_score.py → test_substrate_fork_score.py ===================================================================== Single test file in the tree had a v-prefix in its filename: ``tests/test_v8_fork_score.py``. Renamed via ``git mv`` for consistency with yesterday's substrate refactor (the package is ``arborist/substrate/fork_score.py``; the CLI subcommand is ``arborist substrate score``; the test file should match). No internal code changes needed — the file's imports + assertions were already updated to ``arborist.substrate.*`` paths in yesterday's bae5caf commit. Pure rename. THREAD 2 — close v1 substring discipline gap ============================================= Audit of test_anchor_prg.py + test_phi_alignment_probe.py against docs/calculator-test-patterns.md §2 (versioned-default discipline) found one gap: both files asserted the version string's exact value but neither asserted the ``"v1"`` substring discipline that fox's test_returns_calculator_version_token established. Added ``assert "v1" in PHI_PRG_VERSION`` to test_module_exports_version_string in test_anchor_prg.py. Added ``assert "v1" in PROBE_VERSION`` to test_module_exports_thresholds_and_version in test_phi_alignment_probe.py. Both follow fox's pattern: when the algorithm changes (v2-blake3- expansion, v2-arnoldi-iteration, etc.), the version string MUST change too. The "v1" substring assertion catches a future contributor who refactors without bumping the version constant. THREAD 3 — close CLI subprocess gap on test_substrate_fork_score.py ==================================================================== The renamed file had four CLI tests but all in-process via build_parser() + parse_args() + func(args). That catches argparse- shape drift but NOT entry-point / module-loading / sys.argv drift. Added test_cli_substrate_score_subprocess_invocation: real ``subprocess.run(["python", "-m", "arborist.cli", "substrate", "score", "--parent", ..., "--child", ..., "--out", ...])`` against synthetic bench results. Asserts exit 0 + the --out artifact is written + JSON-parses with valid verdict. Pattern matches fox's test_cli_baseline_runs_clean in test_t3_bound_calculator.py + the 581ad90 KAT-fixture-gap closure. Same hazard fox already hit three times during the substrate rename refactor (85be5eb, 209d670, b320e27): import-only tests silently miss CLI surface drift. CHECKLIST AUDIT — POST-FIX ========================== Three calculator-style test files now all 9-item complete: | t3 | anchor_prg | phi_alignment | substrate_fork | KAT fixture | ✓ | ✓ | ✓ | n/a (different)| VERSION + "v1" | ✓ | ✓ NOW | ✓ NOW | ✓ | Hand-formula | ✓ | ✓ | ✓ | ✓ (synthetic) | Monotonicity | ✓ | ✓ | ✓ | ✓ | Closure / sum-of-parts | ✓ | ✓ | ✓ | ✓ | Parametrized invalid | ✓ | ✓ | ✓ | ~ | CLI subprocess | ✓ | n/a | n/a | ✓ NOW | Doc parity | ✓ | KAT | KAT | KAT | Module-export shape | ✓ | ✓ | ✓ | ✓ | All four files now consistently track the calculator-test-patterns checklist. test_substrate_fork_score.py is structurally different (verifier-adjacent: tests scoring + verdict-band logic, not closed-form math) so some checklist items map differently — KAT fixture replaced by synthetic-input verdict tests (closer to verifier-style), parametrized-invalid is partial (per-verdict- class assertions rather than per-bad-input cone). Acceptable. Test counts: - test_substrate_fork_score.py: 26 → 27 (+1 subprocess test) - test_anchor_prg.py: 27 → 27 (assertion added inline) - test_phi_alignment_probe.py: 23 → 23 (assertion added inline) - t3 file untouched in this commit (581ad90 already at 53) Full suite: 1985 → 1986 (+1 from this commit's only new-test-function addition; the inline assertions don't count as new tests). Hygiene ======= - make test → 1986 passed, 45 skipped. - All four calculator-style test files structurally aligned. - No v-prefixed test filenames remain in tests/ tree. --- tests/test_anchor_prg.py | 5 ++ tests/test_phi_alignment_probe.py | 5 ++ ..._score.py => test_substrate_fork_score.py} | 55 +++++++++++++++++++ 3 files changed, 65 insertions(+) rename tests/{test_v8_fork_score.py => test_substrate_fork_score.py} (89%) diff --git a/tests/test_anchor_prg.py b/tests/test_anchor_prg.py index 7129974..cc6bdbf 100644 --- a/tests/test_anchor_prg.py +++ b/tests/test_anchor_prg.py @@ -311,6 +311,11 @@ def test_phi_prg_rejects_non_int_dim_h(): def test_module_exports_version_string(): assert PHI_PRG_VERSION == "phi-prg-v1-hmac-sha512" + # versioned-default discipline (calculator-test-patterns.md §2): + # "v1" substring present so future major-version rotations + # (v2-blake3-expansion etc.) are detectable at the call site + # without string-comparing module paths. + assert "v1" in PHI_PRG_VERSION def test_placeholder_seed_is_32_bytes(): diff --git a/tests/test_phi_alignment_probe.py b/tests/test_phi_alignment_probe.py index b0b54d3..9853b62 100644 --- a/tests/test_phi_alignment_probe.py +++ b/tests/test_phi_alignment_probe.py @@ -180,6 +180,11 @@ def test_alignment_report_round_trips_via_asdict(): def test_module_exports_thresholds_and_version(): assert PROBE_VERSION == "phi-alignment-v1-lanczos" + # versioned-default discipline (calculator-test-patterns.md §2): + # "v1" substring present so future major-version rotations are + # detectable at the call site without string-comparing module + # paths. + assert "v1" in PROBE_VERSION assert STRUCTURAL_ALIGNMENT_RATIO_FLOOR == 1.5 assert ANTI_ALIGNED_RATIO_CEILING == 0.7 assert DEFAULT_K_TOP == 100 diff --git a/tests/test_v8_fork_score.py b/tests/test_substrate_fork_score.py similarity index 89% rename from tests/test_v8_fork_score.py rename to tests/test_substrate_fork_score.py index 92d7032..0aa5805 100644 --- a/tests/test_v8_fork_score.py +++ b/tests/test_substrate_fork_score.py @@ -14,6 +14,8 @@ Covers: from __future__ import annotations import json +import subprocess +import sys from pathlib import Path import pytest @@ -505,3 +507,56 @@ def test_score_is_dict_serializable(): encoded = json.dumps(sf.to_dict(), default=str) decoded = json.loads(encoded) assert decoded["verdict"] == sf.verdict + + +# --------------------------------------------------------------------- +# Real subprocess-mode CLI test (calculator-test-patterns.md §6). +# The in-process build_parser() tests above catch argparse-shape drift +# but NOT entry-point / module-loading / sys.argv drift. fox's +# test_cli_baseline_runs_clean in test_t3_bound_calculator.py is the +# exemplar; this test mirrors that pattern for `arborist substrate score`. +# --------------------------------------------------------------------- + + +def test_cli_substrate_score_subprocess_invocation(tmp_path): + """Real subprocess-mode invocation of `arborist substrate score`. + + Catches drift at the entry-point + module-loading layer that the + in-process build_parser() tests would miss — e.g. a refactor that + breaks the console-script in pyproject.toml, or that introduces a + sys.argv-handling bug. + + Pattern: docs/calculator-test-patterns.md §6 + the substrate + rename's three-defect lesson (85be5eb fork_score.py import, + 209d670 Makefile bench-fork-score, b320e27 .gitlab-ci.yml job). + """ + parent_path = tmp_path / "parent.json" + child_path = tmp_path / "child.json" + out_path = tmp_path / "report.json" + _write_bench_result(parent_path, { + "5s": {"syntax": {"parse_pass_rate": 0.80}}, + "5t": {}, "5f": {}, + }) + _write_bench_result(child_path, { + "5s": {"syntax": {"parse_pass_rate": 0.85}}, + "5t": {}, "5f": {}, + }) + + cmd = [ + sys.executable, "-m", "arborist.cli", + "substrate", "score", + "--parent", str(parent_path), + "--child", str(child_path), + "--out", str(out_path), + ] + out = subprocess.run( + cmd, capture_output=True, text=True, check=True, + cwd="/home/fox/git/arborist", + ) + # Exit code 0 = ACCEPT or MARGINAL per ScoredFork verdict + # discipline. Subprocess.run(check=True) already asserts it. + assert out_path.exists(), "expected --out artifact written" + artifact = json.loads(out_path.read_text()) + assert "score" in artifact + assert "verdict" in artifact + assert artifact["verdict"] in {"ACCEPT", "MARGINAL", "REJECT"}