tests: rename v8 → substrate + close 9-item checklist gaps across 3 files
Three threads bundled, all surfaced by today's calculator-test- patterns.md audit + fox's directive to remove v-prefix from test filenames: THREAD 1 — rename test_v8_fork_score.py → test_substrate_fork_score.py ===================================================================== Single test file in the tree had a v-prefix in its filename: ``tests/test_v8_fork_score.py``. Renamed via ``git mv`` for consistency with yesterday's substrate refactor (the package is ``arborist/substrate/fork_score.py``; the CLI subcommand is ``arborist substrate score``; the test file should match). No internal code changes needed — the file's imports + assertions were already updated to ``arborist.substrate.*`` paths in yesterday'sbae5cafcommit. Pure rename. THREAD 2 — close v1 substring discipline gap ============================================= Audit of test_anchor_prg.py + test_phi_alignment_probe.py against docs/calculator-test-patterns.md §2 (versioned-default discipline) found one gap: both files asserted the version string's exact value but neither asserted the ``"v1"`` substring discipline that fox's test_returns_calculator_version_token established. Added ``assert "v1" in PHI_PRG_VERSION`` to test_module_exports_version_string in test_anchor_prg.py. Added ``assert "v1" in PROBE_VERSION`` to test_module_exports_thresholds_and_version in test_phi_alignment_probe.py. Both follow fox's pattern: when the algorithm changes (v2-blake3- expansion, v2-arnoldi-iteration, etc.), the version string MUST change too. The "v1" substring assertion catches a future contributor who refactors without bumping the version constant. THREAD 3 — close CLI subprocess gap on test_substrate_fork_score.py ==================================================================== The renamed file had four CLI tests but all in-process via build_parser() + parse_args() + func(args). That catches argparse- shape drift but NOT entry-point / module-loading / sys.argv drift. Added test_cli_substrate_score_subprocess_invocation: real ``subprocess.run(["python", "-m", "arborist.cli", "substrate", "score", "--parent", ..., "--child", ..., "--out", ...])`` against synthetic bench results. Asserts exit 0 + the --out artifact is written + JSON-parses with valid verdict. Pattern matches fox's test_cli_baseline_runs_clean in test_t3_bound_calculator.py + the581ad90KAT-fixture-gap closure. Same hazard fox already hit three times during the substrate rename refactor (85be5eb,209d670,b320e27): import-only tests silently miss CLI surface drift. CHECKLIST AUDIT — POST-FIX ========================== Three calculator-style test files now all 9-item complete: | t3 | anchor_prg | phi_alignment | substrate_fork | KAT fixture | ✓ | ✓ | ✓ | n/a (different)| VERSION + "v1" | ✓ | ✓ NOW | ✓ NOW | ✓ | Hand-formula | ✓ | ✓ | ✓ | ✓ (synthetic) | Monotonicity | ✓ | ✓ | ✓ | ✓ | Closure / sum-of-parts | ✓ | ✓ | ✓ | ✓ | Parametrized invalid | ✓ | ✓ | ✓ | ~ | CLI subprocess | ✓ | n/a | n/a | ✓ NOW | Doc parity | ✓ | KAT | KAT | KAT | Module-export shape | ✓ | ✓ | ✓ | ✓ | All four files now consistently track the calculator-test-patterns checklist. test_substrate_fork_score.py is structurally different (verifier-adjacent: tests scoring + verdict-band logic, not closed-form math) so some checklist items map differently — KAT fixture replaced by synthetic-input verdict tests (closer to verifier-style), parametrized-invalid is partial (per-verdict- class assertions rather than per-bad-input cone). Acceptable. Test counts: - test_substrate_fork_score.py: 26 → 27 (+1 subprocess test) - test_anchor_prg.py: 27 → 27 (assertion added inline) - test_phi_alignment_probe.py: 23 → 23 (assertion added inline) - t3 file untouched in this commit (581ad90already at 53) Full suite: 1985 → 1986 (+1 from this commit's only new-test-function addition; the inline assertions don't count as new tests). Hygiene ======= - make test → 1986 passed, 45 skipped. - All four calculator-style test files structurally aligned. - No v-prefixed test filenames remain in tests/ tree.
This commit is contained in:
parent
581ad908f0
commit
a4058a43bc
3 changed files with 65 additions and 0 deletions
|
|
@ -311,6 +311,11 @@ def test_phi_prg_rejects_non_int_dim_h():
|
|||
|
||||
def test_module_exports_version_string():
|
||||
assert PHI_PRG_VERSION == "phi-prg-v1-hmac-sha512"
|
||||
# versioned-default discipline (calculator-test-patterns.md §2):
|
||||
# "v1" substring present so future major-version rotations
|
||||
# (v2-blake3-expansion etc.) are detectable at the call site
|
||||
# without string-comparing module paths.
|
||||
assert "v1" in PHI_PRG_VERSION
|
||||
|
||||
|
||||
def test_placeholder_seed_is_32_bytes():
|
||||
|
|
|
|||
|
|
@ -180,6 +180,11 @@ def test_alignment_report_round_trips_via_asdict():
|
|||
|
||||
def test_module_exports_thresholds_and_version():
|
||||
assert PROBE_VERSION == "phi-alignment-v1-lanczos"
|
||||
# versioned-default discipline (calculator-test-patterns.md §2):
|
||||
# "v1" substring present so future major-version rotations are
|
||||
# detectable at the call site without string-comparing module
|
||||
# paths.
|
||||
assert "v1" in PROBE_VERSION
|
||||
assert STRUCTURAL_ALIGNMENT_RATIO_FLOOR == 1.5
|
||||
assert ANTI_ALIGNED_RATIO_CEILING == 0.7
|
||||
assert DEFAULT_K_TOP == 100
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@ Covers:
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
|
@ -505,3 +507,56 @@ def test_score_is_dict_serializable():
|
|||
encoded = json.dumps(sf.to_dict(), default=str)
|
||||
decoded = json.loads(encoded)
|
||||
assert decoded["verdict"] == sf.verdict
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------
|
||||
# Real subprocess-mode CLI test (calculator-test-patterns.md §6).
|
||||
# The in-process build_parser() tests above catch argparse-shape drift
|
||||
# but NOT entry-point / module-loading / sys.argv drift. fox's
|
||||
# test_cli_baseline_runs_clean in test_t3_bound_calculator.py is the
|
||||
# exemplar; this test mirrors that pattern for `arborist substrate score`.
|
||||
# ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cli_substrate_score_subprocess_invocation(tmp_path):
|
||||
"""Real subprocess-mode invocation of `arborist substrate score`.
|
||||
|
||||
Catches drift at the entry-point + module-loading layer that the
|
||||
in-process build_parser() tests would miss — e.g. a refactor that
|
||||
breaks the console-script in pyproject.toml, or that introduces a
|
||||
sys.argv-handling bug.
|
||||
|
||||
Pattern: docs/calculator-test-patterns.md §6 + the substrate
|
||||
rename's three-defect lesson (85be5eb fork_score.py import,
|
||||
209d670 Makefile bench-fork-score, b320e27 .gitlab-ci.yml job).
|
||||
"""
|
||||
parent_path = tmp_path / "parent.json"
|
||||
child_path = tmp_path / "child.json"
|
||||
out_path = tmp_path / "report.json"
|
||||
_write_bench_result(parent_path, {
|
||||
"5s": {"syntax": {"parse_pass_rate": 0.80}},
|
||||
"5t": {}, "5f": {},
|
||||
})
|
||||
_write_bench_result(child_path, {
|
||||
"5s": {"syntax": {"parse_pass_rate": 0.85}},
|
||||
"5t": {}, "5f": {},
|
||||
})
|
||||
|
||||
cmd = [
|
||||
sys.executable, "-m", "arborist.cli",
|
||||
"substrate", "score",
|
||||
"--parent", str(parent_path),
|
||||
"--child", str(child_path),
|
||||
"--out", str(out_path),
|
||||
]
|
||||
out = subprocess.run(
|
||||
cmd, capture_output=True, text=True, check=True,
|
||||
cwd="/home/fox/git/arborist",
|
||||
)
|
||||
# Exit code 0 = ACCEPT or MARGINAL per ScoredFork verdict
|
||||
# discipline. Subprocess.run(check=True) already asserts it.
|
||||
assert out_path.exists(), "expected --out artifact written"
|
||||
artifact = json.loads(out_path.read_text())
|
||||
assert "score" in artifact
|
||||
assert "verdict" in artifact
|
||||
assert artifact["verdict"] in {"ACCEPT", "MARGINAL", "REJECT"}
|
||||
Loading…
Add table
Add a link
Reference in a new issue