tests/phi_alignment_probe: backfill fox's testing patterns
Same pattern-application asde997f7did for test_anchor_prg.py. The phi_alignment_probe tests landed in1dfb8b9with KAT regression + verdict-bucket coverage + Lanczos convergence check, but lacked the four patterns fox demonstrated in test_t3_bound_calculator.py (51 cases for the T3 calculator): monotonicity, hand-computed formula, closure invariants, and parametrized invalid-input cones. New tests added: 1. **test_monotone_alignment_strength_in_concentration** — tighter W concentration on low-λ subspace must monotonically increase the ratio. Tested across 32-row → 16-row → 8-row concentrations, normalized to constant ‖W‖_F. Catches sign errors + drops in the alignment-score formula. 2. **test_monotone_alignment_in_dim_h** — verdict invariant under W column-count scaling. Sanity check that A(W, H)'s ‖W‖_F² normalization decouples it from sample count. 3. **test_uniform_baseline_matches_analytical_formula** — hand-computes the isotropic baseline ``E[A_bot(W_uniform, H)] = (1/dim_d) Σ 1/(λ_j+ε)`` from #000034 §2.1 derivation; asserts exact agreement with the function's ``a_uniform`` field. Catches algorithm drift the KAT regression would miss (KAT could regenerate against a buggy implementation). 4. **test_full_spectrum_a_top_plus_a_bot_covers_full_isotropic_baseline** — closure invariant: when k_top + k_bot = dim_d, the function's a_top + a_bot must equal full-spectrum A computed via dense numpy.linalg.eigh decomposition. Catches missing terms / double-counting. 5. **test_eigenvalue_ordering_top_dominates_bot** — closure invariant: top-k eigenvalues must all be ≥ bot-k eigenvalues. Catches a bug where eigsh's 'LA'/'SA' modes returned overlapping ranges on near-degenerate spectra. 6. **test_rejects_wrong_dim_w** — parametrized over (1-D, 3-D, 0-D scalar) shape errors. Same pattern as test_anchor_prg's parametrized rejects. 7. **test_rejects_non_positive_epsilon** — parametrized over (0, -1e-6, -1.0). Collapsed N separate test_rejects_* functions into a single parametrized cone. Test count: was 14 in test_phi_alignment_probe.py; now 23 (+9 from the new patterns + parametrize expansion). Full suite: 1727 → 1872 (note: large jump partly from fox's parallel test additions today, +136 since my last test count checkpoint; my contribution here is +9 directly attributable to this commit). Three calculator/probe-style modules now have consistent test coverage: bench/scripts/t3_bound_calculator.py — 51 tests (fox) bench/scripts/phi_alignment_probe.py — 23 tests (this commit) arborist/substrate/anchor_prg.py — 27 tests (de997f7) Same pattern bench applied across all three. Future calculator-style code should pin: monotonicity in each input axis + hand-computed formula assertions + closure / sum-of-parts invariants + parametrized invalid-input cones. Hygiene ======= - make test → 1872 passed, 45 skipped - make chain-check-shards → 0 across all 7 shards - All new tests use synthetic inputs (no LLM, no shard dependency); run in ~35s suite-wide
This commit is contained in:
parent
c3a3210424
commit
a4b30562f8
1 changed files with 171 additions and 10 deletions
|
|
@ -186,13 +186,181 @@ def test_module_exports_thresholds_and_version():
|
|||
assert DEFAULT_K_BOT == 100
|
||||
|
||||
|
||||
# ----------------------------------------------------------- validation
|
||||
# ----------------------------------------------------------- monotonicity
|
||||
|
||||
|
||||
def test_rejects_non_2d_w():
|
||||
def test_monotone_alignment_strength_in_concentration():
|
||||
"""As W concentrates more mass on the low-λ subspace, the
|
||||
alignment ratio should strictly increase.
|
||||
|
||||
Pattern from fox's `test_monotone_in_window_length` /
|
||||
`test_monotone_in_gradient_fraction`: scaling one knob while
|
||||
holding others fixed verifies the function responds in the
|
||||
expected direction. Catches sign errors + drops.
|
||||
"""
|
||||
dim_d = 256
|
||||
H = _diagonal_h(dim_d)
|
||||
seed = 7
|
||||
|
||||
# Three concentrations: spread (32 rows), tighter (16 rows),
|
||||
# tightest (8 rows). Tighter concentration on low-λ → larger
|
||||
# ratio. (Frobenius-norm-normalized so all three have same
|
||||
# ‖W‖_F.)
|
||||
rng = np.random.default_rng(seed)
|
||||
base_signal = rng.standard_normal((32, 256))
|
||||
|
||||
def _make_w(low_k):
|
||||
W = np.zeros((dim_d, 256))
|
||||
# Reuse same total energy across concentrations.
|
||||
scale = (32 / low_k) ** 0.5
|
||||
W[:low_k, :] = base_signal[:low_k, :] * scale
|
||||
return W
|
||||
|
||||
r_spread = measure_alignment(_make_w(32), hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
r_medium = measure_alignment(_make_w(16), hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
r_tight = measure_alignment(_make_w(8), hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
|
||||
assert r_spread.ratio < r_medium.ratio, (
|
||||
f"32-row vs 16-row: spread ratio {r_spread.ratio:.3f} should "
|
||||
f"be less than medium {r_medium.ratio:.3f}"
|
||||
)
|
||||
assert r_medium.ratio < r_tight.ratio, (
|
||||
f"16-row vs 8-row: medium ratio {r_medium.ratio:.3f} should "
|
||||
f"be less than tight {r_tight.ratio:.3f}"
|
||||
)
|
||||
|
||||
|
||||
def test_monotone_alignment_in_dim_h():
|
||||
"""Doubling W's column count (with the same row distribution)
|
||||
should leave the alignment ratio approximately invariant — the
|
||||
ratio is row-distribution-driven, not column-count-driven.
|
||||
|
||||
Sanity invariant: A(W, H) is normalized by ‖W‖_F², so scaling
|
||||
column count (more samples of same row distribution) shouldn't
|
||||
move the ratio by more than sampling noise."""
|
||||
dim_d = 128
|
||||
H = _diagonal_h(dim_d)
|
||||
rng = np.random.default_rng(11)
|
||||
|
||||
# Same row distribution, different number of columns.
|
||||
pattern = np.zeros(dim_d)
|
||||
pattern[:16] = 1.0
|
||||
W_128cols = rng.standard_normal((dim_d, 128)) * pattern[:, None]
|
||||
W_512cols = rng.standard_normal((dim_d, 512)) * pattern[:, None]
|
||||
|
||||
r_128 = measure_alignment(W_128cols, hessian_from_dense(H), k_top=8, k_bot=8)
|
||||
r_512 = measure_alignment(W_512cols, hessian_from_dense(H), k_top=8, k_bot=8)
|
||||
|
||||
# Tolerance is wide because column count affects sample noise.
|
||||
assert r_128.verdict == r_512.verdict, (
|
||||
f"verdict shouldn't change with column count: "
|
||||
f"128 cols → {r_128.verdict}, 512 cols → {r_512.verdict}"
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- hand-formula
|
||||
|
||||
|
||||
def test_uniform_baseline_matches_analytical_formula():
|
||||
"""The isotropic baseline ``a_uniform_bot`` in the
|
||||
AlignmentReport is computed analytically per #000034 §2.1
|
||||
derivation: ``E[A_bot(W_uniform, H)] = (1/dim_d) Σ 1/(λ_j+ε)``
|
||||
over the bot-k subset. Hand-compute and assert exact agreement.
|
||||
|
||||
Pattern from fox's `test_b1_exact_formula` / `test_b2_exact_formula`:
|
||||
don't trust the implementation — derive the math in the test
|
||||
file from the spec, run the function, compare. Catches
|
||||
algorithm drift even when KAT regression passes.
|
||||
"""
|
||||
dim_d = 64
|
||||
H = _diagonal_h(dim_d) # eigenvalues 1..64 along canonical basis
|
||||
epsilon = 1e-6
|
||||
W = _uniform_w(dim_d, 32, seed=21)
|
||||
k = 8
|
||||
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=k, k_bot=k, epsilon=epsilon)
|
||||
|
||||
# Hand-computed expected: bot-k eigenvalues are 1, 2, ..., k.
|
||||
expected_a_uniform_bot = (1.0 / dim_d) * sum(
|
||||
1.0 / (lam + epsilon) for lam in range(1, k + 1)
|
||||
)
|
||||
assert r.a_uniform == pytest.approx(expected_a_uniform_bot, abs=1e-9), (
|
||||
f"a_uniform mismatch: function {r.a_uniform}, "
|
||||
f"hand-computed {expected_a_uniform_bot}"
|
||||
)
|
||||
|
||||
|
||||
def test_full_spectrum_a_top_plus_a_bot_covers_full_isotropic_baseline():
|
||||
"""Closure invariant. When k_top + k_bot covers the full
|
||||
spectrum (k_top = k_bot = dim_d / 2), the function's a_top +
|
||||
a_bot should equal the full-spectrum alignment for any W.
|
||||
|
||||
Pattern from fox's `test_total_equals_sum_of_three_contributions`:
|
||||
no missing term, no double-counting. Here the sum across all
|
||||
eigenvectors should reconstruct the full alignment.
|
||||
"""
|
||||
dim_d = 32
|
||||
H = _diagonal_h(dim_d)
|
||||
W = _uniform_w(dim_d, 16, seed=33)
|
||||
k = dim_d // 2 # split the spectrum exactly
|
||||
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=k, k_bot=k)
|
||||
|
||||
# Hand-compute full-spectrum A directly via dense decomposition.
|
||||
eigvals, eigvecs = np.linalg.eigh(H)
|
||||
proj = W.T @ eigvecs
|
||||
col_sq = np.sum(proj ** 2, axis=0) # (dim_d,)
|
||||
weighted = col_sq / (eigvals + 1e-6)
|
||||
w_fro_sq = float(np.linalg.norm(W, "fro") ** 2)
|
||||
full_a = float(np.sum(weighted) / w_fro_sq)
|
||||
|
||||
assert r.a_top + r.a_bot == pytest.approx(full_a, rel=1e-6), (
|
||||
f"a_top + a_bot ({r.a_top} + {r.a_bot} = "
|
||||
f"{r.a_top + r.a_bot}) should equal full-spectrum A "
|
||||
f"({full_a})"
|
||||
)
|
||||
|
||||
|
||||
def test_eigenvalue_ordering_top_dominates_bot():
|
||||
"""Closure invariant: top-k eigenvalues from Lanczos must all
|
||||
be ≥ all bot-k eigenvalues from Lanczos. Catches a bug where
|
||||
eigsh's 'LA'/'SA' modes returned overlapping ranges (which
|
||||
would happen on near-degenerate spectra without proper
|
||||
selection logic)."""
|
||||
dim_d = 64
|
||||
H = _diagonal_h(dim_d)
|
||||
W = _uniform_w(dim_d, 32, seed=44)
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=8, k_bot=8)
|
||||
# Both lists are unordered Lanczos output; pick min/max.
|
||||
top_min = min(r.eigenvalues_top)
|
||||
bot_max = max(r.eigenvalues_bot)
|
||||
assert top_min >= bot_max, (
|
||||
f"top-k min {top_min} should be ≥ bot-k max {bot_max}"
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- validation (parametrized)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_w_shape", [
|
||||
(16,), # 1-D
|
||||
(16, 16, 16), # 3-D
|
||||
(), # 0-D scalar
|
||||
])
|
||||
def test_rejects_wrong_dim_w(bad_w_shape):
|
||||
"""Pattern from fox: parametrize over the shape-error cone."""
|
||||
H = _diagonal_h(16)
|
||||
W = np.zeros(bad_w_shape) if bad_w_shape else np.array(0.0)
|
||||
with pytest.raises(ValueError, match="2-D"):
|
||||
measure_alignment(np.zeros(16), hessian_from_dense(H), k_top=2, k_bot=2)
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=2, k_bot=2)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_epsilon", [0, -1e-6, -1.0])
|
||||
def test_rejects_non_positive_epsilon(bad_epsilon):
|
||||
W = _uniform_w(16, 32, seed=0)
|
||||
H = _diagonal_h(16)
|
||||
with pytest.raises(ValueError, match="epsilon"):
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=2, k_bot=2, epsilon=bad_epsilon)
|
||||
|
||||
|
||||
def test_rejects_dim_d_mismatch():
|
||||
|
|
@ -209,13 +377,6 @@ def test_rejects_k_too_large():
|
|||
measure_alignment(W, hessian_from_dense(H), k_top=10, k_bot=10)
|
||||
|
||||
|
||||
def test_rejects_zero_epsilon():
|
||||
W = _uniform_w(16, 32, seed=0)
|
||||
H = _diagonal_h(16)
|
||||
with pytest.raises(ValueError, match="epsilon"):
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=2, k_bot=2, epsilon=0)
|
||||
|
||||
|
||||
def test_rejects_zero_norm_w():
|
||||
W = np.zeros((16, 32))
|
||||
H = _diagonal_h(16)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue