ticket #000034 Phase 1a: φ_linear alignment probe + KAT fixture
Lands the synthetic-ablation infrastructure proposed in fce8826's ticket §7 amendment. Same pattern as #000035 Phase 1: ship the deterministic primitive + unit tests + KAT-pinned fixture on synthetic inputs ahead of v7 deployment ramp-up, so the infrastructure is unit-tested + bench-pinned the moment a real v7 checkpoint becomes available (Phase 1b). bench/scripts/phi_alignment_probe.py ==================================== Implements ``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon) -> AlignmentReport`` per #000034 §3.1: - Lanczos top-k + bottom-k via ``scipy.sparse.linalg.eigsh`` over a user-supplied HVP closure. Probe never materializes H. - Alignment score: A(W, H) = Σ_j (Σ_i ⟨W·e_i, v_j⟩²) / (λ_j+ε) / ‖W‖_F², per ticket §2.1. Computed via W^T @ eigvecs and squared-column-norms (numerically stable + cheap). - Verdict thresholds (§3.3): STRUCTURAL_ALIGNMENT (ratio > 1.5) / NO_ALIGNMENT / ANTI_ALIGNED (ratio < 0.7). Defect caught + fixed during smoke-testing: the original "a_uniform" baseline used the mean of a_top + a_bot, which mechanically over-weights a_bot due to the 1/(λ+ε) term. Fix: analytical isotropic baseline, derived in 2026-05-10 docstring: E[A_k(W_uniform, H)] = (1/dim_d) Σ_{j in k-subset} 1/(λ_j+ε) Under the random-oracle modeling W's columns are isotropic Gaussians with E[‖W^T v_j‖²/‖W‖_F²] = 1/dim_d, so this is the expected score for a uniformly-distributed W. Smoke test post-fix: aligned → STRUCTURAL_ALIGNMENT (ratio ~7.97), uniform → NO_ALIGNMENT (ratio ~1.00), anti → ANTI_ALIGNED (ratio ~0.00). All three classes land cleanly in their expected verdict bucket. Module exports ``PROBE_VERSION = "phi-alignment-v1-lanczos"`` so future algorithm rotations are detectable at the call site without string-comparing module paths. Same convention as #000035's PHI_PRG_VERSION. bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl ======================================================== 30 KAT entries — 10 per class (aligned / uniform / anti) — each pinning (seed, dim_d, k, class) → expected_verdict + observed_ratio for regression coverage. Deterministic-seeded so CI replays exactly. Algorithm change MUST bump PROBE_VERSION + emit a new fixture file under bench/fixtures/phi-alignment/. Class ratio ranges: - aligned: 7.77 - 8.27 (well above 1.5 STRUCTURAL_ALIGNMENT floor) - uniform: 0.95 - 1.04 (cleanly within NO_ALIGNMENT band) - anti: 0.00 (well below 0.7 ANTI_ALIGNED ceiling) tests/test_phi_alignment_probe.py ================================= 14 tests covering #000034 §3.2 + the strict-input-validation surface: - Determinism (verdict + ratio stable across calls within Lanczos float tolerance — eigsh uses randomized initial vectors). - Verdict thresholds (engineered cases land in correct bucket). - Lanczos convergence (top-k matches dense decomposition on synthetic diagonal Hessian within 1e-6). - Module export shape (AlignmentReport JSON-serializable; PROBE_VERSION + thresholds exported). - Validation rejects: non-2D W, dim_d mismatch, k_top+k_bot > dim_d, zero epsilon, zero-norm W, non-square H. - KAT regression against the 30-entry fixture. Tests skip via ``pytest.importorskip`` when ``[hessian]`` extras absent, same fail-soft pattern as the ``[math]``-extras tests for sympy. pyproject.toml — new [hessian] optional-deps block ================================================== Adds ``numpy>=1.26`` + ``scipy>=1.11`` under a new ``[hessian]`` extras gate. Same pattern as ``[math]`` for sympy: kept out of core deps to keep fresh installs lightweight (~80 MB combined). Operators install via ``pip install 'arborist[hessian]'``. #000034 status flip =================== Ticket §7: "open · awaiting go/no-go" → "in progress · Phase 1a landed 2026-05-10; Phase 1b parks for v7 deployment ramp-up". Phase 1b unchanged: closure criterion still requires a real v7 checkpoint measurement that resolves §9.1 of the soft-hash- channel-analysis. TICKETS.md index row refreshed. Hygiene ======= - make test → 1669 passed, 45 skipped (was 1643; +14 anchor_prg not in suite from Phase 1a, +14 phi_alignment from this commit — wait, +12 net since some tests were dropped/renamed in fox's parallel work. Bottom-line: 1669 stable.) - make chain-check-shards → 0 across all 7 shards. - arborist.substrate namespace untouched; this lands under bench/scripts/ since it's a measurement tool, not a substrate primitive — same dir as phi_alignment_probe's intended siblings.
This commit is contained in:
parent
2bc4a15cc7
commit
1dfb8b9b8f
6 changed files with 620 additions and 3 deletions
34
bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl
Normal file
34
bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
# φ_alignment_probe known-answer tests — version phi-alignment-v1-lanczos
|
||||
# Generated 2026-05-10 from synthetic Hessians (diagonal, eigenvalues 1..dim_d).
|
||||
# Each (W, H) pair uses a deterministic seed; algorithm change MUST bump
|
||||
# PROBE_VERSION + emit a new fixture file under bench/fixtures/phi-alignment/.
|
||||
{"label": "aligned/seed=0/dim=256/k=32", "class": "aligned", "seed": 0, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.9766}
|
||||
{"label": "aligned/seed=1/dim=256/k=32", "class": "aligned", "seed": 1, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.7659}
|
||||
{"label": "aligned/seed=2/dim=256/k=32", "class": "aligned", "seed": 2, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 8.0569}
|
||||
{"label": "aligned/seed=3/dim=256/k=32", "class": "aligned", "seed": 3, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.9586}
|
||||
{"label": "aligned/seed=4/dim=256/k=32", "class": "aligned", "seed": 4, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 8.111}
|
||||
{"label": "aligned/seed=5/dim=256/k=32", "class": "aligned", "seed": 5, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.7725}
|
||||
{"label": "aligned/seed=6/dim=256/k=32", "class": "aligned", "seed": 6, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 8.0183}
|
||||
{"label": "aligned/seed=7/dim=256/k=32", "class": "aligned", "seed": 7, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.779}
|
||||
{"label": "aligned/seed=8/dim=256/k=32", "class": "aligned", "seed": 8, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 8.2703}
|
||||
{"label": "aligned/seed=9/dim=256/k=32", "class": "aligned", "seed": 9, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "STRUCTURAL_ALIGNMENT", "expected_ratio_floor": 1.5, "observed_verdict": "STRUCTURAL_ALIGNMENT", "observed_ratio": 7.9949}
|
||||
{"label": "uniform/seed=0/dim=256/k=32", "class": "uniform", "seed": 0, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 0.9998}
|
||||
{"label": "uniform/seed=1/dim=256/k=32", "class": "uniform", "seed": 1, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 0.9714}
|
||||
{"label": "uniform/seed=2/dim=256/k=32", "class": "uniform", "seed": 2, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 1.0098}
|
||||
{"label": "uniform/seed=3/dim=256/k=32", "class": "uniform", "seed": 3, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 1.0081}
|
||||
{"label": "uniform/seed=4/dim=256/k=32", "class": "uniform", "seed": 4, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 0.9993}
|
||||
{"label": "uniform/seed=5/dim=256/k=32", "class": "uniform", "seed": 5, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 0.9862}
|
||||
{"label": "uniform/seed=6/dim=256/k=32", "class": "uniform", "seed": 6, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 1.0061}
|
||||
{"label": "uniform/seed=7/dim=256/k=32", "class": "uniform", "seed": 7, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 0.9492}
|
||||
{"label": "uniform/seed=8/dim=256/k=32", "class": "uniform", "seed": 8, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 1.0381}
|
||||
{"label": "uniform/seed=9/dim=256/k=32", "class": "uniform", "seed": 9, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "NO_ALIGNMENT", "expected_ratio_floor": 0.7, "expected_ratio_ceiling": 1.5, "observed_verdict": "NO_ALIGNMENT", "observed_ratio": 1.0108}
|
||||
{"label": "anti/seed=0/dim=256/k=32", "class": "anti", "seed": 0, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=1/dim=256/k=32", "class": "anti", "seed": 1, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=2/dim=256/k=32", "class": "anti", "seed": 2, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=3/dim=256/k=32", "class": "anti", "seed": 3, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=4/dim=256/k=32", "class": "anti", "seed": 4, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=5/dim=256/k=32", "class": "anti", "seed": 5, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=6/dim=256/k=32", "class": "anti", "seed": 6, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=7/dim=256/k=32", "class": "anti", "seed": 7, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=8/dim=256/k=32", "class": "anti", "seed": 8, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
{"label": "anti/seed=9/dim=256/k=32", "class": "anti", "seed": 9, "dim_d": 256, "n_cols": 256, "k_top": 32, "k_bot": 32, "expected_verdict": "ANTI_ALIGNED", "expected_ratio_ceiling": 0.7, "observed_verdict": "ANTI_ALIGNED", "observed_ratio": 0.0}
|
||||
280
bench/scripts/phi_alignment_probe.py
Normal file
280
bench/scripts/phi_alignment_probe.py
Normal file
|
|
@ -0,0 +1,280 @@
|
|||
"""φ_linear Hessian-alignment probe (ticket #000034 Phase 1a).
|
||||
|
||||
Measures whether the v7 reference frozen-seed linear projection
|
||||
``W`` (dim_d × 256, mapping SHA-256-digest space to the dim_h
|
||||
parameter-vector space) has structural alignment with the
|
||||
low-eigenvalue subspace of the typical training-loss Hessian
|
||||
``H(Θ)``.
|
||||
|
||||
Why this matters — from #000018 §5.1:
|
||||
|
||||
The v7 reference choice (frozen-seed Gaussian linear projection)
|
||||
is **not obviously independent of parameter geometry**. If W's
|
||||
column space aligns with low-eigenvalue directions of the loss
|
||||
Hessian, the L2 anchor pull provides cheap parameter movement
|
||||
in directions correlated with SHA-256 buckets. Cheap = "reachable
|
||||
within a per-step gradient budget" = exploitable by a T2
|
||||
gradient-shaping adversary.
|
||||
|
||||
This module ships only the probe + verdict logic. Phase 1a uses
|
||||
synthetic Hessians (engineered eigenstructure + known W placement)
|
||||
to validate the probe end-to-end. Phase 1b runs the same probe
|
||||
against a real v7 checkpoint when one becomes available — that's
|
||||
when the verdict actually informs the M1 (φ_PRG) vs M2 (anchor
|
||||
nonce) priority decision.
|
||||
|
||||
Construction — the alignment score (#000034 §2.1):
|
||||
|
||||
A(W, H) = Σ_i ⟨W·e_i, v_i⟩² / (λ_i + ε) / ‖W‖_F²
|
||||
|
||||
where the sum runs over the columns ``W·e_i`` of W and the
|
||||
eigenvectors ``v_i`` of H, ``λ_i`` are the matching eigenvalues,
|
||||
and ε is a small regularizer to avoid division by 0 on flat
|
||||
directions. Under random-oracle modeling A is uniform across the
|
||||
spectrum; concentration in the low-λ tail is the alignment
|
||||
signature.
|
||||
|
||||
Verdict thresholds (#000034 §3.3):
|
||||
|
||||
- ``A_low / A_uniform > 1.5`` → ``STRUCTURAL_ALIGNMENT`` (M1
|
||||
priority).
|
||||
- ``A_low / A_uniform`` near ``1.0`` → ``NO_ALIGNMENT`` (M2
|
||||
alone sufficient).
|
||||
- ``A_low / A_uniform < 0.7`` → ``ANTI_ALIGNED`` (φ_linear is
|
||||
actually safe vs the loss landscape).
|
||||
|
||||
This module imports numpy + scipy at function call time (not
|
||||
module load time) so the rest of arborist can import it without
|
||||
the [hessian] extras installed; tests skip via
|
||||
``pytest.importorskip`` when absent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from typing import Callable
|
||||
|
||||
# Module-level marker so callers can detect probe version drift.
|
||||
PROBE_VERSION = "phi-alignment-v1-lanczos"
|
||||
|
||||
# Verdict thresholds per #000034 §3.3.
|
||||
STRUCTURAL_ALIGNMENT_RATIO_FLOOR = 1.5
|
||||
ANTI_ALIGNED_RATIO_CEILING = 0.7
|
||||
|
||||
# Default Lanczos hyperparameters per #000034 §2.2 recommendation.
|
||||
DEFAULT_K_TOP = 100
|
||||
DEFAULT_K_BOT = 100
|
||||
DEFAULT_EPSILON = 1e-6
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AlignmentReport:
|
||||
"""Output of :func:`measure_alignment`. Frozen so a caller's
|
||||
JSON serialization is stable; round-trips via ``asdict``.
|
||||
"""
|
||||
verdict: str # STRUCTURAL_ALIGNMENT | NO_ALIGNMENT | ANTI_ALIGNED
|
||||
ratio: float # A(W, H_low) / A(W, H_uniform)
|
||||
a_top: float # alignment score concentrated on top-k (high λ)
|
||||
a_bot: float # alignment score concentrated on bot-k (low λ)
|
||||
a_uniform: float # baseline assuming uniform alignment
|
||||
eigenvalues_top: tuple[float, ...]
|
||||
eigenvalues_bot: tuple[float, ...]
|
||||
w_frobenius_norm_squared: float
|
||||
k_top: int
|
||||
k_bot: int
|
||||
epsilon: float
|
||||
probe_version: str = field(default_factory=lambda: PROBE_VERSION)
|
||||
|
||||
def as_dict(self) -> dict:
|
||||
d = asdict(self)
|
||||
# tuples → lists for JSON friendliness
|
||||
d["eigenvalues_top"] = list(self.eigenvalues_top)
|
||||
d["eigenvalues_bot"] = list(self.eigenvalues_bot)
|
||||
return d
|
||||
|
||||
|
||||
def measure_alignment(
|
||||
W,
|
||||
hessian_eval: Callable,
|
||||
*,
|
||||
dim_d: int | None = None,
|
||||
k_top: int = DEFAULT_K_TOP,
|
||||
k_bot: int = DEFAULT_K_BOT,
|
||||
epsilon: float = DEFAULT_EPSILON,
|
||||
) -> AlignmentReport:
|
||||
"""Run the alignment probe on ``W`` against the Hessian operator
|
||||
``hessian_eval``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
W
|
||||
Numpy array of shape ``(dim_d, 256)`` per the v7 reference
|
||||
spec — the frozen-seed linear projection from SHA-256-digest
|
||||
space to parameter-vector space.
|
||||
hessian_eval
|
||||
Callable ``x → H @ x`` representing the symmetric Hessian
|
||||
``H(Θ)`` as a linear operator. The probe never materializes
|
||||
the full Hessian; Lanczos only needs matrix-vector products.
|
||||
dim_d
|
||||
Parameter-space dimension. Defaults to ``W.shape[0]``.
|
||||
k_top, k_bot
|
||||
Number of top and bottom eigenpairs to extract. ``k_top +
|
||||
k_bot`` should be ≤ dim_d; per #000034 §2.2,
|
||||
100/100 is empirically tractable for ~10⁸-parameter
|
||||
checkpoints.
|
||||
epsilon
|
||||
Regularizer inside the alignment score's
|
||||
``1 / (λ_i + ε)`` term to avoid division by 0 on flat
|
||||
directions of the loss landscape.
|
||||
|
||||
Returns
|
||||
-------
|
||||
AlignmentReport
|
||||
Per #000034 §3.3 the verdict is
|
||||
``STRUCTURAL_ALIGNMENT`` / ``NO_ALIGNMENT`` /
|
||||
``ANTI_ALIGNED``.
|
||||
|
||||
Raises
|
||||
------
|
||||
ValueError
|
||||
If ``W`` is not 2-D, if its shape doesn't agree with
|
||||
``dim_d``, or if Lanczos can't extract the requested number
|
||||
of eigenpairs.
|
||||
"""
|
||||
import numpy as np
|
||||
from scipy.sparse.linalg import LinearOperator, eigsh
|
||||
|
||||
# Validation — fail loud on shape errors rather than producing a
|
||||
# silent zero alignment.
|
||||
if W.ndim != 2:
|
||||
raise ValueError(
|
||||
f"W must be 2-D (dim_d × 256); got shape {W.shape!r}"
|
||||
)
|
||||
if dim_d is None:
|
||||
dim_d = W.shape[0]
|
||||
if W.shape[0] != dim_d:
|
||||
raise ValueError(
|
||||
f"W.shape[0] = {W.shape[0]} but dim_d = {dim_d}"
|
||||
)
|
||||
if k_top + k_bot > dim_d:
|
||||
raise ValueError(
|
||||
f"k_top ({k_top}) + k_bot ({k_bot}) > dim_d ({dim_d}); "
|
||||
"Lanczos cannot extract more eigenpairs than the matrix dim"
|
||||
)
|
||||
if epsilon <= 0:
|
||||
raise ValueError(f"epsilon must be positive; got {epsilon!r}")
|
||||
|
||||
# Wrap the user closure in a scipy LinearOperator so eigsh can
|
||||
# call it via matvec. The operator is symmetric by construction
|
||||
# (Hessian) so eigsh's symmetric path applies.
|
||||
op = LinearOperator(
|
||||
shape=(dim_d, dim_d),
|
||||
matvec=hessian_eval,
|
||||
dtype=np.float64,
|
||||
)
|
||||
|
||||
# Lanczos top-k via 'LA' (largest algebraic). Bottom-k via 'SA'
|
||||
# (smallest algebraic). Both directions are needed — the
|
||||
# alignment score weights inversely (1/(λ+ε)), so the bottom-k
|
||||
# dominates for any non-flat spectrum but we report top-k to
|
||||
# detect anti-alignment too.
|
||||
eigvals_top, eigvecs_top = eigsh(op, k=k_top, which="LA")
|
||||
eigvals_bot, eigvecs_bot = eigsh(op, k=k_bot, which="SA")
|
||||
|
||||
# Frobenius normalizer.
|
||||
w_fro_sq = float(np.linalg.norm(W, "fro") ** 2)
|
||||
if w_fro_sq == 0.0:
|
||||
raise ValueError("W has zero Frobenius norm; degenerate input")
|
||||
|
||||
# Alignment score on a given (eigvals, eigvecs) pair.
|
||||
# ⟨W·e_i, v_j⟩ = (W^T v_j)_i, so the inner products across all
|
||||
# 256 columns of W are the entries of (W^T @ v_j). Squared, then
|
||||
# weighted by 1/(λ_j + ε) and summed over j, then normalized.
|
||||
def _score(eigvals: "np.ndarray", eigvecs: "np.ndarray") -> float:
|
||||
# eigvecs.shape = (dim_d, k); each column is a v_j.
|
||||
proj = W.T @ eigvecs # (256, k); col j is W^T v_j
|
||||
# Squared norms per column = Σ_i ⟨W·e_i, v_j⟩²
|
||||
col_sq = np.sum(proj ** 2, axis=0) # (k,)
|
||||
weighted = col_sq / (eigvals + epsilon)
|
||||
return float(np.sum(weighted) / w_fro_sq)
|
||||
|
||||
a_top = _score(eigvals_top, eigvecs_top)
|
||||
a_bot = _score(eigvals_bot, eigvecs_bot)
|
||||
|
||||
# Isotropic baseline (#000034 §2.1, derived 2026-05-10).
|
||||
#
|
||||
# Under random-oracle modeling W's columns are isotropically
|
||||
# distributed Gaussians; for each unit-norm eigenvector v_j,
|
||||
# E[‖W^T v_j‖²] = 256 σ² and ‖W‖_F² = dim_d × 256 σ², so
|
||||
# E[‖W^T v_j‖² / ‖W‖_F²] = 1/dim_d. Thus:
|
||||
#
|
||||
# E[A_k(W_uniform, H)] = (1/dim_d) Σ_{j in k-subset} 1/(λ_j+ε)
|
||||
#
|
||||
# This is the per-eigenvalue-subset baseline a uniformly-
|
||||
# distributed W would produce; the verdict ratio is the
|
||||
# observed alignment on the BOTTOM-k normalized by the
|
||||
# bottom-k's isotropic baseline. A previous formulation
|
||||
# (mean(a_top, a_bot)) was wrong because the 1/(λ+ε) weighting
|
||||
# makes a_bot mechanically large for any W shape — so the
|
||||
# ratio came out > 1 even for genuinely uniform W. The
|
||||
# analytical baseline corrects this.
|
||||
a_uniform_top = float(np.sum(1.0 / (eigvals_top + epsilon)) / dim_d)
|
||||
a_uniform_bot = float(np.sum(1.0 / (eigvals_bot + epsilon)) / dim_d)
|
||||
|
||||
if a_uniform_bot == 0.0:
|
||||
# Degenerate — bottom-k baseline is exactly zero (would
|
||||
# require infinite eigenvalues, which Lanczos can't return).
|
||||
# Defensive: treat as NO_ALIGNMENT.
|
||||
ratio = 1.0
|
||||
else:
|
||||
ratio = a_bot / a_uniform_bot
|
||||
|
||||
if ratio > STRUCTURAL_ALIGNMENT_RATIO_FLOOR:
|
||||
verdict = "STRUCTURAL_ALIGNMENT"
|
||||
elif ratio < ANTI_ALIGNED_RATIO_CEILING:
|
||||
verdict = "ANTI_ALIGNED"
|
||||
else:
|
||||
verdict = "NO_ALIGNMENT"
|
||||
|
||||
return AlignmentReport(
|
||||
verdict=verdict,
|
||||
ratio=ratio,
|
||||
a_top=a_top,
|
||||
a_bot=a_bot,
|
||||
a_uniform=a_uniform_bot, # bottom-k isotropic baseline
|
||||
eigenvalues_top=tuple(float(x) for x in eigvals_top),
|
||||
eigenvalues_bot=tuple(float(x) for x in eigvals_bot),
|
||||
w_frobenius_norm_squared=w_fro_sq,
|
||||
k_top=k_top,
|
||||
k_bot=k_bot,
|
||||
epsilon=epsilon,
|
||||
)
|
||||
|
||||
|
||||
def hessian_from_dense(H):
|
||||
"""Helper for unit tests + KAT generation: turn a dense
|
||||
symmetric matrix into a ``hessian_eval`` closure.
|
||||
|
||||
Production callsites (real v7 checkpoints) should pass an
|
||||
HVP closure instead of materializing the Hessian; this helper
|
||||
is for the synthetic Phase 1a path only.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
H = np.asarray(H, dtype=np.float64)
|
||||
if H.ndim != 2 or H.shape[0] != H.shape[1]:
|
||||
raise ValueError(f"H must be square 2-D; got shape {H.shape!r}")
|
||||
return lambda x: H @ x
|
||||
|
||||
|
||||
__all__ = [
|
||||
"PROBE_VERSION",
|
||||
"STRUCTURAL_ALIGNMENT_RATIO_FLOOR",
|
||||
"ANTI_ALIGNED_RATIO_CEILING",
|
||||
"DEFAULT_K_TOP",
|
||||
"DEFAULT_K_BOT",
|
||||
"DEFAULT_EPSILON",
|
||||
"AlignmentReport",
|
||||
"measure_alignment",
|
||||
"hessian_from_dense",
|
||||
]
|
||||
|
|
@ -74,7 +74,7 @@ Newest first. Update on every open/close.
|
|||
| #000037 | Prometheus-Σ recursive falsification controller (bicameral substrate) | open · awaiting go/no-go (doc-only Phase 0) | 2026-05-09 | — |
|
||||
| #000036 | T3 per-window covert-channel budget bound | open · awaiting go/no-go (#000018 follow-up) | 2026-05-09 | — |
|
||||
| #000035 | PRG choice for φ_PRG (HMAC-SHA-512 expansion) | in progress · Phase 1 landed 2026-05-10; v7 §9.10 amendment awaits maintainer review | 2026-05-09 | — |
|
||||
| #000034 | Hessian alignment under φ_linear | open · awaiting go/no-go (#000018 follow-up) | 2026-05-09 | — |
|
||||
| #000034 | Hessian alignment under φ_linear | in progress · Phase 1a landed 2026-05-10 (synthetic-ablation probe + KAT fixture); Phase 1b parks for v7 ramp-up | 2026-05-09 | — |
|
||||
| #000033 | Claim-pack pillar VII (combinatorics) | closed · landed 2026-05-09 (live in shard 000.db; lift verified) | 2026-05-09 | — |
|
||||
| #000032 | combinatorics@v1 π* (integer counting kernel) | closed · landed 2026-05-09 | 2026-05-09 | — |
|
||||
| #000031 | Surface-ingest cited textbooks for claim-pack warrant promotion | closed · Phase 1 + 2 + 2.5 (92/92) + 3 (verifier wiring) + B-1 (via_citation_alias attribution: 18 direct + 74 +alias) + B-2 (source-side title-from-author backfill) all landed 2026-05-09 / 2026-05-10 | 2026-05-09 | — |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000034 — Hessian alignment under φ_linear
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Status:** in progress · Phase 1a landed 2026-05-10 (synthetic-ablation probe + KAT fixture); Phase 1b parks for v7 deployment ramp-up
|
||||
**Opened:** 2026-05-09
|
||||
**Scope:** Empirically measure whether the v7 reference
|
||||
`embed_hard_to_vec` matrix W (frozen-seed linear projection from
|
||||
|
|
@ -176,7 +176,7 @@ side. **Phase 1a (synthetic-ablation) proposed below as a way
|
|||
to land probe infrastructure ahead of v7 ramp-up — same pattern
|
||||
as #000035 Phase 1.**
|
||||
|
||||
### Phase 1a — synthetic-ablation probe (proposed, not landed)
|
||||
### Phase 1a — synthetic-ablation probe (landed 2026-05-10)
|
||||
|
||||
Mirrors the #000035 pattern: ship the deterministic primitive +
|
||||
unit tests + KAT-style verification on a synthetic input ahead
|
||||
|
|
|
|||
|
|
@ -38,6 +38,17 @@ math = [
|
|||
# fresh checkout. Tests skip via pytest.importorskip when absent.
|
||||
"sympy>=1.13",
|
||||
]
|
||||
hessian = [
|
||||
# Phi_alignment_probe (ticket #000034 Phase 1a). Lanczos top-k +
|
||||
# bottom-k eigendecomposition for measuring whether v7's frozen
|
||||
# linear projection W aligns with the loss Hessian's low-eigenvalue
|
||||
# subspace. Numpy + scipy together ~80 MB; gated separately from
|
||||
# core to keep the default install lightweight. Tests skip via
|
||||
# pytest.importorskip when absent. Install with:
|
||||
# pip install 'arborist[hessian]'
|
||||
"numpy>=1.26",
|
||||
"scipy>=1.11",
|
||||
]
|
||||
crawler = [
|
||||
# Verbatim lift from agents.ai.unturf.com/core. Off by default — the
|
||||
# default test suite never imports the crawler. Install with:
|
||||
|
|
|
|||
292
tests/test_phi_alignment_probe.py
Normal file
292
tests/test_phi_alignment_probe.py
Normal file
|
|
@ -0,0 +1,292 @@
|
|||
"""Tests for bench.scripts.phi_alignment_probe (ticket #000034 §3.2).
|
||||
|
||||
Coverage matches the ticket's acceptance criteria:
|
||||
|
||||
- Determinism: same (W, H, seed) → identical ratio + verdict.
|
||||
- Verdict thresholds: engineered cases (aligned / uniform / anti)
|
||||
land in their expected bucket; boundary cases land cleanly.
|
||||
- Lanczos convergence: top-k eigenvalues from scipy.eigsh match
|
||||
a known dense decomposition on a synthetic Hessian.
|
||||
- Module export shape: AlignmentReport dataclass round-trips via
|
||||
``asdict``; ``PROBE_VERSION`` + thresholds exported.
|
||||
- Input validation: shape mismatches, bad k, bad epsilon all raise.
|
||||
- KAT regression: pinned (seed, dim, k, class) → expected_verdict
|
||||
triples replay against the synthetic-checkpoints.jsonl fixture.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
# Module imports skip if numpy/scipy aren't installed (the [hessian]
|
||||
# extras gate per #000034 §3 + pyproject.toml).
|
||||
np = pytest.importorskip("numpy")
|
||||
pytest.importorskip("scipy.sparse.linalg")
|
||||
|
||||
from bench.scripts.phi_alignment_probe import (
|
||||
ANTI_ALIGNED_RATIO_CEILING,
|
||||
DEFAULT_K_TOP,
|
||||
DEFAULT_K_BOT,
|
||||
PROBE_VERSION,
|
||||
STRUCTURAL_ALIGNMENT_RATIO_FLOOR,
|
||||
AlignmentReport,
|
||||
hessian_from_dense,
|
||||
measure_alignment,
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- helpers
|
||||
|
||||
|
||||
def _diagonal_h(dim_d: int, scale: float = 1.0):
|
||||
"""Diagonal Hessian with eigenvalues 1..dim_d * scale.
|
||||
Eigenvectors are the canonical basis."""
|
||||
return np.diag(np.arange(1, dim_d + 1).astype(float) * scale)
|
||||
|
||||
|
||||
def _aligned_w(dim_d: int, n_cols: int, low_k: int, seed: int):
|
||||
"""W with mass concentrated on the bottom-low_k canonical-basis
|
||||
rows — i.e. the low-eigenvalue subspace of _diagonal_h."""
|
||||
rng = np.random.default_rng(seed)
|
||||
W = np.zeros((dim_d, n_cols))
|
||||
W[:low_k, :] = rng.standard_normal((low_k, n_cols))
|
||||
return W
|
||||
|
||||
|
||||
def _uniform_w(dim_d: int, n_cols: int, seed: int):
|
||||
rng = np.random.default_rng(seed)
|
||||
return rng.standard_normal((dim_d, n_cols))
|
||||
|
||||
|
||||
def _anti_w(dim_d: int, n_cols: int, high_k: int, seed: int):
|
||||
"""W with mass concentrated on the top-high_k canonical-basis
|
||||
rows — i.e. the high-eigenvalue subspace."""
|
||||
rng = np.random.default_rng(seed)
|
||||
W = np.zeros((dim_d, n_cols))
|
||||
W[-high_k:, :] = rng.standard_normal((high_k, n_cols))
|
||||
return W
|
||||
|
||||
|
||||
# ----------------------------------------------------------- determinism
|
||||
|
||||
|
||||
def test_measure_alignment_deterministic_same_inputs():
|
||||
"""Same inputs produce the same verdict + ratio within Lanczos
|
||||
float tolerance.
|
||||
|
||||
Strict bit-equality fails because scipy.eigsh's Lanczos
|
||||
iteration uses a randomized starting vector internally; output
|
||||
differs in the ~15th decimal across runs even with identical
|
||||
user inputs. The verdict + alignment-class signal is stable —
|
||||
that's what callers depend on.
|
||||
"""
|
||||
dim_d = 64
|
||||
W = _uniform_w(dim_d, 256, seed=0)
|
||||
H = _diagonal_h(dim_d)
|
||||
r1 = measure_alignment(W, hessian_from_dense(H), k_top=8, k_bot=8)
|
||||
r2 = measure_alignment(W, hessian_from_dense(H), k_top=8, k_bot=8)
|
||||
assert r1.verdict == r2.verdict
|
||||
assert r1.ratio == pytest.approx(r2.ratio, rel=1e-9)
|
||||
assert sorted(r1.eigenvalues_top) == pytest.approx(
|
||||
sorted(r2.eigenvalues_top), rel=1e-6
|
||||
)
|
||||
assert sorted(r1.eigenvalues_bot) == pytest.approx(
|
||||
sorted(r2.eigenvalues_bot), rel=1e-6
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- verdicts
|
||||
|
||||
|
||||
def test_aligned_w_returns_structural_alignment():
|
||||
"""W concentrated on low-λ subspace must produce
|
||||
STRUCTURAL_ALIGNMENT (ratio > 1.5)."""
|
||||
W = _aligned_w(256, 256, low_k=32, seed=42)
|
||||
H = _diagonal_h(256)
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
assert r.verdict == "STRUCTURAL_ALIGNMENT"
|
||||
assert r.ratio > STRUCTURAL_ALIGNMENT_RATIO_FLOOR
|
||||
|
||||
|
||||
def test_uniform_w_returns_no_alignment():
|
||||
"""W with isotropically-distributed entries must produce
|
||||
NO_ALIGNMENT (ratio near 1.0)."""
|
||||
W = _uniform_w(256, 256, seed=42)
|
||||
H = _diagonal_h(256)
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
assert r.verdict == "NO_ALIGNMENT"
|
||||
assert ANTI_ALIGNED_RATIO_CEILING <= r.ratio <= STRUCTURAL_ALIGNMENT_RATIO_FLOOR
|
||||
|
||||
|
||||
def test_anti_aligned_w_returns_anti_aligned():
|
||||
"""W concentrated on high-λ subspace must produce
|
||||
ANTI_ALIGNED (ratio < 0.7)."""
|
||||
W = _anti_w(256, 256, high_k=32, seed=42)
|
||||
H = _diagonal_h(256)
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=32, k_bot=32)
|
||||
assert r.verdict == "ANTI_ALIGNED"
|
||||
assert r.ratio < ANTI_ALIGNED_RATIO_CEILING
|
||||
|
||||
|
||||
# ----------------------------------------------------------- Lanczos convergence
|
||||
|
||||
|
||||
def test_lanczos_top_k_matches_dense_decomposition():
|
||||
"""Synthetic diagonal Hessian: top-k Lanczos eigenvalues must
|
||||
match the analytical top-k eigenvalues exactly (within tol)."""
|
||||
dim_d = 128
|
||||
H = _diagonal_h(dim_d)
|
||||
W = _uniform_w(dim_d, 256, seed=0)
|
||||
k = 16
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=k, k_bot=k)
|
||||
|
||||
# Diagonal H has eigenvalues 1..dim_d, so top-k = dim_d-k+1..dim_d
|
||||
expected_top = list(range(dim_d - k + 1, dim_d + 1))
|
||||
expected_bot = list(range(1, k + 1))
|
||||
|
||||
# Lanczos returns eigenvalues in algebraic order; sort to match
|
||||
# canonical ordering. Allow small numerical tolerance.
|
||||
observed_top = sorted(r.eigenvalues_top)
|
||||
observed_bot = sorted(r.eigenvalues_bot)
|
||||
assert all(
|
||||
abs(o - e) < 1e-6 for o, e in zip(observed_top, expected_top)
|
||||
), f"top: expected {expected_top}, got {observed_top}"
|
||||
assert all(
|
||||
abs(o - e) < 1e-6 for o, e in zip(observed_bot, expected_bot)
|
||||
), f"bot: expected {expected_bot}, got {observed_bot}"
|
||||
|
||||
|
||||
# ----------------------------------------------------------- module shape
|
||||
|
||||
|
||||
def test_alignment_report_round_trips_via_asdict():
|
||||
W = _uniform_w(64, 32, seed=0)
|
||||
H = _diagonal_h(64)
|
||||
r = measure_alignment(W, hessian_from_dense(H), k_top=4, k_bot=4)
|
||||
d = r.as_dict()
|
||||
assert isinstance(d, dict)
|
||||
assert d["verdict"] in {
|
||||
"STRUCTURAL_ALIGNMENT", "NO_ALIGNMENT", "ANTI_ALIGNED",
|
||||
}
|
||||
assert isinstance(d["eigenvalues_top"], list) # not tuple
|
||||
assert isinstance(d["eigenvalues_bot"], list)
|
||||
assert d["probe_version"] == PROBE_VERSION
|
||||
# JSON-serializable
|
||||
json.dumps(d)
|
||||
|
||||
|
||||
def test_module_exports_thresholds_and_version():
|
||||
assert PROBE_VERSION == "phi-alignment-v1-lanczos"
|
||||
assert STRUCTURAL_ALIGNMENT_RATIO_FLOOR == 1.5
|
||||
assert ANTI_ALIGNED_RATIO_CEILING == 0.7
|
||||
assert DEFAULT_K_TOP == 100
|
||||
assert DEFAULT_K_BOT == 100
|
||||
|
||||
|
||||
# ----------------------------------------------------------- validation
|
||||
|
||||
|
||||
def test_rejects_non_2d_w():
|
||||
H = _diagonal_h(16)
|
||||
with pytest.raises(ValueError, match="2-D"):
|
||||
measure_alignment(np.zeros(16), hessian_from_dense(H), k_top=2, k_bot=2)
|
||||
|
||||
|
||||
def test_rejects_dim_d_mismatch():
|
||||
W = np.zeros((16, 256))
|
||||
H = _diagonal_h(32) # different dim
|
||||
with pytest.raises(ValueError, match="dim_d"):
|
||||
measure_alignment(W, hessian_from_dense(H), dim_d=32, k_top=4, k_bot=4)
|
||||
|
||||
|
||||
def test_rejects_k_too_large():
|
||||
W = _uniform_w(8, 8, seed=0)
|
||||
H = _diagonal_h(8)
|
||||
with pytest.raises(ValueError, match="dim_d"):
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=10, k_bot=10)
|
||||
|
||||
|
||||
def test_rejects_zero_epsilon():
|
||||
W = _uniform_w(16, 32, seed=0)
|
||||
H = _diagonal_h(16)
|
||||
with pytest.raises(ValueError, match="epsilon"):
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=2, k_bot=2, epsilon=0)
|
||||
|
||||
|
||||
def test_rejects_zero_norm_w():
|
||||
W = np.zeros((16, 32))
|
||||
H = _diagonal_h(16)
|
||||
with pytest.raises(ValueError, match="Frobenius"):
|
||||
measure_alignment(W, hessian_from_dense(H), k_top=2, k_bot=2)
|
||||
|
||||
|
||||
def test_hessian_from_dense_rejects_non_square():
|
||||
with pytest.raises(ValueError, match="square"):
|
||||
hessian_from_dense(np.zeros((4, 8)))
|
||||
|
||||
|
||||
# ----------------------------------------------------------- KAT regression
|
||||
|
||||
|
||||
KAT_FIXTURE = (
|
||||
Path(__file__).parent.parent
|
||||
/ "bench" / "fixtures" / "phi-alignment"
|
||||
/ "synthetic-checkpoints.jsonl"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not KAT_FIXTURE.exists(),
|
||||
reason="KAT fixture not yet generated",
|
||||
)
|
||||
def test_phi_alignment_known_answer_tests():
|
||||
"""Replay the pinned synthetic-checkpoints.jsonl fixture; every
|
||||
entry's verdict must match what the probe produces today.
|
||||
|
||||
Algorithm change (probe rewrite, threshold tuning, etc.) MUST
|
||||
bump PROBE_VERSION + emit a new fixture file under
|
||||
bench/fixtures/phi-alignment/ — old runs replay against old data.
|
||||
"""
|
||||
cls_to_w_factory = {
|
||||
"aligned": lambda dim_d, n, low_k, seed: _aligned_w(dim_d, n, low_k, seed),
|
||||
"uniform": lambda dim_d, n, _low_k, seed: _uniform_w(dim_d, n, seed),
|
||||
"anti": lambda dim_d, n, high_k, seed: _anti_w(dim_d, n, high_k, seed),
|
||||
}
|
||||
n_kats = 0
|
||||
for line in KAT_FIXTURE.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
kat = json.loads(line)
|
||||
n_kats += 1
|
||||
cls = kat["class"]
|
||||
dim_d = int(kat["dim_d"])
|
||||
n_cols = int(kat["n_cols"])
|
||||
seed = int(kat["seed"])
|
||||
k_top = int(kat["k_top"])
|
||||
k_bot = int(kat["k_bot"])
|
||||
# All synthetic KATs use low_k = high_k = k_top = 32.
|
||||
W = cls_to_w_factory[cls](dim_d, n_cols, k_top, seed)
|
||||
H = _diagonal_h(dim_d)
|
||||
r = measure_alignment(
|
||||
W, hessian_from_dense(H),
|
||||
k_top=k_top, k_bot=k_bot,
|
||||
)
|
||||
assert r.verdict == kat["expected_verdict"], (
|
||||
f"KAT mismatch for {kat['label']!r}: "
|
||||
f"expected {kat['expected_verdict']}, got {r.verdict} "
|
||||
f"(ratio={r.ratio:.4f})"
|
||||
)
|
||||
# Float tolerance for the ratio match (Lanczos has minor
|
||||
# numerical noise across scipy versions).
|
||||
observed = round(r.ratio, 4)
|
||||
recorded = float(kat["observed_ratio"])
|
||||
assert abs(observed - recorded) < 0.05, (
|
||||
f"ratio drift for {kat['label']!r}: recorded {recorded}, "
|
||||
f"observed {observed}"
|
||||
)
|
||||
|
||||
assert n_kats >= 30, f"KAT fixture seems incomplete: only {n_kats} entries"
|
||||
Loading…
Add table
Add a link
Reference in a new issue