From fce88268fdd0bac31ef3df72f27665859990af94 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Sun, 10 May 2026 10:10:15 -0400 Subject: [PATCH] ticket #000034: propose Phase 1a synthetic-ablation probe (doc-only) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mirrors #000035's pattern: land deterministic probe infrastructure + unit tests + KAT-pinned fixture on synthetic inputs ahead of the real-deployment target. v7 plastic-training has no representative checkpoint today (per §5 risk + §7 status); Phase 1a closes that gap by giving the probe a synthetic test surface that's verifiable end-to-end without waiting for v7 ramp-up. What §7 now proposes Phase 1a lands: 1. ``bench/scripts/phi_alignment_probe.py`` (~150 LOC, signature ``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon) → AlignmentReport``). Lanczos top-k + bottom-k via scipy. 2. ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``: ~30 deterministic-seed (W, H) pairs across three classes: aligned (engineered W in low-λ subspace → expect STRUCTURAL_ALIGNMENT), uniform (Gaussian W per v7 reference → expect NO_ALIGNMENT), anti-aligned (W in high-λ subspace → expect ANTI_ALIGNED). Same KAT-regression discipline as #000035's phi_prg fixture. 3. ``tests/test_phi_alignment_probe.py``: round-trip determinism; verdict thresholds; Lanczos convergence vs dense decomposition on synthetic Hessian; module export shape; KAT regression. 4. New ``[hessian]`` optional-dependencies block in pyproject.toml (numpy>=1.26 + scipy>=1.11). Same gating as the existing ``[math]`` block for sympy — keeps core install lightweight. Tests fail-soft via ``pytest.importorskip`` when absent. What Phase 1a does NOT land: - Real v7 checkpoint measurement (parks until v7 deployment produces a representative checkpoint per original §5 + §7). - M1 / M2 implementation (separate tickets). - Adversarial-training experiment (out-of-scope per §4). Phase 1b (real-checkpoint measurement) preserves the original §7 closure criterion: ticket closes when a v7 checkpoint produces a verdict resolving §9.1 of the soft-hash-channel analysis. Phase 1a is prerequisite infrastructure, not closure. This is doc-only — no code change in this commit. fox decision needed before Phase 1a implementation lands. --- ...ket-000034-hessian-alignment-phi-linear.md | 105 +++++++++++++++++- 1 file changed, 99 insertions(+), 6 deletions(-) diff --git a/docs/tickets/ticket-000034-hessian-alignment-phi-linear.md b/docs/tickets/ticket-000034-hessian-alignment-phi-linear.md index 37ba203..e679a42 100644 --- a/docs/tickets/ticket-000034-hessian-alignment-phi-linear.md +++ b/docs/tickets/ticket-000034-hessian-alignment-phi-linear.md @@ -171,10 +171,103 @@ across runs to detect drift in the analysis. **Open · awaiting go/no-go.** Parks until a v7 reference checkpoint is available; the analysis is empirical-only and -useless without representative training data. Recommended -deferral to v7 deployment ramp-up. +useless without representative training data on the V7 deployment +side. **Phase 1a (synthetic-ablation) proposed below as a way +to land probe infrastructure ahead of v7 ramp-up — same pattern +as #000035 Phase 1.** -Closure criterion: `bench/scripts/phi_alignment_probe.py` lands -+ a measurement run against a representative checkpoint -produces a verdict that resolves §9.1 of `docs/soft-hash- -channel-analysis.md`. +### Phase 1a — synthetic-ablation probe (proposed, not landed) + +Mirrors the #000035 pattern: ship the deterministic primitive + +unit tests + KAT-style verification on a synthetic input ahead +of the real-deployment target, so the cryptographic / numerical +infrastructure is unit-tested + bench-pinned the moment v7 +deployment provides a real checkpoint. + +What lands in Phase 1a: + +- ``bench/scripts/phi_alignment_probe.py`` (~150 LOC) per §3.1. + Public surface: + + ```python + def measure_alignment( + W: numpy.ndarray, # dim_d × 256, frozen + hessian_eval: Callable[[numpy.ndarray], numpy.ndarray], + # closure: x → H @ x + *, + k_top: int = 100, + k_bot: int = 100, + epsilon: float = 1e-6, + ) -> AlignmentReport + ``` + + ``AlignmentReport`` carries top-k / bottom-k eigenvalue + spectra, per-eigenvector alignment scores, the aggregate + ``A(W, H_top) / A(W, H_uniform)`` ratio, and a verdict label + per §3.3 thresholds (``STRUCTURAL_ALIGNMENT`` / + ``NO_ALIGNMENT`` / ``ANTI_ALIGNED``). + +- ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl`` + (~30 KAT entries). Each row pins a synthetic ``(W, H)`` pair + with a deterministic seed AND the expected verdict. Three + classes: + + 1. **Aligned**: W's columns engineered to lie in the + low-eigenvalue subspace of a known-eigenstructure quadratic. + Expected verdict: ``STRUCTURAL_ALIGNMENT``. + 2. **Uniform**: W ~ Gaussian(0, 1/dim_d) — the v7 reference + spec — measured against an isotropic Hessian. Expected + verdict: ``NO_ALIGNMENT``. + 3. **Anti-aligned**: W's columns deliberately placed in the + high-eigenvalue subspace. Expected verdict: + ``ANTI_ALIGNED``. + +- ``tests/test_phi_alignment_probe.py``: + + - Round-trip determinism (same seed → same alignment score). + - Verdict thresholds (engineered cases land in their expected + bucket; boundary cases land cleanly). + - Lanczos convergence on the synthetic Hessian (top-k matches + a known dense decomposition). + - Module export shape. + - KAT regression against the fixture. + +- New optional-deps block in ``pyproject.toml``: + + ```toml + [project.optional-dependencies] + hessian = [ + # Lanczos top-k / bottom-k eigendecomposition for the + # phi_alignment_probe (#000034). Pulled into a separate + # extras to keep the core install lightweight; same + # pattern as the [math] extras for sympy. Tests skip via + # pytest.importorskip when absent. + "numpy>=1.26", + "scipy>=1.11", + ] + ``` + + Tests fail-soft via ``pytest.importorskip("scipy.sparse.linalg")``. + Operators run real-checkpoint measurements via + ``pip install 'arborist[hessian]'``. + +What does NOT land in Phase 1a: + +- Real v7 checkpoint measurement (gates on v7 deployment producing + a representative checkpoint per §5 / original §7). +- M1 / M2 implementation (separate tickets). +- Adversarial-training experiment (out-of-scope per §4). + +### Phase 1b — real-checkpoint measurement (parks) + +Original §7 deferral preserved. When a v7 reference checkpoint +becomes available, run ``measure_alignment`` against it; the +verdict resolves §9.1 of ``docs/soft-hash-channel-analysis.md`` +and either closes #000018's open question (NO_ALIGNMENT) or +escalates M1 priority (STRUCTURAL_ALIGNMENT). + +### Closure criterion (unchanged) + +Closes when Phase 1b lands a real-checkpoint measurement that +resolves the §9.1 open question — Phase 1a is the prerequisite +infrastructure, not closure.