ticket #000034: propose Phase 1a synthetic-ablation probe (doc-only)

Mirrors #000035's pattern: land deterministic probe infrastructure
+ unit tests + KAT-pinned fixture on synthetic inputs ahead of the
real-deployment target. v7 plastic-training has no representative
checkpoint today (per §5 risk + §7 status); Phase 1a closes that
gap by giving the probe a synthetic test surface that's verifiable
end-to-end without waiting for v7 ramp-up.

What §7 now proposes Phase 1a lands:

1. ``bench/scripts/phi_alignment_probe.py`` (~150 LOC, signature
   ``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon)
   → AlignmentReport``). Lanczos top-k + bottom-k via scipy.

2. ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``:
   ~30 deterministic-seed (W, H) pairs across three classes:
   aligned (engineered W in low-λ subspace → expect
   STRUCTURAL_ALIGNMENT), uniform (Gaussian W per v7 reference
   → expect NO_ALIGNMENT), anti-aligned (W in high-λ subspace →
   expect ANTI_ALIGNED). Same KAT-regression discipline as
   #000035's phi_prg fixture.

3. ``tests/test_phi_alignment_probe.py``: round-trip determinism;
   verdict thresholds; Lanczos convergence vs dense decomposition
   on synthetic Hessian; module export shape; KAT regression.

4. New ``[hessian]`` optional-dependencies block in pyproject.toml
   (numpy>=1.26 + scipy>=1.11). Same gating as the existing
   ``[math]`` block for sympy — keeps core install lightweight.
   Tests fail-soft via ``pytest.importorskip`` when absent.

What Phase 1a does NOT land:

- Real v7 checkpoint measurement (parks until v7 deployment
  produces a representative checkpoint per original §5 + §7).
- M1 / M2 implementation (separate tickets).
- Adversarial-training experiment (out-of-scope per §4).

Phase 1b (real-checkpoint measurement) preserves the original §7
closure criterion: ticket closes when a v7 checkpoint produces a
verdict resolving §9.1 of the soft-hash-channel analysis. Phase 1a
is prerequisite infrastructure, not closure.

This is doc-only — no code change in this commit. fox decision
needed before Phase 1a implementation lands.
This commit is contained in:
russell@unturf.com 2026-05-10 10:10:15 -04:00
parent a9fcb4251d
commit fce88268fd
No known key found for this signature in database

View file

@ -171,10 +171,103 @@ across runs to detect drift in the analysis.
**Open · awaiting go/no-go.** Parks until a v7 reference
checkpoint is available; the analysis is empirical-only and
useless without representative training data. Recommended
deferral to v7 deployment ramp-up.
useless without representative training data on the V7 deployment
side. **Phase 1a (synthetic-ablation) proposed below as a way
to land probe infrastructure ahead of v7 ramp-up — same pattern
as #000035 Phase 1.**
Closure criterion: `bench/scripts/phi_alignment_probe.py` lands
+ a measurement run against a representative checkpoint
produces a verdict that resolves §9.1 of `docs/soft-hash-
channel-analysis.md`.
### Phase 1a — synthetic-ablation probe (proposed, not landed)
Mirrors the #000035 pattern: ship the deterministic primitive +
unit tests + KAT-style verification on a synthetic input ahead
of the real-deployment target, so the cryptographic / numerical
infrastructure is unit-tested + bench-pinned the moment v7
deployment provides a real checkpoint.
What lands in Phase 1a:
- ``bench/scripts/phi_alignment_probe.py`` (~150 LOC) per §3.1.
Public surface:
```python
def measure_alignment(
W: numpy.ndarray, # dim_d × 256, frozen
hessian_eval: Callable[[numpy.ndarray], numpy.ndarray],
# closure: x → H @ x
*,
k_top: int = 100,
k_bot: int = 100,
epsilon: float = 1e-6,
) -> AlignmentReport
```
``AlignmentReport`` carries top-k / bottom-k eigenvalue
spectra, per-eigenvector alignment scores, the aggregate
``A(W, H_top) / A(W, H_uniform)`` ratio, and a verdict label
per §3.3 thresholds (``STRUCTURAL_ALIGNMENT`` /
``NO_ALIGNMENT`` / ``ANTI_ALIGNED``).
- ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``
(~30 KAT entries). Each row pins a synthetic ``(W, H)`` pair
with a deterministic seed AND the expected verdict. Three
classes:
1. **Aligned**: W's columns engineered to lie in the
low-eigenvalue subspace of a known-eigenstructure quadratic.
Expected verdict: ``STRUCTURAL_ALIGNMENT``.
2. **Uniform**: W ~ Gaussian(0, 1/dim_d) — the v7 reference
spec — measured against an isotropic Hessian. Expected
verdict: ``NO_ALIGNMENT``.
3. **Anti-aligned**: W's columns deliberately placed in the
high-eigenvalue subspace. Expected verdict:
``ANTI_ALIGNED``.
- ``tests/test_phi_alignment_probe.py``:
- Round-trip determinism (same seed → same alignment score).
- Verdict thresholds (engineered cases land in their expected
bucket; boundary cases land cleanly).
- Lanczos convergence on the synthetic Hessian (top-k matches
a known dense decomposition).
- Module export shape.
- KAT regression against the fixture.
- New optional-deps block in ``pyproject.toml``:
```toml
[project.optional-dependencies]
hessian = [
# Lanczos top-k / bottom-k eigendecomposition for the
# phi_alignment_probe (#000034). Pulled into a separate
# extras to keep the core install lightweight; same
# pattern as the [math] extras for sympy. Tests skip via
# pytest.importorskip when absent.
"numpy>=1.26",
"scipy>=1.11",
]
```
Tests fail-soft via ``pytest.importorskip("scipy.sparse.linalg")``.
Operators run real-checkpoint measurements via
``pip install 'arborist[hessian]'``.
What does NOT land in Phase 1a:
- Real v7 checkpoint measurement (gates on v7 deployment producing
a representative checkpoint per §5 / original §7).
- M1 / M2 implementation (separate tickets).
- Adversarial-training experiment (out-of-scope per §4).
### Phase 1b — real-checkpoint measurement (parks)
Original §7 deferral preserved. When a v7 reference checkpoint
becomes available, run ``measure_alignment`` against it; the
verdict resolves §9.1 of ``docs/soft-hash-channel-analysis.md``
and either closes #000018's open question (NO_ALIGNMENT) or
escalates M1 priority (STRUCTURAL_ALIGNMENT).
### Closure criterion (unchanged)
Closes when Phase 1b lands a real-checkpoint measurement that
resolves the §9.1 open question — Phase 1a is the prerequisite
infrastructure, not closure.