ticket #000034: propose Phase 1a synthetic-ablation probe (doc-only)
Mirrors #000035's pattern: land deterministic probe infrastructure + unit tests + KAT-pinned fixture on synthetic inputs ahead of the real-deployment target. v7 plastic-training has no representative checkpoint today (per §5 risk + §7 status); Phase 1a closes that gap by giving the probe a synthetic test surface that's verifiable end-to-end without waiting for v7 ramp-up. What §7 now proposes Phase 1a lands: 1. ``bench/scripts/phi_alignment_probe.py`` (~150 LOC, signature ``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon) → AlignmentReport``). Lanczos top-k + bottom-k via scipy. 2. ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``: ~30 deterministic-seed (W, H) pairs across three classes: aligned (engineered W in low-λ subspace → expect STRUCTURAL_ALIGNMENT), uniform (Gaussian W per v7 reference → expect NO_ALIGNMENT), anti-aligned (W in high-λ subspace → expect ANTI_ALIGNED). Same KAT-regression discipline as #000035's phi_prg fixture. 3. ``tests/test_phi_alignment_probe.py``: round-trip determinism; verdict thresholds; Lanczos convergence vs dense decomposition on synthetic Hessian; module export shape; KAT regression. 4. New ``[hessian]`` optional-dependencies block in pyproject.toml (numpy>=1.26 + scipy>=1.11). Same gating as the existing ``[math]`` block for sympy — keeps core install lightweight. Tests fail-soft via ``pytest.importorskip`` when absent. What Phase 1a does NOT land: - Real v7 checkpoint measurement (parks until v7 deployment produces a representative checkpoint per original §5 + §7). - M1 / M2 implementation (separate tickets). - Adversarial-training experiment (out-of-scope per §4). Phase 1b (real-checkpoint measurement) preserves the original §7 closure criterion: ticket closes when a v7 checkpoint produces a verdict resolving §9.1 of the soft-hash-channel analysis. Phase 1a is prerequisite infrastructure, not closure. This is doc-only — no code change in this commit. fox decision needed before Phase 1a implementation lands.
This commit is contained in:
parent
a9fcb4251d
commit
fce88268fd
1 changed files with 99 additions and 6 deletions
|
|
@ -171,10 +171,103 @@ across runs to detect drift in the analysis.
|
|||
|
||||
**Open · awaiting go/no-go.** Parks until a v7 reference
|
||||
checkpoint is available; the analysis is empirical-only and
|
||||
useless without representative training data. Recommended
|
||||
deferral to v7 deployment ramp-up.
|
||||
useless without representative training data on the V7 deployment
|
||||
side. **Phase 1a (synthetic-ablation) proposed below as a way
|
||||
to land probe infrastructure ahead of v7 ramp-up — same pattern
|
||||
as #000035 Phase 1.**
|
||||
|
||||
Closure criterion: `bench/scripts/phi_alignment_probe.py` lands
|
||||
+ a measurement run against a representative checkpoint
|
||||
produces a verdict that resolves §9.1 of `docs/soft-hash-
|
||||
channel-analysis.md`.
|
||||
### Phase 1a — synthetic-ablation probe (proposed, not landed)
|
||||
|
||||
Mirrors the #000035 pattern: ship the deterministic primitive +
|
||||
unit tests + KAT-style verification on a synthetic input ahead
|
||||
of the real-deployment target, so the cryptographic / numerical
|
||||
infrastructure is unit-tested + bench-pinned the moment v7
|
||||
deployment provides a real checkpoint.
|
||||
|
||||
What lands in Phase 1a:
|
||||
|
||||
- ``bench/scripts/phi_alignment_probe.py`` (~150 LOC) per §3.1.
|
||||
Public surface:
|
||||
|
||||
```python
|
||||
def measure_alignment(
|
||||
W: numpy.ndarray, # dim_d × 256, frozen
|
||||
hessian_eval: Callable[[numpy.ndarray], numpy.ndarray],
|
||||
# closure: x → H @ x
|
||||
*,
|
||||
k_top: int = 100,
|
||||
k_bot: int = 100,
|
||||
epsilon: float = 1e-6,
|
||||
) -> AlignmentReport
|
||||
```
|
||||
|
||||
``AlignmentReport`` carries top-k / bottom-k eigenvalue
|
||||
spectra, per-eigenvector alignment scores, the aggregate
|
||||
``A(W, H_top) / A(W, H_uniform)`` ratio, and a verdict label
|
||||
per §3.3 thresholds (``STRUCTURAL_ALIGNMENT`` /
|
||||
``NO_ALIGNMENT`` / ``ANTI_ALIGNED``).
|
||||
|
||||
- ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``
|
||||
(~30 KAT entries). Each row pins a synthetic ``(W, H)`` pair
|
||||
with a deterministic seed AND the expected verdict. Three
|
||||
classes:
|
||||
|
||||
1. **Aligned**: W's columns engineered to lie in the
|
||||
low-eigenvalue subspace of a known-eigenstructure quadratic.
|
||||
Expected verdict: ``STRUCTURAL_ALIGNMENT``.
|
||||
2. **Uniform**: W ~ Gaussian(0, 1/dim_d) — the v7 reference
|
||||
spec — measured against an isotropic Hessian. Expected
|
||||
verdict: ``NO_ALIGNMENT``.
|
||||
3. **Anti-aligned**: W's columns deliberately placed in the
|
||||
high-eigenvalue subspace. Expected verdict:
|
||||
``ANTI_ALIGNED``.
|
||||
|
||||
- ``tests/test_phi_alignment_probe.py``:
|
||||
|
||||
- Round-trip determinism (same seed → same alignment score).
|
||||
- Verdict thresholds (engineered cases land in their expected
|
||||
bucket; boundary cases land cleanly).
|
||||
- Lanczos convergence on the synthetic Hessian (top-k matches
|
||||
a known dense decomposition).
|
||||
- Module export shape.
|
||||
- KAT regression against the fixture.
|
||||
|
||||
- New optional-deps block in ``pyproject.toml``:
|
||||
|
||||
```toml
|
||||
[project.optional-dependencies]
|
||||
hessian = [
|
||||
# Lanczos top-k / bottom-k eigendecomposition for the
|
||||
# phi_alignment_probe (#000034). Pulled into a separate
|
||||
# extras to keep the core install lightweight; same
|
||||
# pattern as the [math] extras for sympy. Tests skip via
|
||||
# pytest.importorskip when absent.
|
||||
"numpy>=1.26",
|
||||
"scipy>=1.11",
|
||||
]
|
||||
```
|
||||
|
||||
Tests fail-soft via ``pytest.importorskip("scipy.sparse.linalg")``.
|
||||
Operators run real-checkpoint measurements via
|
||||
``pip install 'arborist[hessian]'``.
|
||||
|
||||
What does NOT land in Phase 1a:
|
||||
|
||||
- Real v7 checkpoint measurement (gates on v7 deployment producing
|
||||
a representative checkpoint per §5 / original §7).
|
||||
- M1 / M2 implementation (separate tickets).
|
||||
- Adversarial-training experiment (out-of-scope per §4).
|
||||
|
||||
### Phase 1b — real-checkpoint measurement (parks)
|
||||
|
||||
Original §7 deferral preserved. When a v7 reference checkpoint
|
||||
becomes available, run ``measure_alignment`` against it; the
|
||||
verdict resolves §9.1 of ``docs/soft-hash-channel-analysis.md``
|
||||
and either closes #000018's open question (NO_ALIGNMENT) or
|
||||
escalates M1 priority (STRUCTURAL_ALIGNMENT).
|
||||
|
||||
### Closure criterion (unchanged)
|
||||
|
||||
Closes when Phase 1b lands a real-checkpoint measurement that
|
||||
resolves the §9.1 open question — Phase 1a is the prerequisite
|
||||
infrastructure, not closure.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue