docs/T3 bound + tests/anchor_prg: apply fox's testing patterns
Two related cleanups in one commit, both surfaced by reading fox's test_t3_bound_calculator.py (51 tests for my T3 calculator): 1. Refresh stale §7 numbers in the T3 bound doc ================================================= fox's test_baseline_matches_section_11_doc docstring (lines 56-61) flagged that my §7.1 worked example said 622.7 / 290.0 / 32.7 bits but the calculator's actual closed-form output is 625.87 / 292.48 / 33.39. Same drift in §7.2 (3358 → 3387.72) and §7.3 (247 → 247.14). The numbers were rounded estimates from when I drafted the doc before the calculator existed. Refreshed all three §7 numeric examples to match the calculator's actual output (verified live via t3_bound_bits()). §3 inline approximation likewise updated (290 → 292.48). Added a short note pointing readers at the calculator + tests as the source of truth. 2. Backfill anchor_prg tests with fox's patterns ================================================= fox's test_t3_bound_calculator.py demonstrated four patterns I'd missed in my #000035 phi_prg tests: - **Output prefix invariant** (closure check): phi_prg(h, n+k)[:n] ≡ phi_prg(h, n). Streaming-counter invariant — would catch a bug where a per-call seed mutation broke determinism across dim_h values. - **Output length monotonicity**: len(phi_prg(h, n)) == n exactly. Parametrized over n ∈ {1, 2, 4, 7, 16, 17, 64, 1024}. Catches off-by-one in `_expand` truncation. - **Hand-computed first block**: assert that the first 64 bytes of output equal a direct ``hmac.new(seed, h + b'\\x00\\x00 \\x00\\x00', sha512).digest()``. Pattern from fox's test_b1_exact_formula — don't rely on KAT regression alone; compute the first-principles math in the test file. Catches algorithm drift the KAT (regenerated against a buggy version) would miss. - **Seed-bleed check**: changing the seed must change EVERY output position. Probability of false-positive ≈ 64 · 2^-32 ≈ 2^-26; none expected in practice. - **Parametrized invalid-input tests**: collapsed N separate ``test_rejects_*`` functions into ``@pytest.mark.parametrize`` cones (4 wrong-size-hash cases + 3 non-positive-dim_h cases). Same coverage, fewer test functions. Test count: was 20 in test_anchor_prg.py; now 27 (+7 from parametrize expansion + new patterns). Full suite: 1720 → 1727. Hygiene ======= - make test → 1727 passed, 45 skipped. - make chain-check-shards → 0 across all 7 shards. - All new tests use ``pytest.importorskip`` already at module top (anchor_prg has no extras gate; tests run unconditionally). Lessons captured ================ The patterns to remember for future calculator/probe-style code: 1. KAT regression alone isn't enough. Add hand-computed formula tests so the math itself is asserted in the test file, not just "consistent with a recorded snapshot". 2. Test monotonicity / closure invariants. They catch algorithm drift, sign errors, missing terms. 3. Parametrize invalid-input tests. One function, N cases. 4. Test the doc's numbers against the function. Catches calibration drift in the doc itself (this commit's finding about §7). 5. CLI subprocess tests for end-to-end. Argparse + main() drift the import-only tests miss.
This commit is contained in:
parent
1104cf97ca
commit
de997f7be3
2 changed files with 136 additions and 33 deletions
|
|
@ -148,9 +148,9 @@ B1 contribution ≤ C_B1 · g · W · log₂( SNR_grad + 1 )
|
|||
with C_B1 = 1 (data processing inequality; tight).
|
||||
```
|
||||
|
||||
For typical deployments `g · ‖∇L_max‖ / σ_grad ≈ 0.05 · 1 / 0.1
|
||||
= 0.5`, so `log₂(1.5) ≈ 0.58 bits/step`. With `W = 10000` and
|
||||
`g = 0.05`: `1 · 0.05 · 10000 · 0.58 ≈ 290 bits/window`.
|
||||
For typical deployments `g · ‖∇L_max‖ / σ_grad = 0.05 · 1 / 0.1
|
||||
= 0.5`, so `log₂(1.5) ≈ 0.5850 bits/step`. With `W = 10000` and
|
||||
`g = 0.05`: `1 · 0.05 · 10000 · 0.5850 ≈ 292.48 bits/window`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -260,12 +260,16 @@ g = 0.05, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
|||
K = 100, R = 8, W = 10000, N_b = 1024, E = 1024
|
||||
SNR_grad = 0.5
|
||||
|
||||
B1 ≤ 1 · 0.05 · 10000 · log₂(1.5) ≈ 290 bits
|
||||
B2 ≤ 1 · ⌈10000/100⌉ · log₂(8) = 300 bits
|
||||
B3 ≤ 1 · ⌈10000/1024⌉ · log₂(102.4) / 2 ≈ 33 bits
|
||||
total ≈ 623 bits/window
|
||||
B1 = 1 · 0.05 · 10000 · log₂(1.5) ≈ 292.48 bits
|
||||
B2 = 1 · ⌈10000/100⌉ · log₂(8) = 300.00 bits
|
||||
B3 = 1 · ⌈10000/1024⌉ · log₂(102.4) / 2 ≈ 33.39 bits
|
||||
total ≈ 625.87 bits/window
|
||||
```
|
||||
|
||||
(Numbers are the calculator's closed-form output, not rounded
|
||||
estimates; see ``bench/scripts/t3_bound_calculator.py`` and the
|
||||
unit tests in ``tests/test_t3_bound_calculator.py``.)
|
||||
|
||||
### §7.2 Medium deployment (longer window)
|
||||
|
||||
```
|
||||
|
|
@ -273,10 +277,10 @@ g = 0.05, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
|||
K = 1000, R = 16, W = 100000, N_b = 8192, E = 8192
|
||||
SNR_grad = 0.5
|
||||
|
||||
B1 ≤ 1 · 0.05 · 100000 · log₂(1.5) ≈ 2900 bits
|
||||
B2 ≤ 1 · 100 · log₂(16) = 400 bits
|
||||
B3 ≤ 1 · 12 · log₂(819.2) / 2 ≈ 58 bits
|
||||
total ≈ 3358 bits/window
|
||||
B1 = 1 · 0.05 · 100000 · log₂(1.5) ≈ 2924.81 bits
|
||||
B2 = 1 · ⌈100000/1000⌉ · log₂(16) = 400.00 bits
|
||||
B3 = 1 · ⌈100000/8192⌉ · log₂(819.2) / 2 ≈ 62.91 bits
|
||||
total ≈ 3387.72 bits/window
|
||||
```
|
||||
|
||||
### §7.3 Hardened deployment (lower g, smaller R)
|
||||
|
|
@ -286,10 +290,10 @@ g = 0.01, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
|||
K = 100, R = 4, W = 10000, N_b = 1024, E = 1024
|
||||
SNR_grad = 0.1
|
||||
|
||||
B1 ≤ 1 · 0.01 · 10000 · log₂(1.1) ≈ 13.7 bits
|
||||
B2 ≤ 1 · 100 · log₂(4) = 200 bits
|
||||
B3 ≤ 1 · 10 · log₂(102.4) / 2 ≈ 33 bits
|
||||
total ≈ 247 bits/window
|
||||
B1 = 1 · 0.01 · 10000 · log₂(1.1) ≈ 13.75 bits
|
||||
B2 = 1 · ⌈10000/100⌉ · log₂(4) = 200.00 bits
|
||||
B3 = 1 · ⌈10000/1024⌉ · log₂(102.4) / 2 ≈ 33.39 bits
|
||||
total ≈ 247.14 bits/window
|
||||
```
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -171,17 +171,128 @@ def test_phi_prg_hash_bit_flip_avalanches():
|
|||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- validation
|
||||
# ----------------------------------------------------------- monotonicity / closure
|
||||
|
||||
|
||||
def test_phi_prg_rejects_short_hash():
|
||||
def test_phi_prg_output_length_exactly_dim_h():
|
||||
"""``phi_prg(h, n)`` must produce exactly ``n`` floats.
|
||||
|
||||
Closure check pattern (cf. fox's `test_total_equals_sum_of_three_contributions`
|
||||
in test_t3_bound_calculator.py): the output length is the
|
||||
contract; off-by-one or truncation bugs in `_expand` would
|
||||
surface here. Parametrized to widen the cone."""
|
||||
h = hashlib.sha256(b"len-check").digest()
|
||||
for n in (1, 2, 4, 7, 16, 17, 64, 1024):
|
||||
v = phi_prg(h, dim_h=n)
|
||||
assert len(v) == n, f"dim_h={n}: expected {n} floats, got {len(v)}"
|
||||
|
||||
|
||||
def test_phi_prg_output_is_prefix_extending():
|
||||
"""``phi_prg(h, n)`` must equal the first ``n`` entries of
|
||||
``phi_prg(h, n+k)``.
|
||||
|
||||
Streaming-counter-mode invariant: the HMAC-SHA-512 expansion
|
||||
is deterministic counter-based, so increasing dim_h adds
|
||||
strictly more bytes at the tail without re-deriving the
|
||||
head. A bug that re-keyed HMAC per-call (e.g. seed mutation)
|
||||
would surface here.
|
||||
|
||||
Pattern from fox's `test_monotone_in_window_length`: scaling
|
||||
one input dimension while holding others fixed is a
|
||||
closed-form invariant the function MUST satisfy.
|
||||
"""
|
||||
h = hashlib.sha256(b"prefix-extend").digest()
|
||||
short = phi_prg(h, dim_h=8)
|
||||
long = phi_prg(h, dim_h=24)
|
||||
assert long[:8] == short, (
|
||||
"phi_prg should be prefix-stable: "
|
||||
f"long[:8] = {long[:8][:3]!r}... vs short = {short[:3]!r}..."
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- hand-formula
|
||||
|
||||
|
||||
def test_phi_prg_first_block_matches_direct_hmac():
|
||||
"""First HMAC-SHA-512 block of phi_prg's output should match
|
||||
a direct hmac.new(seed, hard_hash + b'\\x00\\x00\\x00\\x00',
|
||||
sha512).digest() invocation. Hand-computed against the
|
||||
function's spec (#000035 §3.1 + module §1).
|
||||
|
||||
Pattern from fox's `test_b1_exact_formula`: don't rely on
|
||||
KAT regression alone — compute the first-principles math in
|
||||
the test file and assert exact agreement. Catches algorithm
|
||||
drift that KAT regenerated against a buggy version would
|
||||
miss.
|
||||
"""
|
||||
import hmac
|
||||
|
||||
seed = hashlib.sha256(b"hand-formula-seed").digest()
|
||||
h = hashlib.sha256(b"hand-formula-hash").digest()
|
||||
|
||||
# Spec: out = HMAC-SHA-512(seed, hard_hash || counter_be_4) for
|
||||
# counter = 0, 1, 2, ...; concatenated; truncated to dim_h * 4 bytes.
|
||||
expected_block_0 = hmac.new(
|
||||
seed, h + (0).to_bytes(4, "big"), hashlib.sha512
|
||||
).digest()
|
||||
# First block is 64 bytes = 16 uint32s = 16 floats. dim_h=16
|
||||
# consumes exactly the first block.
|
||||
floats = phi_prg(h, dim_h=16, seed=seed)
|
||||
# Convert expected_block_0 to floats per §2.3 spec.
|
||||
expected_floats = []
|
||||
for i in range(16):
|
||||
u32 = int.from_bytes(expected_block_0[4 * i : 4 * (i + 1)], "big")
|
||||
expected_floats.append(2.0 * (u32 / 2 ** 32) - 1.0)
|
||||
for j, (got, exp) in enumerate(zip(floats, expected_floats)):
|
||||
assert got == pytest.approx(exp, abs=1e-12), (
|
||||
f"first-block float mismatch at index {j}: got {got}, "
|
||||
f"expected {exp}"
|
||||
)
|
||||
|
||||
|
||||
def test_phi_prg_seed_changes_every_byte_independently():
|
||||
"""Pattern from fox's monotone tests: scaling one input
|
||||
independently shouldn't bleed into other parts of the output.
|
||||
For phi_prg this is harder to assert directly — HMAC mixes
|
||||
everything — but we can pin: changing the seed should change
|
||||
EVERY output float (not zero of them)."""
|
||||
h = hashlib.sha256(b"seed-bleed-check").digest()
|
||||
seed_a = hashlib.sha256(b"seed-A").digest()
|
||||
seed_b = hashlib.sha256(b"seed-B").digest()
|
||||
out_a = phi_prg(h, dim_h=64, seed=seed_a)
|
||||
out_b = phi_prg(h, dim_h=64, seed=seed_b)
|
||||
# Every position should differ — under HMAC-SHA-512 PRF,
|
||||
# the probability that any specific 32-bit float matches by
|
||||
# chance is 2^-32, so 64 positions × 2^-32 ≈ 2^-26 false
|
||||
# positives expected. None expected in practice.
|
||||
matches = sum(1 for a, b in zip(out_a, out_b) if a == b)
|
||||
assert matches == 0, (
|
||||
f"{matches}/64 positions matched between different seeds; "
|
||||
"PRF bleed check failed"
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------- validation (parametrized)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_hash", [
|
||||
b"too short", # too few bytes
|
||||
b"\x00" * 33, # too many bytes
|
||||
b"", # empty
|
||||
b"\x00" * 31, # off by one short
|
||||
])
|
||||
def test_phi_prg_rejects_wrong_size_hash(bad_hash):
|
||||
"""Pattern from fox: parametrize over the invalid-input cone
|
||||
rather than spawn a separate test function per case."""
|
||||
with pytest.raises(ValueError, match="32 bytes"):
|
||||
phi_prg(b"too short", dim_h=8)
|
||||
phi_prg(bad_hash, dim_h=8)
|
||||
|
||||
|
||||
def test_phi_prg_rejects_long_hash():
|
||||
with pytest.raises(ValueError, match="32 bytes"):
|
||||
phi_prg(b"\x00" * 33, dim_h=8)
|
||||
@pytest.mark.parametrize("bad_dim", [0, -1, -100])
|
||||
def test_phi_prg_rejects_non_positive_dim_h(bad_dim):
|
||||
h = hashlib.sha256(b"x").digest()
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
phi_prg(h, dim_h=bad_dim)
|
||||
|
||||
|
||||
def test_phi_prg_rejects_non_bytes_hash():
|
||||
|
|
@ -189,18 +300,6 @@ def test_phi_prg_rejects_non_bytes_hash():
|
|||
phi_prg("not bytes", dim_h=8) # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_phi_prg_rejects_zero_dim_h():
|
||||
h = hashlib.sha256(b"x").digest()
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
phi_prg(h, dim_h=0)
|
||||
|
||||
|
||||
def test_phi_prg_rejects_negative_dim_h():
|
||||
h = hashlib.sha256(b"x").digest()
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
phi_prg(h, dim_h=-1)
|
||||
|
||||
|
||||
def test_phi_prg_rejects_non_int_dim_h():
|
||||
h = hashlib.sha256(b"x").digest()
|
||||
with pytest.raises(ValueError):
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue