From da62f8047c3864312b43b3df83acd2cfae6d7340 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Mon, 11 May 2026 06:39:27 -0400 Subject: [PATCH] ticket #000036 Tier-1: apply dav1d 2026-05-11 review polish (no math change) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit dav1d's review (RESPONSE_1 + RESPONSE_2) returned 2026-05-11. This lands the Tier-1 items — everything that doesn't change numeric outputs or invalidate the KAT discipline. The Tier-2 B1 conservative- envelope (v2 calculator) is a separate decision and stays a closure blocker. Calculator (bench/scripts/t3_bound_calculator.py): - Recommendation wording: "M2's single-window guarantee is broken" → "this conservative bound CANNOT CERTIFY M2's residual". An upper bound exceeding 256 bits means we cannot certify, NOT that the adversary can steer 256 bits — the prior wording overclaimed. - New structured output fields: b1_model ("effective_control_v1"), certification_status ∈ {CERTIFIED_BY_BOUND, NOT_CERTIFIED_BY_BOUND}, certification_threshold_bits (256), model_assumptions[]. Callers read a machine-readable status, not just prose. - Input validation hardening: _require_finite_float / _require_positive_int helpers reject bools (isinstance(True, int) is True in Python — a real leak risk for a security calculator) and NaN / ±inf for every numeric input and constant. - gradient_fraction = 0 now accepted (no T2 surface; B1 = 0; T3's LR + batch-order channels still contribute) — improves component isolation. CLI help + module docstring updated accordingly. - Numeric outputs UNCHANGED: baseline still 625.8716 / 292.4813 / 300.0 / 33.3904; b1_model stays effective_control_v1; KAT discipline intact. Tests (tests/test_t3_bound_calculator.py, 53 → 75): - Hard-coded cwd="/home/fox/git/arborist" → pathlib.Path(__file__). resolve().parents[1] so the suite runs on any checkout. - New: test_gradient_fraction_zero_accepted, test_bool_rejected_for_int_fields, test_bool_rejected_for_float_fields, test_nonfinite_numbers_rejected, test_output_carries_b1_model_and_certification_fields, test_certification_status_certified_below_threshold. - test_recommendation_exceeds_sha256 now also asserts "CANNOT CERTIFY" + certification_status == NOT_CERTIFIED_BY_BOUND. Doc (docs/soft-hash-channel-t3-bound.md): - §0 reworked into a reviewer brief recording dav1d's findings (§2 accepted, §4 accepted, §5 accepted as model-bound, §3 = closure blocker, wording/validation = applied). - New §3.1: the B1-double-g issue spelled out — effective_control_v1 vs fraction_channels vs aggregate_bias vs max_envelope, with the baseline-spread table (292 / 1730 / 5850 / 5850 bits); v2 path described. - §5: "B3 is a model-bound, not a directly-quoted theorem" note. - §10: items 1-2 are now the closure blockers (B1 envelope v2; active KAT fixture); items 3-7 are tightening paths (#000043). New §10.1 records what the 2026-05-11 hardening pass already landed. - §11: calculator-output example updated to show the new fields + corrected recommendation wording. - §12: references add the dav1d review + clarify Bottou-Bousquet "inspires" (not "underlies") the §5 model-bound. Status (#000036 ticket + TICKETS.md row): review-returned + Tier-1- applied; closure blockers = B1 v2 envelope (awaits fox go/no-go) + active KAT fixture. R2's architectural integrations (Merkle audit- event commitment, SQD canonicalization, CTI clause-lattice, 5F trigger, ForkScore security-risk) noted as out-of-scope (separate tickets if wanted). AUTOCOUNT markers in docs/calculator-test-patterns.md + docs/warrant-substrate-cookbook.md bumped 53 → 75. Full suite: 2264 passed, 28 skipped. --- bench/scripts/t3_bound_calculator.py | 134 ++++++--- docs/TICKETS.md | 2 +- docs/calculator-test-patterns.md | 4 +- docs/soft-hash-channel-t3-bound.md | 265 ++++++++++++++---- .../ticket-000036-t3-per-window-bound.md | 2 +- docs/warrant-substrate-cookbook.md | 4 +- tests/test_t3_bound_calculator.py | 100 ++++++- 7 files changed, 406 insertions(+), 105 deletions(-) diff --git a/bench/scripts/t3_bound_calculator.py b/bench/scripts/t3_bound_calculator.py index db8cd75..ee1f4fb 100644 --- a/bench/scripts/t3_bound_calculator.py +++ b/bench/scripts/t3_bound_calculator.py @@ -23,6 +23,16 @@ processing inequality; see §6 + §10). Operators with deployment- specific empirical measurements of the constants pass them via ``--c-b1`` / ``--c-b2`` / ``--c-b3`` flags. +**B1 model.** This v1 calculator uses ``b1_model="effective_control_v1"``: +``g`` is read as an *effective gradient-control coefficient* that +simultaneously bounds the fraction of steerable directions AND the +amplitude shrinkage of the aggregate adversarial gradient. Under that +reading the B1 formula ``C_B1 · g · W · log₂(1 + g·G/σ)`` is sound, +but it is **not the most-conservative gradient adversary** — see +#000036 §3 + §10 for the alternative ``b1_model="max_envelope"`` +path (deferred to a v2 bump). For now ``certification_status`` is +relative to this v1 effective-control model. + Pure stdlib. No numpy / scipy dependency — the math is arithmetic + math.log2. @@ -48,6 +58,33 @@ import sys CALCULATOR_VERSION = "t3-bound-v1-bottou-refinement" +B1_MODEL = "effective_control_v1" +CERTIFICATION_THRESHOLD_BITS = 256 + + +def _require_finite_float(name: str, value: object) -> float: + """Reject bools, non-numbers, NaN, and infinities; return float. + + Python's ``isinstance(True, int)`` is True, so a bare ``isinstance`` + check lets ``True`` / ``False`` leak into a numeric calculation — + unacceptable for a security calculator. Reject bools explicitly. + """ + if isinstance(value, bool) or not isinstance(value, (int, float)): + raise ValueError(f"{name} must be a finite number; got {value!r}") + fv = float(value) + if not math.isfinite(fv): + raise ValueError(f"{name} must be finite (no NaN / inf); got {value!r}") + return fv + + +def _require_positive_int(name: str, value: object) -> int: + """Reject bools and non-ints; require value > 0. + + ``type(value) is int`` (not ``isinstance``) rejects ``True`` / ``False``. + """ + if type(value) is not int or value <= 0: + raise ValueError(f"{name} must be a positive int; got {value!r}") + return value def t3_bound_bits( @@ -71,34 +108,45 @@ def t3_bound_bits( contributions, the constants in use, and a sentence of operator guidance based on the bound vs the SHA-256 (256-bit) baseline. """ - # Validation — the bound is meaningful only on positive inputs. - if not 0 < gradient_fraction <= 1: + # Validation — reject bools / NaN / inf; coerce numeric types. + gradient_fraction = _require_finite_float("gradient_fraction", gradient_fraction) + if not 0 <= gradient_fraction <= 1: + # 0 is allowed: a deployment with no T2 gradient surface still + # has T3's LR + batch-order channels (B2, B3), so B1=0 is a + # meaningful component-isolation case. raise ValueError( - f"gradient_fraction must be in (0, 1]; got {gradient_fraction}" + f"gradient_fraction must be in [0, 1]; got {gradient_fraction}" ) + gradient_norm_max = _require_finite_float("gradient_norm_max", gradient_norm_max) if gradient_norm_max <= 0: - raise ValueError( - f"gradient_norm_max must be > 0; got {gradient_norm_max}" - ) + raise ValueError(f"gradient_norm_max must be > 0; got {gradient_norm_max}") + gradient_noise_stddev = _require_finite_float( + "gradient_noise_stddev", gradient_noise_stddev + ) if gradient_noise_stddev <= 0: raise ValueError( f"gradient_noise_stddev must be > 0; got {gradient_noise_stddev}" ) - for name, v in [ - ("lr_decision_interval", lr_decision_interval), - ("lr_grid_size", lr_grid_size), - ("window_length", window_length), - ("batches_per_epoch", batches_per_epoch), - ("steps_per_epoch", steps_per_epoch), - ]: - if not isinstance(v, int) or v <= 0: - raise ValueError(f"{name} must be a positive int; got {v!r}") + lr_decision_interval = _require_positive_int( + "lr_decision_interval", lr_decision_interval + ) + lr_grid_size = _require_positive_int("lr_grid_size", lr_grid_size) + window_length = _require_positive_int("window_length", window_length) + batches_per_epoch = _require_positive_int("batches_per_epoch", batches_per_epoch) + steps_per_epoch = _require_positive_int("steps_per_epoch", steps_per_epoch) + c_b1 = _require_finite_float("c_b1", c_b1) + c_b2 = _require_finite_float("c_b2", c_b2) + c_b3 = _require_finite_float("c_b3", c_b3) for name, c in [("c_b1", c_b1), ("c_b2", c_b2), ("c_b3", c_b3)]: if not 0 <= c <= 1: raise ValueError(f"{name} must be in [0, 1]; got {c}") - # B1 — gradient bias. Per-step Fano bound on adversary-controlled - # parameter shift; SNR is g · ‖∇L_max‖ / σ_grad (#000036 §3). + # B1 — gradient bias. Per-step discrete channel-capacity bound on + # the adversary-controlled parameter shift (NOT Fano's inequality — + # see #000036 §3): the per-step shift falls in one of ~SNR_grad+1 + # distinguishable buckets, capacity ≤ log₂(SNR_grad+1). Under the + # b1_model="effective_control_v1" reading, g bounds both the + # steerable-direction fraction and the aggregate amplitude shrinkage. snr_grad = gradient_fraction * gradient_norm_max / gradient_noise_stddev b1_per_step = math.log2(snr_grad + 1) b1 = c_b1 * gradient_fraction * window_length * b1_per_step @@ -124,35 +172,45 @@ def t3_bound_bits( total = b1 + b2 + b3 - # Operator guidance — translate the bound to "windows needed to - # brute-force a 256-bit target". - sha256_bits = 256 + # Operator guidance — I_window is an UPPER BOUND. If it exceeds the + # 256-bit threshold we do NOT know the adversary can steer 256 bits; + # we know only that this bound is too loose to certify safety. The + # wording reflects that (per dav1d review 2026-05-11). + sha256_bits = CERTIFICATION_THRESHOLD_BITS if total <= 0: + certification_status = "CERTIFIED_BY_BOUND" recommendation = ( - "I_window ≤ 0 (degenerate inputs); bound is vacuously safe." + "I_window upper bound ≤ 0 (degenerate inputs); the bound is " + "vacuously below the 256-bit certification threshold." ) elif total >= sha256_bits: - # Adversary can in principle steer the full SHA-256 output - # within one window — i.e. M2's nonce-window discipline is - # too loose at this W. Operators must reduce W (or g, K, R). + certification_status = "NOT_CERTIFIED_BY_BOUND" recommendation = ( - f"I_window ≈ {total:.1f} bits/window EXCEEDS the " - f"SHA-256 ({sha256_bits} bit) output size. M2's " - f"single-window guarantee is broken at this W. " - f"Reduce W (or reduce g / R / increase K) until " - f"I_window < 256 bits/window." + f"I_window upper bound ≈ {total:.1f} bits/window EXCEEDS the " + f"SHA-256 ({sha256_bits} bit) certification threshold. This " + f"conservative bound CANNOT CERTIFY M2's single-window " + f"residual at this W — it does not prove the adversary can " + f"steer {sha256_bits} bits, only that the bound is too loose " + f"to certify safety. Reduce W (or reduce g / R / increase K, " + f"or tighten C_B* empirically) until the certified bound is " + f"< {sha256_bits} bits/window." ) else: + certification_status = "CERTIFIED_BY_BOUND" windows_to_brute_force = 2 ** (sha256_bits - total) recommendation = ( - f"I_window ≈ {total:.1f} bits/window. To steer C(M) " - f"to a specific 256-bit target, adversary needs " - f"≥ 2^{sha256_bits - total:.1f} ≈ {windows_to_brute_force:.2e} " - f"windows." + f"I_window upper bound ≈ {total:.1f} bits/window. Certified " + f"residual ≥ {sha256_bits - total:.1f} bits per independent " + f"nonce window under this leakage model; a {sha256_bits}-bit " + f"specific-target search remains bounded below by ≈ " + f"2^{sha256_bits - total:.1f} ≈ {windows_to_brute_force:.2e} " + f"independent windows (assumes fresh nonce per window, no " + f"structural advantage beyond the bounded channel)." ) return { "calculator_version": CALCULATOR_VERSION, + "b1_model": B1_MODEL, "I_window_bits_upper_bound": round(total, 4), "B1_contribution": round(b1, 4), "B2_contribution": round(b2, 4), @@ -161,6 +219,14 @@ def t3_bound_bits( "decisions_in_window": decisions_in_window, "epochs_in_window": epochs_in_window, "constants": {"C_B1": c_b1, "C_B2": c_b2, "C_B3": c_b3}, + "certification_status": certification_status, + "certification_threshold_bits": sha256_bits, + "model_assumptions": [ + "M2_nonce_per_window", + "SHA256_random_oracle_baseline", + "B1_effective_control_model_v1", + "B3_gradient_noise_refinement", + ], "inputs": { "gradient_fraction": gradient_fraction, "gradient_norm_max": gradient_norm_max, @@ -179,7 +245,7 @@ def main(argv: list[str] | None = None) -> int: p = argparse.ArgumentParser(description=__doc__.splitlines()[0]) p.add_argument( "--gradient-fraction", type=float, required=True, - help="g — fraction of gradients the T2 adversary controls; in (0, 1]", + help="g — effective gradient-control coefficient (b1_model=effective_control_v1); in [0, 1], 0 = no T2 surface", ) p.add_argument( "--gradient-norm-max", type=float, required=True, diff --git a/docs/TICKETS.md b/docs/TICKETS.md index ff76994..eaf4fba 100644 --- a/docs/TICKETS.md +++ b/docs/TICKETS.md @@ -101,7 +101,7 @@ Newest first. Update on every open/close. | #000039 | Optional `sqlite-vec` retrieval backend (A/B vs FTS5, hybrid not replacement) | open · awaiting go/no-go (doc-only Phase 0) | 2026-05-09 | — | | #000038 | Phase 4 content acquisition — proprietary textbook license decisions for warrant coverage | closed · obviated 2026-05-10 by alias-substitution sprint under #000031 (74 rows in #000041 + 13 rows in #000042); 92/92 records now resolve. Residue (multilingual PD, Hilbert-Ackermann OCR, Knuth permission, personal-copy path B) preserved as design log §8 | 2026-05-09 | — | | #000037 | Prometheus-Σ recursive falsification controller (bicameral substrate) | in progress · Phases 0 + 1 + 1.b + 1.c + 2 landed 2026-05-10; **§12 Trigger 2 fired** (divergence variance 0.575 / N=37); §22 Findings 2 + 3 RESOLVED (kernel/llm cost split + sweep_weights §15.4 + per-mode τ_qa); `controller_events` carries 4 event kinds (decision · difficulty · budget_allocation · falsification_proposal) feeding `arborist controller-events` inspector + live-harvest third bucket in `bench/scripts/harvest_falsification_proposals.py`; Phase 3 sleep-sweep scheduler tracked under #000045 (gating ticket) | 2026-05-09 | — | -| #000036 | T3 per-window covert-channel budget bound | in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; pre-review polish `8916bf3`; **math review in flight with dav1d** (forwarded 2026-05-10 — Tier 2 bundle) | 2026-05-09 | — | +| #000036 | T3 per-window covert-channel budget bound | in progress · Phase 1 + dav1d review returned 2026-05-11; Tier-1 polish applied (wording → "CANNOT CERTIFY", structured `certification_status`/`b1_model` fields, bool/NaN/inf validation, `g=0` accepted, test cwd fix; 53 → 75 tests); **closure blockers: B1 conservative-envelope v2 calculator (awaits fox go/no-go) + active KAT fixture** | 2026-05-09 | — | | #000035 | PRG choice for φ_PRG (HMAC-SHA-512 expansion) | in progress · Phase 1 landed 2026-05-10; v7 §9.10 amendment awaits maintainer review | 2026-05-09 | — | | #000034 | Hessian alignment under φ_linear | in progress · Phase 1a landed 2026-05-10 (synthetic-ablation probe + KAT fixture); Phase 1b parks for v7 ramp-up | 2026-05-09 | — | | #000033 | Claim-pack pillar VII (combinatorics) | closed · landed 2026-05-09 (live in shard 000.db; lift verified) | 2026-05-09 | — | diff --git a/docs/calculator-test-patterns.md b/docs/calculator-test-patterns.md index 3c49252..6e79907 100644 --- a/docs/calculator-test-patterns.md +++ b/docs/calculator-test-patterns.md @@ -13,7 +13,7 @@ bench across three modules: closed-form bound for #000036 The exemplar test file is ``tests/test_t3_bound_calculator.py`` -(53 cases as of 2026-05-10; fox shipped 51 in the +(75 cases as of 2026-05-10; fox shipped 51 in the initial cut and the +2 KAT-fixture-gap closure landed in ``581ad90``). The other two test files (``tests/test_anchor_prg.py`` and @@ -378,7 +378,7 @@ shape exposes the corresponding surface. Many calculator modules Exemplar test files: - ``tests/test_t3_bound_calculator.py`` — - 53 tests covering + 75 tests covering items 1-9 for the T3 closed-form bound. - ``tests/test_pi_star_arithmetic.py`` — 56 tests covering items diff --git a/docs/soft-hash-channel-t3-bound.md b/docs/soft-hash-channel-t3-bound.md index a772614..0ef5312 100644 --- a/docs/soft-hash-channel-t3-bound.md +++ b/docs/soft-hash-channel-t3-bound.md @@ -2,16 +2,20 @@ **Ticket**: #000036 **Source analysis**: `docs/soft-hash-channel-analysis.md` -**Date**: 2026-05-10 -**Status**: first-cut formal derivation; awaiting fox + cryptographer -review of constants. The framework is the deliverable; the named -constants below are conservative-but-loose first estimates that -future tightening can replace without changing the call sites of -the closed-form bound. +**Date**: 2026-05-10 (review pass 2026-05-11) +**Status**: formal derivation + calculator landed; **dav1d review +returned 2026-05-11**. Tier-1 polish applied (recommendation +wording, structured `certification_status` output, validation +hardening, B1-model labelling — see §3.1 + §10.1). **Closure +blockers remaining**: B1 conservative-envelope (v2 calculator, +§10 item 1) and active KAT fixture (§10 item 2). The framework is +the deliverable; the named constants are conservative starting +estimates that empirical tightening (#000043) can replace without +changing call sites. --- -## §0. What we're asking the reviewer to confirm +## §0. Reviewer brief (dav1d review returned 2026-05-11) This document derives an upper bound on the per-window mutual information a T3 (hyperparameter) adversary can steer into the @@ -20,36 +24,36 @@ channel across T3's three control surfaces (gradient bias, LR selection, batch order) and combines them into a closed-form bound consumed by `bench/scripts/t3_bound_calculator.py`. -Three specific things to check: +**dav1d's 2026-05-11 review** (`RESPONSE_1` + `RESPONSE_2`) +landed these findings: -1. **§2 decomposition.** Is the Markov-chain DPI step (`A → - Θ_{t+1} → C(M_{t+1})`) correctly applied, and is the - T1 + T2 baseline inherited from `soft-hash-channel-analysis.md` - §4 cleanly separated from the T3 capacity bound this doc - adds? -2. **§§ 3-5 derivations.** Is each per-surface bound (`C_B1` - gradient-bias, `C_B2` LR selection, `C_B3` batch order) - derived with a sound information-theoretic argument? §3 uses - discrete channel-capacity counting on the per-step - parameter-shift; §4 uses categorical-channel capacity on the - LR grid; §5 uses the Bottou-Bousquet adversarial-order - refinement. -3. **Conservative-constant choice.** Are `C_B1 = C_B2 = C_B3 = 1` - genuinely upper-bounding (never optimistic)? Where would you - tighten? Empirical tightening paths are catalogued in §10 - and tracked under #000043. +1. **§2 decomposition** — accepted. Markov-chain DPI on + `A → Θ_{t+1} → C(M_{t+1})` correctly applied; T1+T2 baseline + cleanly separated from the T3 capacity term. +2. **§4 (C_B2 LR selection)** — accepted. Clean categorical-channel + capacity bound. +3. **§5 (C_B3 batch order)** — accepted *as a model-bound, not a + theorem* (§5's "model-bound" note reflects this). +4. **§3 (C_B1 gradient bias)** — **closure blocker**. `g` appears + twice in the per-window expression, which is only sound under + the `effective_control_v1` reading (§3.1). A worst-case + gradient adversary needs the `max_envelope` model — the v2 + calculator path (§10 item 1). +5. Wording / validation / output-schema fixes — applied (§10.1). -What's *not* being asked: empirical validation against an actual -adversarial-training run (out of scope per §10 item 5; opens as a -future ticket once v7 plastic-training has a deployment target). +**What's *not* being asked**: empirical validation against an +actual adversarial-training run (out of scope per §10 item 7; +opens as a future ticket once v7 plastic-training has a deployment +target). The headline numeric example in §7 + §11 yields -`I_window ≈ 626 bits/window` for a default deployment config — -exceeding SHA-256's 256-bit output, which is the calculator's -"reduce `W`" trigger condition. The framework's job is to make -that trigger derivable from operator-measurable inputs; the -constants' job is to be honest worst-case starting points -operators can refine per-deployment. +`I_window ≈ 626 bits/window` for a default deployment config +**under the v1 effective-control B1 model** — exceeding SHA-256's +256-bit threshold, so `certification_status = NOT_CERTIFIED_BY_BOUND`. +A conservative `max_envelope` B1 model would produce ~5850 bits. +The framework's job is to make the trigger derivable from +operator-measurable inputs; the v2 path makes the bound +worst-case-conservative rather than effective-control-relative. --- @@ -205,10 +209,10 @@ C_B1 · g · W · log₂(N_dir) ``` where the ticket's `log₂(N_dir)` factor is the per-step bit -**budget** (capped at log₂(SNR_grad + 1) per the Fano bound -above, which by data-processing-inequality is much smaller than -log₂ of the full direction count `2^256`). Replace the ticket -sketch's `log₂(N_dir)` with `log₂(SNR_grad + 1)`: +**budget** (capped at log₂(SNR_grad + 1) per the discrete +channel-capacity bound above, which by data-processing-inequality +is much smaller than log₂ of the full direction count `2^256`). +Replace the ticket sketch's `log₂(N_dir)` with `log₂(SNR_grad + 1)`: ``` B1 contribution ≤ C_B1 · g · W · log₂( SNR_grad + 1 ) @@ -220,6 +224,59 @@ For typical deployments `g · ‖∇L_max‖ / σ_grad = 0.05 · 1 / 0.1 = 0.5`, so `log₂(1.5) ≈ 0.5850 bits/step`. With `W = 10000` and `g = 0.05`: `1 · 0.05 · 10000 · 0.5850 ≈ 292.48 bits/window`. +### §3.1 The B1 model — `g` appears twice (dav1d review 2026-05-11) + +Note that `g` enters the per-window B1 expression **twice**: once +as the outer multiplier `g · W` (number of adversary-controlled +steps) and once inside `log₂(SNR_grad + 1)` where +`SNR_grad = g · ‖∇L_max‖ / σ_grad`. That double use is only sound +under a specific reading: + +``` +b1_model = "effective_control_v1": + g is an effective gradient-control coefficient that + simultaneously bounds (a) the fraction of steerable + directions per step AND (b) the amplitude shrinkage of + the aggregate adversarial gradient signal. +``` + +If instead `g` means **only** "fraction of gradient computations +controlled" (with full per-channel amplitude), the conservative +shape is larger. dav1d's review spelled out the spread on the +baseline (`g=0.05, ‖∇L_max‖=1, σ_grad=0.1, W=10000`): + +| B1 model | Formula | Baseline B1 | +|---|---|---| +| `effective_control_v1` (current) | `g · W · log₂(1 + g·G/σ)` | **292.5 bits** | +| `fraction_channels` (g = channel fraction only) | `g · W · log₂(1 + G/σ)` | 1 729.7 bits | +| `aggregate_bias` (g = amplitude shrinkage only) | `W · log₂(1 + g·G/σ)` | 5 849.6 bits | +| `max_envelope` (conservative) | `max(fraction_channels, aggregate_bias)` | 5 849.6 bits | + +The current v1 calculator uses `effective_control_v1`. The +`certification_status` it emits is therefore **relative to that +model** — it is an operational risk score, not a worst-case upper +bound across all gradient-adversary interpretations. + +**The conservative `max_envelope` model is the v2 path.** When the +team wants the calculator to certify against the worst-case +gradient adversary, it should: + +``` +1. add a b1_model parameter (default "max_envelope" for v2) +2. bump CALCULATOR_VERSION → "t3-bound-v2-conservative-envelope" +3. emit a new KAT fixture under + bench/fixtures/t3-bound/v2-conservative-envelope/ +4. keep effective_control_v1 reachable as a backward-compat mode + so v1 rows stay replayable +``` + +That v2 work is **not in this ticket** — see §10. Until it lands, +read the calculator's output as: "under the v1 effective-control +model, the baseline evaluates to ~626 bits/window; this exceeds +the 256-bit certification threshold so the calculator CANNOT +CERTIFY M2 residual safety at W=10000; a conservative B1 envelope +would produce a larger bound (~5850 bits in the example above)." + --- ## §4. C_B2 — learning-rate selection @@ -309,6 +366,20 @@ For `W = 10000`, `E = 1024`, the contribution is `1 · 9.77 · 3.34 ≈ 32.6 bits/window` — substantially less than the naive `log₂(N_b!)` bound of `~8000+ bits/epoch`. +**B3 is a model-bound, not a directly-quoted theorem (dav1d +review 2026-05-11).** The expression above is a *gradient-noise- +scale conservative modelling term* inspired by Bottou-Bousquet / +Hardt-Recht-Singer shuffle-stability arguments. It is **not** a +statement that "the Bottou-Bousquet theorem proves every +batch-order adversary is bounded by this exact expression." The +calculator's `test_b3_exact_formula` pins the implementation to +the model; it does not prove the model. The floor at 0 (when +`N_b · σ_grad / ‖∇L_max‖ ≤ 1`) does not mean batch order has +literally zero effect — it means this upper-bound formula gives +no positive distinguishable capacity above the noise floor in +that regime. Empirical tightening (random-shuffle classification) +remains Phase 2 / #000043. + --- ## §6. Combined per-window bound @@ -430,31 +501,55 @@ is the explicit form. Reference: this doc §6. ## §10. Open questions + future work -The framework is conservative; tightening any of these refines -the bound without invalidating it: +Items 1-2 below are the **closure blockers** flagged by dav1d's +2026-05-11 review; items 3-6 are tightening paths that refine +the bound without invalidating it. -1. **C_B1 below the data-processing limit.** The discrete +1. **B1 conservative envelope (v2 calculator) — closure blocker.** + The current `b1_model="effective_control_v1"` (§3.1) reads `g` + as a combined direction-fraction + amplitude-shrinkage + coefficient, which makes the headline B1 ~292 bits on the + baseline. A worst-case gradient adversary needs the + `max_envelope` model (~5850 bits on the same baseline). Until + the v2 calculator with `b1_model` selection lands — + `CALCULATOR_VERSION → t3-bound-v2-conservative-envelope`, new + KAT fixture under `bench/fixtures/t3-bound/v2-conservative-envelope/`, + `effective_control_v1` kept as a backward-compat mode — the + calculator's `certification_status` is relative to the v1 + model only and should be read as an operational risk score, + not a certified worst-case bound. **Do not close #000036 + until this is resolved** (either by landing v2 or by an + explicit team decision to keep v1 with the narrowed-semantics + caveat documented in §3.1). +2. **Active KAT fixture for closure.** `bench/fixtures/t3-bound/known-answer-tests.jsonl` + must exist and `test_t3_bound_known_answer_tests` must run + (not skip) with ≥ 5 entries before #000036 closes. If the v2 + B1 model lands first, the KAT fixture goes under the + versioned subdirectory. +3. **C_B1 below the data-processing limit.** The discrete channel-capacity bound `log₂(SNR_grad + 1)` is a per-step ceiling derived under uniform-prior signal levels; tighter bounds are possible if the loss surface has reduced adversary-controllable directions (e.g. Hessian rank deficiency). Empirical measurement via #000034's probe could tighten C_B1 by 1-2 orders of magnitude on typical - deployments. -2. **C_B2 below 1.** Many LR choices map to similar + deployments. (Tracked under #000043.) +4. **C_B2 below 1.** Many LR choices map to similar trajectories; a deployment-specific empirical measurement of LR-trajectory diversity (1-Wasserstein distance between `(LR_1, LR_2, … LR_R)` final-checkpoint distributions) - yields C_B2 < 1. -3. **C_B3 closer to the random-shuffle baseline.** If the + yields C_B2 < 1. (Tracked under #000043.) +5. **C_B3 closer to the random-shuffle baseline.** If the deployment's SGD is random-shuffle (most modern training is), the adversarial-order bound used here over-estimates by - `O(√N_b)` factor. C_B3 → `O(1/√N_b)`. -4. **Future B4-B5 control surfaces.** Adaptive optimizer state + `O(√N_b)` factor. C_B3 → `O(1/√N_b)`. (Tracked under #000043; + the cheapest of the three constant-tightening paths — needs + only a DataLoader-config audit, no checkpoint.) +6. **Future B4-B5 control surfaces.** Adaptive optimizer state manipulation (momentum, second-moment estimates) is not in the §1 model. The framework here generalizes — add new B_i terms as new T3 control surfaces are documented. -5. **Empirical validation.** This bound has not been validated +7. **Empirical validation.** This bound has not been validated against an actual adversarial-training experiment. The acceptance criterion (§5 of the source ticket) explicitly marks empirical validation as out-of-scope; landing the @@ -462,6 +557,32 @@ the bound without invalidating it: #000034's Phase 1b would feed directly into a future empirical-validation ticket. +### §10.1 Calculator hardening landed 2026-05-11 (dav1d review) + +Not "open" — already done in the `8916bf3` follow-up + the +2026-05-11 review pass: + +- Recommendation wording: "M2's single-window guarantee is + broken" → "this conservative bound CANNOT CERTIFY M2's + residual" (an upper bound exceeding 256 means we cannot + certify, not that the adversary can steer 256 bits). +- Structured output fields: `b1_model`, `certification_status` + ∈ {`CERTIFIED_BY_BOUND`, `NOT_CERTIFIED_BY_BOUND`}, + `certification_threshold_bits`, `model_assumptions[]` — so + callers read a machine-readable status, not just prose. +- Input validation: bools rejected for both int and float fields + (`isinstance(True, int)` is True in Python — a real leak risk + for a security calculator); NaN / ±inf rejected for every + numeric input and constant. +- `gradient_fraction = 0` now accepted (no T2 surface; B1 = 0; + T3's LR + batch-order channels still contribute) — improves + component isolation. +- Tests: hard-coded `cwd="/home/fox/git/arborist"` replaced with + `pathlib.Path(__file__).resolve().parents[1]` so the suite + runs on any checkout. New bool/NaN/inf rejection tests, + `g=0` acceptance test, `certification_status` field tests. + Suite count 53 → 75. + --- ## §11. Calculator script @@ -480,6 +601,8 @@ $ python -m bench.scripts.t3_bound_calculator \ --batches-per-epoch 1024 \ --steps-per-epoch 1024 { + "calculator_version": "t3-bound-v1-bottou-refinement", + "b1_model": "effective_control_v1", "I_window_bits_upper_bound": 625.8716, "B1_contribution": 292.4813, "B2_contribution": 300.0, @@ -488,18 +611,35 @@ $ python -m bench.scripts.t3_bound_calculator \ "decisions_in_window": 100, "epochs_in_window": 10, "constants": {"C_B1": 1.0, "C_B2": 1.0, "C_B3": 1.0}, - "recommendation": "I_window ≈ 625.9 bits/window EXCEEDS the - SHA-256 (256 bit) output size. M2's - single-window guarantee is broken at this W. - Reduce W (or reduce g / R / increase K) until - I_window < 256 bits/window." + "certification_status": "NOT_CERTIFIED_BY_BOUND", + "certification_threshold_bits": 256, + "model_assumptions": [ + "M2_nonce_per_window", + "SHA256_random_oracle_baseline", + "B1_effective_control_model_v1", + "B3_gradient_noise_refinement" + ], + "inputs": { "...echoed input tuple..." }, + "recommendation": "I_window upper bound ≈ 625.9 bits/window + EXCEEDS the SHA-256 (256 bit) certification + threshold. This conservative bound CANNOT + CERTIFY M2's single-window residual at this W + — it does not prove the adversary can steer + 256 bits, only that the bound is too loose to + certify safety. Reduce W (or reduce g / R / + increase K, or tighten C_B* empirically) until + the certified bound is < 256 bits/window." } ``` -The above example shows a deployment whose per-window budget -exceeds 256 bits — the W of 10000 is too large for a 1-window -SHA-256-resistance guarantee. Operators read the calculator -output and adjust W (or g, R, K) to tune. +The above example shows a deployment whose per-window bound +exceeds 256 bits **under the `effective_control_v1` B1 model** — +`certification_status` is `NOT_CERTIFIED_BY_BOUND` and the W of +10000 is too large for a 1-window SHA-256-resistance certification. +A conservative `max_envelope` B1 model (§3.1) would produce a +larger bound (~5850 bits in the worked example). Operators read +the structured `certification_status` field, not just the prose, +and adjust W (or g, R, K) to tune. --- @@ -512,8 +652,13 @@ output and adjust W (or g, R, K) to tune. - Ticket #000035 — φ_PRG construction; closes the random-oracle modeling gap independent of T3. - Bottou & Bousquet (2008), "The Tradeoffs of Large Scale - Learning" — gradient-noise / batch-order bound argument. + Learning" — gradient-noise / batch-order argument that + **inspires** the §5 C_B3 model-bound (not a direct quotation; + see §5's "model-bound, not theorem" note). - Hardt, Recht & Singer (2016), "Train Faster, Generalize - Better: Stability of Stochastic Gradient Descent" — formal - stability framework underlying the C_B3 random-shuffle - bound. + Better: Stability of Stochastic Gradient Descent" — shuffle- + stability framework the §5 C_B3 random-shuffle reasoning draws on. +- dav1d review 2026-05-11 (`RESPONSE_1` + `RESPONSE_2`) — the + B1-double-`g` finding (§3.1), the recommendation-wording + correction, validation hardening, and the v2 conservative- + envelope path (§10 item 1). diff --git a/docs/tickets/ticket-000036-t3-per-window-bound.md b/docs/tickets/ticket-000036-t3-per-window-bound.md index c7ef7f1..2e7066f 100644 --- a/docs/tickets/ticket-000036-t3-per-window-bound.md +++ b/docs/tickets/ticket-000036-t3-per-window-bound.md @@ -1,6 +1,6 @@ # Ticket #000036 — T3 per-window covert-channel budget bound -**Status:** in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; pre-review polish pass committed 2026-05-10 (`8916bf3`); **math review in flight with dav1d** (forwarded 2026-05-10 Asia/Kuala_Lumpur — Tier 2 bundle: `t3-bound.md` + `soft-hash-analysis.md` + `t3_bound_calculator.py` + `test_t3_bound_calculator.py` + this ticket); empirical tightening tracked separately under #000043 +**Status:** in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; **dav1d review returned 2026-05-11** (`RESPONSE_1` + `RESPONSE_2`); **Tier-1 polish applied 2026-05-11** (recommendation wording → "CANNOT CERTIFY", structured `certification_status` / `b1_model` / `model_assumptions` output fields, bool/NaN/inf validation hardening, `g=0` accepted, hard-coded test cwd removed; test suite 53 → 75); **closure blockers remaining**: (1) B1 conservative-envelope = v2 calculator (`b1_model=max_envelope`, `CALCULATOR_VERSION → t3-bound-v2-conservative-envelope`, versioned KAT fixture) — awaits fox go/no-go on Tier 2; (2) active KAT fixture (`bench/fixtures/t3-bound/known-answer-tests.jsonl` must exist + test must run not skip, ≥ 5 entries). Empirical constant-tightening tracked separately under #000043. R2's architectural integrations (Merkle audit-event commitment, SQD canonicalization, CTI clause-lattice, 5F trigger, ForkScore security-risk) are out-of-scope here — separate tickets if wanted. **Opened:** 2026-05-09 **Scope:** Derive an explicit closed-form upper bound on the covert- channel capacity under threat model T3 (hyperparameter adversary), diff --git a/docs/warrant-substrate-cookbook.md b/docs/warrant-substrate-cookbook.md index 5356f1a..fe79bc6 100644 --- a/docs/warrant-substrate-cookbook.md +++ b/docs/warrant-substrate-cookbook.md @@ -600,7 +600,7 @@ than waiting for bench-time STRICT-rate drift to surface it. W concentration + dim_h, closure (a_top + a_bot ≡ full-spectrum on dense decomposition), Lanczos eigenvalue ordering invariant. -- `tests/test_t3_bound_calculator.py` — **53 tests** (was 51; +- `tests/test_t3_bound_calculator.py` — **75 tests** (was 51; +2 from `581ad90` 2026-05-10 KAT-fixture-gap closure) for the T3 per-window covert-channel bound calculator (#000036 §11); pins the closed-form B1/B2/B3 formulas, monotonicity in each @@ -674,7 +674,7 @@ than waiting for bench-time STRICT-rate drift to surface it. | aliases.py | 512 | 469 (28 tests) | 0.92 | | warrant_resolver.py | ~800 | ~430 (combined) | 0.54 | | warrant_chain.py | 89 | 320 (9 tests) | 3.6 | -| t3_bound_calculator.py | 249 | 446 (53 tests) | 1.79 | +| t3_bound_calculator.py | 249 | 446 (75 tests) | 1.79 | | fork_score.py | 386 | 609 (23 tests) | 1.58 | | weights.py | 73 | 180 (16 tests) | 2.5 | | pi_star/protocol+registry | 124 | 280 (21 tests) | 2.3 | diff --git a/tests/test_t3_bound_calculator.py b/tests/test_t3_bound_calculator.py index 0d2ea22..addc3d2 100644 --- a/tests/test_t3_bound_calculator.py +++ b/tests/test_t3_bound_calculator.py @@ -30,6 +30,11 @@ from bench.scripts.t3_bound_calculator import ( ) +# Repo root for CLI subprocess tests — derived, not hard-coded, so the +# suite runs on any checkout (dav1d review 2026-05-11). +REPO_ROOT = pathlib.Path(__file__).resolve().parents[1] + + # --- baseline (§11 worked example) ----------------------------------- @@ -158,13 +163,72 @@ def test_constants_recorded(): # --- input validation ------------------------------------------------ -@pytest.mark.parametrize("g", [0.0, -0.1, 1.5, 2.0]) +@pytest.mark.parametrize("g", [-0.1, 1.5, 2.0]) def test_invalid_gradient_fraction_rejected(g): args = {**_BASELINE, "gradient_fraction": g} with pytest.raises(ValueError, match="gradient_fraction"): t3_bound_bits(**args) +def test_gradient_fraction_zero_accepted(): + """g = 0 means no T2 gradient surface — B1 = 0, but T3's LR + + batch-order channels (B2, B3) still fire. Component-isolation case + (dav1d review 2026-05-11: allow 0 ≤ g ≤ 1).""" + r = t3_bound_bits(**{**_BASELINE, "gradient_fraction": 0.0}) + assert r["B1_contribution"] == pytest.approx(0.0, abs=1e-9) + assert r["snr_grad"] == pytest.approx(0.0, abs=1e-9) + assert r["B2_contribution"] == pytest.approx(300.0, abs=1e-3) + assert r["I_window_bits_upper_bound"] == pytest.approx( + r["B2_contribution"] + r["B3_contribution"], abs=1e-3 + ) + + +@pytest.mark.parametrize("name,bad", [ + ("window_length", True), + ("window_length", False), + ("lr_grid_size", True), + ("batches_per_epoch", False), + ("lr_decision_interval", True), + ("steps_per_epoch", True), +]) +def test_bool_rejected_for_int_fields(name, bad): + """Python's isinstance(True, int) is True — a bare isinstance check + lets bools leak into a security calculation. Reject explicitly via + type(value) is int (dav1d review 2026-05-11).""" + with pytest.raises(ValueError, match=name): + t3_bound_bits(**{**_BASELINE, name: bad}) + + +@pytest.mark.parametrize("name,bad", [ + ("gradient_fraction", True), + ("gradient_norm_max", False), + ("gradient_noise_stddev", True), + ("c_b1", True), + ("c_b2", False), +]) +def test_bool_rejected_for_float_fields(name, bad): + with pytest.raises(ValueError, match=name): + t3_bound_bits(**{**_BASELINE, name: bad}) + + +@pytest.mark.parametrize("name,bad", [ + ("gradient_fraction", float("nan")), + ("gradient_fraction", float("inf")), + ("gradient_norm_max", float("nan")), + ("gradient_norm_max", float("inf")), + ("gradient_noise_stddev", float("inf")), + ("gradient_noise_stddev", float("nan")), + ("c_b1", float("nan")), + ("c_b2", float("inf")), + ("c_b3", float("-inf")), +]) +def test_nonfinite_numbers_rejected(name, bad): + """NaN / ±inf in any numeric field → ValueError, not a poisoned + bound that emits inf in the recommendation (dav1d review 2026-05-11).""" + with pytest.raises(ValueError, match=name): + t3_bound_bits(**{**_BASELINE, name: bad}) + + @pytest.mark.parametrize("name,value", [ ("gradient_norm_max", 0.0), ("gradient_norm_max", -1.0), @@ -237,12 +301,38 @@ def test_recommendation_below_sha256(): def test_recommendation_exceeds_sha256(): - """Baseline (W=10000) → 622 bits > 256 — recommendation should - flag M2's single-window guarantee as broken.""" + """Baseline (W=10000) → ~626 bits > 256 — recommendation flags that + the conservative bound CANNOT CERTIFY the residual (not 'guarantee + broken'; dav1d review 2026-05-11: I_window is an upper bound, so + exceeding 256 means we cannot certify, not that the adversary can + steer 256 bits).""" r = t3_bound_bits(**_BASELINE) assert r["I_window_bits_upper_bound"] > 256 assert "EXCEEDS" in r["recommendation"] + assert "CANNOT CERTIFY" in r["recommendation"] assert "256" in r["recommendation"] + assert r["certification_status"] == "NOT_CERTIFIED_BY_BOUND" + + +def test_output_carries_b1_model_and_certification_fields(): + """Structured machine-readable fields added per dav1d review + 2026-05-11: b1_model, certification_status, + certification_threshold_bits, model_assumptions.""" + r = t3_bound_bits(**_BASELINE) + assert r["b1_model"] == "effective_control_v1" + assert r["certification_threshold_bits"] == 256 + assert r["certification_status"] in { + "CERTIFIED_BY_BOUND", "NOT_CERTIFIED_BY_BOUND" + } + assert "B1_effective_control_model_v1" in r["model_assumptions"] + assert r["certification_status"] == "NOT_CERTIFIED_BY_BOUND" # baseline > 256 + + +def test_certification_status_certified_below_threshold(): + """A config landing in (0, 256) → CERTIFIED_BY_BOUND.""" + r = t3_bound_bits(**{**_BASELINE, "window_length": 1000}) + assert 0 < r["I_window_bits_upper_bound"] < 256 + assert r["certification_status"] == "CERTIFIED_BY_BOUND" # --- CLI surface (subprocess invocation) ----------------------------- @@ -264,7 +354,7 @@ def test_cli_baseline_runs_clean(tmp_path): "--steps-per-epoch", "1024", ] out = subprocess.run(cmd, capture_output=True, text=True, check=True, - cwd="/home/fox/git/arborist") + cwd=REPO_ROOT) j = json.loads(out.stdout) assert j["calculator_version"] == CALCULATOR_VERSION # Closed-form actual; doc §11's 622.7 is approximate. @@ -285,7 +375,7 @@ def test_cli_invalid_input_exits_2(): "--steps-per-epoch", "1024", ] out = subprocess.run(cmd, capture_output=True, text=True, - cwd="/home/fox/git/arborist") + cwd=REPO_ROOT) assert out.returncode == 2 assert "gradient_fraction" in out.stderr