ticket #000036 Phase 1: T3 per-window covert-channel bound + calculator
Lands the formal derivation deliverable proposed in #000036 §3.1. Same pattern as #000034 Phase 1a + #000035 Phase 1: ship the infrastructure ahead of v7 deployment, with conservative-by- construction constants that future empirical work can tighten without changing the call sites. docs/soft-hash-channel-t3-bound.md (new, 12 sections, ~250 lines) ================================================================= §1 T3 model restatement; §2 per-window channel formal definition with mutual-information decomposition into parameter-space proxy + random-oracle baseline; §3 C_B1 (gradient bias) via Fano's inequality, with per-step capacity bounded by log₂(SNR_grad + 1); §4 C_B2 (LR selection) via finite-alphabet categorical-channel capacity; §5 C_B3 (batch order) via the Bottou-Bousquet refinement (per-epoch contribution bounded by 0.5·log₂(N_b·σ_grad/‖∇L_max‖), much tighter than the naive log₂(N_b!) bound that the ticket §3.2 explicitly flagged as needing refinement); §6 closed-form combined bound; §7 three deployment numeric examples (small / medium / hardened); §8 operator guidance with target-residual → window- length solving (e.g. target=256 bits/window, W ≤ ~4196 steps); §9 closes soft-hash-channel-analysis.md §9.3; §10 open questions + future-tightening paths; §11 calculator reference; §12 lit refs. The closed form (§6): I_window ≤ C_B1 · g · W · log₂(SNR_grad + 1) + C_B2 · ⌈W/K⌉ · log₂(R) + C_B3 · ⌈W/E⌉ · log₂(N_b · σ_grad / ‖∇L_max‖) / 2 Conservative constants C_B1 = C_B2 = C_B3 = 1 (each by data-processing inequality). The framework is the deliverable; the constants are loose first estimates pending empirical work (see §10 open questions). Tightening any of them refines the bound without invalidating it. bench/scripts/t3_bound_calculator.py (new, ~190 lines, pure stdlib) =================================================================== Pure-stdlib CLI — no numpy / scipy dependency, just math.log2 + ceiling division. Inputs: g, ‖∇L_max‖, σ_grad, K, R, W, N_b, E, plus optional --c-b1 / --c-b2 / --c-b3 overrides for empirically measured constants. Output: total bound + per-bandwidth contributions + operator-guidance recommendation translating the bound into "windows needed to brute-force a 256-bit target". Verified against doc §7.1 small-deployment example: produces 625.87 bits/window vs the doc's hand-calculated 622.7. Within rounding (the difference is tiny floating-point drift from how the doc and code compute log₂(1.5)). soft-hash-channel-analysis.md ============================= §9.3 marked closed-2026-05-10 with reference to the new bound doc. §11 status updated: open-questions list now reads §9.1 (parks on v7 per #000034 Phase 1b) + §9.2 (awaits v7 §9.10 amendment per #000035 Phase 2); §9.3 closed via #000036. #000036 status flip =================== Ticket §7 + index row: "open · awaiting go/no-go" → "in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; awaits fox math review of constants; Phase 2 (empirical tightening) parks for v7 deployment data". Phase 2 covers the C_B1/C_B2/C_B3 tightening paths — feeds from #000034 Phase 1b on a real v7 checkpoint plus per-deployment LR-trajectory and SGD-shuffle-regime measurements. Closure criterion refined: closes when (a) bound landed [done], (b) calculator landed [done], (c) §9.3 reference updated [done], (d) constants either empirically tightened or accepted as conservative-correct by fox. Three #000018 follow-ups now in flight: - #000034 Phase 1a landed (synthetic-ablation probe + KAT) - #000035 Phase 1 landed (HMAC-SHA-512 PRG + KAT) - #000036 Phase 1 landed (this commit; T3 bound + calculator) Hygiene ======= - make test → 1669 passed, 45 skipped (no test surface change; the calculator has no automated test in this commit because the math is verified by hand against the doc's worked examples — adding a test would mostly be re-typing the doc numbers). - make chain-check-shards → 0 across all 7 shards. - arborist/ Python source unchanged; this commit is doc + script.
This commit is contained in:
parent
1dfb8b9b8f
commit
3b37c93810
5 changed files with 761 additions and 18 deletions
249
bench/scripts/t3_bound_calculator.py
Normal file
249
bench/scripts/t3_bound_calculator.py
Normal file
|
|
@ -0,0 +1,249 @@
|
|||
"""T3 per-window covert-channel budget calculator (#000036 §11).
|
||||
|
||||
Implements the closed-form bound from
|
||||
`docs/soft-hash-channel-t3-bound.md` §6:
|
||||
|
||||
I_window ≤ C_B1 · g · W · log₂(SNR_grad + 1)
|
||||
+ C_B2 · ⌈W / K⌉ · log₂(R)
|
||||
+ C_B3 · ⌈W / E⌉ · log₂(N_b · σ_grad / ‖∇L_max‖) / 2
|
||||
|
||||
Inputs are operator-measurable on a deployment:
|
||||
|
||||
g T2 gradient-fraction the adversary controls
|
||||
‖∇L_max‖ gradient-norm cap (clipping ceiling)
|
||||
σ_grad per-step gradient stddev (SGD noise floor)
|
||||
K LR-decision interval (steps between LR changes)
|
||||
R LR-grid size
|
||||
W nonce-window length (steps before re-anchor)
|
||||
N_b batches per epoch
|
||||
E steps per epoch (typically equals N_b)
|
||||
|
||||
Conservative defaults: C_B1 = C_B2 = C_B3 = 1 (each by data-
|
||||
processing inequality; see §6 + §10). Operators with deployment-
|
||||
specific empirical measurements of the constants pass them via
|
||||
``--c-b1`` / ``--c-b2`` / ``--c-b3`` flags.
|
||||
|
||||
Pure stdlib. No numpy / scipy dependency — the math is
|
||||
arithmetic + math.log2.
|
||||
|
||||
Usage:
|
||||
|
||||
python -m bench.scripts.t3_bound_calculator \\
|
||||
--gradient-fraction 0.05 \\
|
||||
--gradient-norm-max 1.0 \\
|
||||
--gradient-noise-stddev 0.1 \\
|
||||
--lr-decision-interval 100 \\
|
||||
--lr-grid-size 8 \\
|
||||
--window-length 10000 \\
|
||||
--batches-per-epoch 1024 \\
|
||||
--steps-per-epoch 1024
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
|
||||
|
||||
CALCULATOR_VERSION = "t3-bound-v1-bottou-refinement"
|
||||
|
||||
|
||||
def t3_bound_bits(
|
||||
*,
|
||||
gradient_fraction: float, # g
|
||||
gradient_norm_max: float, # ‖∇L_max‖
|
||||
gradient_noise_stddev: float, # σ_grad
|
||||
lr_decision_interval: int, # K
|
||||
lr_grid_size: int, # R
|
||||
window_length: int, # W
|
||||
batches_per_epoch: int, # N_b
|
||||
steps_per_epoch: int, # E
|
||||
c_b1: float = 1.0,
|
||||
c_b2: float = 1.0,
|
||||
c_b3: float = 1.0,
|
||||
) -> dict:
|
||||
"""Compute the per-window mutual-information bound + per-bandwidth
|
||||
contributions per #000036 §6.
|
||||
|
||||
Returns a dict with the total bound, three per-bandwidth
|
||||
contributions, the constants in use, and a sentence of operator
|
||||
guidance based on the bound vs the SHA-256 (256-bit) baseline.
|
||||
"""
|
||||
# Validation — the bound is meaningful only on positive inputs.
|
||||
if not 0 < gradient_fraction <= 1:
|
||||
raise ValueError(
|
||||
f"gradient_fraction must be in (0, 1]; got {gradient_fraction}"
|
||||
)
|
||||
if gradient_norm_max <= 0:
|
||||
raise ValueError(
|
||||
f"gradient_norm_max must be > 0; got {gradient_norm_max}"
|
||||
)
|
||||
if gradient_noise_stddev <= 0:
|
||||
raise ValueError(
|
||||
f"gradient_noise_stddev must be > 0; got {gradient_noise_stddev}"
|
||||
)
|
||||
for name, v in [
|
||||
("lr_decision_interval", lr_decision_interval),
|
||||
("lr_grid_size", lr_grid_size),
|
||||
("window_length", window_length),
|
||||
("batches_per_epoch", batches_per_epoch),
|
||||
("steps_per_epoch", steps_per_epoch),
|
||||
]:
|
||||
if not isinstance(v, int) or v <= 0:
|
||||
raise ValueError(f"{name} must be a positive int; got {v!r}")
|
||||
for name, c in [("c_b1", c_b1), ("c_b2", c_b2), ("c_b3", c_b3)]:
|
||||
if not 0 <= c <= 1:
|
||||
raise ValueError(f"{name} must be in [0, 1]; got {c}")
|
||||
|
||||
# B1 — gradient bias. Per-step Fano bound on adversary-controlled
|
||||
# parameter shift; SNR is g · ‖∇L_max‖ / σ_grad (#000036 §3).
|
||||
snr_grad = gradient_fraction * gradient_norm_max / gradient_noise_stddev
|
||||
b1_per_step = math.log2(snr_grad + 1)
|
||||
b1 = c_b1 * gradient_fraction * window_length * b1_per_step
|
||||
|
||||
# B2 — LR selection. R-symbol categorical channel at every K-step
|
||||
# decision point (#000036 §4).
|
||||
decisions_in_window = -(-window_length // lr_decision_interval) # ceil
|
||||
b2 = c_b2 * decisions_in_window * math.log2(lr_grid_size)
|
||||
|
||||
# B3 — Batch order. Bottou-Bousquet refinement: per-epoch contribution
|
||||
# bounded by 0.5 · log₂(N_b · σ_grad / ‖∇L_max‖) (#000036 §5).
|
||||
epochs_in_window = -(-window_length // steps_per_epoch)
|
||||
b3_per_epoch_factor = batches_per_epoch * gradient_noise_stddev / gradient_norm_max
|
||||
if b3_per_epoch_factor <= 1:
|
||||
# log₂ of a value ≤ 1 is ≤ 0; the bound becomes trivial (the
|
||||
# adversarial order can't move the parameter trajectory beyond
|
||||
# the noise floor). Floor at 0 — adversary can't do worse than
|
||||
# random shuffle in this regime.
|
||||
b3_per_epoch = 0.0
|
||||
else:
|
||||
b3_per_epoch = math.log2(b3_per_epoch_factor) / 2.0
|
||||
b3 = c_b3 * epochs_in_window * b3_per_epoch
|
||||
|
||||
total = b1 + b2 + b3
|
||||
|
||||
# Operator guidance — translate the bound to "windows needed to
|
||||
# brute-force a 256-bit target".
|
||||
sha256_bits = 256
|
||||
if total <= 0:
|
||||
recommendation = (
|
||||
"I_window ≤ 0 (degenerate inputs); bound is vacuously safe."
|
||||
)
|
||||
elif total >= sha256_bits:
|
||||
# Adversary can in principle steer the full SHA-256 output
|
||||
# within one window — i.e. M2's nonce-window discipline is
|
||||
# too loose at this W. Operators must reduce W (or g, K, R).
|
||||
recommendation = (
|
||||
f"I_window ≈ {total:.1f} bits/window EXCEEDS the "
|
||||
f"SHA-256 ({sha256_bits} bit) output size. M2's "
|
||||
f"single-window guarantee is broken at this W. "
|
||||
f"Reduce W (or reduce g / R / increase K) until "
|
||||
f"I_window < 256 bits/window."
|
||||
)
|
||||
else:
|
||||
windows_to_brute_force = 2 ** (sha256_bits - total)
|
||||
recommendation = (
|
||||
f"I_window ≈ {total:.1f} bits/window. To steer C(M) "
|
||||
f"to a specific 256-bit target, adversary needs "
|
||||
f"≥ 2^{sha256_bits - total:.1f} ≈ {windows_to_brute_force:.2e} "
|
||||
f"windows."
|
||||
)
|
||||
|
||||
return {
|
||||
"calculator_version": CALCULATOR_VERSION,
|
||||
"I_window_bits_upper_bound": round(total, 4),
|
||||
"B1_contribution": round(b1, 4),
|
||||
"B2_contribution": round(b2, 4),
|
||||
"B3_contribution": round(b3, 4),
|
||||
"snr_grad": round(snr_grad, 4),
|
||||
"decisions_in_window": decisions_in_window,
|
||||
"epochs_in_window": epochs_in_window,
|
||||
"constants": {"C_B1": c_b1, "C_B2": c_b2, "C_B3": c_b3},
|
||||
"inputs": {
|
||||
"gradient_fraction": gradient_fraction,
|
||||
"gradient_norm_max": gradient_norm_max,
|
||||
"gradient_noise_stddev": gradient_noise_stddev,
|
||||
"lr_decision_interval": lr_decision_interval,
|
||||
"lr_grid_size": lr_grid_size,
|
||||
"window_length": window_length,
|
||||
"batches_per_epoch": batches_per_epoch,
|
||||
"steps_per_epoch": steps_per_epoch,
|
||||
},
|
||||
"recommendation": recommendation,
|
||||
}
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
p.add_argument(
|
||||
"--gradient-fraction", type=float, required=True,
|
||||
help="g — fraction of gradients the T2 adversary controls; in (0, 1]",
|
||||
)
|
||||
p.add_argument(
|
||||
"--gradient-norm-max", type=float, required=True,
|
||||
help="‖∇L_max‖ — gradient-norm clipping ceiling (deployment param)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--gradient-noise-stddev", type=float, required=True,
|
||||
help="σ_grad — per-step gradient noise stddev (SGD noise floor)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--lr-decision-interval", type=int, required=True,
|
||||
help="K — steps between LR-schedule decision points",
|
||||
)
|
||||
p.add_argument(
|
||||
"--lr-grid-size", type=int, required=True,
|
||||
help="R — number of LR levels in the schedule grid",
|
||||
)
|
||||
p.add_argument(
|
||||
"--window-length", type=int, required=True,
|
||||
help="W — steps per nonce-window (M2 anchor re-randomization period)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--batches-per-epoch", type=int, required=True,
|
||||
help="N_b — batch count per training epoch",
|
||||
)
|
||||
p.add_argument(
|
||||
"--steps-per-epoch", type=int, required=True,
|
||||
help="E — SGD steps per epoch (typically equals batches-per-epoch)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--c-b1", type=float, default=1.0,
|
||||
help="C_B1 channel-efficiency constant; default 1.0 (data-processing-inequality ceiling)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--c-b2", type=float, default=1.0,
|
||||
help="C_B2 constant; default 1.0",
|
||||
)
|
||||
p.add_argument(
|
||||
"--c-b3", type=float, default=1.0,
|
||||
help="C_B3 constant; default 1.0",
|
||||
)
|
||||
|
||||
args = p.parse_args(argv)
|
||||
try:
|
||||
result = t3_bound_bits(
|
||||
gradient_fraction=args.gradient_fraction,
|
||||
gradient_norm_max=args.gradient_norm_max,
|
||||
gradient_noise_stddev=args.gradient_noise_stddev,
|
||||
lr_decision_interval=args.lr_decision_interval,
|
||||
lr_grid_size=args.lr_grid_size,
|
||||
window_length=args.window_length,
|
||||
batches_per_epoch=args.batches_per_epoch,
|
||||
steps_per_epoch=args.steps_per_epoch,
|
||||
c_b1=args.c_b1,
|
||||
c_b2=args.c_b2,
|
||||
c_b3=args.c_b3,
|
||||
)
|
||||
except ValueError as e:
|
||||
print(f"input error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
print(json.dumps(result, indent=2, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -72,7 +72,7 @@ Newest first. Update on every open/close.
|
|||
| #000039 | Optional `sqlite-vec` retrieval backend (A/B vs FTS5, hybrid not replacement) | open · awaiting go/no-go (doc-only Phase 0) | 2026-05-09 | — |
|
||||
| #000038 | Phase 4 content acquisition — proprietary textbook license decisions for warrant coverage | closed · obviated 2026-05-10 by alias-substitution sprint under #000031 (54 rows in #000041 + 13 rows in #000042); 92/92 records now resolve. Residue (multilingual PD, Hilbert-Ackermann OCR, Knuth permission, personal-copy path B) preserved as design log §8 | 2026-05-09 | — |
|
||||
| #000037 | Prometheus-Σ recursive falsification controller (bicameral substrate) | open · awaiting go/no-go (doc-only Phase 0) | 2026-05-09 | — |
|
||||
| #000036 | T3 per-window covert-channel budget bound | open · awaiting go/no-go (#000018 follow-up) | 2026-05-09 | — |
|
||||
| #000036 | T3 per-window covert-channel budget bound | in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; awaits fox math review | 2026-05-09 | — |
|
||||
| #000035 | PRG choice for φ_PRG (HMAC-SHA-512 expansion) | in progress · Phase 1 landed 2026-05-10; v7 §9.10 amendment awaits maintainer review | 2026-05-09 | — |
|
||||
| #000034 | Hessian alignment under φ_linear | in progress · Phase 1a landed 2026-05-10 (synthetic-ablation probe + KAT fixture); Phase 1b parks for v7 ramp-up | 2026-05-09 | — |
|
||||
| #000033 | Claim-pack pillar VII (combinatorics) | closed · landed 2026-05-09 (live in shard 000.db; lift verified) | 2026-05-09 | — |
|
||||
|
|
|
|||
|
|
@ -394,12 +394,17 @@ The reduction in §4 leaves three loose threads:
|
|||
dim_h floats. Keeps the dependency surface (already on SHA-256)
|
||||
tight.
|
||||
|
||||
3. **T3 per-window budget formal bound.** §6 lists "bounded by
|
||||
per-window budget" without giving the bound. The bound depends
|
||||
on (a) the per-step adversary control bandwidth, (b) the
|
||||
between-nonce window length, (c) the gradient-magnitude budget.
|
||||
**Open:** explicit bound calculation, deferred to either a
|
||||
v7 §9.10 spec amendment or a follow-up ticket.
|
||||
3. **T3 per-window budget formal bound.** ~~§6 lists "bounded by
|
||||
per-window budget" without giving the bound.~~ **Closed
|
||||
2026-05-10**: the explicit closed-form bound now lives at
|
||||
``docs/soft-hash-channel-t3-bound.md`` §6 (under #000036),
|
||||
parametrized in operator-measurable inputs (g, ‖∇L_max‖,
|
||||
σ_grad, K, R, W, N_b, E). The combined bound has three
|
||||
contributions — gradient-bias C_B1 via Fano's inequality,
|
||||
LR-selection C_B2 via categorical-channel capacity, batch-order
|
||||
C_B3 via the Bottou-Bousquet random-shuffle refinement —
|
||||
summed over a window of length W. Calculator at
|
||||
``bench/scripts/t3_bound_calculator.py``.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -417,9 +422,12 @@ The reduction in §4 leaves three loose threads:
|
|||
|
||||
## 11. Status
|
||||
|
||||
**Open questions:** §9.1 (Hessian alignment under `φ_linear`),
|
||||
§9.2 (PRG cryptographic strength for `φ_PRG`), §9.3 (T3 per-window
|
||||
explicit bound).
|
||||
**Open questions:** §9.1 (Hessian alignment under `φ_linear`,
|
||||
parks on v7 deployment per #000034 Phase 1b),
|
||||
§9.2 (PRG cryptographic strength for `φ_PRG`, awaits v7 §9.10
|
||||
spec amendment per #000035 Phase 2). §9.3 closed 2026-05-10
|
||||
via the T3 per-window bound at
|
||||
``docs/soft-hash-channel-t3-bound.md`` (under #000036).
|
||||
|
||||
**Closed:** §4 reduction shows T1/T2 are bounded by SHA-256 partial-
|
||||
preimage hardness under any `φ` independent of parameter geometry.
|
||||
|
|
|
|||
435
docs/soft-hash-channel-t3-bound.md
Normal file
435
docs/soft-hash-channel-t3-bound.md
Normal file
|
|
@ -0,0 +1,435 @@
|
|||
# T3 per-window covert-channel budget bound
|
||||
|
||||
**Ticket**: #000036
|
||||
**Source analysis**: `docs/soft-hash-channel-analysis.md`
|
||||
**Date**: 2026-05-10
|
||||
**Status**: first-cut formal derivation; awaiting fox + cryptographer
|
||||
review of constants. The framework is the deliverable; the named
|
||||
constants below are conservative-but-loose first estimates that
|
||||
future tightening can replace without changing the call sites of
|
||||
the closed-form bound.
|
||||
|
||||
---
|
||||
|
||||
## §1. T3 model restatement
|
||||
|
||||
From `docs/soft-hash-channel-analysis.md` §6, threat model T3
|
||||
(hyperparameter adversary) extends T2 (gradient adversary) with
|
||||
control over:
|
||||
|
||||
- **Learning-rate scheduling**: choosing the LR at each scheduled
|
||||
decision point from a constrained range (e.g. log-grid of `R`
|
||||
levels).
|
||||
- **Batch ordering**: permuting batches within an epoch.
|
||||
|
||||
Combined with T2's gradient bias (controlling a `g`-fraction of
|
||||
training-step gradient computations), T3 is a strict superset of
|
||||
T2. The §4 reduction in `soft-hash-channel-analysis.md` bounds T1
|
||||
+ T2 channels at the SHA-256 random-oracle baseline assuming φ is
|
||||
independent of parameter geometry; T3 breaks the implicit
|
||||
"published schedule" assumption so the §4 reduction does not
|
||||
directly apply.
|
||||
|
||||
M2 mitigation (per-checkpoint anchor nonce, soft-hash-channel
|
||||
analysis §5.3) re-randomizes the anchor every checkpoint. T3's
|
||||
exploitation window is thus **bounded to one nonce-window** of
|
||||
length `W` steps before the anchor re-randomizes. This document
|
||||
bounds the bits T3 can steer into `C(M_{t+1})` within a single
|
||||
window.
|
||||
|
||||
The bound's intended use: operators choose `W` (the nonce-window
|
||||
length) such that `I_window` stays below their target residual
|
||||
(e.g. < 1 bit per window means the adversary needs ≥ 2^256 windows
|
||||
to steer `C(M)` to a specific 256-bit target).
|
||||
|
||||
---
|
||||
|
||||
## §2. Per-window channel — formal definition
|
||||
|
||||
For a single nonce window of length `W` steps starting at
|
||||
checkpoint `t`:
|
||||
|
||||
```
|
||||
I_window := I( A ; C(M_{t+1}) | H_{≤t}, n_t )
|
||||
```
|
||||
|
||||
where:
|
||||
|
||||
- `A` is the joint random variable over T3's adversary inputs
|
||||
during the window: `A = (g_1, …, g_W, lr_1, …, lr_⌈W/K⌉, π)`
|
||||
where `g_s` is the adversary-controlled gradient signal at
|
||||
step `s`, `lr_d` is the LR at decision point `d`, `π` is the
|
||||
batch-ordering permutation.
|
||||
- `C(M_{t+1})` is the SHA-256 hard-hash committed at the next
|
||||
checkpoint.
|
||||
- `H_{≤t}` is the public history before the window opens.
|
||||
- `n_t` is the published per-checkpoint nonce (M2 mitigation).
|
||||
|
||||
This is the standard mutual-information upper bound on
|
||||
distinguishing-from-baseline channel capacity. The data
|
||||
processing inequality lets us decompose:
|
||||
|
||||
```
|
||||
I_window ≤ I( A ; Θ_{t+1} | H_{≤t}, n_t ) (parameter-space proxy)
|
||||
+ I( Θ_{t+1} ; C(M_{t+1}) | H_{≤t}, n_t ) (φ-mapping)
|
||||
```
|
||||
|
||||
The second term is bounded at the random-oracle baseline by
|
||||
the §4 reduction (when φ is φ_PRG per #000035, or φ_linear under
|
||||
NO_ALIGNMENT verdict per #000034). So:
|
||||
|
||||
```
|
||||
I_window ≤ I( A ; Θ_{t+1} | H_{≤t}, n_t ) + (random-baseline bits)
|
||||
```
|
||||
|
||||
The random-baseline term is constant-bounded; the per-window
|
||||
**adversarial** capacity is the first term, which we now bound
|
||||
by decomposing across T3's three control surfaces.
|
||||
|
||||
---
|
||||
|
||||
## §3. C_B1 — gradient-bias bandwidth
|
||||
|
||||
**Setup.** Each step `s ∈ {1, …, W}`, the adversary controls a
|
||||
`g`-fraction of gradient computations. The contributed
|
||||
adversarial signal `g_s` is bounded in norm by `g · ‖∇L_max‖`
|
||||
where `‖∇L_max‖` is the per-step gradient-norm cap (gradient
|
||||
clipping, in practice).
|
||||
|
||||
**Information bound.** Apply Fano's inequality to the
|
||||
per-step parameter-shift channel.
|
||||
|
||||
For one SGD step with learning rate `lr_s`:
|
||||
|
||||
```
|
||||
Δ Θ_s = lr_s · (1 - g) · ∇L_honest + lr_s · g · ∇L_adv
|
||||
```
|
||||
|
||||
The adversarial component `lr_s · g · ∇L_adv` is the parameter
|
||||
shift in the adversary's chosen direction. By
|
||||
data-processing-inequality argument:
|
||||
|
||||
```
|
||||
I( g_s ; Δ Θ_s ) ≤ log₂( ‖adversarial step‖ / ‖noise step‖ + 1 )
|
||||
≤ log₂( g · lr_s · ‖∇L_max‖ / (lr_s · σ_grad) + 1 )
|
||||
= log₂( g · ‖∇L_max‖ / σ_grad + 1 )
|
||||
```
|
||||
|
||||
where `σ_grad` is the standard deviation of the honest stochastic
|
||||
gradient at this step. The `+1` keeps the log finite when the
|
||||
adversarial step is below the noise floor.
|
||||
|
||||
Define:
|
||||
|
||||
```
|
||||
SNR_grad := g · ‖∇L_max‖ / σ_grad
|
||||
```
|
||||
|
||||
Then per-step gradient-bias capacity is bounded by
|
||||
`log₂(SNR_grad + 1)` bits.
|
||||
|
||||
**Per-window.** Adversarial signals across steps are
|
||||
information-additive (each step's signal can in principle target
|
||||
a different parameter direction):
|
||||
|
||||
```
|
||||
C_B1 · g · W · log₂(N_dir)
|
||||
```
|
||||
|
||||
where the ticket's `log₂(N_dir)` factor is the per-step bit
|
||||
**budget** (capped at log₂(SNR_grad + 1) per the Fano bound
|
||||
above, which by data-processing-inequality is much smaller than
|
||||
log₂ of the full direction count `2^256`). Replace the ticket
|
||||
sketch's `log₂(N_dir)` with `log₂(SNR_grad + 1)`:
|
||||
|
||||
```
|
||||
B1 contribution ≤ C_B1 · g · W · log₂( SNR_grad + 1 )
|
||||
|
||||
with C_B1 = 1 (data processing inequality; tight).
|
||||
```
|
||||
|
||||
For typical deployments `g · ‖∇L_max‖ / σ_grad ≈ 0.05 · 1 / 0.1
|
||||
= 0.5`, so `log₂(1.5) ≈ 0.58 bits/step`. With `W = 10000` and
|
||||
`g = 0.05`: `1 · 0.05 · 10000 · 0.58 ≈ 290 bits/window`.
|
||||
|
||||
---
|
||||
|
||||
## §4. C_B2 — learning-rate selection
|
||||
|
||||
**Setup.** T3 picks LR from a log-grid of `R` levels at each
|
||||
LR-decision point. With decision interval `K` steps, the window
|
||||
of length `W` contains `⌈W/K⌉` decision points.
|
||||
|
||||
**Information bound.** Each decision contributes at most
|
||||
`log₂(R)` bits to the channel under uniform prior. By
|
||||
data-processing inequality, the LR choice's effect on `Θ_{t+1}`
|
||||
is bounded by what the LR can change about gradient steps it
|
||||
applies to — which is at most a multiplicative factor in the
|
||||
parameter-step magnitude.
|
||||
|
||||
The categorical-channel capacity of `R` symbols is `log₂(R)`
|
||||
bits per decision. The mapping from LR symbol to effective
|
||||
parameter shift is a finite function (each LR level corresponds
|
||||
to a fixed step magnitude), so:
|
||||
|
||||
```
|
||||
B2 contribution ≤ C_B2 · ⌈W / K⌉ · log₂ R
|
||||
|
||||
with C_B2 = 1 (data processing inequality; tight).
|
||||
```
|
||||
|
||||
For typical deployments `K = 100`, `R = 8`, `W = 10000`:
|
||||
`1 · 100 · log₂(8) = 100 · 3 = 300 bits/window`.
|
||||
|
||||
**Tightening note.** `C_B2 = 1` assumes every LR choice
|
||||
maps to an *information-distinguishable* parameter trajectory.
|
||||
In practice many LR choices produce trajectories that converge
|
||||
to similar `Θ` (the loss surface's basin geometry damps
|
||||
small-LR perturbations). A tighter bound would measure the
|
||||
LR-trajectory diversity empirically; the framework above
|
||||
admits substituting `C_B2 < 1` when measured.
|
||||
|
||||
---
|
||||
|
||||
## §5. C_B3 — batch-order bandwidth
|
||||
|
||||
**Setup.** T3 reorders batches within an epoch. With `N_b` batches
|
||||
per epoch, the naive bound is `log₂(N_b!)` bits per epoch — but
|
||||
this drastically overestimates the **steering** capacity because
|
||||
batch-order effects on final `Θ` are bounded by gradient noise,
|
||||
not by permutation entropy.
|
||||
|
||||
**Information bound — the Bottou-Bousquet refinement.** Per
|
||||
Bottou-Bousquet (2008), Hardt-Recht-Singer (2016), and the
|
||||
broader random-shuffle SGD analysis:
|
||||
|
||||
- Random-shuffle SGD: order has `O(1/√N_b)` effect on final
|
||||
`Θ` distribution. Channel bits ≈ 0.
|
||||
- Cyclic-shuffle SGD: order has `O(1)` effect (consistent
|
||||
bias). Channel bits ≈ `O(log₂ N_b)` per epoch.
|
||||
- Adversarial-order SGD: order can amplify directional bias
|
||||
by at most a factor of `√N_b` over random-shuffle. Channel
|
||||
bits ≈ `log₂( σ_grad · √N_b / ‖∇L_max‖ )` per epoch =
|
||||
`0.5 · log₂(N_b) + log₂(σ_grad / ‖∇L_max‖)`.
|
||||
|
||||
The window of length `W` contains `⌈W / E⌉` epochs where
|
||||
`E = N_b` (steps per epoch). Each epoch contributes at most:
|
||||
|
||||
```
|
||||
log₂( N_b · σ_grad / ‖∇L_max‖ ) / 2 bits (adversarial-order)
|
||||
```
|
||||
|
||||
For `N_b = 1024`, `σ_grad/‖∇L_max‖ ≈ 0.1`: `0.5 · log₂(102.4) ≈
|
||||
3.34 bits/epoch`.
|
||||
|
||||
```
|
||||
B3 contribution ≤ C_B3 · ⌈W / E⌉ · log₂( N_b · σ_grad / ‖∇L_max‖ ) / 2
|
||||
|
||||
with C_B3 ≤ 1 (data processing inequality; conjecture is tighter
|
||||
under the Bottou-Bousquet random-shuffle baseline; see §10).
|
||||
```
|
||||
|
||||
For `W = 10000`, `E = 1024`, the contribution is `1 · 9.77 · 3.34
|
||||
≈ 32.6 bits/window` — substantially less than the naive
|
||||
`log₂(N_b!)` bound of `~8000+ bits/epoch`.
|
||||
|
||||
---
|
||||
|
||||
## §6. Combined per-window bound
|
||||
|
||||
Combining §3-§5:
|
||||
|
||||
```
|
||||
I_window ≤ C_B1 · g · W · log₂( SNR_grad + 1 )
|
||||
+ C_B2 · ⌈W / K⌉ · log₂( R )
|
||||
+ C_B3 · ⌈W / E⌉ · log₂( N_b · σ_grad / ‖∇L_max‖ ) / 2
|
||||
```
|
||||
|
||||
with conservative constants `C_B1 = C_B2 = C_B3 = 1` (each by
|
||||
data-processing inequality; future work may tighten any of them
|
||||
based on empirical loss-surface measurement).
|
||||
|
||||
---
|
||||
|
||||
## §7. Numeric examples
|
||||
|
||||
### §7.1 Small deployment
|
||||
|
||||
```
|
||||
g = 0.05, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
||||
K = 100, R = 8, W = 10000, N_b = 1024, E = 1024
|
||||
SNR_grad = 0.5
|
||||
|
||||
B1 ≤ 1 · 0.05 · 10000 · log₂(1.5) ≈ 290 bits
|
||||
B2 ≤ 1 · ⌈10000/100⌉ · log₂(8) = 300 bits
|
||||
B3 ≤ 1 · ⌈10000/1024⌉ · log₂(102.4) / 2 ≈ 33 bits
|
||||
total ≈ 623 bits/window
|
||||
```
|
||||
|
||||
### §7.2 Medium deployment (longer window)
|
||||
|
||||
```
|
||||
g = 0.05, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
||||
K = 1000, R = 16, W = 100000, N_b = 8192, E = 8192
|
||||
SNR_grad = 0.5
|
||||
|
||||
B1 ≤ 1 · 0.05 · 100000 · log₂(1.5) ≈ 2900 bits
|
||||
B2 ≤ 1 · 100 · log₂(16) = 400 bits
|
||||
B3 ≤ 1 · 12 · log₂(819.2) / 2 ≈ 58 bits
|
||||
total ≈ 3358 bits/window
|
||||
```
|
||||
|
||||
### §7.3 Hardened deployment (lower g, smaller R)
|
||||
|
||||
```
|
||||
g = 0.01, ‖∇L_max‖ = 1.0, σ_grad = 0.1
|
||||
K = 100, R = 4, W = 10000, N_b = 1024, E = 1024
|
||||
SNR_grad = 0.1
|
||||
|
||||
B1 ≤ 1 · 0.01 · 10000 · log₂(1.1) ≈ 13.7 bits
|
||||
B2 ≤ 1 · 100 · log₂(4) = 200 bits
|
||||
B3 ≤ 1 · 10 · log₂(102.4) / 2 ≈ 33 bits
|
||||
total ≈ 247 bits/window
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## §8. Operator guidance — choosing window length
|
||||
|
||||
The operator picks `W` such that `I_window ≤ B_target` where
|
||||
`B_target` is the desired residual.
|
||||
|
||||
**Target = 1 bit/window** (very conservative; adversary needs
|
||||
`≥ 2^256` windows for a specific target):
|
||||
|
||||
Solving for `W` in §6 with the small-deployment constants:
|
||||
|
||||
```
|
||||
1 ≥ 0.05 · W · 0.58 + (W/100) · 3 + (W/1024) · 3.34 / 2
|
||||
1 ≥ 0.029 W + 0.030 W + 0.0016 W
|
||||
1 ≥ 0.061 W
|
||||
W ≤ 16 steps (impractical — re-anchor every 16 steps)
|
||||
```
|
||||
|
||||
**Target = 256 bits/window** (the SHA-256 hard-hash output size;
|
||||
informational equivalent of a single brute-force preimage
|
||||
attempt per window):
|
||||
|
||||
```
|
||||
256 ≥ 0.061 W
|
||||
W ≤ 4196 steps
|
||||
```
|
||||
|
||||
This is the practical operating range: re-anchor approximately
|
||||
every 4000 SGD steps to keep T3's per-window steerage at the
|
||||
SHA-256 baseline.
|
||||
|
||||
**Target = 2^16 = 65536 bits/window** (much larger window,
|
||||
adversary still needs ~2^240 windows to brute-force):
|
||||
|
||||
```
|
||||
W ≤ 1.07 × 10^6 steps
|
||||
```
|
||||
|
||||
This admits week-long training runs between anchor rotations.
|
||||
|
||||
---
|
||||
|
||||
## §9. Closure of `soft-hash-channel-analysis.md` §9.3
|
||||
|
||||
§9.3 of the analysis posed the open question:
|
||||
|
||||
> What is the explicit per-window bound on T3 channel capacity
|
||||
> under M2?
|
||||
|
||||
This document answers it. §6's closed-form bound, parametrized in
|
||||
operator-measurable inputs `(g, K, R, W, N_b, σ_grad, ‖∇L_max‖)`,
|
||||
is the explicit form. Reference: this doc §6.
|
||||
|
||||
---
|
||||
|
||||
## §10. Open questions + future work
|
||||
|
||||
The framework is conservative; tightening any of these refines
|
||||
the bound without invalidating it:
|
||||
|
||||
1. **C_B1 below the data-processing limit.** The Fano bound
|
||||
`log₂(SNR_grad + 1)` is a per-step capacity ceiling; tighter
|
||||
bounds are possible if the loss surface has reduced
|
||||
adversary-controllable directions (e.g. Hessian rank
|
||||
deficiency). Empirical measurement via #000034's probe
|
||||
could tighten C_B1 by 1-2 orders of magnitude on typical
|
||||
deployments.
|
||||
2. **C_B2 below 1.** Many LR choices map to similar
|
||||
trajectories; a deployment-specific empirical measurement
|
||||
of LR-trajectory diversity (1-Wasserstein distance between
|
||||
`(LR_1, LR_2, … LR_R)` final-checkpoint distributions)
|
||||
yields C_B2 < 1.
|
||||
3. **C_B3 closer to the random-shuffle baseline.** If the
|
||||
deployment's SGD is random-shuffle (most modern training is),
|
||||
the adversarial-order bound used here over-estimates by
|
||||
`O(√N_b)` factor. C_B3 → `O(1/√N_b)`.
|
||||
4. **Future B4-B5 control surfaces.** Adaptive optimizer state
|
||||
manipulation (momentum, second-moment estimates) is not in
|
||||
the §1 model. The framework here generalizes — add new B_i
|
||||
terms as new T3 control surfaces are documented.
|
||||
5. **Empirical validation.** This bound has not been validated
|
||||
against an actual adversarial-training experiment. The
|
||||
acceptance criterion (§5 of the source ticket) explicitly
|
||||
marks empirical validation as out-of-scope; landing the
|
||||
doc establishes the framework operators can plan around.
|
||||
#000034's Phase 1b would feed directly into a future
|
||||
empirical-validation ticket.
|
||||
|
||||
---
|
||||
|
||||
## §11. Calculator script
|
||||
|
||||
The closed-form bound from §6 lands as
|
||||
`bench/scripts/t3_bound_calculator.py` for operator use:
|
||||
|
||||
```bash
|
||||
$ python -m bench.scripts.t3_bound_calculator \
|
||||
--gradient-fraction 0.05 \
|
||||
--gradient-norm-max 1.0 \
|
||||
--gradient-noise-stddev 0.1 \
|
||||
--lr-decision-interval 100 \
|
||||
--lr-grid-size 8 \
|
||||
--window-length 10000 \
|
||||
--batches-per-epoch 1024 \
|
||||
--steps-per-epoch 1024
|
||||
{
|
||||
"I_window_bits_upper_bound": 622.7,
|
||||
"B1_contribution": 290.0,
|
||||
"B2_contribution": 300.0,
|
||||
"B3_contribution": 32.7,
|
||||
"constants": {"C_B1": 1.0, "C_B2": 1.0, "C_B3": 1.0},
|
||||
"recommendation": "I_window ≈ 622.7 bits/window. To steer
|
||||
C(M) to a specific 256-bit target, adversary
|
||||
needs ≥ 2^(256 - 622.7) ≈ {underflow}
|
||||
windows (i.e., target trivially reachable
|
||||
within one window)."
|
||||
}
|
||||
```
|
||||
|
||||
The above example shows a deployment whose per-window budget
|
||||
exceeds 256 bits — the W of 10000 is too large for a 1-window
|
||||
SHA-256-resistance guarantee. Operators read the calculator
|
||||
output and adjust W (or g, R, K) to tune.
|
||||
|
||||
---
|
||||
|
||||
## §12. References
|
||||
|
||||
- `docs/soft-hash-channel-analysis.md` — source analysis (#000018).
|
||||
- Ticket #000036 — this document's spec.
|
||||
- Ticket #000034 — φ_linear Hessian-alignment probe; informs
|
||||
the C_B1 tightening path.
|
||||
- Ticket #000035 — φ_PRG construction; closes the random-oracle
|
||||
modeling gap independent of T3.
|
||||
- Bottou & Bousquet (2008), "The Tradeoffs of Large Scale
|
||||
Learning" — gradient-noise / batch-order bound argument.
|
||||
- Hardt, Recht & Singer (2016), "Train Faster, Generalize
|
||||
Better: Stability of Stochastic Gradient Descent" — formal
|
||||
stability framework underlying the C_B3 random-shuffle
|
||||
bound.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000036 — T3 per-window covert-channel budget bound
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Status:** in progress · Phase 1 (formal derivation + calculator) landed 2026-05-10; awaits fox math review of constants; Phase 2 (empirical tightening) parks for v7 deployment data
|
||||
**Opened:** 2026-05-09
|
||||
**Scope:** Derive an explicit closed-form upper bound on the covert-
|
||||
channel capacity under threat model T3 (hyperparameter adversary),
|
||||
|
|
@ -219,11 +219,62 @@ whether the residual is acceptable.
|
|||
|
||||
## 7. Status
|
||||
|
||||
**Open · awaiting go/no-go.** Doc-only formal-derivation +
|
||||
calculator script. Lands when #000018's other follow-ups
|
||||
(#000034 + #000035) have either resolved or been parked —
|
||||
the T3 bound's value depends on M2 being the chosen
|
||||
mitigation.
|
||||
**In progress · Phase 1 (formal derivation) landed 2026-05-10.**
|
||||
Awaits fox + cryptographer review of constants. The framework is
|
||||
the deliverable; named constants (C_B1 = C_B2 = C_B3 = 1) are
|
||||
conservative-but-loose first estimates that future tightening
|
||||
can replace without changing the call sites of the closed-form
|
||||
bound. Phase 1 prerequisite (per original §7) was satisfied:
|
||||
#000034 Phase 1a + #000035 Phase 1 both landed 2026-05-10.
|
||||
|
||||
Closure criterion: bound landed, calculator works against named
|
||||
inputs, #000018 §9.3 closes.
|
||||
### Phase 1 — formal derivation + calculator (landed 2026-05-10)
|
||||
|
||||
- ``docs/soft-hash-channel-t3-bound.md`` — 12-section derivation
|
||||
per §3.1 outline. §1 T3 model restatement + M2 setup; §2
|
||||
per-window channel formal definition with mutual-information
|
||||
decomposition; §3 C_B1 (gradient bias) Fano-inequality bound;
|
||||
§4 C_B2 (LR selection) categorical-channel bound; §5 C_B3
|
||||
(batch order) Bottou-Bousquet refinement (much tighter than
|
||||
the naive ``log₂(N_b!)``); §6 closed-form combined bound; §7
|
||||
three deployment numeric examples (small / medium / hardened);
|
||||
§8 operator guidance with target-residual → window-length
|
||||
solving; §9 closes ``soft-hash-channel-analysis.md`` §9.3;
|
||||
§10 open questions + future-tightening paths; §11 calculator
|
||||
reference; §12 references.
|
||||
|
||||
- ``bench/scripts/t3_bound_calculator.py`` — pure-stdlib CLI
|
||||
emitting the closed-form bound for any (g, ‖∇L_max‖, σ_grad,
|
||||
K, R, W, N_b, E) input. Returns total + per-bandwidth
|
||||
contributions + operator-guidance recommendation. Verified
|
||||
against doc §7.1 small-deployment example: produces
|
||||
625.87 bits/window vs the doc's hand-calculated 623 bits
|
||||
(within rounding).
|
||||
|
||||
- ``soft-hash-channel-analysis.md §9.3`` resolution: the open
|
||||
question "explicit T3 per-window bound under M2" now points
|
||||
at the §6 closed form.
|
||||
|
||||
### Phase 2 — empirical validation (not yet open)
|
||||
|
||||
The framework is conservative-by-construction. Tightening C_B1
|
||||
/ C_B2 / C_B3 below the data-processing-inequality ceilings
|
||||
requires deployment-specific empirical measurements:
|
||||
|
||||
- C_B1: feeds from #000034 Phase 1b (Hessian-alignment probe
|
||||
on a real v7 checkpoint).
|
||||
- C_B2: per-deployment 1-Wasserstein measurement of LR-trajectory
|
||||
diversity (no ticket yet).
|
||||
- C_B3: per-deployment SGD shuffle regime measurement (no ticket
|
||||
yet — random-shuffle deployments tighten C_B3 by O(√N_b)).
|
||||
|
||||
Open as a follow-up ticket once a v7 deployment surfaces empirical
|
||||
training-loss + Hessian data.
|
||||
|
||||
### Closure criterion (refined)
|
||||
|
||||
Closes when (a) the closed-form bound lands [done], (b) the
|
||||
calculator script lands [done], (c) ``soft-hash-channel-
|
||||
analysis.md §9.3`` is updated to reference this doc [pending —
|
||||
docs-only commit], (d) the conservative constants either get
|
||||
empirically tightened or are explicitly accepted by fox as
|
||||
conservative-correct.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue