ticket #000036 Tier-2: dav1d Option B (conservative B1 envelope) applied in v1
Per fox: apply the conservative max_envelope B1 model by changing the
v1 calculator's default — NOT by forking a v2. CALCULATOR_VERSION stays
"t3-bound-v1-bottou-refinement" (the descriptor names the unchanged B3
term); b1_model is echoed in the output AND the inputs dict so KAT
replays are unambiguous about which model produced a row.
Calculator (bench/scripts/t3_bound_calculator.py):
- New b1_model kwarg + --b1-model CLI flag, choices:
max_envelope (default) max(fraction_channels, aggregate_bias)
fraction_channels g · W · log₂(1 + G/σ)
aggregate_bias W · log₂(1 + g·G/σ)
effective_control_v1 g · W · log₂(1 + g·G/σ) (old non-worst-case)
- Default is now max_envelope — genuinely upper-bounding across both
interpretations of g (dav1d review §3 closure blocker, RESOLVED).
- Every output reports all three concrete B1 variants
(B1_fraction_channels / B1_aggregate_bias / B1_effective_control_v1),
b1_selected, and both SNR readings (snr_grad = g·G/σ,
snr_per_channel = G/σ) regardless of which b1_model was requested.
- model_assumptions[] now carries f"B1_model_{b1_model}".
- inputs echo now includes c_b1/c_b2/c_b3/b1_model (replay-complete).
- Invalid b1_model rejected with a ValueError naming the field.
- Baseline I_window: 625.8716 (effective_control_v1) → 6183.0154
(max_envelope: B1=aggregate_bias 5849.63 dominates fraction_channels
1729.72), certification_status NOT_CERTIFIED_BY_BOUND at W=10000.
KAT fixture (bench/fixtures/t3-bound/known-answer-tests.jsonl):
- Regenerated 2026-05-11 — 12 entries: the 8 §7-derived configs under
the new max_envelope default, a g=0 edge case, plus explicit-mode
pins for effective_control_v1 / fraction_channels / aggregate_bias.
- Each entry carries b1_model, expected_b1_selected,
expected_b1_{fraction_channels,aggregate_bias,effective_control_v1},
expected_snr_per_channel, expected_certification_status.
Tests (tests/test_t3_bound_calculator.py, 75 → 83):
- test_t3_bound_known_answer_tests no longer skips (fixture active);
pins b1_model, b1_selected, certification_status + numbers, tolerates
optional new fields on older fixtures.
- New: test_b1_max_envelope_exact_formula, test_invalid_b1_model_rejected,
test_cli_b1_model_flag (effective_control_v1 / fraction_channels /
aggregate_bias). test_b1_exact_formula renamed
test_b1_effective_control_v1_exact_formula and now passes the explicit
model. Updated baseline / below-256 / CLI tests for the new numbers.
Doc (docs/soft-hash-channel-t3-bound.md):
- Header + §0 + §3.1 + §6 + §7 (worked examples) + §8 (operator
guidance W-solving) + §10 (closure blockers RESOLVED) + §10.1 +
§11 (calculator schema) + §12 all updated for the max_envelope
default. §8: target-256 W drops from ~4196 to ~415 steps under the
conservative model — the ~10× cost of not assuming which g-reading
holds; operators who can measure effective-control applies can use
--b1-model effective_control_v1 for the looser W (a calibration
claim they must justify, not a default).
Status (#000036 ticket + TICKETS.md): both prior dav1d closure
blockers cleared (B1 worst-case model + active KAT fixture); remaining
= fox's final close-or-iterate call.
AUTOCOUNT markers bumped 75 → 83. Full suite: 2288 passed, 28 skipped.
This commit is contained in:
parent
dbe824944a
commit
1d1a942d57
8 changed files with 501 additions and 258 deletions
|
|
@ -51,32 +51,45 @@ _BASELINE = dict(
|
|||
|
||||
|
||||
def test_baseline_matches_section_11_doc():
|
||||
"""The §11 worked example baseline. Calculator's actual numbers:
|
||||
"""The §11 worked example baseline under the default b1_model=
|
||||
"max_envelope" (dav1d review 2026-05-11). Calculator's actual numbers:
|
||||
|
||||
I_window_bits_upper_bound: 625.8716
|
||||
B1_contribution: 292.4813
|
||||
B2_contribution: 300.0
|
||||
B3_contribution: 33.3904
|
||||
snr_grad: 0.5
|
||||
I_window_bits_upper_bound: 6183.0154
|
||||
B1_contribution: 5849.625 (aggregate_bias selected)
|
||||
B1_fraction_channels: 1729.7158
|
||||
B1_aggregate_bias: 5849.625
|
||||
B1_effective_control_v1: 292.4813 (the older non-worst-case)
|
||||
B2_contribution: 300.0
|
||||
B3_contribution: 33.3904
|
||||
snr_grad: 0.5 (g·G/σ)
|
||||
snr_per_channel: 10.0 (G/σ)
|
||||
|
||||
Doc §11's quoted 622.7 / 290.0 / 32.7 are first-cut rounded
|
||||
estimates; calculator's actual closed-form values differ by
|
||||
~3 bits total due to the doc's 290 vs actual 292.5 in B1 and
|
||||
32.7 vs actual 33.4 in B3. This test pins the closed-form
|
||||
truth (the doc text needs a calibration pass; tracked in this
|
||||
test's docstring as a follow-up).
|
||||
The conservative envelope takes max(fraction_channels, aggregate_bias);
|
||||
at this baseline aggregate_bias (5849.6) dominates fraction_channels
|
||||
(1729.7). The old effective_control formula (292.5) is still reported
|
||||
as B1_effective_control_v1 for comparison.
|
||||
"""
|
||||
r = t3_bound_bits(**_BASELINE)
|
||||
# Total: ⟨B1+B2+B3⟩ ≈ 292.48 + 300.0 + 33.39 = 625.87
|
||||
assert r["I_window_bits_upper_bound"] == pytest.approx(625.87, abs=0.05)
|
||||
assert r["b1_model"] == "max_envelope"
|
||||
assert r["b1_selected"] == "aggregate_bias"
|
||||
# Total: ⟨B1+B2+B3⟩ ≈ 5849.63 + 300.0 + 33.39 = 6183.02
|
||||
assert r["I_window_bits_upper_bound"] == pytest.approx(6183.02, abs=0.05)
|
||||
# B2 is exact: ⌈10000/100⌉ · log₂(8) = 100 · 3 = 300
|
||||
assert r["B2_contribution"] == pytest.approx(300.0, abs=1e-3)
|
||||
# B3: epochs = ⌈10000/1024⌉ = 10; per-epoch factor = 1024·0.1/1.0 = 102.4;
|
||||
# log₂(102.4)/2 ≈ 3.339; total ≈ 33.39 bits.
|
||||
assert r["B3_contribution"] == pytest.approx(33.39, abs=0.05)
|
||||
# B1: C_B1 · g · W · log₂(SNR+1) where SNR = 0.05·1.0/0.1 = 0.5
|
||||
# → log₂(1.5) ≈ 0.585; total ≈ 1.0 · 0.05 · 10000 · 0.585 = 292.48
|
||||
assert r["B1_contribution"] == pytest.approx(292.48, abs=0.05)
|
||||
# B1 (max_envelope): aggregate_bias = W·log₂(1 + g·G/σ) = 10000·log₂(1.5)
|
||||
# ≈ 10000·0.585 = 5849.63 (> fraction_channels 1729.72).
|
||||
assert r["B1_contribution"] == pytest.approx(5849.63, abs=0.05)
|
||||
assert r["B1_aggregate_bias"] == pytest.approx(5849.63, abs=0.05)
|
||||
# fraction_channels = g·W·log₂(1 + G/σ) = 0.05·10000·log₂(11) ≈ 1729.72
|
||||
assert r["B1_fraction_channels"] == pytest.approx(1729.72, abs=0.05)
|
||||
# effective_control_v1 (old formula) still reported: g·W·log₂(1+g·G/σ)
|
||||
# = 0.05·10000·log₂(1.5) ≈ 292.48
|
||||
assert r["B1_effective_control_v1"] == pytest.approx(292.48, abs=0.05)
|
||||
assert r["snr_grad"] == pytest.approx(0.5, abs=1e-6) # g·G/σ
|
||||
assert r["snr_per_channel"] == pytest.approx(10.0, abs=1e-6) # G/σ
|
||||
|
||||
|
||||
def test_returns_calculator_version_token():
|
||||
|
|
@ -289,15 +302,17 @@ def test_recommendation_total_le_zero():
|
|||
|
||||
|
||||
def test_recommendation_below_sha256():
|
||||
"""Baseline (~622 bits) actually exceeds 256 — wrong band for
|
||||
this test. Use a config that lands in the (0, 256) range."""
|
||||
args = {**_BASELINE, "window_length": 1000}
|
||||
"""The default-baseline bound exceeds 256 under max_envelope; use a
|
||||
small W that lands in the (0, 256) certified band. At W=300:
|
||||
aggregate_bias ≈ 175.5, B2 = 9, B3 ≈ 3.3 → total ≈ 187.8."""
|
||||
args = {**_BASELINE, "window_length": 300}
|
||||
r = t3_bound_bits(**args)
|
||||
assert 0 < r["I_window_bits_upper_bound"] < 256, (
|
||||
f"window_length=1000 should land in (0, 256); got {r['I_window_bits_upper_bound']}"
|
||||
f"window_length=300 should land in (0, 256); got {r['I_window_bits_upper_bound']}"
|
||||
)
|
||||
assert "windows" in r["recommendation"]
|
||||
assert "256" in r["recommendation"]
|
||||
assert r["certification_status"] == "CERTIFIED_BY_BOUND"
|
||||
|
||||
|
||||
def test_recommendation_exceeds_sha256():
|
||||
|
|
@ -315,22 +330,29 @@ def test_recommendation_exceeds_sha256():
|
|||
|
||||
|
||||
def test_output_carries_b1_model_and_certification_fields():
|
||||
"""Structured machine-readable fields added per dav1d review
|
||||
2026-05-11: b1_model, certification_status,
|
||||
certification_threshold_bits, model_assumptions."""
|
||||
"""Structured machine-readable fields (dav1d review 2026-05-11):
|
||||
b1_model, b1_selected, certification_status,
|
||||
certification_threshold_bits, model_assumptions, plus all three
|
||||
B1 model variants + both SNR readings."""
|
||||
r = t3_bound_bits(**_BASELINE)
|
||||
assert r["b1_model"] == "effective_control_v1"
|
||||
assert r["b1_model"] == "max_envelope" # the new default
|
||||
assert r["b1_selected"] in {"fraction_channels", "aggregate_bias"}
|
||||
assert r["certification_threshold_bits"] == 256
|
||||
assert r["certification_status"] in {
|
||||
"CERTIFIED_BY_BOUND", "NOT_CERTIFIED_BY_BOUND"
|
||||
}
|
||||
assert "B1_effective_control_model_v1" in r["model_assumptions"]
|
||||
assert "B1_model_max_envelope" in r["model_assumptions"]
|
||||
# all three B1 variants reported regardless of which is selected
|
||||
for k in ("B1_fraction_channels", "B1_aggregate_bias",
|
||||
"B1_effective_control_v1", "snr_per_channel"):
|
||||
assert k in r, f"missing output field {k!r}"
|
||||
assert r["certification_status"] == "NOT_CERTIFIED_BY_BOUND" # baseline > 256
|
||||
|
||||
|
||||
def test_certification_status_certified_below_threshold():
|
||||
"""A config landing in (0, 256) → CERTIFIED_BY_BOUND."""
|
||||
r = t3_bound_bits(**{**_BASELINE, "window_length": 1000})
|
||||
"""A config landing in (0, 256) → CERTIFIED_BY_BOUND. W=300 under
|
||||
max_envelope yields ~188 bits."""
|
||||
r = t3_bound_bits(**{**_BASELINE, "window_length": 300})
|
||||
assert 0 < r["I_window_bits_upper_bound"] < 256
|
||||
assert r["certification_status"] == "CERTIFIED_BY_BOUND"
|
||||
|
||||
|
|
@ -357,8 +379,34 @@ def test_cli_baseline_runs_clean(tmp_path):
|
|||
cwd=REPO_ROOT)
|
||||
j = json.loads(out.stdout)
|
||||
assert j["calculator_version"] == CALCULATOR_VERSION
|
||||
# Closed-form actual; doc §11's 622.7 is approximate.
|
||||
assert j["I_window_bits_upper_bound"] == pytest.approx(625.87, abs=0.05)
|
||||
assert j["b1_model"] == "max_envelope" # CLI default
|
||||
# Closed-form actual under max_envelope: 5849.63 + 300.0 + 33.39 = 6183.02
|
||||
assert j["I_window_bits_upper_bound"] == pytest.approx(6183.02, abs=0.05)
|
||||
|
||||
|
||||
def test_cli_b1_model_flag():
|
||||
"""--b1-model effective_control_v1 reverts to the older formula;
|
||||
--b1-model fraction_channels picks that envelope term."""
|
||||
base_cmd = [
|
||||
sys.executable, "-m", "bench.scripts.t3_bound_calculator",
|
||||
"--gradient-fraction", "0.05", "--gradient-norm-max", "1.0",
|
||||
"--gradient-noise-stddev", "0.1", "--lr-decision-interval", "100",
|
||||
"--lr-grid-size", "8", "--window-length", "10000",
|
||||
"--batches-per-epoch", "1024", "--steps-per-epoch", "1024",
|
||||
]
|
||||
for model, expected_b1 in [
|
||||
("effective_control_v1", 292.48),
|
||||
("fraction_channels", 1729.72),
|
||||
("aggregate_bias", 5849.63),
|
||||
]:
|
||||
out = subprocess.run(base_cmd + ["--b1-model", model],
|
||||
capture_output=True, text=True, check=True,
|
||||
cwd=REPO_ROOT)
|
||||
j = json.loads(out.stdout)
|
||||
assert j["b1_model"] == model
|
||||
assert j["B1_contribution"] == pytest.approx(expected_b1, abs=0.05), (
|
||||
f"b1_model={model}: B1={j['B1_contribution']} vs expected {expected_b1}"
|
||||
)
|
||||
|
||||
|
||||
def test_cli_invalid_input_exits_2():
|
||||
|
|
@ -404,24 +452,59 @@ def test_b2_exact_formula():
|
|||
)
|
||||
|
||||
|
||||
def test_b1_exact_formula():
|
||||
"""B1 = C_B1 · g · W · log₂(SNR + 1) where SNR = g·‖∇L_max‖/σ_grad.
|
||||
def test_b1_effective_control_v1_exact_formula():
|
||||
"""b1_model="effective_control_v1": B1 = C_B1 · g · W · log₂(SNR+1)
|
||||
where SNR = g·‖∇L_max‖/σ_grad (the older non-worst-case formula).
|
||||
Hand-compute for one config and check exact agreement."""
|
||||
g, gnm, sigma, W = 0.1, 2.0, 0.5, 5000
|
||||
snr = g * gnm / sigma # 0.4
|
||||
expected = 1.0 * g * W * math.log2(snr + 1)
|
||||
r = t3_bound_bits(**{
|
||||
**_BASELINE,
|
||||
"gradient_fraction": g,
|
||||
"gradient_norm_max": gnm,
|
||||
"gradient_noise_stddev": sigma,
|
||||
"window_length": W,
|
||||
"gradient_fraction": g, "gradient_norm_max": gnm,
|
||||
"gradient_noise_stddev": sigma, "window_length": W,
|
||||
"b1_model": "effective_control_v1",
|
||||
})
|
||||
# Output rounded to 4 dp inside the calculator.
|
||||
assert r["b1_model"] == "effective_control_v1"
|
||||
assert r["b1_selected"] == "effective_control_v1"
|
||||
assert r["B1_contribution"] == pytest.approx(expected, abs=1e-3)
|
||||
assert r["B1_effective_control_v1"] == pytest.approx(expected, abs=1e-3)
|
||||
assert r["snr_grad"] == pytest.approx(snr, abs=1e-3)
|
||||
|
||||
|
||||
def test_b1_max_envelope_exact_formula():
|
||||
"""b1_model="max_envelope" (default): B1 = max(fraction_channels,
|
||||
aggregate_bias) where
|
||||
fraction_channels = C_B1 · g · W · log₂(1 + G/σ)
|
||||
aggregate_bias = C_B1 · W · log₂(1 + g·G/σ)
|
||||
Hand-compute for one config; verify the larger term is selected."""
|
||||
g, gnm, sigma, W = 0.1, 2.0, 0.5, 5000
|
||||
snr_per_channel = gnm / sigma # 4.0
|
||||
snr_grad = g * gnm / sigma # 0.4
|
||||
fc = 1.0 * g * W * math.log2(snr_per_channel + 1) # 0.1·5000·log₂(5) ≈ 1160.96
|
||||
ab = 1.0 * W * math.log2(snr_grad + 1) # 5000·log₂(1.4) ≈ 2426.26
|
||||
expected_b1 = max(fc, ab)
|
||||
expected_selected = "aggregate_bias" if ab >= fc else "fraction_channels"
|
||||
r = t3_bound_bits(**{
|
||||
**_BASELINE,
|
||||
"gradient_fraction": g, "gradient_norm_max": gnm,
|
||||
"gradient_noise_stddev": sigma, "window_length": W,
|
||||
# b1_model defaults to max_envelope
|
||||
})
|
||||
assert r["b1_model"] == "max_envelope"
|
||||
assert r["b1_selected"] == expected_selected
|
||||
assert r["B1_contribution"] == pytest.approx(expected_b1, abs=1e-3)
|
||||
assert r["B1_fraction_channels"] == pytest.approx(fc, abs=1e-3)
|
||||
assert r["B1_aggregate_bias"] == pytest.approx(ab, abs=1e-3)
|
||||
assert r["snr_per_channel"] == pytest.approx(snr_per_channel, abs=1e-3)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", ["v2", "envelope", "", "MAX_ENVELOPE", None, 1])
|
||||
def test_invalid_b1_model_rejected(bad):
|
||||
with pytest.raises(ValueError, match="b1_model"):
|
||||
t3_bound_bits(**{**_BASELINE, "b1_model": bad})
|
||||
|
||||
|
||||
# --- integration with §6 framework -----------------------------------
|
||||
|
||||
|
||||
|
|
@ -504,6 +587,21 @@ def test_t3_bound_known_answer_tests():
|
|||
Closes the calculator-test-patterns.md §1 checklist gap (no
|
||||
KAT fixture for this calculator).
|
||||
"""
|
||||
# Numeric fields checked on every KAT entry.
|
||||
_NUMERIC = [
|
||||
("expected_total", "I_window_bits_upper_bound"),
|
||||
("expected_b1", "B1_contribution"),
|
||||
("expected_b2", "B2_contribution"),
|
||||
("expected_b3", "B3_contribution"),
|
||||
("expected_snr_grad", "snr_grad"),
|
||||
]
|
||||
# Optional numeric fields (present in 2026-05-11+ fixtures).
|
||||
_NUMERIC_OPT = [
|
||||
("expected_b1_fraction_channels", "B1_fraction_channels"),
|
||||
("expected_b1_aggregate_bias", "B1_aggregate_bias"),
|
||||
("expected_b1_effective_control_v1", "B1_effective_control_v1"),
|
||||
("expected_snr_per_channel", "snr_per_channel"),
|
||||
]
|
||||
n_kats = 0
|
||||
for line in KAT_FIXTURE.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
|
|
@ -517,13 +615,27 @@ def test_t3_bound_known_answer_tests():
|
|||
f"recorded {kat['calculator_version']}, "
|
||||
f"function {r['calculator_version']}"
|
||||
)
|
||||
for field, key in [
|
||||
("expected_total", "I_window_bits_upper_bound"),
|
||||
("expected_b1", "B1_contribution"),
|
||||
("expected_b2", "B2_contribution"),
|
||||
("expected_b3", "B3_contribution"),
|
||||
("expected_snr_grad", "snr_grad"),
|
||||
]:
|
||||
# b1_model + b1_selected pinned when recorded.
|
||||
if "b1_model" in kat:
|
||||
assert r["b1_model"] == kat["b1_model"], (
|
||||
f"KAT b1_model mismatch on {kat['label']!r}: "
|
||||
f"recorded {kat['b1_model']}, function {r['b1_model']}"
|
||||
)
|
||||
if "expected_b1_selected" in kat:
|
||||
assert r["b1_selected"] == kat["expected_b1_selected"], (
|
||||
f"KAT b1_selected mismatch on {kat['label']!r}: "
|
||||
f"recorded {kat['expected_b1_selected']}, "
|
||||
f"function {r['b1_selected']}"
|
||||
)
|
||||
if "expected_certification_status" in kat:
|
||||
assert r["certification_status"] == kat["expected_certification_status"], (
|
||||
f"KAT certification_status mismatch on {kat['label']!r}: "
|
||||
f"recorded {kat['expected_certification_status']}, "
|
||||
f"function {r['certification_status']}"
|
||||
)
|
||||
for field, key in _NUMERIC + _NUMERIC_OPT:
|
||||
if field not in kat:
|
||||
continue
|
||||
recorded = float(kat[field])
|
||||
observed = float(r[key])
|
||||
assert observed == pytest.approx(recorded, abs=1e-3), (
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue