The doc-drift pattern recurred four times today on 2026-05-10 (commits6cbbf95,14bcb99,5c21e83,30a9488). Each fix was the same shape: walk a doc, find a count that drifted from live truth during the hours after the doc was written, refresh it. Cost: ~5 min per drift × 4 = 20 min of manual catching, with no guarantee the next drift gets caught before someone external reads it. Per fox's selection: regression test that makes drift loud at test time instead of relying on visual catching. == Mechanism == `tests/test_doc_counts.py` scans `docs/**/*.md` for AUTOCOUNT tags of the form: <!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT--> Two metrics supported: - `tests` — pytest collected count for path. Batches every tagged path into one `pytest --collect-only` subprocess (~0.5s total). - `fixture-rows` — non-blank-non-comment line count in a JSONL fixture. GitHub and most markdown renderers strip HTML comments, so readers see only `N`. The tags are invisible in rendered output but make the claim machine-checkable. Three tests in the file: 1. `test_doc_autocount_claims_match_live` — the core invariant 2. `test_autocount_tags_are_well_formed` — open/close balance 3. `test_autocount_metric_names_are_documented` — fail-closed on undocumented metrics (catches typos) Failure message names the doc file, line number, and the claimed-vs-live diff. Example: `docs/foo.md:42 AUTOCOUNT(tests:tests/test_x.py) claims 23, live is 27` == 29 tags installed across 5 docs == While installing tags I had to read the surrounding prose, which surfaced six stale counts that had drifted same-day: `docs/soft-hash-channel-analysis.md`: - L392 14 → 23 tests for phi_alignment_probe - L417 20 → 27 tests for anchor_prg - L463 14 → 23 tests for phi_alignment_probe (status section) `docs/seven-point-program.md`: - L77 68 → 58 tests for metacognition (drift -10; the file shed tests during a refactor and the doc didn't catch up) - L78 9 tests for `test_dag.py::test_preflight_*` — removed count entirely; pytest selector subsets aren't currently supported by the AUTOCOUNT metric set (would need a `tests-matching` metric; not worth the surface for one claim). - L110 24 → 33 tests for test_dag.py `docs/calculator-test-patterns.md`: - L35 33 → 23 tests for warrant_resolver - L35 10 → 9 tests for warrant_chain - L16, L265 51 → 53 tests for t3_bound_calculator (kept initial-shipment provenance in prose) == Coverage installed == calculator-test-patterns.md 3 tagged claims soft-hash-channel-analysis.md 5 tagged claims warrant-substrate-cookbook.md 14 tagged claims seven-point-program.md 3 tagged claims tickets/ticket-000006-bench-... 4 tagged claims --- 29 tagged claims Every count that drifted today is now tagged. Future drift fires the regression test at the next pytest run instead of waiting for human catching. == Discipline pattern == Walk this pattern for any new doc that names a count: 1. Surround the number with the tag pair: `<!--AUTOCOUNT:tests:tests/test_foo.py-->N<!--/AUTOCOUNT-->` 2. Run `pytest tests/test_doc_counts.py` (~3.5s) 3. If it passes, the claim is now machine-verified Aim to tag counts on first authorship. Retrofitting is cheap but only catches drift after the fact. == Out of scope == Test counts inside source code (docstrings, CLI --help) are not scanned — would expand the test surface significantly and the drift pattern hasn't manifested there. Add `**/*.py` scope when that pattern surfaces. Alias-row counts and claim-pack-record counts could be tagged with new `db-rows:<table>` and `db-where:<sql>` metrics; deferred until the next drift on those numbers (none caught today after 30a9488's cookbook refresh). == Verification == $ .venv/bin/pytest tests/test_doc_counts.py -v 3 passed in 3.89s $ .venv/bin/pytest -q 2276 passed, 54 skipped in 153.21s No new dependencies. No schema changes. No source-code changes.
20 KiB
Soft-hash covert channel analysis
Ticket: #000018 Authored: 2026-05-09 Audience: fox + future blackops shifts + downstream Merkle-AGI v7+ authors Stance: defensive only — detection / flagging / bound-derivation, never generation or concealment.
0. Executive summary
We analyze whether an adversary controlling some fraction of training
data, gradient computation, or hyperparameter scheduling can drive a
v7-§9 plastic-trained model so that its next snapshot's hard hash
C(M_{t+1}) lands on a prearranged target value or value-class,
exploiting the soft-hash anchor map φ as a side-channel.
Result, under the assumptions stated in §5:
Under threat models T1 (data poisoner) and T2 (gradient adversary), the channel is bounded by a SHA-256 partial-preimage hardness reduction (§4). Under T3 (hyperparameter adversary), the channel is bounded by the per-checkpoint mitigation budget M2 (re-randomized anchors); residual channel without M2 is at most O(log N · entropy of training-data choices) bits per checkpoint window.
Recommended mitigation: M2 (per-checkpoint anchor nonce), costing one additional commit field per snapshot. Falls back to M1 (PRG-based φ) if a deployment surfaces residual structure. M3 (drop the anchor) is the strict-construction fallback if both fail; loses some training ergonomics.
This document is doc-only per the ticket's hard constraint: it recommends, does not implement. A follow-up ticket lands the chosen mitigation.
1. Threat model
Three adversary capabilities, increasing in strength.
1.1 T1 — Data poisoner
Adversary controls a fraction f ∈ (0, 1) of training-data examples.
Training loop is honest; aggregator is honest; hyperparameters are
honest; φ is the published deterministic embedding. Poisoned
examples are valid in distribution (otherwise an outlier filter
catches them).
Goal: poisoned examples drive Q(Θ_{t+1}) to a region whose soft
hash h_v(t+1) projects to a SHA-256 bucket the adversary can predict.
Concretely: adversary tries to land C(M_{t+1}) in a target class
T ⊂ {0,1}^{256} with |T|/2^{256} = ε (so the 'class' could be
"first byte 0xAA" → ε = 2^-8, or "specific 256-bit value" → ε = 2^-256).
1.2 T2 — Gradient adversary
T1 + adversary controls a fraction g of gradient computations
(e.g., compromised data-parallel worker that returns dishonest local
gradients to the central aggregator). Aggregator is still honest.
Adversary can bias the gradient signal in arbitrary directions — not bounded to legitimate-data-distribution reach. Tighter than T1.
1.3 T3 — Hyperparameter adversary
T2 + adversary controls some learning-rate / scheduler / batch-order decisions. Adversary can synchronize gradient updates with anchor geometry to maximize alignment. Strictly stronger than T2.
1.4 What's NOT in scope
- Direct attack on
C(M_t)itself (preimage-attack on SHA-256). Out of scope by A3 in v7 §3 (collision-resistant hash assumption). - Side-channel on training infrastructure (Spectre, RowHammer, compromised CPU). Different threat model.
- Adversary controlling the published
φseed retroactively. Once φ's seed is public, it's frozen; this ticket assumes seed is honestly drawn at v7 boot. - Compromised aggregator that lies about the snapshot bytes. Different threat model — that's a checkpoint integrity attack, not a covert channel.
2. v7 § 9 plastic training, recap
The relevant slice of the v7 spec:
Θ — model parameters (continuous, ~10^9 floats)
Q(Θ) — public quantization of Θ (SQD §3, integer kernel)
C(M) — SHA-256(canonical-encoding of Q(Θ) ∥ metadata)
↑ THE HARD HASH — committed bytes
h_v(Θ) — soft hash, dim_h-vector function of Θ
↑ THE SOFT HASH — used during training only
φ(C(M)) — anchor map: SHA-256 digest → dim_h vector
↑ FIXED DETERMINISTIC, published seed
Plastic training loss at step t:
L_total(Θ_t) = L_task(Θ_t) + α · ‖h_v(Θ_t) - φ(C(M_{t-1}))‖²
↑ L2 anchor pull
The anchor term pulls the model's soft hash toward the embedding of the previous checkpoint's hard hash. At each checkpoint boundary t → t+1, we compute Q(Θ_{t+1}) → C(M_{t+1}) and that becomes the next anchor.
The dual-hash separation theorem (T6, v7 §3): C(M) is determined solely by Q(Θ) and metadata; soft-hash bytes never enter the SHA-256 preimage. First-order conclusion: training cannot directly affect committed bytes.
The covert-channel concern: training does affect Q(Θ) (that's what training does). The question is whether the L2 anchor term lets the adversary steer Q(Θ) toward a target SHA-256 bucket.
3. Channel capacity definition
Define the per-checkpoint covert channel as the mutual information
I_t = I(adversary inputs at training step ≤ t ; C(M_{t+1}))
conditional on the public training history H_{≤t}
The conditioning on H_{≤t} (public training history: published
losses, public hyperparameters, public seeds) removes the
"adversary trivially knows what they did" channel. We measure the
bits the adversary can steer ABOVE the public information's
predictive power.
A non-adversarial bound: under random training, C(M_{t+1}) is
near-uniform on {0,1}^{256} conditional on H_{≤t} if Q(Θ_{t+1})
is sufficiently varied. Adversarial bound is the question.
3.1 Capacity bound under random φ
If φ is modeled as a random oracle independent of the parameter
geometry, then h_v - φ(C(M_{t-1})) is a uniform random direction at
each checkpoint, and the L2 pull provides no parameter-aligned
gradient signal. Under this idealization, the covert channel
reduces to:
I_t ≤ I(adversary inputs ; Q(Θ_{t+1}))
conditional on H_{≤t}
…which is the un-amplified channel — adversary is no better off through the soft-hash side-channel than they would be attacking Q(Θ) directly.
3.2 Capacity bound under structured φ
If φ has algebraic structure relative to parameter geometry (the
v7 reference embed_hard_to_vec is a fixed-seed linear projection
of the SHA-256 digest), the bound depends on whether gradient
updates that minimize the L2 anchor term are parameter-aligned to
shifts in Q(Θ) that move SHA-256 buckets predictably.
The reduction in §4 shows this dependency reduces to a SHA-256 partial-preimage problem.
4. Reduction to SHA-256 partial-preimage
4.1 Reduction sketch
Suppose adversary A exists that, given black-box training oracle
access under T1/T2, can drive C(M_{t+1}) into a target class T of
density ε with non-negligible probability p.
Construct algorithm B that solves SHA-256 partial-preimage:
B(target T):
1. Initialize a fresh v7 plastic-training run with random seed.
2. Run A as a sub-routine, supplying T as the target class.
3. After A signals success on checkpoint t+1, output Q(Θ_{t+1}).
4. Q(Θ_{t+1}) is then a partial preimage of T under SHA-256.
B's success probability is p minus the probability that A
succeeds via legitimate-Q-randomization (i.e., B would have
solved the problem by random Q-search anyway). Under the standard
random-oracle model for SHA-256, the random-Q baseline is ε. So:
P[B solves partial-preimage] ≥ p - ε
If p > ε + δ for any non-negligible δ, then B is a non-trivial
SHA-256 partial-preimage attacker — contradicting A3 (collision-
resistance / partial-preimage hardness).
Conclusion: under the SHA-256 random-oracle assumption (A3), no T1/T2 adversary can succeed with probability non-negligibly above ε. The channel exists but is bounded at the random baseline.
4.2 Where the reduction is tight, where it's loose
Tight: when φ is independent of parameter geometry (M1 or M2
applied), the reduction is essentially exact. T1/T2 are bounded to
the random baseline.
Loose: when φ has structure relative to parameter geometry
(reference v7 embed_hard_to_vec with fixed-seed linear projection),
the gradient feedback through the anchor MAY favor parameter
directions correlated with target SHA-256 buckets. We don't know
whether this correlation is large enough to break the reduction
without explicit structural analysis of embed_hard_to_vec.
This is the residual gap §6 addresses.
4.3 T3 caveat
T3 (hyperparameter adversary) is NOT bounded by the §4.1 reduction directly. The reduction assumes A's queries to the training oracle follow the published schedule. T3 violates that. Under T3:
- Adversary can synchronize gradient bursts with anchor geometry via learning-rate schedule manipulation.
- Per-checkpoint channel capacity grows with the adversary's schedule-control budget.
T3 needs M2 (re-randomized per-checkpoint anchor) or M3 (drop anchor). M1 alone may not suffice.
5. Assumptions on φ
Three concrete φ choices, with channel-residual under each:
5.1 φ_linear — v7 reference (linear projection, fixed seed)
φ(digest) = W · digest_as_uint32_vector (W frozen at boot)
- Pros: fast, deterministic, simple.
- Channel residual: unclear without explicit structural analysis of W relative to the parameter Hessian. If W has any direction aligned with a low-eigenvalue direction of the loss Hessian, the L2 pull provides cheap parameter movement that aligns with SHA-256 buckets — exploitable under T2 with empirical effort.
- Verdict: the reduction in §4 is loose under
φ_linear.
5.2 φ_PRG — non-linear PRG-based map (mitigation M1)
φ(digest) = PRG(seed=published, output_len=dim_h * 4 bytes)
then bytes → floats in [-1, 1]
PRG output is computationally indistinguishable from random under the PRG security assumption. Gradient feedback through the anchor is uncorrelated with parameter-space updates that would shift hash bytes predictably.
- Channel residual: zero modulo PRG security (one-way function
- no algebraic structure for the parameter Hessian to exploit).
- Verdict: §4 reduction is tight under
φ_PRG.
5.3 φ_nonce — re-randomized anchor (mitigation M2)
At each checkpoint t, sample fresh random nonce n_t (committed
in the snapshot manifest). Compute anchor as
φ(C(M_{t-1}) ∥ n_t)
where φ is φ_linear or φ_PRG. Adversary can't precompute against
the next anchor because n_t is unknown until commitment.
- Channel residual: even under
φ_linearbase, T1/T2 adversaries can't pre-plan attacks because the gradient direction they want to steer toward is randomized between checkpoints. - T3 residual: still bounded — T3 adversary controls timing,
not the nonce. Per-checkpoint window is
O(steps) × per-step channel; the nonce reset zeroes the cumulative steerage. - Verdict: lightweight; only adds 32 bytes to the manifest per checkpoint. Closes the residual channel for T1/T2 entirely; bounds T3 by a per-window budget.
5.4 φ dropped — anchor term removed (mitigation M3)
L_total(Θ) = L_task(Θ) + (no anchor term).
- Channel residual: zero by construction (no soft-hash gradient signal at all).
- Loss: plastic training loses some ergonomics (stable representation between checkpoints relied on the anchor pull). Quantified loss: depends on deployment; estimated in v7 §9.7 as 5-15% slower convergence on representation-stability metrics.
- Verdict: strict-construction fallback. Recommend only if M1
- M2 prove insufficient empirically.
6. Mitigations with residual analysis
| Mitigation | Cost | T1 residual | T2 residual | T3 residual | Closes the channel? |
|---|---|---|---|---|---|
| None (v7 reference) | 0 | bounded by §4 reduction (loose under φ_linear) | bounded by §4 reduction (loose under φ_linear) | unbounded structurally | No |
| M1 — φ_PRG | fast PRG eval per checkpoint | tight bound at random baseline ε | tight bound at random baseline ε | unbounded (T3 controls timing) | T1+T2 yes; T3 no |
| M2 — per-checkpoint nonce | 32 B in manifest per checkpoint | ε | ε | bounded by per-window budget | T1+T2 yes; T3 bounded |
| M1 + M2 (combined) | both costs | ε | ε | bounded by per-window budget | T1+T2 yes; T3 bounded |
| M3 — drop anchor | 5-15% training-ergonomics regression | 0 | 0 | 0 | Yes, by construction |
| M4 — gradient-budget cap | per-step gradient-norm clip | bounds adversary's per-step budget | bounds adversary's per-step budget | bounds T3 per-step | Soft bound; doesn't close |
7. Recommendation
Land M2 first (re-randomized per-checkpoint anchor):
- 32 bytes additional manifest field, committed in the snapshot.
- Closes the T1 + T2 channel under any
φchoice. - Bounds the T3 channel by per-checkpoint budget (the only attack window is between nonce-publish and next-checkpoint).
- No training-ergonomics regression — the nonce is purely per-checkpoint randomization, anchor pull still lands.
Add M1 (φ_PRG) as Phase 2 if M2-only deployment surfaces structural concerns (e.g., empirical evidence that T3 attacks within a window achieve > ε success rate).
M3 (drop anchor) stays in reserve as the strict-construction fallback. v7 §9 spec amendment proposed:
v7 §9.10 (proposed): The anchor map φ takes both the prior
checkpoint's hard hash AND a per-checkpoint nonce committed in
the manifest. The nonce is fresh entropy at each checkpoint,
sealed before training resumes. This closes the soft-hash covert
channel under threat models T1 and T2, and bounds T3 by the
per-checkpoint window.
8. Empirical validation (out of scope; follow-up)
A small experiment to sanity-check the analysis:
- Run v7 §9 plastic training with a deliberately-poisoned dataset
targeting "first byte of
C(M)equals 0xAA" — that'sε = 2^-8. - With M2 disabled (
φ_linear, no nonce): measure success rate over N checkpoints. Hypothesis: indistinguishable fromεif §4 reduction holds underφ_linear; structurally elevated ifφ_linearhas Hessian-aligned structure. - With M2 enabled (
φ_linear, per-checkpoint nonce): measure again. Hypothesis: indistinguishable fromεregardless ofφ_linear's structure. - Same experiment under
φ_PRG: should matchεbaseline with or without M2.
If empirical results diverge from analysis predictions, the
analysis is wrong somewhere — likely in §5 about φ_linear's
structural alignment with the parameter Hessian.
This experiment is a follow-up ticket; not in scope here.
9. Open questions
The reduction in §4 leaves three loose threads:
-
Hessian alignment under
φ_linear. Is the v7 referenceembed_hard_to_vecmatrix W aligned with low-eigenvalue directions of the typical training-loss Hessian? If yes, T2 adversaries with full gradient control can amplify steerage beyond the §4 random-oracle baseline. Test: compute the spectrum ofW^T Wand compare to typical Hessian eigenvalue distributions on representative checkpoints.Phase 1a landed 2026-05-10 under #000034: synthetic- ablation infrastructure at
bench/scripts/phi_alignment_probe.py(measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon) → AlignmentReport; Lanczos top-k + bottom-k viascipy.sparse.linalg.eigsh). 23 tests intests/test_phi_alignment_probe.pycovering verdict classification (STRUCTURAL_ALIGNMENT,NO_ALIGNMENT,ANTI_ALIGNED), boundary cases, monotonicity, closure (full-spectrum sum), parametrized invalid-input cones, and KAT regression. KAT fixture (30 vectors: 10 aligned + 10 uniform + 10 anti-aligned synthetic checkpoints) atbench/fixtures/phi-alignment/synthetic-checkpoints.jsonlpins synthetic-checkpoint verdicts so the probe is regression- guarded ahead of v7 deployment ramp-up. Lands underbench/scripts/(measurement tool, not a substrate primitive likearborist/substrate/anchor_prg.py). Phase 1b runs the probe against a real v7 checkpoint when one becomes available; the alignment-score distribution then resolves this open question and feeds C_B1 tightening per #000043. -
PRG modeling for
φ_PRG. The §4 reduction treatsφ_PRGas a random oracle. In practice it's a deterministic PRG with public seed. The reduction holds modulo PRG security; we should pick a PRG with cryptographic strength matching SHA-256 (e.g., AES-256-CTR with public IV; or HMAC-SHA-512 expansion). Decision: use HMAC-SHA-512(seed, digest ∥ counter) for dim_h floats. Keeps the dependency surface (already on SHA-256) tight.Phase 1 landed 2026-05-10 under #000035: reference implementation at
arborist/substrate/anchor_prg.py(HMAC-SHA-512 counter-mode KDF, pure stdlib —hashlib+hmac, no third-party crypto dep). 27 tests intests/test_anchor_prg.pycovering determinism, range invariants, chi² uniformity, dim_h boundary (1, 16384), seed-bit-flip and hash-bit-flip avalanche, hand-computed HMAC-SHA-512 first-block formula, prefix-extension closure, parametrized invalid-input cones, input validation, module-export shape, and KAT regression. 10 pinned KAT vectors atbench/fixtures/phi-prg/known-answer-tests.jsonlcovering block-boundary cases (dim_h=16 = one HMAC block; dim_h=17 = two blocks with truncation), one-bit-flip variants, and a 4096-element counter-rollover stress sample. Module exportsPHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"so future algorithm rotation is detectable at the call site without string-comparing module paths. Spec amendment text drafted at #000035 §3.4; Phase 2 lands it into a v7 plastic-training spec when one gains an active deployment target. -
T3 per-window budget formal bound.
§6 lists "bounded by per-window budget" without giving the bound.Closed 2026-05-10: the explicit closed-form bound now lives atdocs/soft-hash-channel-t3-bound.md§6 (under #000036), parametrized in operator-measurable inputs (g, ‖∇L_max‖, σ_grad, K, R, W, N_b, E). The combined bound has three contributions — gradient-bias C_B1 via Fano's inequality, LR-selection C_B2 via categorical-channel capacity, batch-order C_B3 via the Bottou-Bousquet random-shuffle refinement — summed over a window of length W. Calculator atbench/scripts/t3_bound_calculator.py.
10. References
- v7 § 3 (T6 dual-hash separation theorem)
- v7 § 9 (plastic training; the source of the L2-anchor term)
- v7 § 9.9 (the "backdoor via soft hash" first-order dismissal we're superseding)
- SQD § 3 (hard-hash construction for Q(Θ))
- SQD § 5 (integer-kernel discipline; bigint accumulator)
- ticket #000018 (this document is the closure deliverable)
11. Status
Open questions: §9.1 (Hessian alignment under φ_linear:
synthetic-ablation infrastructure landed 2026-05-10 per #000034
Phase 1a — probe + 23 tests + KAT fixture; parks on v7
deployment data per #000034 Phase 1b for the actual checkpoint
measurement),
§9.2 (PRG cryptographic strength for φ_PRG: decision pinned +
Phase 1 reference implementation landed 2026-05-10 per #000035 —
HMAC-SHA-512 with 32-byte committed seed, KAT-pinned; awaits v7
§9.10 spec amendment when v7 plastic-training spec gains an
active deployment target). §9.3 closed 2026-05-10 via the T3
per-window bound at docs/soft-hash-channel-t3-bound.md
(under #000036).
Closed: §4 reduction shows T1/T2 are bounded by SHA-256 partial-
preimage hardness under any φ independent of parameter geometry.
M2 (per-checkpoint nonce) closes residual channels at minimal cost.
Recommendation: v7 spec amendment §9.10 lands the M2 nonce. Implementation ticket follows.
Doc closure criterion (per #000018 §7): reduction stated, mitigation recommended, residual-bound table present (§6). Done.