arborist/docs/soft-hash-channel-analysis.md
russell@unturf.com fc5ba507dc
tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix)
The doc-drift pattern recurred four times today on 2026-05-10
(commits 6cbbf95, 14bcb99, 5c21e83, 30a9488). Each fix was the
same shape: walk a doc, find a count that drifted from live truth
during the hours after the doc was written, refresh it. Cost: ~5
min per drift × 4 = 20 min of manual catching, with no guarantee
the next drift gets caught before someone external reads it.

Per fox's selection: regression test that makes drift loud at
test time instead of relying on visual catching.

== Mechanism ==

`tests/test_doc_counts.py` scans `docs/**/*.md` for AUTOCOUNT
tags of the form:

  <!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->

Two metrics supported:

- `tests` — pytest collected count for path. Batches every
  tagged path into one `pytest --collect-only` subprocess
  (~0.5s total).
- `fixture-rows` — non-blank-non-comment line count in a JSONL
  fixture.

GitHub and most markdown renderers strip HTML comments, so
readers see only `N`. The tags are invisible in rendered output
but make the claim machine-checkable. Three tests in the file:

1. `test_doc_autocount_claims_match_live` — the core invariant
2. `test_autocount_tags_are_well_formed` — open/close balance
3. `test_autocount_metric_names_are_documented` — fail-closed on
   undocumented metrics (catches typos)

Failure message names the doc file, line number, and the
claimed-vs-live diff. Example:
`docs/foo.md:42 AUTOCOUNT(tests:tests/test_x.py) claims 23, live is 27`

== 29 tags installed across 5 docs ==

While installing tags I had to read the surrounding prose, which
surfaced six stale counts that had drifted same-day:

`docs/soft-hash-channel-analysis.md`:
- L392 14 → 23 tests for phi_alignment_probe
- L417 20 → 27 tests for anchor_prg
- L463 14 → 23 tests for phi_alignment_probe (status section)

`docs/seven-point-program.md`:
- L77 68 → 58 tests for metacognition (drift -10; the file
  shed tests during a refactor and the doc didn't catch up)
- L78 9 tests for `test_dag.py::test_preflight_*` — removed
  count entirely; pytest selector subsets aren't currently
  supported by the AUTOCOUNT metric set (would need a
  `tests-matching` metric; not worth the surface for one claim).
- L110 24 → 33 tests for test_dag.py

`docs/calculator-test-patterns.md`:
- L35 33 → 23 tests for warrant_resolver
- L35 10 → 9 tests for warrant_chain
- L16, L265 51 → 53 tests for t3_bound_calculator (kept
  initial-shipment provenance in prose)

== Coverage installed ==

  calculator-test-patterns.md           3 tagged claims
  soft-hash-channel-analysis.md         5 tagged claims
  warrant-substrate-cookbook.md        14 tagged claims
  seven-point-program.md                3 tagged claims
  tickets/ticket-000006-bench-...      4 tagged claims
                                      ---
                                       29 tagged claims

Every count that drifted today is now tagged. Future drift
fires the regression test at the next pytest run instead of
waiting for human catching.

== Discipline pattern ==

Walk this pattern for any new doc that names a count:

1. Surround the number with the tag pair:
   `<!--AUTOCOUNT:tests:tests/test_foo.py-->N<!--/AUTOCOUNT-->`
2. Run `pytest tests/test_doc_counts.py` (~3.5s)
3. If it passes, the claim is now machine-verified

Aim to tag counts on first authorship. Retrofitting is cheap
but only catches drift after the fact.

== Out of scope ==

Test counts inside source code (docstrings, CLI --help) are not
scanned — would expand the test surface significantly and the
drift pattern hasn't manifested there. Add `**/*.py` scope when
that pattern surfaces.

Alias-row counts and claim-pack-record counts could be tagged
with new `db-rows:<table>` and `db-where:<sql>` metrics; deferred
until the next drift on those numbers (none caught today after
30a9488's cookbook refresh).

== Verification ==

  $ .venv/bin/pytest tests/test_doc_counts.py -v
  3 passed in 3.89s

  $ .venv/bin/pytest -q
  2276 passed, 54 skipped in 153.21s

No new dependencies. No schema changes. No source-code changes.
2026-05-10 16:15:52 -04:00

20 KiB
Raw Blame History

Soft-hash covert channel analysis

Ticket: #000018 Authored: 2026-05-09 Audience: fox + future blackops shifts + downstream Merkle-AGI v7+ authors Stance: defensive only — detection / flagging / bound-derivation, never generation or concealment.


0. Executive summary

We analyze whether an adversary controlling some fraction of training data, gradient computation, or hyperparameter scheduling can drive a v7-§9 plastic-trained model so that its next snapshot's hard hash C(M_{t+1}) lands on a prearranged target value or value-class, exploiting the soft-hash anchor map φ as a side-channel.

Result, under the assumptions stated in §5:

Under threat models T1 (data poisoner) and T2 (gradient adversary), the channel is bounded by a SHA-256 partial-preimage hardness reduction (§4). Under T3 (hyperparameter adversary), the channel is bounded by the per-checkpoint mitigation budget M2 (re-randomized anchors); residual channel without M2 is at most O(log N · entropy of training-data choices) bits per checkpoint window.

Recommended mitigation: M2 (per-checkpoint anchor nonce), costing one additional commit field per snapshot. Falls back to M1 (PRG-based φ) if a deployment surfaces residual structure. M3 (drop the anchor) is the strict-construction fallback if both fail; loses some training ergonomics.

This document is doc-only per the ticket's hard constraint: it recommends, does not implement. A follow-up ticket lands the chosen mitigation.


1. Threat model

Three adversary capabilities, increasing in strength.

1.1 T1 — Data poisoner

Adversary controls a fraction f ∈ (0, 1) of training-data examples. Training loop is honest; aggregator is honest; hyperparameters are honest; φ is the published deterministic embedding. Poisoned examples are valid in distribution (otherwise an outlier filter catches them).

Goal: poisoned examples drive Q(Θ_{t+1}) to a region whose soft hash h_v(t+1) projects to a SHA-256 bucket the adversary can predict. Concretely: adversary tries to land C(M_{t+1}) in a target class T ⊂ {0,1}^{256} with |T|/2^{256} = ε (so the 'class' could be "first byte 0xAA" → ε = 2^-8, or "specific 256-bit value" → ε = 2^-256).

1.2 T2 — Gradient adversary

T1 + adversary controls a fraction g of gradient computations (e.g., compromised data-parallel worker that returns dishonest local gradients to the central aggregator). Aggregator is still honest.

Adversary can bias the gradient signal in arbitrary directions — not bounded to legitimate-data-distribution reach. Tighter than T1.

1.3 T3 — Hyperparameter adversary

T2 + adversary controls some learning-rate / scheduler / batch-order decisions. Adversary can synchronize gradient updates with anchor geometry to maximize alignment. Strictly stronger than T2.

1.4 What's NOT in scope

  • Direct attack on C(M_t) itself (preimage-attack on SHA-256). Out of scope by A3 in v7 §3 (collision-resistant hash assumption).
  • Side-channel on training infrastructure (Spectre, RowHammer, compromised CPU). Different threat model.
  • Adversary controlling the published φ seed retroactively. Once φ's seed is public, it's frozen; this ticket assumes seed is honestly drawn at v7 boot.
  • Compromised aggregator that lies about the snapshot bytes. Different threat model — that's a checkpoint integrity attack, not a covert channel.

2. v7 § 9 plastic training, recap

The relevant slice of the v7 spec:

Θ        — model parameters (continuous, ~10^9 floats)
Q(Θ)     — public quantization of Θ (SQD §3, integer kernel)
C(M)     — SHA-256(canonical-encoding of Q(Θ) ∥ metadata)
                  ↑ THE HARD HASH — committed bytes
h_v(Θ)   — soft hash, dim_h-vector function of Θ
                  ↑ THE SOFT HASH — used during training only
φ(C(M))  — anchor map: SHA-256 digest → dim_h vector
                  ↑ FIXED DETERMINISTIC, published seed

Plastic training loss at step t:

L_total(Θ_t) = L_task(Θ_t) + α · ‖h_v(Θ_t) - φ(C(M_{t-1}))‖²
                                     ↑ L2 anchor pull

The anchor term pulls the model's soft hash toward the embedding of the previous checkpoint's hard hash. At each checkpoint boundary t → t+1, we compute Q(Θ_{t+1}) → C(M_{t+1}) and that becomes the next anchor.

The dual-hash separation theorem (T6, v7 §3): C(M) is determined solely by Q(Θ) and metadata; soft-hash bytes never enter the SHA-256 preimage. First-order conclusion: training cannot directly affect committed bytes.

The covert-channel concern: training does affect Q(Θ) (that's what training does). The question is whether the L2 anchor term lets the adversary steer Q(Θ) toward a target SHA-256 bucket.


3. Channel capacity definition

Define the per-checkpoint covert channel as the mutual information

I_t = I(adversary inputs at training step ≤ t  ;  C(M_{t+1}))
      conditional on the public training history H_{≤t}

The conditioning on H_{≤t} (public training history: published losses, public hyperparameters, public seeds) removes the "adversary trivially knows what they did" channel. We measure the bits the adversary can steer ABOVE the public information's predictive power.

A non-adversarial bound: under random training, C(M_{t+1}) is near-uniform on {0,1}^{256} conditional on H_{≤t} if Q(Θ_{t+1}) is sufficiently varied. Adversarial bound is the question.

3.1 Capacity bound under random φ

If φ is modeled as a random oracle independent of the parameter geometry, then h_v - φ(C(M_{t-1})) is a uniform random direction at each checkpoint, and the L2 pull provides no parameter-aligned gradient signal. Under this idealization, the covert channel reduces to:

I_t ≤ I(adversary inputs ; Q(Θ_{t+1}))
       conditional on H_{≤t}

…which is the un-amplified channel — adversary is no better off through the soft-hash side-channel than they would be attacking Q(Θ) directly.

3.2 Capacity bound under structured φ

If φ has algebraic structure relative to parameter geometry (the v7 reference embed_hard_to_vec is a fixed-seed linear projection of the SHA-256 digest), the bound depends on whether gradient updates that minimize the L2 anchor term are parameter-aligned to shifts in Q(Θ) that move SHA-256 buckets predictably.

The reduction in §4 shows this dependency reduces to a SHA-256 partial-preimage problem.


4. Reduction to SHA-256 partial-preimage

4.1 Reduction sketch

Suppose adversary A exists that, given black-box training oracle access under T1/T2, can drive C(M_{t+1}) into a target class T of density ε with non-negligible probability p.

Construct algorithm B that solves SHA-256 partial-preimage:

B(target T):
    1. Initialize a fresh v7 plastic-training run with random seed.
    2. Run A as a sub-routine, supplying T as the target class.
    3. After A signals success on checkpoint t+1, output Q(Θ_{t+1}).
    4. Q(Θ_{t+1}) is then a partial preimage of T under SHA-256.

B's success probability is p minus the probability that A succeeds via legitimate-Q-randomization (i.e., B would have solved the problem by random Q-search anyway). Under the standard random-oracle model for SHA-256, the random-Q baseline is ε. So:

P[B solves partial-preimage] ≥ p - ε

If p > ε + δ for any non-negligible δ, then B is a non-trivial SHA-256 partial-preimage attacker — contradicting A3 (collision- resistance / partial-preimage hardness).

Conclusion: under the SHA-256 random-oracle assumption (A3), no T1/T2 adversary can succeed with probability non-negligibly above ε. The channel exists but is bounded at the random baseline.

4.2 Where the reduction is tight, where it's loose

Tight: when φ is independent of parameter geometry (M1 or M2 applied), the reduction is essentially exact. T1/T2 are bounded to the random baseline.

Loose: when φ has structure relative to parameter geometry (reference v7 embed_hard_to_vec with fixed-seed linear projection), the gradient feedback through the anchor MAY favor parameter directions correlated with target SHA-256 buckets. We don't know whether this correlation is large enough to break the reduction without explicit structural analysis of embed_hard_to_vec.

This is the residual gap §6 addresses.

4.3 T3 caveat

T3 (hyperparameter adversary) is NOT bounded by the §4.1 reduction directly. The reduction assumes A's queries to the training oracle follow the published schedule. T3 violates that. Under T3:

  • Adversary can synchronize gradient bursts with anchor geometry via learning-rate schedule manipulation.
  • Per-checkpoint channel capacity grows with the adversary's schedule-control budget.

T3 needs M2 (re-randomized per-checkpoint anchor) or M3 (drop anchor). M1 alone may not suffice.


5. Assumptions on φ

Three concrete φ choices, with channel-residual under each:

5.1 φ_linear — v7 reference (linear projection, fixed seed)

φ(digest) = W · digest_as_uint32_vector  (W frozen at boot)
  • Pros: fast, deterministic, simple.
  • Channel residual: unclear without explicit structural analysis of W relative to the parameter Hessian. If W has any direction aligned with a low-eigenvalue direction of the loss Hessian, the L2 pull provides cheap parameter movement that aligns with SHA-256 buckets — exploitable under T2 with empirical effort.
  • Verdict: the reduction in §4 is loose under φ_linear.

5.2 φ_PRG — non-linear PRG-based map (mitigation M1)

φ(digest) = PRG(seed=published, output_len=dim_h * 4 bytes)
            then bytes → floats in [-1, 1]

PRG output is computationally indistinguishable from random under the PRG security assumption. Gradient feedback through the anchor is uncorrelated with parameter-space updates that would shift hash bytes predictably.

  • Channel residual: zero modulo PRG security (one-way function
    • no algebraic structure for the parameter Hessian to exploit).
  • Verdict: §4 reduction is tight under φ_PRG.

5.3 φ_nonce — re-randomized anchor (mitigation M2)

At each checkpoint t, sample fresh random nonce n_t (committed
in the snapshot manifest). Compute anchor as
        φ(C(M_{t-1}) ∥ n_t)
where φ is φ_linear or φ_PRG. Adversary can't precompute against
the next anchor because n_t is unknown until commitment.
  • Channel residual: even under φ_linear base, T1/T2 adversaries can't pre-plan attacks because the gradient direction they want to steer toward is randomized between checkpoints.
  • T3 residual: still bounded — T3 adversary controls timing, not the nonce. Per-checkpoint window is O(steps) × per-step channel; the nonce reset zeroes the cumulative steerage.
  • Verdict: lightweight; only adds 32 bytes to the manifest per checkpoint. Closes the residual channel for T1/T2 entirely; bounds T3 by a per-window budget.

5.4 φ dropped — anchor term removed (mitigation M3)

L_total(Θ) = L_task(Θ) + (no anchor term).

  • Channel residual: zero by construction (no soft-hash gradient signal at all).
  • Loss: plastic training loses some ergonomics (stable representation between checkpoints relied on the anchor pull). Quantified loss: depends on deployment; estimated in v7 §9.7 as 5-15% slower convergence on representation-stability metrics.
  • Verdict: strict-construction fallback. Recommend only if M1
    • M2 prove insufficient empirically.

6. Mitigations with residual analysis

Mitigation Cost T1 residual T2 residual T3 residual Closes the channel?
None (v7 reference) 0 bounded by §4 reduction (loose under φ_linear) bounded by §4 reduction (loose under φ_linear) unbounded structurally No
M1 — φ_PRG fast PRG eval per checkpoint tight bound at random baseline ε tight bound at random baseline ε unbounded (T3 controls timing) T1+T2 yes; T3 no
M2 — per-checkpoint nonce 32 B in manifest per checkpoint ε ε bounded by per-window budget T1+T2 yes; T3 bounded
M1 + M2 (combined) both costs ε ε bounded by per-window budget T1+T2 yes; T3 bounded
M3 — drop anchor 5-15% training-ergonomics regression 0 0 0 Yes, by construction
M4 — gradient-budget cap per-step gradient-norm clip bounds adversary's per-step budget bounds adversary's per-step budget bounds T3 per-step Soft bound; doesn't close

7. Recommendation

Land M2 first (re-randomized per-checkpoint anchor):

  • 32 bytes additional manifest field, committed in the snapshot.
  • Closes the T1 + T2 channel under any φ choice.
  • Bounds the T3 channel by per-checkpoint budget (the only attack window is between nonce-publish and next-checkpoint).
  • No training-ergonomics regression — the nonce is purely per-checkpoint randomization, anchor pull still lands.

Add M1 (φ_PRG) as Phase 2 if M2-only deployment surfaces structural concerns (e.g., empirical evidence that T3 attacks within a window achieve > ε success rate).

M3 (drop anchor) stays in reserve as the strict-construction fallback. v7 §9 spec amendment proposed:

v7 §9.10 (proposed): The anchor map φ takes both the prior
checkpoint's hard hash AND a per-checkpoint nonce committed in
the manifest. The nonce is fresh entropy at each checkpoint,
sealed before training resumes. This closes the soft-hash covert
channel under threat models T1 and T2, and bounds T3 by the
per-checkpoint window.

8. Empirical validation (out of scope; follow-up)

A small experiment to sanity-check the analysis:

  1. Run v7 §9 plastic training with a deliberately-poisoned dataset targeting "first byte of C(M) equals 0xAA" — that's ε = 2^-8.
  2. With M2 disabled (φ_linear, no nonce): measure success rate over N checkpoints. Hypothesis: indistinguishable from ε if §4 reduction holds under φ_linear; structurally elevated if φ_linear has Hessian-aligned structure.
  3. With M2 enabled (φ_linear, per-checkpoint nonce): measure again. Hypothesis: indistinguishable from ε regardless of φ_linear's structure.
  4. Same experiment under φ_PRG: should match ε baseline with or without M2.

If empirical results diverge from analysis predictions, the analysis is wrong somewhere — likely in §5 about φ_linear's structural alignment with the parameter Hessian.

This experiment is a follow-up ticket; not in scope here.


9. Open questions

The reduction in §4 leaves three loose threads:

  1. Hessian alignment under φ_linear. Is the v7 reference embed_hard_to_vec matrix W aligned with low-eigenvalue directions of the typical training-loss Hessian? If yes, T2 adversaries with full gradient control can amplify steerage beyond the §4 random-oracle baseline. Test: compute the spectrum of W^T W and compare to typical Hessian eigenvalue distributions on representative checkpoints.

    Phase 1a landed 2026-05-10 under #000034: synthetic- ablation infrastructure at bench/scripts/phi_alignment_probe.py (measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon) → AlignmentReport; Lanczos top-k + bottom-k via scipy.sparse.linalg.eigsh). 23 tests in tests/test_phi_alignment_probe.py covering verdict classification (STRUCTURAL_ALIGNMENT, NO_ALIGNMENT, ANTI_ALIGNED), boundary cases, monotonicity, closure (full-spectrum sum), parametrized invalid-input cones, and KAT regression. KAT fixture (30 vectors: 10 aligned + 10 uniform + 10 anti-aligned synthetic checkpoints) at bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl pins synthetic-checkpoint verdicts so the probe is regression- guarded ahead of v7 deployment ramp-up. Lands under bench/scripts/ (measurement tool, not a substrate primitive like arborist/substrate/anchor_prg.py). Phase 1b runs the probe against a real v7 checkpoint when one becomes available; the alignment-score distribution then resolves this open question and feeds C_B1 tightening per #000043.

  2. PRG modeling for φ_PRG. The §4 reduction treats φ_PRG as a random oracle. In practice it's a deterministic PRG with public seed. The reduction holds modulo PRG security; we should pick a PRG with cryptographic strength matching SHA-256 (e.g., AES-256-CTR with public IV; or HMAC-SHA-512 expansion). Decision: use HMAC-SHA-512(seed, digest ∥ counter) for dim_h floats. Keeps the dependency surface (already on SHA-256) tight.

    Phase 1 landed 2026-05-10 under #000035: reference implementation at arborist/substrate/anchor_prg.py (HMAC-SHA-512 counter-mode KDF, pure stdlib — hashlib + hmac, no third-party crypto dep). 27 tests in tests/test_anchor_prg.py covering determinism, range invariants, chi² uniformity, dim_h boundary (1, 16384), seed-bit-flip and hash-bit-flip avalanche, hand-computed HMAC-SHA-512 first-block formula, prefix-extension closure, parametrized invalid-input cones, input validation, module-export shape, and KAT regression. 10 pinned KAT vectors at bench/fixtures/phi-prg/known-answer-tests.jsonl covering block-boundary cases (dim_h=16 = one HMAC block; dim_h=17 = two blocks with truncation), one-bit-flip variants, and a 4096-element counter-rollover stress sample. Module exports PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512" so future algorithm rotation is detectable at the call site without string-comparing module paths. Spec amendment text drafted at #000035 §3.4; Phase 2 lands it into a v7 plastic-training spec when one gains an active deployment target.

  3. T3 per-window budget formal bound. §6 lists "bounded by per-window budget" without giving the bound. Closed 2026-05-10: the explicit closed-form bound now lives at docs/soft-hash-channel-t3-bound.md §6 (under #000036), parametrized in operator-measurable inputs (g, ‖∇L_max‖, σ_grad, K, R, W, N_b, E). The combined bound has three contributions — gradient-bias C_B1 via Fano's inequality, LR-selection C_B2 via categorical-channel capacity, batch-order C_B3 via the Bottou-Bousquet random-shuffle refinement — summed over a window of length W. Calculator at bench/scripts/t3_bound_calculator.py.


10. References

  • v7 § 3 (T6 dual-hash separation theorem)
  • v7 § 9 (plastic training; the source of the L2-anchor term)
  • v7 § 9.9 (the "backdoor via soft hash" first-order dismissal we're superseding)
  • SQD § 3 (hard-hash construction for Q(Θ))
  • SQD § 5 (integer-kernel discipline; bigint accumulator)
  • ticket #000018 (this document is the closure deliverable)

11. Status

Open questions: §9.1 (Hessian alignment under φ_linear: synthetic-ablation infrastructure landed 2026-05-10 per #000034 Phase 1a — probe + 23 tests + KAT fixture; parks on v7 deployment data per #000034 Phase 1b for the actual checkpoint measurement), §9.2 (PRG cryptographic strength for φ_PRG: decision pinned + Phase 1 reference implementation landed 2026-05-10 per #000035 — HMAC-SHA-512 with 32-byte committed seed, KAT-pinned; awaits v7 §9.10 spec amendment when v7 plastic-training spec gains an active deployment target). §9.3 closed 2026-05-10 via the T3 per-window bound at docs/soft-hash-channel-t3-bound.md (under #000036).

Closed: §4 reduction shows T1/T2 are bounded by SHA-256 partial- preimage hardness under any φ independent of parameter geometry. M2 (per-checkpoint nonce) closes residual channels at minimal cost.

Recommendation: v7 spec amendment §9.10 lands the M2 nonce. Implementation ticket follows.

Doc closure criterion (per #000018 §7): reduction stated, mitigation recommended, residual-bound table present (§6). Done.