From b2209ac33720c760589a254f32bf075476f8af7e Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 5 Jun 2026 15:32:32 -0400 Subject: [PATCH] cuda-fanout plans: Day-2/3 progress + Form D pivot RESULTS docs Captures empirical record from three agents that finished today: form-A-day2-progress.md (STOPPED, no merge) v2 Montgomery batch inversion regressed v1 by 0.63x-0.94x across n in {10k, 100k, 1M}. Byte-identity PASS at every N; math correct. Root cause: at this N, scalar_mul (256 Jacobian doubles x 5-6 ModMult each) dominates, NOT _ModInv. v2's Phase B/D used 1 thread/block leaving ~97% of SMs idle. v1 baseline re-measured at 7.88 Mkeys/s at n=1M (catalog upward correction from initial 6.51). Day-3 path: warp-level prefix scan OR windowed-G ladder. form-D-axis-flip-RESULTS.md (Form D opt 1 shipped foxhop 1f7ac9d) Per-candidate axis kernel measured 217 Mops/s at K=32 M=4 on 3090; 23.7x over per-shot N=4 at same M. Both axes saturate at the same ~220-250 Mops/s, refuting the bandwidth-bound diagnosis. Axis flip's win is occupancy-amortization at small M, not bandwidth redistribution. form-D-build-progress.md (Form D AG kernel parked) Aaronson-Gottesman tableau dead-end: our point-add circuit contains no H or S, so state never leaves the computational basis and AG buys nothing. Three pivot options proposed; fox picked options 1 + 3 in parallel. Day-1 binary at lumbda 7661788 stays canonical for cuda-secp256k1-batched-mul; v2 working-tree code lives uncommitted on the build host as Day-3 scratch. --- .../cuda-fanout/plans/form-A-day2-progress.md | 80 ++++++++++ .../plans/form-D-axis-flip-RESULTS.md | 141 ++++++++++++++++++ .../plans/form-D-build-progress.md | 127 ++++++++++++++++ 3 files changed, 348 insertions(+) create mode 100644 examples/cuda-fanout/plans/form-A-day2-progress.md create mode 100644 examples/cuda-fanout/plans/form-D-axis-flip-RESULTS.md create mode 100644 examples/cuda-fanout/plans/form-D-build-progress.md diff --git a/examples/cuda-fanout/plans/form-A-day2-progress.md b/examples/cuda-fanout/plans/form-A-day2-progress.md new file mode 100644 index 0000000..1e33572 --- /dev/null +++ b/examples/cuda-fanout/plans/form-A-day2-progress.md @@ -0,0 +1,80 @@ +# Form A Day-2 progress — Montgomery batch inversion landed correctly but regressed throughput + +Status: **STOPPED, no commit.** v2 kernel validates byte-identically against coincurve through n=1M but runs slower than Day-1 v1 across all tested batch sizes on 3090. + +## What shipped (uncommitted, working tree) + +`secp256k1-batch-mul.cu` now carries two kernel variants: + +- `secp_mul_batch_kernel_v1` (renamed from `secp_mul_batch_kernel`) — Day-1 path, per-thread `_ModInv`. +- `v2_phase_a` / `v2_phase_b` / `v2_phase_c` / `v2_phase_d` — 4-launch Montgomery batch inversion pipeline. + +CLI gates v2 via `--batch-inv` (default ON) or `--no-batch-inv` (selects v1). Stderr labels each run `[v1-per-thread-inv]` or `[v2-batch-inv]`. + +## Algorithm landed correctly + +Block decomposition: K = ceil(n / B) where B = 256. + +1. **Phase A** (full-parallel, N threads): per-thread scalar_mul, writes (Jx, Jy, Jz) to global scratch `jac[n*12]`; writes effective z[i] = Jz (or 1 for infinity) to scratch `z[n*4]`. +2. **Phase B** (1 thread/block, K blocks): per-block prefix product c[i] = z[base..i] mod p; writes block_prod[b] = c[end-1]. +3. **Phase C** (single thread total): walks left-to-right computing G_left[b] = product of block_prod[0..b-1]; one `_ModInv` on the global product G; walks right-to-left to produce u_entry[b] = (block_prod[0..b])^-1. +4. **Phase D** (1 thread/block, K blocks): walks each block backward i = end-1..base, computing z_inv[i] = u * c_global[i-1] where c_global[i-1] = G_left[b] * c[i-1] for i > base, else G_left[b] (b > 0) or 1 (b == 0); updates u *= z[i]; finishes affine projection (Zi^2, Zi^3, Ax, Ay); emits BE bytes. + +## Validation — passes byte-identical + +Coincurve oracle on 3090-ai (after `python3 -m venv ~/venv-secp && ~/venv-secp/bin/pip install coincurve`): + +``` +n=8 known-small PASS +n=32 random PASS +n=1000 random PASS +n=10000 random PASS +n=100000 random PASS +n=1000000 random PASS +``` + +Math is correct. The "STOPPED" condition is throughput, not correctness. + +## Throughput — v2 regresses across all N + +Three runs each on 3090-ai, kernel ms reported by the worker: + +| n | v1 Mkeys/s | v1 kernel ms | v2 Mkeys/s | v2 kernel ms | v2 / v1 | +|---------|------------|--------------|------------|--------------|---------| +| 10,000 | 4.37 | 2.29 | 2.76 | 3.62 | 0.63x | +| 100,000 | 7.43 | 13.47 | 5.97 | 16.74 | 0.80x | +| 1,000,000 | 7.88 | 126.85 | 7.39 | 139.42 | 0.94x | + +(Note: today's measured Day-1 baseline at n=1M is 7.88 Mkeys/s — higher than the 6.51 Mkeys/s cited in the task prompt. v1 was re-benchmarked on the same box in the same session.) + +## Root cause + +`_ModInv` on a 3090 is not the bottleneck at this n. v1 already saturates at ~7.9 Mkeys/s at n=1M because **scalar_mul** dominates the kernel: 256 Jacobian doubles + ~128 adds per scalar, each calling 5-6 `_ModMult` / `_ModSqr`. That's ~2000 field mults per scalar in scalar_mul vs ~20-30 field mults amortized for the inversion. Replacing _ModInv with batch inversion saves a small fraction of total work. + +The v2 implementation as designed makes the situation worse because: + +- Phase B and Phase D use **1 thread per block** (block size 256). At n=1M that's 3906 active threads vs v1's 1M threads. 3090 has 82 SMs × 1536 max-resident-threads = ~125k concurrent threads; v2 leaves >97% of SM throughput idle in the prefix and suffix passes. +- Per-block serial walk in Phase D performs ~256 ModMult4 calls sequentially per thread; v1 spreads ModInv (~20-30 ModMult-equivalent ops) across all N threads in parallel. +- v2 adds ~120 MB of device scratch (`jac`, `z`, `c`, `block_prod`, `G_left`, `u_entry`) at n=1M plus extra kernel launches. + +## What v2 needs to become a win + +Two non-obvious follow-ups, neither in scope today: + +1. **Warp-level prefix scan inside Phase B + suffix walk in Phase D.** Use Kogge-Stone or Sklansky scan with 32 threads/warp on the prefix product. At n=1M with B=256, each warp scans 8 elements — turns the serial 256-mult chain into 8 mults of warp-coordinated work. Block_size could grow to 1024 (warp == 32, scan 32 elements/warp). +2. **Move scalar_mul to a windowed ladder (precomputed G, 2G, ..., 256G).** Cuts scalar_mul cost ~4x and shifts the kernel's bottleneck onto inversion, where batching actually wins. Upstream `GPUGroup.h` carries the precomputed tables — vendoring that header is the Day-3+ move flagged in the original plan §2. + +Neither follow-up was in the Day-2 mandate. Logical halt point. + +## Files touched (uncommitted, in working tree) + +- `examples/cuda-fanout/secp256k1-batch-mul.cu` — split v1/v2 kernels, added `--batch-inv` flag, v2 default-ON in flag wiring (but never reached commit because throughput failed). + +## Rebuild + revert path + +``` +ssh 3090-ai.foxhop.net 'cd /home/fox/git/lumbda/examples/cuda-fanout && rm -f secp256k1-batch-mul && make secp256k1-batch-mul' +git -C ~/git/lumbda checkout -- examples/cuda-fanout/secp256k1-batch-mul.cu # if reverting +``` + +Day-1 binary @ commit 7661788 stays canonical until Day-3 lands either the warp-scan inner kernel or the windowed-ladder scalar_mul. diff --git a/examples/cuda-fanout/plans/form-D-axis-flip-RESULTS.md b/examples/cuda-fanout/plans/form-D-axis-flip-RESULTS.md new file mode 100644 index 0000000..9415620 --- /dev/null +++ b/examples/cuda-fanout/plans/form-D-axis-flip-RESULTS.md @@ -0,0 +1,141 @@ +# Form D Option 1 — axis-flip refactor RESULTS + +Companion to `form-D-build-progress.md` (which halted the AG-tableau +Form D before kernel write & recommended an axis-flip refactor of +`sim_gpu.cu`). This document records what landed. + +## What shipped + +| artifact | path | role | +|----------|------|------| +| `sim_gpu_axis.cu` | `ecdsa/cuda/sim_gpu_axis.cu` | per-candidate axis kernel, single launch covers K×M lanes | +| `main_axis_demo.c` | `ecdsa/cuda/main_axis_demo.c` | host driver, CPU/GPU byte-identity per lane, portal output | +| `Makefile` (+demo_axis target) | `ecdsa/cuda/Makefile` | builds via `make demo_axis` | + +`sim_gpu.cu` (per-shot striped) is untouched. `ops_loader.c` & `sim_cpu.c` +are untouched. Axis kernel reuses both as canonical references. + +## Hardware & workload + +- Host: `3090-ai.foxhop.net`, RTX 3090 (24 GB), CUDA 12.0, sm_86. +- Workload: `/tmp/ops.bin` (716 MB, 12,788,119 ops, 1355 qubits, 1,300,679 + bits, 2,100,266 RNG ops — point-add-toy Phase B emission). +- RNG: deterministic LFSR per `(candidate, lane)` seed — shared by CPU & + GPU so byte-identity holds. + +## Byte-identity + +Every reported configuration passed `cpu phase==gpu phase && +cpu cliff==gpu cliff && cpu toff==gpu toff` for every lane. + +| config | total lanes | result | +|--------|-------------|--------| +| K=1, M=4 | 4 | PASS | +| K=4, M=4 | 16 | PASS | +| K=16, M=4 | 64 | PASS | +| K=32, M=4 | 128 | PASS | + +K=64 M=4 OOMed at device alloc (bits arena alone = 2.7 GB × the rest +exceeded 24 GB on 3090). No incorrect result — kernel never reached +launch. + +## Throughput + +Apples-to-apples: same total quantum work in ops (`Σ_c n_ops_c × M_c` += `12.788M × total_lanes`). + +| kernel | K | M | total lanes | kernel ms | Mops/s | GPU speedup vs CPU | +|--------|---|---|-------------|-----------|--------|-------------------| +| per-shot striped (existing `sim_gpu.cu`) | 1 | 4 | 4 | 5,590 | 9.15 | 0.03× | +| **per-cand axis (new `sim_gpu_axis.cu`)** | 1 | 4 | 4 | 5,780 | 8.85 | 0.03× | +| **per-cand axis** | 4 | 4 | 16 | 6,938 | 29.49 | 0.08× | +| **per-cand axis** | 16 | 4 | 64 | 7,402 | 110.58 | 0.32× | +| **per-cand axis** | 32 | 4 | 128 | 7,546 | 216.91 | 0.61× | +| per-shot striped | 1 | 128 | 128 | 6,604 | 247.86 | 1.06× | + +CPU reference scales near-linearly (~350 Mops/s steady) — single-core +LLVM-vectorized bit ops on host's Zen. + +## Reading the numbers + +1. **Both kernels reach the same throughput ceiling at full saturation.** + - Per-shot at 128 lanes: 247 Mops/s. + - Axis at 128 lanes (K=32, M=4): 217 Mops/s. + - Within 12% of each other. The flip did **not** unlock new headroom. +2. **Small-M underutilization: axis wins by candidate-stacking.** + - At M=4 with per-shot kernel (one candidate), only 4 lanes / 4 + threads run — kernel is launch-overhead-dominated, 9 Mops/s. + - Axis K=32 at M=4 fills the same 128-lane budget by stacking 32 + candidates → **23.7× more throughput** than per-shot at the same + per-candidate M (5,590 ms → 7,546 ms while doing 32× more work). + - This is the only regime where the axis flip pays. +3. **Per-candidate ms drops cleanly with K.** + - K=1: 5,780 ms/cand. + - K=4: 1,735 ms/cand. + - K=16: 463 ms/cand. + - K=32: 236 ms/cand. → 24.5× per-candidate amortization vs K=1. + - Useful when a lumbda search loop sweeps many candidates with + few shots each (typical early-screen pattern: M=1–4, K=many). + +## Did the bandwidth-bound diagnosis hold? + +**Partially — and the failure mode matters.** + +The form-D-build-progress.md diagnosis was: "1.07× ceiling traces to +memory bandwidth on per-shot striped state." If that were the dominant +constraint, swapping the striping axis should redistribute the +bandwidth pressure & raise the ceiling. + +What the data shows: + +- At fully-saturated launch (128 lanes), both layouts land at ~220–250 + Mops/s. The ceiling did **not** move materially. → bandwidth was not + uniquely bound to the striping axis; redistributing across candidates + buys nothing once the device is full. +- At underutilized launch (M=4, 4 lanes), the axis flip's win is + 100% occupancy-driven (more lanes per launch), not bandwidth-driven. + Same speedup would have come from any axis that filled SMs. + +So the diagnosis was wrong about the **mechanism** but the **refactor +still pays** in the regime the lumbda search actually targets: many +candidates × few shots each. The win is occupancy-amortization, not +bandwidth-redistribution. + +## Where this leaves us + +- **Use axis kernel** when sweeping K ≥ 16 candidates with M ≤ 16 + shots each — lumbda's early-screen pattern. Per-candidate ms drops + ~20–25× vs per-shot kernel run sequentially per candidate. +- **Use per-shot kernel** when one candidate needs many shots (M ≥ 64) + — its at-saturation throughput is 14% higher & memory layout is + proven across thousands of test runs. +- **Don't pursue further axis variants** chasing the 1.07× → Nx + speedup. The 1.07× ceiling is a saturation ceiling on this device + for this op mix, not an axis-choice artifact. Future wins live in: + - ops.bin packing (Op = 28 bytes → 16 bytes halves op-stream + bandwidth; per `sim.h` most fields are NO_SLOT for most kinds). + - Persistent kernel + ops streaming from device-resident pool. + - True multi-GPU dispatch via lumbda's TCP-portal primitive (cammy + + guile + 4090-ai/ai are sitting idle for this workload). + +## Reproducing + +On the GPU host (`3090-ai.foxhop.net` in this run): + +```bash +cd /tmp/ecdsa-cuda +make demo_axis NVCC=/usr/bin/nvcc ARCH=sm_86 + +# K=1, M=4 — byte-identity check, baseline +./demo_axis /tmp/ops.bin --shots 4 + +# K=32, M=4 — fully-saturated axis run (needs --force-mem ~3.5 GB) +./demo_axis $(printf '/tmp/ops.bin %.0s' {1..32}) \ + --shots 4 --force-mem --portal /tmp/axis-K32-M4.portal + +# baseline per-shot comparison at same total lanes +./demo_ops /tmp/ops.bin 128 +``` + +Portal format `(cuda-axis-result ...)` — lumbda search loops ingest +this directly. Not yet wired into `gpu-worker.lsp` per task scope. diff --git a/examples/cuda-fanout/plans/form-D-build-progress.md b/examples/cuda-fanout/plans/form-D-build-progress.md new file mode 100644 index 0000000..6fa0470 --- /dev/null +++ b/examples/cuda-fanout/plans/form-D-build-progress.md @@ -0,0 +1,127 @@ +# Form D — build progress, halted 2026-06-05 + +Plan: `form-D-clifford-stabilizer.md`. +Status: **HALTED before kernel-write.** No code committed. Reason +below. + +## What ran + +Step 0 of plan workflow only: Toffoli-fraction measurement on +`/tmp/ops.bin` (716 MB, 12,788,119 ops, Phase B point-add-toy emission +on this host). + +Per-kind histogram: + +| kind | name | count | % | +|------|-----------|--------------|--------| +| 1 | REGISTER | 4 | 0.00% | +| 2 | APPEND_REG| 1,024 | 0.01% | +| 6 | X | 1,810,486 | 14.16% | +| 8 | CX | 5,336,861 | 41.73% | +| 9 | CZ | 1,319,391 | 10.32% | +| 10 | SWAP | 447,076 | 3.50% | +| 11 | R | 800,099 | 6.26% | +| 12 | HMR | 1,300,167 | 10.17% | +| 13 | CCX | 1,773,011 | 13.86% | + +- Toffoli fraction of all quantum ops: **13.87%** — well under our 40% + STOP threshold. Plan §3's "~half" suspicion did not hold. +- Amdahl ceiling on overall speedup, given Toffoli stays on CPU: + `1 / (0.139 + 0.861 / S)` where `S` is the Clifford-only speedup. At + the plan's 186× target, overall hits ~6.9× — still meaningful, but + the headline number does not apply. + +That part of the answer is clean: **fraction is fine, proceed gated +on the next finding.** + +## Why I stopped before writing the kernel + +Re-reading `ecdsa/cuda/sim.h` & `sim_gpu.cu` against the plan exposed +a structural mismatch: + +1. Upstream Roetteler point-add (kinds 0–17 enumerated in `sim.h`) + **carries no Hadamard, no S, no Y, no measurement-onto-basis-rotation + op.** Search across `ecdsa/lumbda/`, `ecdsa/cuda/`, and + `ecdsafail-challenge/` for any H gate emission returned zero hits. + The op histogram above is the entire op alphabet that ever fires. +2. With no H or S in the circuit, the quantum state never leaves the + computational basis. The "stabilizer simulation" reduces to + **classical reversible computation + a global phase ledger**. + That is exactly what `sim_gpu.cu`'s per-shot bit-packed simulator + already does — and proven exactly correct for this gateset, not an + approximation. +3. Aaronson-Gottesman tableau over 2n × (2n+1) bits gains us nothing + we don't already have. Its parallelism win (row-XOR primitives, + warp-cooperative tableau updates) shows up only when **superposition + matters** — i.e., when H or S introduce mixed-basis states the + per-shot simulator cannot represent in one bit per shot. We have + none of that. +4. The existing simulator's 1.07× ceiling on a 3090 (per plan + reference to `sim_gpu.cu`) traces to **memory bandwidth on + per-shot striped state**, not to algorithm choice. Switching the + algorithm to AG cannot help; switching the parallelism axis (per- + candidate over per-shot, as the plan correctly identifies) might — + but the axis flip belongs to the existing kernel, not a clean-room + AG re-implementation. + +## What this means for Form D + +The plan's headline target (186× from STABSim-class Clifford speedup) +does not transfer because: + +- Our circuit is a strict subset of Clifford that the existing kernel + already handles exactly in one bit per shot. +- AG's wins come from compressing exponentially-large stabilizer + state into 2n bits per shot; we already compress further (1 bit + per qubit per shot) because we never leave |0⟩/|1⟩. + +The **correct Form D**, if we still want a per-candidate axis flip, +is a refactor of `ecdsa/cuda/sim_gpu.cu` that: + +- maps one CUDA block to one candidate circuit instead of one thread + to one batch of shots, +- keeps the existing bit-packed phase/qubits/bits state, +- moves cross-thread cooperation onto the candidate dimension (warp + shares work per gate), +- continues to count CCX symbolically alongside the dynamic counter. + +That is **not** a clean-room Aaronson-Gottesman implementation. It is +"sim_gpu.cu, axis-flipped." I should not write a binary that pretends +to be the former when our real bottleneck wants the latter. + +## Recommendation to fox + +Three options, your call: + +1. **Re-scope Form D** to "axis-flip refactor of existing sim_gpu.cu + kernel, per-candidate parallelism." Drop AG framing entirely. + Drop BSTB/BSTR magic — reuse BSHK with a new `op_id`. Aim for the + kickmix speedup the plan actually wants. Effort ~3 days. + +2. **Build Form D as written** (clean-room AG tableau) for circuits + that **do** have H/S — e.g. error-correction codes, surface-code + patches, future research workloads. Keep the kernel in `cuda-fanout/` + as a general capability not coupled to our point-add metric. Effort + ~10 days, throughput gain on point-add is zero (no AG-eligible + structure to exploit). + +3. **Park Form D**, redirect engineering to options the kickmix + actually rewards: ops.bin packing (reduce per-op bytes from 56 → + 16 to halve global-memory traffic), or per-candidate `sim_gpu.cu` + refactor (option 1 above), or revisit Form B/C numbers. + +Hard rule from session: I do not commit broken code & I do not write +a kernel that misroutes around the actual bottleneck. Halting here. + +## Files touched + +None — no commit, no kernel write. This progress doc is the only +artifact. + +## Numbers for CATALOG once a path forward is chosen + +- Toffoli fraction (point-add-toy): 13.87% of quantum ops +- Op-mix dominated by CX (41.7%), X (14.2%), CCX (13.9%), CZ (10.3%), + HMR (10.2%), R (6.3%), SWAP (3.5%) +- No H, no S, no Y in upstream point-add gateset — confirms + computational-basis simulator is exact, not approximate.