bend: planning docs for forms A, B, D (CATALOG.md build runway)

Three implementation plans land per CATALOG.md "Build order —
recommended 3-form runway":

  form-B-bignum-cgbn.md (3-5 dev-days, fewest unknowns):
    direct NVlabs CGBN consumption. 9-op dispatcher
    (mod-add/sub/mul/sqr/inv/exp/reduce + add-no-mod +
    mul-no-mod-2x-width). BSHK/BSHR mirrors shake256-fanout.
    15k crossover (wire overhead vs raw kernel). Risks:
    CGBN maintenance freshness + sm_86/sm_89 verification.

  form-A-secp256k1-batched-mul.md (6 dev-days):
    FixedPaul/VanitySearch-Bitcrack picked (AGPL-3, active
    2025-05-16) over JeanLucPons (GPL-3, stale 2022-04-04).
    Vendor 3 headers ~2400 LoC + write secp_mul_batch_kernel.
    BSPK/BSPR wire mirrors BSHK. Kernel extraction is the
    dominant risk.

  form-D-clifford-stabilizer.md (10 dev-days):
    STABSim & Qimax not open-source; clean-room
    Aaronson-Gottesman tableau recommended. Reuse QECCOPS1
    ops.bin over BSHK op_id=0x02 — no .stim translation.
    Toffoli option (b): score symbolically, simulate
    Clifford-only — gated on first measuring Toffoli fraction
    (point-add suspected ~50%, Amdahl eats gains if worse).

Plans live as actionable instructions in second-person voice;
each cites URLs for every claim. CATALOG.md & bend.html
untouched here — plans inform future surveys without inventing
numbers.
This commit is contained in:
russell@unturf.com 2026-06-05 13:25:37 -04:00
parent a33ae4331d
commit 4c5e04b07f
No known key found for this signature in database
3 changed files with 768 additions and 0 deletions

View file

@ -0,0 +1,272 @@
# Form A — `cuda-secp256k1-batched-mul` implementation plan
Status: design only. No binary built. No remote GPU touched.
Reference catalog entry: `../CATALOG.md` § A. Target: a `secp256k1-batch-mul`
daemon binary that speaks the BSHK binary wire protocol the worker already
drives for `shake256-fanout`, sitting next to it in `examples/cuda-fanout/`.
## 1. Upstream selection — pick FixedPaul/VanitySearch-Bitcrack
You would fork FixedPaul/VanitySearch-Bitcrack, not JeanLucPons. Reasons:
- **License.** FixedPaul ships AGPL-3.0
(https://github.com/FixedPaul/VanitySearch-Bitcrack), matching
`shake256-fanout.cu`'s AGPLv3 header. JeanLucPons is GPL-3.0
(https://github.com/JeanLucPons/VanitySearch). AGPL ↔ AGPL stays a
clean derivative.
- **Activity.** FixedPaul last touched master 2025-05-16; JeanLucPons
last commit 2022-04-04 (commits pages on each repo). Three years
stale vs actively maintained.
- **Speed.** FixedPaul publishes 6.9 Gkeys/s on 4090, 8.8 Gkeys/s on
5090 (repo README). Their "optimized CUDA modular math" patch over
upstream is the win we want.
No 3090 number published; you would measure on `3090-ai.foxhop.net`
once a binary exists. Expect ~3 Gkeys/s extrapolating from 3080's 2.65
Gkeys/s in our catalog.
## 2. Kernel extraction — files you need
Upstream layout
(https://github.com/JeanLucPons/VanitySearch/tree/master/GPU; fork
preserves shape):
- **`GPU/GPUMath.h`** (~900 lines) — `__device__` modular arithmetic
with inline PTX: `_ModMult`, `_ModSqr`, `ModSub256`, `_ModInv`, and
crucially `_ModInvGrouped` (Montgomery batched inversion: one inverse
per `GRP_SIZE/2+1` points). Keep verbatim.
- **`GPU/GPUGroup.h`** (~1024 lines) — precomputed `Gx[][]`, `Gy[][]`
for G, 2G, … 512G plus `_2Gnx`/`_2Gny`. Windowed-mul ladder. Keep
verbatim.
- **`GPU/GPUCompute.h`** (~450 lines) — `ComputeKeys()` Shamir-trick
loop (P+iG and P-iG share one delta-x). Keep field arithmetic;
**strip** SHA256 + RIPEMD160 + base58 + address-prefix branches.
- **`GPU/GPUEngine.cu`** (~920 lines, 5 `__global__` kernels:
`comp_keys`, `comp_keys_p2sh`, `comp_keys_comp`, `comp_keys_pattern`,
`comp_keys_p2sh_pattern`) — **drop entirely.** Write one new
`secp_mul_batch_kernel(scalars[N], base, points[N])` calling
`_ModInvGrouped`.
Drop: `main.cpp`, `Vanity.cpp`, `Base58.cpp`, `Bech32.cpp`,
`Wildcard.cpp`, `hash/*` (SSE host hashes), `GPUBase58.h`, `GPUHash.h`,
`GPUWildcard.h`, `GPUGenerate.cpp` — 19 .cpp files per upstream
Makefile, zero kept.
License: copy each file's header verbatim, add SPDX line, drop a
`LICENSE-vanitysearch-bitcrack` beside our existing license note.
## 3. Wire protocol — BSHK request, BSHR response
Mirror the existing shake protocol (`shake256-fanout.cu` lines 286-292):
Request, little-endian throughout, no alignment padding:
```
magic u8[4] = "BSHK"
op_id u32 = 0x01 (secp_mul_batch)
n u32 = number of scalars
base_x u8[32] base point X (big-endian field element, secp256k1 native)
base_y u8[32] base point Y
scalars u8[n*32] big-endian 256-bit ints
```
Response:
```
magic u8[4] = "BSHR" on success, "BERR" on failure
status u32 = 0 ok, nonzero error code (1 bad-magic, 2 bad-n,
3 cuda-launch-fail, 4 truncated-input)
n u32 = echoed scalar count
points u8[n*64] N points, each (X32 || Y32) big-endian; affine,
infinity encoded as all-zero
```
Endianness: LE u32 headers (matches shake binary at
`shake256-fanout.cu:286`), BE field elements (matches secp256k1 native;
ships straight to openssl/python-ecdsa for validation). Worker's
`wire-send-raw` treats payload as opaque (`gpu-worker.lsp:264`), no
change there.
## 4. `gpu-worker.lsp` handler sketch
You add one handler analogous to `handle-binary-shake` at
`gpu-worker.lsp:253`. The daemon stays warm for the whole worker
lifetime — same pattern, same `daemon-process` helper:
```scheme
(define *binary-secp-mul* "./secp256k1-batch-mul")
;; in main: register at boot, paying CUDA init once.
(register-daemon! 'cuda-secp-mul-batch *binary-secp-mul*)
;; new binary dispatcher: payload[0..4] = "BSPK" magic to disambiguate
;; from "BSHK" shake requests.
(define (handle-binary-secp-mul client payload)
(let* ((daemon (cdr (assoc 'cuda-secp-mul-batch *daemons*)))
(in-path (gensym-path "/tmp/bend-secp-in" ".bin"))
(out-path (gensym-path "/tmp/bend-secp-out" ".bin"))
(blob (substring payload 4 (string-length payload))))
(write-binary-file in-path blob)
(let ((status (daemon-process daemon in-path out-path #t)))
(cond
((eq? status 'ok)
(let ((result (read-binary-file out-path)))
(delete-file in-path) (delete-file out-path)
(wire-send-raw client (string-append "BSPR" result))))
(else
(delete-file in-path) (delete-file out-path)
(wire-send-raw client (string-append "BERR" (cdr status))))))))
```
Extend `handle-one` (`gpu-worker.lsp:271`) to branch on first 4 bytes:
`BSHK` → shake, `BSPK` → secp-mul. S-expression path stays for `ping`
and any text-mode ops. CUDA context init runs twice at startup, never
per request.
## 5. Daemon protocol — match shake256-fanout exactly
`secp256k1-batch-mul --daemon` reads stdin one cmd per line, emits
one response per line, identical to `shake256-fanout.cu:226-269`:
```
ready ; once at startup after cudaFree(0)
process-bin <in> <out> ; binary request
done <out> ; success
error <reason> ; failure
quit / bye ; shutdown
```
`daemon-process` in `gpu-worker.lsp:71` already writes
`process-bin <in> <out>\n` & expects `done`/`error` — zero handler-side
change. `start-daemon` (line 62) waits for `ready` before serving, so
CUDA init blocks worker readiness deterministically.
CLI fallback: `secp256k1-batch-mul --binary <in> <out>` for one-shot
test (matches `shake256-fanout.cu:272`).
## 6. Build recipe
Two options:
- **A — submodule.** `git submodule add
https://github.com/FixedPaul/VanitySearch-Bitcrack
third_party/vanitysearch-bitcrack`. Pro: trivial upstream-bump.
Con: ~12 MB of unused .cpp / .vcxproj noise.
- **B — vendor three headers.** Copy `GPUMath.h`, `GPUGroup.h`,
`GPUCompute.h` into
`examples/cuda-fanout/vendor/secp256k1-vanitysearch/` with
AGPL-3.0 + SPDX preserved, add a `PROVENANCE.md` carrying source
commit SHA. Pro: lean, audit-friendly. Con: manual sync.
Pick B — ~2400 lines, mirrors how `shake256-fanout.cu` carries its
tiny-sha3 distillation inline.
Makefile additions:
```
NVCCFLAGS_SECP ?= -O3 -arch=sm_86 -Xcompiler="-O3 -Wall" \
-maxrregcount=0 --ptxas-options=-v
all: shake256-fanout secp256k1-batch-mul
secp256k1-batch-mul: secp256k1-batch-mul.cu \
vendor/secp256k1-vanitysearch/GPUMath.h \
vendor/secp256k1-vanitysearch/GPUGroup.h \
vendor/secp256k1-vanitysearch/GPUCompute.h
$(NVCC) $(NVCCFLAGS_SECP) -Ivendor/secp256k1-vanitysearch -o $@ $<
test-secp: secp256k1-batch-mul test_secp_roundtrip.py
./secp256k1-batch-mul --binary /tmp/secp-in.bin /tmp/secp-out.bin
python3 test_secp_roundtrip.py /tmp/secp-in.bin /tmp/secp-out.bin
bench-secp: secp256k1-batch-mul bench_secp.py
python3 bench_secp.py ./secp256k1-batch-mul
```
`sm_86` covers 3090; `sm_89` covers 4090. Build a fat binary with
`-gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89`
for both fleet hosts.
Worker auto-registration: one line, `(register-daemon!
'cuda-secp-mul-batch *binary-secp-mul*)` next to the shake
registration at `gpu-worker.lsp:299`.
## 7. Validation plan
**Correctness.** `test_secp_roundtrip.py` runs N=100 known scalars
(1, 2, 3, …, plus edge cases: curve order n → infinity, n-1, scalars
near 2^128, 2^256-1). Computes `k*G` on CPU via `coincurve` (or
`ecdsa.SECP256k1.generator * k`), byte-compares GPU output. Halt on
first mismatch. Cross-check at N=10 against openssl `EC_POINT_mul` to
catch a coincurve defect masking a kernel defect.
**Performance.** `bench_secp.py` sweeps N ∈ {1k, 10k, 100k, 1M},
records `wire_ms`, `kernel_ms` (extend response to carry `u32
kernel_us` after `n`), `gpu/cpu` ratio. CPU baseline: in-process
`coincurve` loop. Median over 20 runs. Format mirrors
`bench_tiers.py:174` so the summary table renders unchanged.
Expected: 3090 ~3 Gkeys/s, 4090 ~6.9 Gkeys/s. CPU baseline ~50
kkeys/s single-thread coincurve → ~60,000× per core, ~5,000× vs
12-thread. Below 1,000× signals a defect — most likely PCIe transfer
saturating (1M × 64 B = 64 MB out, ~2 ms at PCIe 4.0).
## 8. Risks & open questions
- **License audit.** FixedPaul declares AGPL-3.0 at repo level but
`GPU/*.h` headers may still carry GPLv3 inherited from JeanLucPons.
Audit each vendored file before commit. If a file is GPLv3-only,
our binary becomes GPLv3 (still compatible with our AGPLv3 worker
via §7 of GPLv3).
- **Endomorphism in fork.** FixedPaul claims "optimized CUDA modular
math" but lists no specifics. If the speedup is a private
`GPUMath.h` patch, vendoring upstream headers alone loses it.
Diff `FixedPaul/GPU/GPUMath.h` vs `JeanLucPons/GPU/GPUMath.h` first;
take FixedPaul's.
- **Host RAM at N=1M.** 1M × 64 B = 64 MB out, 1M × 32 B = 32 MB in.
Worker holds ~3 copies (in-portal + daemon + out-portal): peak ~300
MB transient. Bounded on 24 GB / 80 GB hosts.
- **GRP_SIZE preservation.** Upstream's `GRP_SIZE=1024` means each
thread does 1024 keys sequentially sharing one `_ModInvGrouped` call.
A naive one-thread-per-scalar kernel loses the amortisation — ~50×
worse. New kernel launches `ceil(N/1024)` threads, each handling a
1024-scalar group; N rounds up to multiple of 1024 with sentinel
scalar = 1, pad outputs discarded.
- **Device memory.** 1M-point request allocates ~96 MB device-side
(in/out + scratch). `cudaFree` between requests to avoid fragmentation
over 10k+ requests. Add explicit free in `process_one_bin`.
## 9. Effort estimate
Experienced CUDA dev, no fleet-side work counted:
| step | days |
|---|---|
| vendor headers, audit licenses, write PROVENANCE.md | 0.5 |
| write `secp_mul_batch_kernel` reusing `_ModInvGrouped` group loop | 1.5 |
| daemon shell (stdin loop, --binary, --daemon modes) — copy shake | 0.5 |
| binary I/O, BSHK/BSHR parse + emit, host-side scalar layout | 0.5 |
| Makefile, smoke test on local nvcc (host has no GPU — compile only) | 0.5 |
| `test_secp_roundtrip.py` + coincurve oracle + openssl cross-check | 1.0 |
| `bench_secp.py` + sweeps + first 3090/4090 measurement | 1.0 |
| `gpu-worker.lsp` handler + dispatch branch + integration test | 0.5 |
| catalog promotion (live row, measured speedup) | 0.0 (fox's call) |
| **total** | **6 days** |
Concentration: ~40% kernel-extraction (steps 1+2), ~25% validation,
~15% bench harness, ~10% daemon/wire, ~10% handler. Kernel-extraction
risk dominates; everything else mirrors shake256-fanout.
## URLs cited
- Catalog § A: ../CATALOG.md
- Existing shake daemon: ../shake256-fanout.cu
- Worker handler pattern: ../gpu-worker.lsp lines 62, 71, 253, 264, 299
- Wire bench reference: ../bench_tiers.py lines 150-158, 174
- VanitySearch original: https://github.com/JeanLucPons/VanitySearch
- VanitySearch GPU folder: https://github.com/JeanLucPons/VanitySearch/tree/master/GPU
- VanitySearch last commit: https://github.com/JeanLucPons/VanitySearch/commits/master
- Bitcrack fork: https://github.com/FixedPaul/VanitySearch-Bitcrack
- Bitcrack last commit: https://github.com/FixedPaul/VanitySearch-Bitcrack/commits/master
- gECC paper (speedup reference): https://arxiv.org/pdf/2501.03245

View file

@ -0,0 +1,240 @@
# Form B — `cuda-bignum-cgbn` implementation plan
You wrap NVlabs CGBN (Cooperative Groups Big Numbers) as a BSHK-speaking
daemon under our existing bend-form pattern. Output: a single CUDA
binary `cgbn-batch-worker`, a Scheme handler in `gpu-worker.lsp`, a
Makefile target, & a validation harness. Cross-cite: catalog entry
`cuda-bignum-cgbn` in `CATALOG.md` (form B, build order step 3), source
NVlabs/CGBN at https://github.com/NVlabs/CGBN, midsize-int benchmark at
https://arxiv.org/pdf/2405.14642.
## 1. CGBN model → wire model
CGBN groups 4/8/16/32 threads into one **instance**; each instance owns
one bignum (32-bit to 32 K-bit). The library handles cooperative-group
dispatch (https://github.com/NVlabs/CGBN/blob/master/samples/sample_01_add/add.cu);
you write a kernel templated on `cgbn_env_t<context_t, BITS>` & call
`cgbn_load / cgbn_mul / cgbn_modular_inverse / cgbn_store`.
For our 256-bit secp256k1 case: `TPI=8` (threads per instance) at 256 bits
is the published sweet spot per CGBN README. One warp (32 threads) =
4 instances. A 3090 SM holds 1024 threads = 128 instances per SM × 82 SMs
= **10,496 in-flight 256-bit ops per kernel wave**. Map: one BSHK
request = one launch = N instances. We never expose CGBN's cooperative-
group primitive across our wire; we expose **op-coded batches**.
## 2. Operation dispatcher — `cgbn-batch-worker`
Single binary, op-coded. Op table (one byte each, room to grow):
| op_id | mnemonic | inputs | output | kernel |
|------:|----------------|--------|--------|---------------------------------------|
| 0x01 | mod-add | a, b | a+b mod m | `cgbn_add` + `cgbn_rem` |
| 0x02 | mod-sub | a, b | a-b mod m | `cgbn_sub` + conditional `cgbn_add` |
| 0x03 | mod-mul | a, b | a*b mod m | `cgbn_mont_mul` (precomputed `m`) |
| 0x04 | mod-sqr | a | a^2 mod m | `cgbn_mont_sqr` |
| 0x05 | mod-inv | a | a^-1 mod m | `cgbn_modular_inverse` |
| 0x06 | mod-exp | a, b | a^b mod m | `cgbn_modular_power` (binary ladder) |
| 0x07 | mod-reduce | a | a mod m | `cgbn_rem` standalone |
| 0x08 | add-no-mod | a, b | a+b | `cgbn_add` straight |
| 0x09 | mul-no-mod | a, b | a*b (lift to 2 × width) | `cgbn_mul_wide` |
Kernel dispatch via single `__global__` switch on op_id. CGBN environment
template is bit-width parameterized; you compile one specialization per
expected bit-width (256, 384, 521) & route via a runtime `switch
(bitwidth)` over the templated launchers. Default bit-widths to ship:
256, 384. Wider widths land on demand.
## 3. Wire protocol — BSHK / BSHR for CGBN
Request (little-endian throughout, matching `bench_tiers.py`
`make_payload_binary` in this dir):
```
"BSHK" (4 bytes magic)
u32 op_id
u32 bitwidth ; one of {256, 384}
u32 n_instances
u8[bitwidth/8] modulus m (omitted for op 0x08, 0x09)
u8[n * bitwidth/8] operand_a
u8[n * bitwidth/8] operand_b (omitted for 0x04, 0x05, 0x07)
```
Response:
```
"BSHR" (4 bytes magic — matches handle-binary-shake in gpu-worker.lsp)
u32 status ; 0 = ok; nonzero = error code
u32 n_instances
u8[n * width_out/8] result ; width_out = bitwidth or 2*bitwidth
```
For 256-bit secp256k1 case @ N=1M instances: input ~64 MB
(modulus + 2 operand arrays), output ~32 MB. CGBN documents 32K-bit max
per instance (see `cgbn.h` `BITS` template parameter
https://github.com/NVlabs/CGBN/blob/master/include/cgbn/cgbn.h); 256-bit
fits trivially. TPI=8 at 256 bits leaves no SM under-utilization on
3090/4090 because each SM still hosts 128 in-flight instances.
## 4. `gpu-worker.lsp` handler
You add `handle-cuda-bignum-cgbn` modeled on `handle-cuda-shake-fanout`
plus the binary path in `handle-binary-shake` already in this dir's
`gpu-worker.lsp`. Two surfaces:
S-expression (small N, dev convenience):
```scheme
(cgbn-batch
(op . mod-mul)
(bitwidth . 256)
(modulus . "fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f")
(a . ("..." "..."))
(b . ("..." "...")))
```
Binary (production): client sends `BSHK | u32 op_id | …` blob exactly as
above. Worker writes blob to `/tmp/bend-cgbn-in-<ts>.bin`, calls daemon
`process-bin <in> <out>`, reads `/tmp/bend-cgbn-out-<ts>.bin`, prepends
`"BSHR"`, sends back via `wire-send-raw`. Register the daemon at startup
next to `register-daemon!` for `'cuda-shake-fanout`:
```scheme
(define *binary-cgbn-batch*
(or (get-environment-variable "CGBN_BATCH_WORKER")
"./cgbn-batch-worker"))
(register-daemon! 'cuda-bignum-cgbn *binary-cgbn-batch*)
```
Dispatch in `handle-request`: `((eq? op 'cgbn-batch) (handle-cuda-bignum-cgbn args))`.
## 5. Daemon protocol
Inherited verbatim from `shake256-fanout.cu --daemon` (this dir): stdin
line-protocol `process-bin <in> <out>` / `quit`, stdout `ready` once
CUDA context warms, `done <out>` / `error <reason>` per command, `bye`
on quit. CUDA context init pays once per worker startup, never per BSHK
request — same architectural point the SHAKE form already proved.
## 6. Build recipe
```makefile
CGBN_DIR := vendor/CGBN
NVCC_ARCH_3090 := -arch=sm_86
NVCC_ARCH_4090 := -arch=sm_89
NVCC_FLAGS := -O3 -I$(CGBN_DIR)/include --extended-lambda
cgbn-batch-worker: cgbn-batch-worker.cu $(CGBN_DIR)/include/cgbn/cgbn.h
nvcc $(NVCC_FLAGS) $(NVCC_ARCH_3090) -gencode=arch=compute_89,code=sm_89 \
-o $@ cgbn-batch-worker.cu -lgmp
vendor/CGBN/include/cgbn/cgbn.h:
git submodule add https://github.com/NVlabs/CGBN vendor/CGBN
cgbn-bench: cgbn-batch-worker
python3 bench_cgbn.py --small 1000 --medium 100000 --huge 1000000
cgbn-test: cgbn-batch-worker
python3 test_cgbn_known_answers.py
```
CGBN ships as header-only inside `include/cgbn/` — no separate
`libcgbn.so` build. Vendor the headers; do not pull the full repo into
our build graph. License: BSD 3-Clause per https://github.com/NVlabs/CGBN/blob/master/LICENSE
— compatible with our AGPLv3 binary distribution (CGBN headers ship
permissively, our wrapper binary stays AGPLv3).
## 7. Validation plan
Three layers, cheapest first:
1. **openssl bignum CLI** for op 0x03 mod-mul on tiny operands. Drive
from `test_cgbn_known_answers.py`: pass N=32 hand-picked triples
(a, b, m, expected = a*b mod m). Sanity, not throughput.
2. **python gmpy2** for batches at N ∈ {1k, 10k, 100k}. Generate
random a, b ∈ [0, m); compute reference on host (`gmpy2.f_mod(a*b, m)`),
compare byte-for-byte against GPU result. Cover ops 0x01, 0x03, 0x04,
0x05, 0x06. gmpy2 docs: https://gmpy2.readthedocs.io.
3. **secp256k1 round-trip**: m = secp256k1 field prime
`0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f`,
load N random 256-bit a, b; compute a*b mod m through CGBN; verify
against Python `coincurve` reference (https://github.com/ofek/coincurve).
Halt promotion if any byte mismatches.
Cross-tier portal validation: identical BSHK input bytes to Python,
C tree-walker, asm-gc workers via our `bench_tiers.py` shape (already
binary-wire-aware) must produce byte-identical BSHR responses. Mismatch
on any tier halts promotion per `ecdsa/CLAUDE.md` cross-tier rule.
## 8. Performance projection
Sources: midsize-int benchmark https://arxiv.org/pdf/2405.14642 (Table 3
reports ~100×+ on 256-bit dense mul vs Xeon-20c GMP+OpenMP on V100);
NVlabs CGBN README claims 5-10× over GMP per-op on Volta. A 3090 is
~3× V100 throughput on dense integer work (per
https://www.nvidia.com/en-us/data-center/a100/ comparison & 3090
whitepaper).
Rough numbers, to be confirmed on hardware:
- **mod-mul 256-bit**: ~3-5 Gops/s on a 3090 (TPI=8 at warp-saturated
occupancy, kernel-only). 4090 ~1.6× that. Reference: midsize-int paper
Fig 7, scaled by 3090/V100 fp32 ratio.
- **mod-inv 256-bit** (CGBN's binary GCD path): ~50-100 Mops/s on 3090.
Bernstein-Yang batched variant (form E in catalog) wins ~3-5× over
this once we add it.
- **mod-exp 256-bit** (256-bit ladder): ~10-30 Mops/s on 3090.
- **wire-overhead crossover**: BSHK round-trip pays ~3 ms median per call
per `bench_tiers.py` binary-mode at small N (this dir's existing
measurements). At 3 Gops/s kernel rate, you break even with CPU GMP
(~5 Mops/s single-thread) around **N ≈ 15,000 instances per BSHK
call**. Stay above N=100k for clean 100×.
## 9. Risks & open questions
- **CGBN maintenance.** Last public commit on https://github.com/NVlabs/CGBN
is 2021-era. NVlabs has not formally archived it. You must verify
it compiles under CUDA 12.x on Ampere & Ada (sm_86, sm_89). If not,
patch `cgbn.h` toolchain checks ourselves & vendor the patched copy.
- **License.** BSD 3-Clause per https://github.com/NVlabs/CGBN/blob/master/LICENSE.
No AGPL incompatibility for our binary distribution.
- **sm_86 / sm_89 support.** CGBN documents Volta + Turing as primary
targets. Ampere (sm_86) cooperative-groups API is a superset, so the
cooperative-thread-array primitives CGBN calls remain valid. Open
risk: warp-shuffle reductions in `cgbn_add` may need width
adjustment for newer warp-scheduler behavior. Mitigation: ship
validation suite first; any divergence shows up against gmpy2 ref.
- **Under-utilization at 256-bit / TPI=8.** Each warp owns 4
independent instances. At N=1M & 82 SMs × 128 in-flight, you get
~95 kernel waves on a 3090. Each wave is ~µs; total kernel ~100 µs.
Memory bandwidth (32 MB input, 32 MB output → 64 MB at 936 GB/s =
~70 µs) becomes the floor. Kernel & PCIe transfer are co-dominant.
- **PCIe transfer budget.** 64 MB host→device + 32 MB back via PCIe
Gen4 x16 (32 GB/s) = ~3 ms. Already comparable to wire overhead;
pinned-host memory & overlap with kernel via streams cuts this in
half.
## 10. Effort estimate
You should land this in **3-5 days** end-to-end as an experienced CUDA
developer. Breakdown:
- Day 1: vendor CGBN, build sample_01_add unmodified on 3090 host,
confirm sm_86 compatibility, build baseline `cgbn-batch-worker.cu`
with op 0x03 mod-mul only.
- Day 2: daemon mode (clone shake256-fanout `--daemon` skeleton), add
ops 0x01, 0x04, 0x05, wire validation against gmpy2 at N=1k, 10k.
- Day 3: gpu-worker.lsp handler, BSHK dispatch, cross-tier validation
(Python + C + asm via `bench_tiers.py`-style harness), portal test
for secp256k1 mod-mul.
- Day 4: remaining ops (0x02, 0x06, 0x07, 0x08, 0x09), bench_cgbn.py
driver covering all ops at N ∈ {1k, 100k, 1M}, telemetry lines in
Scheme handler (wall-ms / cpu-ms / gpu-ms / gpu/cpu ratio).
- Day 5 (buffer): polish, README, CATALOG.md row promotion to `live`,
Makefile install/test targets, runs/lumbda-cgbn-001/ baseline
artifacts.
Lower than form A (`cuda-secp256k1-batched-mul`) since CGBN is a
**direct API consumption** — no extraction effort from a research
codebase, no curve-specific Montgomery tuning to port. Form A risks 7-10
days; form B caps at 5.

View file

@ -0,0 +1,256 @@
# Form D — `cuda-clifford-stabilizer` implementation plan
You will not build this binary yet. This document gives you (fox) a
fully-specified runway you can hand to a CUDA + quantum-sim engineer or
to a future blackops session. Catalog entry lives at
`CATALOG.md` (Form D). Numbers cited here all carry URLs.
## 1. Reference selection — what you can fork vs what you must write
Three candidates surveyed 2026-06-05:
- **STABSim** (Garner, Liu, Wang, Stein, Li — UW lead, seangarn@uw.edu)
arXiv 2507.03092 https://arxiv.org/abs/2507.03092 . Warp-level CUDA
primitives, scales better than CPU on QEC workloads. **No public
repo.** GitHub search for `stabsim cuda` returns zero results
https://github.com/search?q=stabsim+cuda&type=repositories . Treat as
reference design only. Email seangarn@uw.edu before sinking weeks
into a clean-room port. Our CATALOG's 186× figure derives from the
equivalence-checking benchmark — re-extract once you have full PDF
in hand.
- **Qimax** (Vu Tuan Hai et al, NAIST) arXiv 2505.03307
https://arxiv.org/abs/2505.03307 . Extended-stabilizer on multi-core
/ GPU; compares vs Qiskit & PennyLane, **not vs Stim**. No GitHub
on arXiv landing. A re-implementation matching Qimax does not yet
beat our CPU floor.
- **Clean-room Aaronson-Gottesman tableau on CUDA**
https://arxiv.org/abs/quant-ph/0406196 . ~500 LoC CUDA for a binary
symplectic tableau (2n × 2n bits + sign), gates H/S/CNOT/X/Z + M —
every Clifford gate boils down to row-XOR + sign twiddle. **Our
recommended target** because (a) you own our wire format, (b) STABSim
& Qimax did not release source, (c) Aaronson-Gottesman runs at known
O(n²) per gate, maps cleanly onto a CUDA warp.
Recommendation: write our own, citing STABSim's per-warp parallelism
as inspiration. Re-evaluate STABSim source if UW responds.
## 2. Circuit format — translate, do not adopt
Our circuit lives as Phase B lumbda Scheme & emits to QECCOPS1
`ops.bin` (op kinds in `ecdsa/cuda/sim.h:21-38`). Stim's `.stim` text
format is the de-facto wire for stabilizer circuits
(https://github.com/quantumlib/Stim , gate names `H S CX CZ SWAP R M
TICK`).
Pick **(b)** — `demo_stab_sim_ops.cu` accepts our existing QECCOPS1
`ops.bin` natively, treats Clifford ops as primary. Reasons:
- `ops.bin` already exists, generated by every lumbda tier; a `.stim`
translator adds a redundant emission path.
- Op kinds `OP_X, OP_Z, OP_CX, OP_CZ, OP_SWAP, OP_R, OP_HMR` map 1:1
onto Aaronson-Gottesman tableau ops. `OP_CCX, OP_CCZ` route to our
Toffoli handler (§3).
- Cross-validation against Stim CPU still happens via a one-shot
`emit-stim-circuit.lsp` test utility under `ecdsa/tests/` — small
circuits only, byte-equivalence spot checks, never our production
wire.
## 3. Toffoli problem — pick option (b), score symbolically
Stabilizer formalism handles Clifford only. Our Roetteler 12-step
point-add (`ecdsa/lumbda/point-add-real.lsp`) carries both Clifford &
Toffoli. Our CUDA sim already tracks them separately:
`ecdsa/cuda/sim_gpu.cu:155` counts `OP_CCZ / OP_CCX` as Toffoli, all
others as Clifford.
Three options surfaced in our brief; you pick **(b)**:
- **(a) Decompose each CCX into 7T.** Inflates op count ~7× per
Toffoli, requires magic-state tracking. Wrong for a scoring sim.
- **(b) Run Clifford fragment only on GPU; fold Toffoli count from
circuit structure.** Our score is a *count*, not a measurement
outcome. Toffoli ops are symbolic — you tally them at circuit-emit
time, skip them in our stabilizer kernel. GPU lane handles only
Clifford ops; CPU emits Σ-Toffoli alongside as a static circuit
property. **This is sufficient for our metric.** Speedup numbers in
CATALOG (186× equivalence-checking) come from Clifford-only
benchmarks, so this matches what published work measures.
- **(c) Extended stabilizer / magic-state sim.** Qimax territory.
Slower & more code; only revisit if our scoring metric changes to
require sampled outcomes through Toffoli gates.
Phase B point-add Toffoli fraction (from existing kernel counters):
roughly half of all gates per circuit are CCX / CCZ at current ancilla
widths. Confirm exact ratio by running `demo_ops` against
`point-add.bin` — GPU counters at `sim_gpu.cu:238-239` emit
`cliff_out` & `toff_out` per batch. Record this ratio before you sink
time into Form D: if Toffoli > 90% of ops, a Clifford-only sim
accelerates only 10% of work & a 186× fragment win yields 1.1× overall
(Amdahl).
## 4. Wire protocol — BSHK reuse
Reuse the BSHK / BSHR framing already wired through `gpu-worker.lsp`
(handler `handle-binary-shake` at lines 253-267). Layout:
- Request: `BSHK | u32 op_id=0x02 | u32 n_qubits | u32 n_shots |
u32 circuit_blob_size | circuit_blob[N]`
- circuit_blob: our existing QECCOPS1 `ops.bin` bytes, no translation.
- Response: `BSHR | u32 status | u32 n_shots |
samples[n_shots * ceil(n_qubits/8)]` bit-packed measurement outcomes.
Confirm match: STABSim & Qimax did not publish wire schemas, so you
own this protocol fully — design freedom, no compatibility tax.
`op_id` discriminator lets the daemon serve both `cuda-sim-ops-bin`
(kickmix, `op_id=0x01`) and `cuda-clifford-stabilizer`
(`op_id=0x02`) without renegotiating magic bytes.
## 5. `gpu-worker.lsp` handler sketch
Mirror `handle-cuda-sim-ops-bin` (lines 187-233). Pseudocode:
```scheme
(define *binary-stab-sim*
(or (get-environment-variable "STAB_SIM")
"/home/fox/git/www.foxhop.net/ecdsa/cuda/demo_stab_sim_ops"))
(define (handle-cuda-clifford-stabilizer args)
(let* ((ops-path (car args))
(n-qubits (car (cdr args)))
(n-shots (car (cdr (cdr args))))
(portal (gensym-path "/tmp/bend-stab" ".portal"))
(t-start (current-time-ms))
(argv (list ops-path
(number->string n-qubits)
(number->string n-shots)
"--portal" portal))
(daemon (cdr (assoc 'cuda-clifford-stabilizer *daemons*))))
(daemon-process daemon ops-path portal #t)
;; portal carries (cuda-stab-result (samples blob) (sigma-clifford k)
;; (sigma-toffoli k) (timing-ms ...))
;; same shape as cuda-sim-ops-bin, parse via existing portal-find-number.
...))
```
Dispatch line in `handle-request`:
```scheme
((eq? op 'cuda-clifford-stabilizer)
(handle-cuda-clifford-stabilizer args))
```
Register daemon at startup:
```scheme
(register-daemon! 'cuda-clifford-stabilizer *binary-stab-sim*)
```
## 6. Daemon protocol
Mirror `shake256-fanout --daemon` (called via `start-daemon` line 62).
On stdin one line per request, on stdout one line per response.
Two modes — text (`process`) for dev, binary (`process-bin`) for hot
path:
```
> process-bin /tmp/in.bin /tmp/out.bin\n
< done\n (or `error <message>\n`)
```
On start, daemon prints `ready\n` after CUDA context init &
tableau-row buffer allocation. CUDA context warm-up pays once per
worker lifetime.
## 7. Build recipe
Add to `ecdsa/cuda/Makefile`:
```make
demo_stab_sim_ops: stab_sim_gpu.o ops_loader.o main_stab_sim.o
nvcc -O3 -arch=sm_80 -o $@ $^
stab_sim_gpu.o: stab_sim_gpu.cu
nvcc -O3 -arch=sm_80 -c $< -o $@
```
`arch=sm_80` covers RTX 3090 (sm_86) & RTX 4090 (sm_89) via JIT;
hard-code at deploy time if you measure a perf loss.
Source files to write fresh:
- `stab_sim_gpu.cu` — Aaronson-Gottesman tableau kernel, one CUDA
block per circuit (= per candidate), threads cooperate on the
tableau rows. Reference: quant-ph/0406196 §3 update rules.
https://arxiv.org/abs/quant-ph/0406196
- `main_stab_sim.c` — daemon loop, ops.bin load, dispatch.
- Reuse `ops_loader.c` from existing simulator (already parses
QECCOPS1).
## 8. Validation plan
Two axes:
- **Σ-Clifford equivalence vs Stim CPU.** Generate a 16-qubit
toy circuit covering every supported gate; run our sim on
CPU+GPU; emit the same circuit through
`emit-stim-circuit.lsp` (one-shot translator, test-only); run
Stim CPU via `stim sample --in toy.stim` (Stim install on
permacomputer fleet, not on host). Σ-Clifford & measurement
outcomes must match byte-for-byte over 1024 shots seeded
identically.
- **Per-candidate axis flip benchmark.** Run N independent candidate
circuits as N GPU blocks (= N lanes). Compare:
- existing `demo_ops` per-shot: N candidates × 8192 shots each = 8192N
threads, one thread per shot of a single candidate;
- new `demo_stab_sim_ops` per-candidate: N candidates × 1 lane each,
each lane runs its candidate to completion serially in tableau.
At N=256 candidates expect Stim CPU to take ~2 s/candidate (512 s
total); Form D target on RTX 3090 ~2.7 s total based on a 186×
scaling factor. Record actual numbers; if our ratio < 30× you
surface our crossover N on CATALOG.
Bench target: a 16-qubit, 1000-Clifford-gate test circuit (small
enough to avoid memory blowup on first run; large enough that
kernel-launch overhead amortizes).
## 9. Risks & open questions
- **STABSim source.** No public repo. Either email seangarn@uw.edu &
wait, or build clean-room from quant-ph/0406196. Clean-room is our
default path. Risk: clean-room reaches Aaronson-Gottesman baseline,
not STABSim's published wins (likely 10-30× over CPU, not 186×).
- **Per-candidate axis vs per-shot axis.** STABSim hints at warp-level
parallelism without specifying whether warps map to circuits or
shots. Read PDF §III before committing — our axis-flip thesis
depends on per-circuit being the right grain.
- **Toffoli ratio.** If point-add ends up >90% Toffoli, Form D wins
only 1.1× overall (Amdahl). Measure Toffoli fraction before build.
- **Memory.** A 16-qubit tableau is ~1 KB per candidate; a 256-qubit
point-add fragment is 128 KB per candidate. N=1024 × 128 KB = 128
MB GPU state — fits. Cap N×state at 8 GB on 3090.
- **Stim install footprint.** Validation needs Stim on fleet;
`pip install stim`. Test-only dep, not a worker dep.
## 10. Effort estimate
For one experienced CUDA + quantum-sim engineer (assumes prior
Aaronson-Gottesman familiarity):
- Day 1: read STABSim PDF, read Aaronson-Gottesman paper, run
existing `demo_ops` to measure Toffoli ratio, decide go/no-go.
- Days 2-4: `stab_sim_gpu.cu` kernel — tableau rep, X/Z/H/S/CX/M
update rules, warp-level row-XOR primitive.
- Day 5: `main_stab_sim.c` daemon loop, BSHK plumbing.
- Day 6: `gpu-worker.lsp` handler, daemon registration, smoke run.
- Days 7-8: validation vs Stim CPU on 16-qubit toy, fix divergence.
- Days 9-10: per-candidate benchmark, record numbers, update
CATALOG.
**Total: ~10 working days.** Halve to 5 if STABSim source surfaces;
double to 20 if Toffoli folding turns out insufficient & you must add
extended-stabilizer (option c) support.
You decide go/no-go after Day 1 measurement.