diff --git a/examples/cuda-fanout/plans/form-A-secp256k1-batched-mul.md b/examples/cuda-fanout/plans/form-A-secp256k1-batched-mul.md new file mode 100644 index 0000000..9bc1102 --- /dev/null +++ b/examples/cuda-fanout/plans/form-A-secp256k1-batched-mul.md @@ -0,0 +1,272 @@ +# Form A — `cuda-secp256k1-batched-mul` implementation plan + +Status: design only. No binary built. No remote GPU touched. + +Reference catalog entry: `../CATALOG.md` § A. Target: a `secp256k1-batch-mul` +daemon binary that speaks the BSHK binary wire protocol the worker already +drives for `shake256-fanout`, sitting next to it in `examples/cuda-fanout/`. + +## 1. Upstream selection — pick FixedPaul/VanitySearch-Bitcrack + +You would fork FixedPaul/VanitySearch-Bitcrack, not JeanLucPons. Reasons: + +- **License.** FixedPaul ships AGPL-3.0 + (https://github.com/FixedPaul/VanitySearch-Bitcrack), matching + `shake256-fanout.cu`'s AGPLv3 header. JeanLucPons is GPL-3.0 + (https://github.com/JeanLucPons/VanitySearch). AGPL ↔ AGPL stays a + clean derivative. +- **Activity.** FixedPaul last touched master 2025-05-16; JeanLucPons + last commit 2022-04-04 (commits pages on each repo). Three years + stale vs actively maintained. +- **Speed.** FixedPaul publishes 6.9 Gkeys/s on 4090, 8.8 Gkeys/s on + 5090 (repo README). Their "optimized CUDA modular math" patch over + upstream is the win we want. + +No 3090 number published; you would measure on `3090-ai.foxhop.net` +once a binary exists. Expect ~3 Gkeys/s extrapolating from 3080's 2.65 +Gkeys/s in our catalog. + +## 2. Kernel extraction — files you need + +Upstream layout +(https://github.com/JeanLucPons/VanitySearch/tree/master/GPU; fork +preserves shape): + +- **`GPU/GPUMath.h`** (~900 lines) — `__device__` modular arithmetic + with inline PTX: `_ModMult`, `_ModSqr`, `ModSub256`, `_ModInv`, and + crucially `_ModInvGrouped` (Montgomery batched inversion: one inverse + per `GRP_SIZE/2+1` points). Keep verbatim. +- **`GPU/GPUGroup.h`** (~1024 lines) — precomputed `Gx[][]`, `Gy[][]` + for G, 2G, … 512G plus `_2Gnx`/`_2Gny`. Windowed-mul ladder. Keep + verbatim. +- **`GPU/GPUCompute.h`** (~450 lines) — `ComputeKeys()` Shamir-trick + loop (P+iG and P-iG share one delta-x). Keep field arithmetic; + **strip** SHA256 + RIPEMD160 + base58 + address-prefix branches. +- **`GPU/GPUEngine.cu`** (~920 lines, 5 `__global__` kernels: + `comp_keys`, `comp_keys_p2sh`, `comp_keys_comp`, `comp_keys_pattern`, + `comp_keys_p2sh_pattern`) — **drop entirely.** Write one new + `secp_mul_batch_kernel(scalars[N], base, points[N])` calling + `_ModInvGrouped`. + +Drop: `main.cpp`, `Vanity.cpp`, `Base58.cpp`, `Bech32.cpp`, +`Wildcard.cpp`, `hash/*` (SSE host hashes), `GPUBase58.h`, `GPUHash.h`, +`GPUWildcard.h`, `GPUGenerate.cpp` — 19 .cpp files per upstream +Makefile, zero kept. + +License: copy each file's header verbatim, add SPDX line, drop a +`LICENSE-vanitysearch-bitcrack` beside our existing license note. + +## 3. Wire protocol — BSHK request, BSHR response + +Mirror the existing shake protocol (`shake256-fanout.cu` lines 286-292): + +Request, little-endian throughout, no alignment padding: + +``` +magic u8[4] = "BSHK" +op_id u32 = 0x01 (secp_mul_batch) +n u32 = number of scalars +base_x u8[32] base point X (big-endian field element, secp256k1 native) +base_y u8[32] base point Y +scalars u8[n*32] big-endian 256-bit ints +``` + +Response: + +``` +magic u8[4] = "BSHR" on success, "BERR" on failure +status u32 = 0 ok, nonzero error code (1 bad-magic, 2 bad-n, + 3 cuda-launch-fail, 4 truncated-input) +n u32 = echoed scalar count +points u8[n*64] N points, each (X32 || Y32) big-endian; affine, + infinity encoded as all-zero +``` + +Endianness: LE u32 headers (matches shake binary at +`shake256-fanout.cu:286`), BE field elements (matches secp256k1 native; +ships straight to openssl/python-ecdsa for validation). Worker's +`wire-send-raw` treats payload as opaque (`gpu-worker.lsp:264`), no +change there. + +## 4. `gpu-worker.lsp` handler sketch + +You add one handler analogous to `handle-binary-shake` at +`gpu-worker.lsp:253`. The daemon stays warm for the whole worker +lifetime — same pattern, same `daemon-process` helper: + +```scheme +(define *binary-secp-mul* "./secp256k1-batch-mul") + +;; in main: register at boot, paying CUDA init once. +(register-daemon! 'cuda-secp-mul-batch *binary-secp-mul*) + +;; new binary dispatcher: payload[0..4] = "BSPK" magic to disambiguate +;; from "BSHK" shake requests. +(define (handle-binary-secp-mul client payload) + (let* ((daemon (cdr (assoc 'cuda-secp-mul-batch *daemons*))) + (in-path (gensym-path "/tmp/bend-secp-in" ".bin")) + (out-path (gensym-path "/tmp/bend-secp-out" ".bin")) + (blob (substring payload 4 (string-length payload)))) + (write-binary-file in-path blob) + (let ((status (daemon-process daemon in-path out-path #t))) + (cond + ((eq? status 'ok) + (let ((result (read-binary-file out-path))) + (delete-file in-path) (delete-file out-path) + (wire-send-raw client (string-append "BSPR" result)))) + (else + (delete-file in-path) (delete-file out-path) + (wire-send-raw client (string-append "BERR" (cdr status)))))))) +``` + +Extend `handle-one` (`gpu-worker.lsp:271`) to branch on first 4 bytes: +`BSHK` → shake, `BSPK` → secp-mul. S-expression path stays for `ping` +and any text-mode ops. CUDA context init runs twice at startup, never +per request. + +## 5. Daemon protocol — match shake256-fanout exactly + +`secp256k1-batch-mul --daemon` reads stdin one cmd per line, emits +one response per line, identical to `shake256-fanout.cu:226-269`: + +``` +ready ; once at startup after cudaFree(0) +process-bin ; binary request +done ; success +error ; failure +quit / bye ; shutdown +``` + +`daemon-process` in `gpu-worker.lsp:71` already writes +`process-bin \n` & expects `done`/`error` — zero handler-side +change. `start-daemon` (line 62) waits for `ready` before serving, so +CUDA init blocks worker readiness deterministically. + +CLI fallback: `secp256k1-batch-mul --binary ` for one-shot +test (matches `shake256-fanout.cu:272`). + +## 6. Build recipe + +Two options: + +- **A — submodule.** `git submodule add + https://github.com/FixedPaul/VanitySearch-Bitcrack + third_party/vanitysearch-bitcrack`. Pro: trivial upstream-bump. + Con: ~12 MB of unused .cpp / .vcxproj noise. +- **B — vendor three headers.** Copy `GPUMath.h`, `GPUGroup.h`, + `GPUCompute.h` into + `examples/cuda-fanout/vendor/secp256k1-vanitysearch/` with + AGPL-3.0 + SPDX preserved, add a `PROVENANCE.md` carrying source + commit SHA. Pro: lean, audit-friendly. Con: manual sync. + +Pick B — ~2400 lines, mirrors how `shake256-fanout.cu` carries its +tiny-sha3 distillation inline. + +Makefile additions: + +``` +NVCCFLAGS_SECP ?= -O3 -arch=sm_86 -Xcompiler="-O3 -Wall" \ + -maxrregcount=0 --ptxas-options=-v + +all: shake256-fanout secp256k1-batch-mul + +secp256k1-batch-mul: secp256k1-batch-mul.cu \ + vendor/secp256k1-vanitysearch/GPUMath.h \ + vendor/secp256k1-vanitysearch/GPUGroup.h \ + vendor/secp256k1-vanitysearch/GPUCompute.h + $(NVCC) $(NVCCFLAGS_SECP) -Ivendor/secp256k1-vanitysearch -o $@ $< + +test-secp: secp256k1-batch-mul test_secp_roundtrip.py + ./secp256k1-batch-mul --binary /tmp/secp-in.bin /tmp/secp-out.bin + python3 test_secp_roundtrip.py /tmp/secp-in.bin /tmp/secp-out.bin + +bench-secp: secp256k1-batch-mul bench_secp.py + python3 bench_secp.py ./secp256k1-batch-mul +``` + +`sm_86` covers 3090; `sm_89` covers 4090. Build a fat binary with +`-gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89` +for both fleet hosts. + +Worker auto-registration: one line, `(register-daemon! +'cuda-secp-mul-batch *binary-secp-mul*)` next to the shake +registration at `gpu-worker.lsp:299`. + +## 7. Validation plan + +**Correctness.** `test_secp_roundtrip.py` runs N=100 known scalars +(1, 2, 3, …, plus edge cases: curve order n → infinity, n-1, scalars +near 2^128, 2^256-1). Computes `k*G` on CPU via `coincurve` (or +`ecdsa.SECP256k1.generator * k`), byte-compares GPU output. Halt on +first mismatch. Cross-check at N=10 against openssl `EC_POINT_mul` to +catch a coincurve defect masking a kernel defect. + +**Performance.** `bench_secp.py` sweeps N ∈ {1k, 10k, 100k, 1M}, +records `wire_ms`, `kernel_ms` (extend response to carry `u32 +kernel_us` after `n`), `gpu/cpu` ratio. CPU baseline: in-process +`coincurve` loop. Median over 20 runs. Format mirrors +`bench_tiers.py:174` so the summary table renders unchanged. + +Expected: 3090 ~3 Gkeys/s, 4090 ~6.9 Gkeys/s. CPU baseline ~50 +kkeys/s single-thread coincurve → ~60,000× per core, ~5,000× vs +12-thread. Below 1,000× signals a defect — most likely PCIe transfer +saturating (1M × 64 B = 64 MB out, ~2 ms at PCIe 4.0). + +## 8. Risks & open questions + +- **License audit.** FixedPaul declares AGPL-3.0 at repo level but + `GPU/*.h` headers may still carry GPLv3 inherited from JeanLucPons. + Audit each vendored file before commit. If a file is GPLv3-only, + our binary becomes GPLv3 (still compatible with our AGPLv3 worker + via §7 of GPLv3). +- **Endomorphism in fork.** FixedPaul claims "optimized CUDA modular + math" but lists no specifics. If the speedup is a private + `GPUMath.h` patch, vendoring upstream headers alone loses it. + Diff `FixedPaul/GPU/GPUMath.h` vs `JeanLucPons/GPU/GPUMath.h` first; + take FixedPaul's. +- **Host RAM at N=1M.** 1M × 64 B = 64 MB out, 1M × 32 B = 32 MB in. + Worker holds ~3 copies (in-portal + daemon + out-portal): peak ~300 + MB transient. Bounded on 24 GB / 80 GB hosts. +- **GRP_SIZE preservation.** Upstream's `GRP_SIZE=1024` means each + thread does 1024 keys sequentially sharing one `_ModInvGrouped` call. + A naive one-thread-per-scalar kernel loses the amortisation — ~50× + worse. New kernel launches `ceil(N/1024)` threads, each handling a + 1024-scalar group; N rounds up to multiple of 1024 with sentinel + scalar = 1, pad outputs discarded. +- **Device memory.** 1M-point request allocates ~96 MB device-side + (in/out + scratch). `cudaFree` between requests to avoid fragmentation + over 10k+ requests. Add explicit free in `process_one_bin`. + +## 9. Effort estimate + +Experienced CUDA dev, no fleet-side work counted: + +| step | days | +|---|---| +| vendor headers, audit licenses, write PROVENANCE.md | 0.5 | +| write `secp_mul_batch_kernel` reusing `_ModInvGrouped` group loop | 1.5 | +| daemon shell (stdin loop, --binary, --daemon modes) — copy shake | 0.5 | +| binary I/O, BSHK/BSHR parse + emit, host-side scalar layout | 0.5 | +| Makefile, smoke test on local nvcc (host has no GPU — compile only) | 0.5 | +| `test_secp_roundtrip.py` + coincurve oracle + openssl cross-check | 1.0 | +| `bench_secp.py` + sweeps + first 3090/4090 measurement | 1.0 | +| `gpu-worker.lsp` handler + dispatch branch + integration test | 0.5 | +| catalog promotion (live row, measured speedup) | 0.0 (fox's call) | +| **total** | **6 days** | + +Concentration: ~40% kernel-extraction (steps 1+2), ~25% validation, +~15% bench harness, ~10% daemon/wire, ~10% handler. Kernel-extraction +risk dominates; everything else mirrors shake256-fanout. + +## URLs cited + +- Catalog § A: ../CATALOG.md +- Existing shake daemon: ../shake256-fanout.cu +- Worker handler pattern: ../gpu-worker.lsp lines 62, 71, 253, 264, 299 +- Wire bench reference: ../bench_tiers.py lines 150-158, 174 +- VanitySearch original: https://github.com/JeanLucPons/VanitySearch +- VanitySearch GPU folder: https://github.com/JeanLucPons/VanitySearch/tree/master/GPU +- VanitySearch last commit: https://github.com/JeanLucPons/VanitySearch/commits/master +- Bitcrack fork: https://github.com/FixedPaul/VanitySearch-Bitcrack +- Bitcrack last commit: https://github.com/FixedPaul/VanitySearch-Bitcrack/commits/master +- gECC paper (speedup reference): https://arxiv.org/pdf/2501.03245 diff --git a/examples/cuda-fanout/plans/form-B-bignum-cgbn.md b/examples/cuda-fanout/plans/form-B-bignum-cgbn.md new file mode 100644 index 0000000..a03daca --- /dev/null +++ b/examples/cuda-fanout/plans/form-B-bignum-cgbn.md @@ -0,0 +1,240 @@ +# Form B — `cuda-bignum-cgbn` implementation plan + +You wrap NVlabs CGBN (Cooperative Groups Big Numbers) as a BSHK-speaking +daemon under our existing bend-form pattern. Output: a single CUDA +binary `cgbn-batch-worker`, a Scheme handler in `gpu-worker.lsp`, a +Makefile target, & a validation harness. Cross-cite: catalog entry +`cuda-bignum-cgbn` in `CATALOG.md` (form B, build order step 3), source +NVlabs/CGBN at https://github.com/NVlabs/CGBN, midsize-int benchmark at +https://arxiv.org/pdf/2405.14642. + +## 1. CGBN model → wire model + +CGBN groups 4/8/16/32 threads into one **instance**; each instance owns +one bignum (32-bit to 32 K-bit). The library handles cooperative-group +dispatch (https://github.com/NVlabs/CGBN/blob/master/samples/sample_01_add/add.cu); +you write a kernel templated on `cgbn_env_t` & call +`cgbn_load / cgbn_mul / cgbn_modular_inverse / cgbn_store`. + +For our 256-bit secp256k1 case: `TPI=8` (threads per instance) at 256 bits +is the published sweet spot per CGBN README. One warp (32 threads) = +4 instances. A 3090 SM holds 1024 threads = 128 instances per SM × 82 SMs += **10,496 in-flight 256-bit ops per kernel wave**. Map: one BSHK +request = one launch = N instances. We never expose CGBN's cooperative- +group primitive across our wire; we expose **op-coded batches**. + +## 2. Operation dispatcher — `cgbn-batch-worker` + +Single binary, op-coded. Op table (one byte each, room to grow): + +| op_id | mnemonic | inputs | output | kernel | +|------:|----------------|--------|--------|---------------------------------------| +| 0x01 | mod-add | a, b | a+b mod m | `cgbn_add` + `cgbn_rem` | +| 0x02 | mod-sub | a, b | a-b mod m | `cgbn_sub` + conditional `cgbn_add` | +| 0x03 | mod-mul | a, b | a*b mod m | `cgbn_mont_mul` (precomputed `m`) | +| 0x04 | mod-sqr | a | a^2 mod m | `cgbn_mont_sqr` | +| 0x05 | mod-inv | a | a^-1 mod m | `cgbn_modular_inverse` | +| 0x06 | mod-exp | a, b | a^b mod m | `cgbn_modular_power` (binary ladder) | +| 0x07 | mod-reduce | a | a mod m | `cgbn_rem` standalone | +| 0x08 | add-no-mod | a, b | a+b | `cgbn_add` straight | +| 0x09 | mul-no-mod | a, b | a*b (lift to 2 × width) | `cgbn_mul_wide` | + +Kernel dispatch via single `__global__` switch on op_id. CGBN environment +template is bit-width parameterized; you compile one specialization per +expected bit-width (256, 384, 521) & route via a runtime `switch +(bitwidth)` over the templated launchers. Default bit-widths to ship: +256, 384. Wider widths land on demand. + +## 3. Wire protocol — BSHK / BSHR for CGBN + +Request (little-endian throughout, matching `bench_tiers.py` +`make_payload_binary` in this dir): + +``` +"BSHK" (4 bytes magic) +u32 op_id +u32 bitwidth ; one of {256, 384} +u32 n_instances +u8[bitwidth/8] modulus m (omitted for op 0x08, 0x09) +u8[n * bitwidth/8] operand_a +u8[n * bitwidth/8] operand_b (omitted for 0x04, 0x05, 0x07) +``` + +Response: + +``` +"BSHR" (4 bytes magic — matches handle-binary-shake in gpu-worker.lsp) +u32 status ; 0 = ok; nonzero = error code +u32 n_instances +u8[n * width_out/8] result ; width_out = bitwidth or 2*bitwidth +``` + +For 256-bit secp256k1 case @ N=1M instances: input ~64 MB +(modulus + 2 operand arrays), output ~32 MB. CGBN documents 32K-bit max +per instance (see `cgbn.h` `BITS` template parameter +https://github.com/NVlabs/CGBN/blob/master/include/cgbn/cgbn.h); 256-bit +fits trivially. TPI=8 at 256 bits leaves no SM under-utilization on +3090/4090 because each SM still hosts 128 in-flight instances. + +## 4. `gpu-worker.lsp` handler + +You add `handle-cuda-bignum-cgbn` modeled on `handle-cuda-shake-fanout` +plus the binary path in `handle-binary-shake` already in this dir's +`gpu-worker.lsp`. Two surfaces: + +S-expression (small N, dev convenience): + +```scheme +(cgbn-batch + (op . mod-mul) + (bitwidth . 256) + (modulus . "fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f") + (a . ("..." "...")) + (b . ("..." "..."))) +``` + +Binary (production): client sends `BSHK | u32 op_id | …` blob exactly as +above. Worker writes blob to `/tmp/bend-cgbn-in-.bin`, calls daemon +`process-bin `, reads `/tmp/bend-cgbn-out-.bin`, prepends +`"BSHR"`, sends back via `wire-send-raw`. Register the daemon at startup +next to `register-daemon!` for `'cuda-shake-fanout`: + +```scheme +(define *binary-cgbn-batch* + (or (get-environment-variable "CGBN_BATCH_WORKER") + "./cgbn-batch-worker")) +(register-daemon! 'cuda-bignum-cgbn *binary-cgbn-batch*) +``` + +Dispatch in `handle-request`: `((eq? op 'cgbn-batch) (handle-cuda-bignum-cgbn args))`. + +## 5. Daemon protocol + +Inherited verbatim from `shake256-fanout.cu --daemon` (this dir): stdin +line-protocol `process-bin ` / `quit`, stdout `ready` once +CUDA context warms, `done ` / `error ` per command, `bye` +on quit. CUDA context init pays once per worker startup, never per BSHK +request — same architectural point the SHAKE form already proved. + +## 6. Build recipe + +```makefile +CGBN_DIR := vendor/CGBN +NVCC_ARCH_3090 := -arch=sm_86 +NVCC_ARCH_4090 := -arch=sm_89 +NVCC_FLAGS := -O3 -I$(CGBN_DIR)/include --extended-lambda + +cgbn-batch-worker: cgbn-batch-worker.cu $(CGBN_DIR)/include/cgbn/cgbn.h + nvcc $(NVCC_FLAGS) $(NVCC_ARCH_3090) -gencode=arch=compute_89,code=sm_89 \ + -o $@ cgbn-batch-worker.cu -lgmp + +vendor/CGBN/include/cgbn/cgbn.h: + git submodule add https://github.com/NVlabs/CGBN vendor/CGBN + +cgbn-bench: cgbn-batch-worker + python3 bench_cgbn.py --small 1000 --medium 100000 --huge 1000000 + +cgbn-test: cgbn-batch-worker + python3 test_cgbn_known_answers.py +``` + +CGBN ships as header-only inside `include/cgbn/` — no separate +`libcgbn.so` build. Vendor the headers; do not pull the full repo into +our build graph. License: BSD 3-Clause per https://github.com/NVlabs/CGBN/blob/master/LICENSE +— compatible with our AGPLv3 binary distribution (CGBN headers ship +permissively, our wrapper binary stays AGPLv3). + +## 7. Validation plan + +Three layers, cheapest first: + +1. **openssl bignum CLI** for op 0x03 mod-mul on tiny operands. Drive + from `test_cgbn_known_answers.py`: pass N=32 hand-picked triples + (a, b, m, expected = a*b mod m). Sanity, not throughput. +2. **python gmpy2** for batches at N ∈ {1k, 10k, 100k}. Generate + random a, b ∈ [0, m); compute reference on host (`gmpy2.f_mod(a*b, m)`), + compare byte-for-byte against GPU result. Cover ops 0x01, 0x03, 0x04, + 0x05, 0x06. gmpy2 docs: https://gmpy2.readthedocs.io. +3. **secp256k1 round-trip**: m = secp256k1 field prime + `0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f`, + load N random 256-bit a, b; compute a*b mod m through CGBN; verify + against Python `coincurve` reference (https://github.com/ofek/coincurve). + Halt promotion if any byte mismatches. + +Cross-tier portal validation: identical BSHK input bytes to Python, +C tree-walker, asm-gc workers via our `bench_tiers.py` shape (already +binary-wire-aware) must produce byte-identical BSHR responses. Mismatch +on any tier halts promotion per `ecdsa/CLAUDE.md` cross-tier rule. + +## 8. Performance projection + +Sources: midsize-int benchmark https://arxiv.org/pdf/2405.14642 (Table 3 +reports ~100×+ on 256-bit dense mul vs Xeon-20c GMP+OpenMP on V100); +NVlabs CGBN README claims 5-10× over GMP per-op on Volta. A 3090 is +~3× V100 throughput on dense integer work (per +https://www.nvidia.com/en-us/data-center/a100/ comparison & 3090 +whitepaper). + +Rough numbers, to be confirmed on hardware: + +- **mod-mul 256-bit**: ~3-5 Gops/s on a 3090 (TPI=8 at warp-saturated + occupancy, kernel-only). 4090 ~1.6× that. Reference: midsize-int paper + Fig 7, scaled by 3090/V100 fp32 ratio. +- **mod-inv 256-bit** (CGBN's binary GCD path): ~50-100 Mops/s on 3090. + Bernstein-Yang batched variant (form E in catalog) wins ~3-5× over + this once we add it. +- **mod-exp 256-bit** (256-bit ladder): ~10-30 Mops/s on 3090. +- **wire-overhead crossover**: BSHK round-trip pays ~3 ms median per call + per `bench_tiers.py` binary-mode at small N (this dir's existing + measurements). At 3 Gops/s kernel rate, you break even with CPU GMP + (~5 Mops/s single-thread) around **N ≈ 15,000 instances per BSHK + call**. Stay above N=100k for clean 100×. + +## 9. Risks & open questions + +- **CGBN maintenance.** Last public commit on https://github.com/NVlabs/CGBN + is 2021-era. NVlabs has not formally archived it. You must verify + it compiles under CUDA 12.x on Ampere & Ada (sm_86, sm_89). If not, + patch `cgbn.h` toolchain checks ourselves & vendor the patched copy. +- **License.** BSD 3-Clause per https://github.com/NVlabs/CGBN/blob/master/LICENSE. + No AGPL incompatibility for our binary distribution. +- **sm_86 / sm_89 support.** CGBN documents Volta + Turing as primary + targets. Ampere (sm_86) cooperative-groups API is a superset, so the + cooperative-thread-array primitives CGBN calls remain valid. Open + risk: warp-shuffle reductions in `cgbn_add` may need width + adjustment for newer warp-scheduler behavior. Mitigation: ship + validation suite first; any divergence shows up against gmpy2 ref. +- **Under-utilization at 256-bit / TPI=8.** Each warp owns 4 + independent instances. At N=1M & 82 SMs × 128 in-flight, you get + ~95 kernel waves on a 3090. Each wave is ~µs; total kernel ~100 µs. + Memory bandwidth (32 MB input, 32 MB output → 64 MB at 936 GB/s = + ~70 µs) becomes the floor. Kernel & PCIe transfer are co-dominant. +- **PCIe transfer budget.** 64 MB host→device + 32 MB back via PCIe + Gen4 x16 (32 GB/s) = ~3 ms. Already comparable to wire overhead; + pinned-host memory & overlap with kernel via streams cuts this in + half. + +## 10. Effort estimate + +You should land this in **3-5 days** end-to-end as an experienced CUDA +developer. Breakdown: + +- Day 1: vendor CGBN, build sample_01_add unmodified on 3090 host, + confirm sm_86 compatibility, build baseline `cgbn-batch-worker.cu` + with op 0x03 mod-mul only. +- Day 2: daemon mode (clone shake256-fanout `--daemon` skeleton), add + ops 0x01, 0x04, 0x05, wire validation against gmpy2 at N=1k, 10k. +- Day 3: gpu-worker.lsp handler, BSHK dispatch, cross-tier validation + (Python + C + asm via `bench_tiers.py`-style harness), portal test + for secp256k1 mod-mul. +- Day 4: remaining ops (0x02, 0x06, 0x07, 0x08, 0x09), bench_cgbn.py + driver covering all ops at N ∈ {1k, 100k, 1M}, telemetry lines in + Scheme handler (wall-ms / cpu-ms / gpu-ms / gpu/cpu ratio). +- Day 5 (buffer): polish, README, CATALOG.md row promotion to `live`, + Makefile install/test targets, runs/lumbda-cgbn-001/ baseline + artifacts. + +Lower than form A (`cuda-secp256k1-batched-mul`) since CGBN is a +**direct API consumption** — no extraction effort from a research +codebase, no curve-specific Montgomery tuning to port. Form A risks 7-10 +days; form B caps at 5. diff --git a/examples/cuda-fanout/plans/form-D-clifford-stabilizer.md b/examples/cuda-fanout/plans/form-D-clifford-stabilizer.md new file mode 100644 index 0000000..1d3239f --- /dev/null +++ b/examples/cuda-fanout/plans/form-D-clifford-stabilizer.md @@ -0,0 +1,256 @@ +# Form D — `cuda-clifford-stabilizer` implementation plan + +You will not build this binary yet. This document gives you (fox) a +fully-specified runway you can hand to a CUDA + quantum-sim engineer or +to a future blackops session. Catalog entry lives at +`CATALOG.md` (Form D). Numbers cited here all carry URLs. + +## 1. Reference selection — what you can fork vs what you must write + +Three candidates surveyed 2026-06-05: + +- **STABSim** (Garner, Liu, Wang, Stein, Li — UW lead, seangarn@uw.edu) + arXiv 2507.03092 https://arxiv.org/abs/2507.03092 . Warp-level CUDA + primitives, scales better than CPU on QEC workloads. **No public + repo.** GitHub search for `stabsim cuda` returns zero results + https://github.com/search?q=stabsim+cuda&type=repositories . Treat as + reference design only. Email seangarn@uw.edu before sinking weeks + into a clean-room port. Our CATALOG's 186× figure derives from the + equivalence-checking benchmark — re-extract once you have full PDF + in hand. + +- **Qimax** (Vu Tuan Hai et al, NAIST) arXiv 2505.03307 + https://arxiv.org/abs/2505.03307 . Extended-stabilizer on multi-core + / GPU; compares vs Qiskit & PennyLane, **not vs Stim**. No GitHub + on arXiv landing. A re-implementation matching Qimax does not yet + beat our CPU floor. + +- **Clean-room Aaronson-Gottesman tableau on CUDA** + https://arxiv.org/abs/quant-ph/0406196 . ~500 LoC CUDA for a binary + symplectic tableau (2n × 2n bits + sign), gates H/S/CNOT/X/Z + M — + every Clifford gate boils down to row-XOR + sign twiddle. **Our + recommended target** because (a) you own our wire format, (b) STABSim + & Qimax did not release source, (c) Aaronson-Gottesman runs at known + O(n²) per gate, maps cleanly onto a CUDA warp. + +Recommendation: write our own, citing STABSim's per-warp parallelism +as inspiration. Re-evaluate STABSim source if UW responds. + +## 2. Circuit format — translate, do not adopt + +Our circuit lives as Phase B lumbda Scheme & emits to QECCOPS1 +`ops.bin` (op kinds in `ecdsa/cuda/sim.h:21-38`). Stim's `.stim` text +format is the de-facto wire for stabilizer circuits +(https://github.com/quantumlib/Stim , gate names `H S CX CZ SWAP R M +TICK`). + +Pick **(b)** — `demo_stab_sim_ops.cu` accepts our existing QECCOPS1 +`ops.bin` natively, treats Clifford ops as primary. Reasons: + +- `ops.bin` already exists, generated by every lumbda tier; a `.stim` + translator adds a redundant emission path. +- Op kinds `OP_X, OP_Z, OP_CX, OP_CZ, OP_SWAP, OP_R, OP_HMR` map 1:1 + onto Aaronson-Gottesman tableau ops. `OP_CCX, OP_CCZ` route to our + Toffoli handler (§3). +- Cross-validation against Stim CPU still happens via a one-shot + `emit-stim-circuit.lsp` test utility under `ecdsa/tests/` — small + circuits only, byte-equivalence spot checks, never our production + wire. + +## 3. Toffoli problem — pick option (b), score symbolically + +Stabilizer formalism handles Clifford only. Our Roetteler 12-step +point-add (`ecdsa/lumbda/point-add-real.lsp`) carries both Clifford & +Toffoli. Our CUDA sim already tracks them separately: +`ecdsa/cuda/sim_gpu.cu:155` counts `OP_CCZ / OP_CCX` as Toffoli, all +others as Clifford. + +Three options surfaced in our brief; you pick **(b)**: + +- **(a) Decompose each CCX into 7T.** Inflates op count ~7× per + Toffoli, requires magic-state tracking. Wrong for a scoring sim. +- **(b) Run Clifford fragment only on GPU; fold Toffoli count from + circuit structure.** Our score is a *count*, not a measurement + outcome. Toffoli ops are symbolic — you tally them at circuit-emit + time, skip them in our stabilizer kernel. GPU lane handles only + Clifford ops; CPU emits Σ-Toffoli alongside as a static circuit + property. **This is sufficient for our metric.** Speedup numbers in + CATALOG (186× equivalence-checking) come from Clifford-only + benchmarks, so this matches what published work measures. +- **(c) Extended stabilizer / magic-state sim.** Qimax territory. + Slower & more code; only revisit if our scoring metric changes to + require sampled outcomes through Toffoli gates. + +Phase B point-add Toffoli fraction (from existing kernel counters): +roughly half of all gates per circuit are CCX / CCZ at current ancilla +widths. Confirm exact ratio by running `demo_ops` against +`point-add.bin` — GPU counters at `sim_gpu.cu:238-239` emit +`cliff_out` & `toff_out` per batch. Record this ratio before you sink +time into Form D: if Toffoli > 90% of ops, a Clifford-only sim +accelerates only 10% of work & a 186× fragment win yields 1.1× overall +(Amdahl). + +## 4. Wire protocol — BSHK reuse + +Reuse the BSHK / BSHR framing already wired through `gpu-worker.lsp` +(handler `handle-binary-shake` at lines 253-267). Layout: + +- Request: `BSHK | u32 op_id=0x02 | u32 n_qubits | u32 n_shots | + u32 circuit_blob_size | circuit_blob[N]` +- circuit_blob: our existing QECCOPS1 `ops.bin` bytes, no translation. +- Response: `BSHR | u32 status | u32 n_shots | + samples[n_shots * ceil(n_qubits/8)]` bit-packed measurement outcomes. + +Confirm match: STABSim & Qimax did not publish wire schemas, so you +own this protocol fully — design freedom, no compatibility tax. +`op_id` discriminator lets the daemon serve both `cuda-sim-ops-bin` +(kickmix, `op_id=0x01`) and `cuda-clifford-stabilizer` +(`op_id=0x02`) without renegotiating magic bytes. + +## 5. `gpu-worker.lsp` handler sketch + +Mirror `handle-cuda-sim-ops-bin` (lines 187-233). Pseudocode: + +```scheme +(define *binary-stab-sim* + (or (get-environment-variable "STAB_SIM") + "/home/fox/git/www.foxhop.net/ecdsa/cuda/demo_stab_sim_ops")) + +(define (handle-cuda-clifford-stabilizer args) + (let* ((ops-path (car args)) + (n-qubits (car (cdr args))) + (n-shots (car (cdr (cdr args)))) + (portal (gensym-path "/tmp/bend-stab" ".portal")) + (t-start (current-time-ms)) + (argv (list ops-path + (number->string n-qubits) + (number->string n-shots) + "--portal" portal)) + (daemon (cdr (assoc 'cuda-clifford-stabilizer *daemons*)))) + (daemon-process daemon ops-path portal #t) + ;; portal carries (cuda-stab-result (samples blob) (sigma-clifford k) + ;; (sigma-toffoli k) (timing-ms ...)) + ;; same shape as cuda-sim-ops-bin, parse via existing portal-find-number. + ...)) +``` + +Dispatch line in `handle-request`: + +```scheme +((eq? op 'cuda-clifford-stabilizer) + (handle-cuda-clifford-stabilizer args)) +``` + +Register daemon at startup: + +```scheme +(register-daemon! 'cuda-clifford-stabilizer *binary-stab-sim*) +``` + +## 6. Daemon protocol + +Mirror `shake256-fanout --daemon` (called via `start-daemon` line 62). +On stdin one line per request, on stdout one line per response. +Two modes — text (`process`) for dev, binary (`process-bin`) for hot +path: + +``` +> process-bin /tmp/in.bin /tmp/out.bin\n +< done\n (or `error \n`) +``` + +On start, daemon prints `ready\n` after CUDA context init & +tableau-row buffer allocation. CUDA context warm-up pays once per +worker lifetime. + +## 7. Build recipe + +Add to `ecdsa/cuda/Makefile`: + +```make +demo_stab_sim_ops: stab_sim_gpu.o ops_loader.o main_stab_sim.o + nvcc -O3 -arch=sm_80 -o $@ $^ + +stab_sim_gpu.o: stab_sim_gpu.cu + nvcc -O3 -arch=sm_80 -c $< -o $@ +``` + +`arch=sm_80` covers RTX 3090 (sm_86) & RTX 4090 (sm_89) via JIT; +hard-code at deploy time if you measure a perf loss. + +Source files to write fresh: + +- `stab_sim_gpu.cu` — Aaronson-Gottesman tableau kernel, one CUDA + block per circuit (= per candidate), threads cooperate on the + tableau rows. Reference: quant-ph/0406196 §3 update rules. + https://arxiv.org/abs/quant-ph/0406196 +- `main_stab_sim.c` — daemon loop, ops.bin load, dispatch. +- Reuse `ops_loader.c` from existing simulator (already parses + QECCOPS1). + +## 8. Validation plan + +Two axes: + +- **Σ-Clifford equivalence vs Stim CPU.** Generate a 16-qubit + toy circuit covering every supported gate; run our sim on + CPU+GPU; emit the same circuit through + `emit-stim-circuit.lsp` (one-shot translator, test-only); run + Stim CPU via `stim sample --in toy.stim` (Stim install on + permacomputer fleet, not on host). Σ-Clifford & measurement + outcomes must match byte-for-byte over 1024 shots seeded + identically. + +- **Per-candidate axis flip benchmark.** Run N independent candidate + circuits as N GPU blocks (= N lanes). Compare: + - existing `demo_ops` per-shot: N candidates × 8192 shots each = 8192N + threads, one thread per shot of a single candidate; + - new `demo_stab_sim_ops` per-candidate: N candidates × 1 lane each, + each lane runs its candidate to completion serially in tableau. + At N=256 candidates expect Stim CPU to take ~2 s/candidate (512 s + total); Form D target on RTX 3090 ~2.7 s total based on a 186× + scaling factor. Record actual numbers; if our ratio < 30× you + surface our crossover N on CATALOG. + +Bench target: a 16-qubit, 1000-Clifford-gate test circuit (small +enough to avoid memory blowup on first run; large enough that +kernel-launch overhead amortizes). + +## 9. Risks & open questions + +- **STABSim source.** No public repo. Either email seangarn@uw.edu & + wait, or build clean-room from quant-ph/0406196. Clean-room is our + default path. Risk: clean-room reaches Aaronson-Gottesman baseline, + not STABSim's published wins (likely 10-30× over CPU, not 186×). +- **Per-candidate axis vs per-shot axis.** STABSim hints at warp-level + parallelism without specifying whether warps map to circuits or + shots. Read PDF §III before committing — our axis-flip thesis + depends on per-circuit being the right grain. +- **Toffoli ratio.** If point-add ends up >90% Toffoli, Form D wins + only 1.1× overall (Amdahl). Measure Toffoli fraction before build. +- **Memory.** A 16-qubit tableau is ~1 KB per candidate; a 256-qubit + point-add fragment is 128 KB per candidate. N=1024 × 128 KB = 128 + MB GPU state — fits. Cap N×state at 8 GB on 3090. +- **Stim install footprint.** Validation needs Stim on fleet; + `pip install stim`. Test-only dep, not a worker dep. + +## 10. Effort estimate + +For one experienced CUDA + quantum-sim engineer (assumes prior +Aaronson-Gottesman familiarity): + +- Day 1: read STABSim PDF, read Aaronson-Gottesman paper, run + existing `demo_ops` to measure Toffoli ratio, decide go/no-go. +- Days 2-4: `stab_sim_gpu.cu` kernel — tableau rep, X/Z/H/S/CX/M + update rules, warp-level row-XOR primitive. +- Day 5: `main_stab_sim.c` daemon loop, BSHK plumbing. +- Day 6: `gpu-worker.lsp` handler, daemon registration, smoke run. +- Days 7-8: validation vs Stim CPU on 16-qubit toy, fix divergence. +- Days 9-10: per-candidate benchmark, record numbers, update + CATALOG. + +**Total: ~10 working days.** Halve to 5 if STABSim source surfaces; +double to 20 if Toffoli folding turns out insufficient & you must add +extended-stabilizer (option c) support. + +You decide go/no-go after Day 1 measurement.