From f8705f2e0d07aa7292e9f76048dfd4c6622d22e5 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 5 Jun 2026 14:14:56 -0400 Subject: [PATCH] cuda-fanout plans: Form D Option 3 (ops.bin packing) results MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QECCOPS2 packed-op format landed in foxhop ecdsa/cuda/ at commit 90484ca. 28 → 24 B per op in VRAM, 56 → 24 B on disk (2.33× shrink). n_batches 16/64/128 on RTX 3090: ~7% kernel speedup, byte-identical CPU vs unpacked-GPU vs packed-GPU. Bandwidth-bound diagnosis from form-D-build-progress.md §4 stands — per-shot state traffic (~85× larger than op stream) owns the 1.07× ceiling. Per-candidate axis- flip (sim_gpu_axis.cu, foxhop 1f7ac9d) remains the open lever; this ~7% stacks on top. --- .../plans/form-D-packing-RESULTS.md | 165 ++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 examples/cuda-fanout/plans/form-D-packing-RESULTS.md diff --git a/examples/cuda-fanout/plans/form-D-packing-RESULTS.md b/examples/cuda-fanout/plans/form-D-packing-RESULTS.md new file mode 100644 index 0000000..66d3fd0 --- /dev/null +++ b/examples/cuda-fanout/plans/form-D-packing-RESULTS.md @@ -0,0 +1,165 @@ +# Form D — Option 3 (ops.bin packing) results, 2026-06-05 + +Plan referent: `form-D-build-progress.md` § Recommendation, option 3 +("ops.bin packing — reduce per-op bytes from 56 → 16 to halve global- +memory traffic"). + +Status: **landed, measured, & byte-identical**. Speedup matches the +build-progress diagnosis (per-shot state, not per-op stream, owns our +ceiling); does NOT match the task-author's 3.5× hypothesis. + +## Findings up front + +| n_batches | unpacked kernel ms | packed kernel ms | packed speedup | +|-----------|-------------------:|-----------------:|---------------:| +| 16 | 5842.2 | 5427.9 | **1.076×** | +| 64 | 6225.9 | 5830.2 | **1.068×** | +| 128 | 6604.2 | 6201.0 | **1.065×** | + +- Byte-identity: **PASS** at every n_batches across CPU vs unpacked-GPU + vs packed-GPU. Phase, clifford & toffoli counters, full qubit & bit + state vectors — all match. +- Diagnosis from `form-D-build-progress.md` § 4 ("per-shot striped + state owns the 1.07× ceiling") **stands**. Packing the op stream + does not flip the bottleneck. + +## Why 1.07× & not 3.5× + +The task-author's number was anchored to "56 B → 16 B = 3.5×" on the +**on-disk** layout. Two corrections found in code review before the +kernel write: + +1. `sim.h` Op struct is **already 28 B** in VRAM (narrowed u64 → u32 + in `ops_loader.c::load_ops_bin`). The 56 B figure is QECCOPS1 wire + format only. Maximum-feasible VRAM reduction is therefore + 28 → 24 B = 1.17×, not 3.5×. +2. Op fetches `Op op = ops[i]` warp-broadcast: every thread in a warp + reads the same address each iteration, so L1/L2 already absorbs + per-op traffic. The diverging reads are per-shot state, i.e. + `qubits[op.q_target * n_batches + b]`, where every thread hits a + distinct address. + +Result: packing op records reduces op-stream traffic by 14% but barely +moves the kernel because op fetches were never the bottleneck. The +measured 1.07× tracks the upper bound nearly exactly — packing is +working, the bottleneck is elsewhere. + +## Confirmed slot-index ranges on /tmp/ops.bin + +Walked all 12,788,119 ops via `convert_ops_bin`: + +- `n_ops` : 12,788,119 +- `num_qubits` : 1,355 (max index 1,354 — fits u11) +- `num_bits` : 1,300,679 (max index 1,300,678 — fits u21) +- `max_qubit` : 1,354 (well under u32 = 4.29 billion) +- `max_bit` : 1,300,678 (also well under u32) + +**Conclusion: u32 slot indices are safe for this circuit & for any +realistic point-add-toy descendant.** The packing scheme is sound on +index width. + +The original task-spec's proposal of `u16 c_target_or_condition_*` +would **fail** — max bit 1,300,678 > 65,535. + +## QECCOPS2 format spec (what landed) + +``` +magic : "QECCOPS2" (8 bytes) +n_ops : u64 LE (8 bytes) +ops : n_ops × 24 bytes: + [0] u8 kind (0..=17 validated) + [1] u8 flags bit0=has_q_c2, bit1=has_q_c1, + bit2=has_q_t, bit3=has_c_t, + bit4=has_c_cond. bits 5-7 reserved + [2-3] u16 reserved (zero, alignment) + [4-7] u32 q_control2 (0 if !has_q_c2) + [8-11] u32 q_control1 + [12-15] u32 q_target + [16-19] u32 c_target + [20-23] u32 c_condition +``` + +- `r_target` is **omitted**. The kernel (`sim_gpu.cu`) never reads it, + so we drop it from VRAM-resident state. Upstream QECCOPS1 still + carries it for round-trip with `eval_circuit.rs`. +- File-size ratio at n_ops = 12.8M: + - QECCOPS1 : 716,134,680 B = 716 MB + - QECCOPS2 : 306,914,872 B = 307 MB + - Ratio : **2.33× smaller on disk**. + +## VRAM resident bandwidth — packed vs unpacked + +Per-shot state remains the dominant traffic. Concrete numbers from the +n_batches=128 run: + +- Op stream traffic (warp-broadcast, mostly L2/L1): 12.8 M × 24 B = + 307 MB per shot (packed) vs 12.8 M × 28 B = 358 MB (unpacked) — a + 51 MB delta which the cache absorbs. +- State traffic (per-shot, all threads diverge): each op reads ~2 u64 + state lanes × n_batches threads = 16 × 128 = 2048 B per op = + 12.8 M × 2048 B = 26 GB streamed per shot at n_batches=128. + +The per-shot state stream is ~85× larger than the op stream. Reducing +the op stream by 14% delivers ~14% × (op-fraction of total) speedup. +Op fraction ≈ 358 MB / (358 MB + 26 GB) ≈ 1.4%. So packing's +theoretical max impact at this n_batches is ~ 1.014×. Measured 1.065× +exceeds that, suggesting cache-line alignment & smaller register +spill (the kernel pulls a smaller struct into registers per op) also +contribute marginally. + +## Did the bandwidth-bound diagnosis flip to compute-bound? + +**No.** Per-shot state traffic still dominates. To flip the diagnosis +we would need to (a) packed-bit-state storage (8 shots per byte +instead of 64 shots per u64 — wrong direction, that's the same lane +width); or (b) change the parallelism axis from per-shot to per- +candidate, which is the **option 1** refactor in `form-D-build- +progress.md`. That refactor remains the open lever for moving the +1.07× ceiling. Op packing is a clean ~7% boost that stacks on top of +whichever axis-flip lands next. + +## Memory safety + +- Per-side state at n_batches=128: 3.5 GB device + 3.5 GB host. +- Two device contexts simultaneous (unpacked + packed) — peak ≈ + 10.5 GB. Stayed under 16 GB cap. +- 3090 has 23561 MB free; we used ~10.5 GB; no OOM, no QEMU envelope + needed for this CUDA workload (3090-ai host runs no concurrent + search loop while we benchmark). + +## Files landed under ecdsa/cuda/ + +- `sim_packed.h` — `PackedOp` (24 B) struct + QECCOPS2 magic / flags +- `ops_loader_v2.c` — `load_ops_bin_v2` (→ unpacked Op) + + `load_packed_ops_bin_v2` (→ PackedOp) +- `ops_loader_v2_writer.c` — `convert_ops_bin ` (QECCOPS1 → + QECCOPS2) +- `sim_gpu_packed.cu` — `sim_kernel_packed` reading PackedOp directly +- `main_packed_demo.c` — drives unpacked & packed kernels on the same + circuit + CPU ground-truth, reports byte-identity + timings +- `Makefile` — `demo_packed`, `convert_ops_bin`, `remote-packed` + targets + +`ops_loader.c`, `sim_cpu.c`, `sim_gpu.cu`, `sim.h`, & the lumbda +`emit-ops-bin.lsp` walker remain **untouched**. Upstream contract is +preserved; QECCOPS2 lives only under our cuda/ tree. + +## Follow-on (deferred) + +- `emit-ops-bin-v2.lsp` (lumbda walker emitting QECCOPS2 natively) — + defer until we see a workload where the ~14% on-disk shrink matters + more than maintaining a single emit path. +- Per-candidate axis-flip refactor of `sim_gpu.cu` (build-progress doc + option 1) — that is the lever that can actually move 1.07× → useful. + Packing stacks on top once the axis lands. +- Bend / GPU-worker `gpu-worker.lsp` integration with QECCOPS2 — only + worth wiring once a search loop wants the 7% kernel boost + consistently. + +## Conclusion + +Packing landed clean. Byte-identical. ~7% kernel speedup at every +batch size, ~2.33× on-disk shrink. The 3.5× headline hypothesis was +anchored to the on-disk format, not the VRAM-resident format the +kernel actually consumes; it does not apply. The build-progress +diagnosis (per-shot state owns the ceiling) survives unchallenged.