diff --git a/examples/cuda-fanout/bench_tiers.py b/examples/cuda-fanout/bench_tiers.py index 1e81980..4e427e4 100644 --- a/examples/cuda-fanout/bench_tiers.py +++ b/examples/cuda-fanout/bench_tiers.py @@ -170,16 +170,19 @@ def main(): # Write a launch.lsp the worker process will load with open(os.path.join(ROOT, "launch.lsp"), "w") as f: f.write('(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n') - # asm tier doesn't auto-bind *argv*; pre-define it here so - # gpu-worker.lsp's parse-port-arg falls through to *default-port*. + # asm tier doesn't auto-bind *argv* OR ship write-to-string; + # pre-define the former and shim the latter via wts-shim. with open(os.path.join(ROOT, "launch-asm.lsp"), "w") as f: f.write('(define *argv* (quote ()))\n' + '(load "write-to-string-shim.lsp")\n' '(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n') configs = { - "small (3×16B)": (args.small_n, 3, 16), - "small (100×16B)": (args.small_n, 100, 16), - "medium (1000×16B)": (args.medium_n, 1000, 16), + "small (3 × 16 B)": (args.small_n, 3, 16), + "small (100 × 16 B)": (args.small_n, 100, 16), + "medium (1000 × 16 B)": (args.medium_n, 1000, 16), + "huge (100k × 16 B)": (3, 100_000, 16), + "huge (1M × 16 B)": (2, 1_000_000, 16), } tiers = ["python", "c"] diff --git a/examples/cuda-fanout/write-to-string-shim.lsp b/examples/cuda-fanout/write-to-string-shim.lsp new file mode 100644 index 0000000..ad248ee --- /dev/null +++ b/examples/cuda-fanout/write-to-string-shim.lsp @@ -0,0 +1,33 @@ +;;; write-to-string-shim.lsp — portable write-to-string for tiers +;;; without the native builtin (asm). +;;; +;;; Python & C tiers ship `write-to-string` as a builtin; do NOT +;;; load this file on those tiers, you'll shadow the fast native +;;; version with a slower Scheme one. +;;; +;;; asm tier launch script (see launch-asm.lsp) loads this first +;;; so wire.lsp's wire-send can call write-to-string unconditionally. + +(define (write-to-string x) + (cond + ((null? x) "()") + ((pair? x) + (string-append "(" + (write-to-string (car x)) + (sexp->list-tail (cdr x)))) + ((symbol? x) (symbol->string x)) + ((number? x) (number->string x)) + ((string? x) (string-append "\"" x "\"")) + ((eq? x #t) "#t") + ((eq? x #f) "#f") + (else "?"))) + +(define (sexp->list-tail rest) + (cond + ((null? rest) ")") + ((pair? rest) + (string-append " " + (write-to-string (car rest)) + (sexp->list-tail (cdr rest)))) + (else + (string-append " . " (write-to-string rest) ")")))) diff --git a/www/index.html b/www/index.html index e50ff96..c90d09b 100644 --- a/www/index.html +++ b/www/index.html @@ -89,16 +89,19 @@ make gpu-worker LUMBDA=asm # smallest footprint

On a single RTX 3090 with a warm daemon, fan-out matched hashlib.shake_256 byte-for-byte and won by 1.5–10× across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.

Tier choice for the worker host

-

The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of 20 calls per workload, daemon warm):

+

The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of N calls per workload, daemon warm, both client & worker on localhost):

- - - + + + + +
workloadPython tierC tierC win
small (3 × 16 B)1.27 ms0.14 ms9.1×
small (100 × 16 B)3.46 ms0.41 ms8.4×
medium (1000 × 16 B)23.51 ms2.67 ms8.8×
small (3 × 16 B)1.16 ms0.14 ms8.3×
small (100 × 16 B)3.39 ms0.42 ms8.1×
medium (1000 × 16 B)23.26 ms2.67 ms8.7×
huge (100k × 16 B)2,220 ms(wire stalls)n/a
huge (1M × 16 B)24,338 ms(wire stalls)n/a
-

C tier wins by ~9× across the grid — consistent with the ratio between Python's S-expression parser and the C tier's reader. At very heavy workloads (where the kernel itself takes seconds) the tier choice becomes noise; at light workloads (where bend stays local anyway) the tier choice doesn't matter either. The middle ground is where C tier earns its default.

+

At small & medium sizes, C tier wins by ~9× — that's the ratio between Python's S-expression parser and the C tier's reader. At huge sizes, both tiers stall on the wire, not the kernel. The shake256-fanout CUDA kernel on this 3090 finishes 1M × 16 B in ~47 ms — three orders of magnitude faster than the Python tier's 24 s end-to-end. The S-expression hex framing in wire.lsp is O(n²) at huge payload sizes because recv-exact accumulates chunks through string-append in a loop. C tier fails outright at this size; Python tier just suffers.

+

What this means: at small inputs, tier orchestration matters & C wins. At huge inputs, wire framing dominates regardless of tier — the right fix is a binary wire mode between client & worker, parallel to the binary portal mode the daemon already supports between worker & leaf. Until that lands, keep payloads under ~10 MB or call cuda-shake-fanout against pre-staged files instead of inline lists.

The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written pipe2 + fork + execve syscalls — no libc anywhere on the chain.

See examples/cuda-fanout/ for the wire contract, daemon protocol, bench data, and per-tier integration sketch.