From 373c8ea0d3dcbde50f650217b193da744287b655 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 5 Jun 2026 08:50:36 -0400 Subject: [PATCH] bench + www: honest huge-workload numbers + wire bottleneck found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added a write-to-string-shim.lsp for asm tier (which lacks the native builtin); asm launch script pre-defines *argv* + loads the shim so wire.lsp's wire-send finds a write-to-string definition. Python/C tiers keep the native builtin — the shim is opt-in. Bench extended with two huge workloads (100k × 16 B, 1M × 16 B). Real numbers, 3090-ai, daemon warm, both ends localhost: workload Python C tier C win small (3 × 16 B) 1.16 ms 0.14 ms 8.3× small (100 × 16 B) 3.39 ms 0.42 ms 8.1× medium (1000 × 16 B) 23.26 ms 2.67 ms 8.7× huge (100k × 16 B) 2,220 ms STALL n/a huge (1M × 16 B) 24,338 ms STALL n/a THE FINDING: at huge sizes, the bottleneck is the S-expression text wire format, not the CUDA kernel. shake256-fanout finishes 1M × 16B in ~47 ms; the Python worker takes 24 SECONDS end-to-end because wire.lsp's recv-exact accumulates chunks via string-append in a loop — O(n²) at multi-MB payload sizes. C tier fails outright. The right fix is binary wire framing between client + worker, parallel to the binary portal format the daemon + leaf already use. That's a separate piece of work; today's Web page edit calls it out honestly so visitors know when bend is the right tool. asm tier worker hosting still has process-management quirks (doesn't survive nohup detachment in this environment); bench ships with --skip-asm by default in this run. --- examples/cuda-fanout/bench_tiers.py | 13 +++++--- examples/cuda-fanout/write-to-string-shim.lsp | 33 +++++++++++++++++++ www/index.html | 13 +++++--- 3 files changed, 49 insertions(+), 10 deletions(-) create mode 100644 examples/cuda-fanout/write-to-string-shim.lsp diff --git a/examples/cuda-fanout/bench_tiers.py b/examples/cuda-fanout/bench_tiers.py index 1e81980..4e427e4 100644 --- a/examples/cuda-fanout/bench_tiers.py +++ b/examples/cuda-fanout/bench_tiers.py @@ -170,16 +170,19 @@ def main(): # Write a launch.lsp the worker process will load with open(os.path.join(ROOT, "launch.lsp"), "w") as f: f.write('(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n') - # asm tier doesn't auto-bind *argv*; pre-define it here so - # gpu-worker.lsp's parse-port-arg falls through to *default-port*. + # asm tier doesn't auto-bind *argv* OR ship write-to-string; + # pre-define the former and shim the latter via wts-shim. with open(os.path.join(ROOT, "launch-asm.lsp"), "w") as f: f.write('(define *argv* (quote ()))\n' + '(load "write-to-string-shim.lsp")\n' '(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n') configs = { - "small (3×16B)": (args.small_n, 3, 16), - "small (100×16B)": (args.small_n, 100, 16), - "medium (1000×16B)": (args.medium_n, 1000, 16), + "small (3 × 16 B)": (args.small_n, 3, 16), + "small (100 × 16 B)": (args.small_n, 100, 16), + "medium (1000 × 16 B)": (args.medium_n, 1000, 16), + "huge (100k × 16 B)": (3, 100_000, 16), + "huge (1M × 16 B)": (2, 1_000_000, 16), } tiers = ["python", "c"] diff --git a/examples/cuda-fanout/write-to-string-shim.lsp b/examples/cuda-fanout/write-to-string-shim.lsp new file mode 100644 index 0000000..ad248ee --- /dev/null +++ b/examples/cuda-fanout/write-to-string-shim.lsp @@ -0,0 +1,33 @@ +;;; write-to-string-shim.lsp — portable write-to-string for tiers +;;; without the native builtin (asm). +;;; +;;; Python & C tiers ship `write-to-string` as a builtin; do NOT +;;; load this file on those tiers, you'll shadow the fast native +;;; version with a slower Scheme one. +;;; +;;; asm tier launch script (see launch-asm.lsp) loads this first +;;; so wire.lsp's wire-send can call write-to-string unconditionally. + +(define (write-to-string x) + (cond + ((null? x) "()") + ((pair? x) + (string-append "(" + (write-to-string (car x)) + (sexp->list-tail (cdr x)))) + ((symbol? x) (symbol->string x)) + ((number? x) (number->string x)) + ((string? x) (string-append "\"" x "\"")) + ((eq? x #t) "#t") + ((eq? x #f) "#f") + (else "?"))) + +(define (sexp->list-tail rest) + (cond + ((null? rest) ")") + ((pair? rest) + (string-append " " + (write-to-string (car rest)) + (sexp->list-tail (cdr rest)))) + (else + (string-append " . " (write-to-string rest) ")")))) diff --git a/www/index.html b/www/index.html index e50ff96..c90d09b 100644 --- a/www/index.html +++ b/www/index.html @@ -89,16 +89,19 @@ make gpu-worker LUMBDA=asm # smallest footprint

On a single RTX 3090 with a warm daemon, fan-out matched hashlib.shake_256 byte-for-byte and won by 1.5–10× across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.

Tier choice for the worker host

-

The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of 20 calls per workload, daemon warm):

+

The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of N calls per workload, daemon warm, both client & worker on localhost):

- - - + + + + +
workloadPython tierC tierC win
small (3 × 16 B)1.27 ms0.14 ms9.1×
small (100 × 16 B)3.46 ms0.41 ms8.4×
medium (1000 × 16 B)23.51 ms2.67 ms8.8×
small (3 × 16 B)1.16 ms0.14 ms8.3×
small (100 × 16 B)3.39 ms0.42 ms8.1×
medium (1000 × 16 B)23.26 ms2.67 ms8.7×
huge (100k × 16 B)2,220 ms(wire stalls)n/a
huge (1M × 16 B)24,338 ms(wire stalls)n/a
-

C tier wins by ~9× across the grid — consistent with the ratio between Python's S-expression parser and the C tier's reader. At very heavy workloads (where the kernel itself takes seconds) the tier choice becomes noise; at light workloads (where bend stays local anyway) the tier choice doesn't matter either. The middle ground is where C tier earns its default.

+

At small & medium sizes, C tier wins by ~9× — that's the ratio between Python's S-expression parser and the C tier's reader. At huge sizes, both tiers stall on the wire, not the kernel. The shake256-fanout CUDA kernel on this 3090 finishes 1M × 16 B in ~47 ms — three orders of magnitude faster than the Python tier's 24 s end-to-end. The S-expression hex framing in wire.lsp is O(n²) at huge payload sizes because recv-exact accumulates chunks through string-append in a loop. C tier fails outright at this size; Python tier just suffers.

+

What this means: at small inputs, tier orchestration matters & C wins. At huge inputs, wire framing dominates regardless of tier — the right fix is a binary wire mode between client & worker, parallel to the binary portal mode the daemon already supports between worker & leaf. Until that lands, keep payloads under ~10 MB or call cuda-shake-fanout against pre-staged files instead of inline lists.

The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written pipe2 + fork + execve syscalls — no libc anywhere on the chain.

See examples/cuda-fanout/ for the wire contract, daemon protocol, bench data, and per-tier integration sketch.