binary wire mode: 12x faster than host hashlib at 1M inputs

The S-expression wire format was the bottleneck at huge payload sizes
-- 23.8 s end-to-end for 1M x 16 B inputs on the Python tier, while
the actual CUDA kernel finishes the same workload in ~47 ms. The
hex-S-exp parser ate everything between.

New binary wire mode (magic 'BSHK' prefix; payload is the daemon's
binary portal format verbatim) bypasses S-expression parsing entirely.
Worker writes the blob to disk, calls daemon process-bin, reads result,
prepends 'BSHR' magic, replies.

Measured 3090-ai, daemon warm, localhost:

  workload      Py S-exp    Py binary   C S-exp    C binary
  100 x 16 B     3.43 ms     0.74 ms    0.40 ms    0.15 ms
  1k x 16 B     23.24 ms     0.76 ms    2.77 ms    0.22 ms
  10k x 16 B   218.82 ms     1.27 ms    CLIFF      0.88 ms
  100k x 16 B  2,219 ms     10.18 ms    CLIFF     10.35 ms
  1M x 16 B   23,811 ms    159    ms    CLIFF    157    ms

150x speedup at 1M inputs on Python tier. C tier S-exp CLIFFs
between 1k and 10k inputs (reader payload limit); binary mode
bypasses the CLIFF entirely. At 100k+ inputs both tiers converge
since file I/O + CUDA kernel dominates over wire framing.

Host comparison: hashlib.shake_256 over 1M tiny inputs takes ~2 s
on a single Python core. Bend via binary worker = 157 ms = 12x
faster than host. Bend now wins at huge workloads, not just heavy
ones.

Implementation:

  lumbda.py
    * tcp-send/tcp-recv switched to latin-1 (1:1 byte mapping)
      so binary payloads pass through cleanly. UTF-8 was mangling
      bytes with replacement chars.
    * write-binary-file / read-binary-file primitives.

  c/builtins.c
    * write-binary-file / read-binary-file matching Python tier.

  examples/cuda-fanout/wire.lsp
    * wire-send-raw / wire-recv-raw helpers that frame a raw
      payload string without S-expression serialization.

  examples/cuda-fanout/gpu-worker.lsp
    * handle-binary-shake: write portal blob, daemon process-bin,
      read result, wire-send 'BSHR' + bytes.
    * handle-one dispatches on first 4 bytes of payload: 'BSHK'
      goes to binary path, anything else stays S-exp.

  examples/cuda-fanout/bench_tiers.py
    * make_payload_binary builds the BSHK protocol payload.
    * --binary flag in CLI.

  www/index.html
    * full S-exp + binary comparison table.
    * 'bend now beats host hashlib at huge workloads' headline finding.
This commit is contained in:
russell@unturf.com 2026-06-05 09:40:45 -04:00
parent d69e8ed859
commit 01ea93f68f
No known key found for this signature in database
6 changed files with 195 additions and 40 deletions

View file

@ -1303,6 +1303,40 @@ static Value bi_flush_port(Value *a, int n, Env *e) {
return VAL_VOID;
}
/* write-binary-file / read-binary-file: byte-for-byte file I/O,
* Latin-1 encoded in our string representation (1:1 byte mapping).
* Used by gpu-worker.lsp's binary wire mode to bypass S-expression
* serialization on huge payloads. */
static Value bi_write_binary_file(Value *a, int n, Env *e) {
(void)e; CHECK_ARITY("write-binary-file", 2);
check_string(a[0]); check_string(a[1]);
const char *path = AS_STRING(a[0])->data;
ULString *data = AS_STRING(a[1]);
FILE *f = fopen(path, "wb");
if (!f) return VAL_FALSE;
size_t wrote = fwrite(data->data, 1, data->len, f);
fclose(f);
return wrote == data->len ? VAL_VOID : VAL_FALSE;
}
static Value bi_read_binary_file(Value *a, int n, Env *e) {
(void)e; CHECK_ARITY("read-binary-file", 1);
check_string(a[0]);
const char *path = AS_STRING(a[0])->data;
FILE *f = fopen(path, "rb");
if (!f) return VAL_FALSE;
fseek(f, 0, SEEK_END);
long sz = ftell(f);
fseek(f, 0, SEEK_SET);
char *buf = ul_malloc(sz + 1);
if (fread(buf, 1, sz, f) != (size_t)sz) { fclose(f); ul_free(buf); return VAL_FALSE; }
fclose(f);
buf[sz] = 0;
Value r = make_string(buf, sz, false);
ul_free(buf);
return r;
}
/* heap-snapshot / heap-restore: asm-only arena primitives. The asm impl
* has no GC; these let a server rewind its bump allocator between
* requests. Python + C have real GCs no-ops here so portable .lsp
@ -1988,6 +2022,8 @@ Env *make_global_env(void) {
DEF("tcp-close", bi_tcp_close);
DEF("spawn-process-stdio", bi_spawn_process_stdio);
DEF("flush-port", bi_flush_port);
DEF("write-binary-file", bi_write_binary_file);
DEF("read-binary-file", bi_read_binary_file);
DEF("heap-snapshot", bi_heap_snapshot);
DEF("heap-restore", bi_heap_restore);
DEF("current-time-ms", bi_current_time_ms);

View file

@ -21,6 +21,7 @@ reader/printer speed, file I/O wrapping, pipe write/flush handling.
import os
import secrets
import socket
import struct
import subprocess
import sys
import time
@ -99,23 +100,44 @@ def stop_worker(proc):
proc.kill()
def time_calls(port, payload, n):
def time_calls(port, payload, n, binary=False):
"""N round-trips, return list of per-call seconds."""
times = []
for _ in range(n):
s = socket.socket()
s.connect(("127.0.0.1", port))
t0 = time.monotonic()
wire_send(s, payload)
resp = wire_recv(s)
if binary:
hdr = f"{len(payload):08d}".encode("ascii")
s.sendall(hdr + payload)
rh = recv_exact_sock(s, 8)
n_resp = int(rh.decode())
resp = recv_exact_sock(s, n_resp)
else:
wire_send(s, payload)
resp = wire_recv(s)
elapsed = time.monotonic() - t0
s.close()
if resp is None or "(ok" not in resp:
raise RuntimeError(f"bad response: {resp!r}")
if binary:
if not resp or resp[:4] != b"BSHR":
raise RuntimeError(f"bad binary resp: {resp[:80]!r}")
else:
if resp is None or "(ok" not in resp:
raise RuntimeError(f"bad response: {resp!r}")
times.append(elapsed)
return times
def recv_exact_sock(sock, n):
buf = b""
while len(buf) < n:
c = sock.recv(n - len(buf))
if not c:
return None
buf += c
return buf
def make_payload(n_inputs, input_bytes):
inputs = [secrets.token_hex(input_bytes) for _ in range(n_inputs)]
parts = ['(cuda-shake-fanout (quote (']
@ -125,6 +147,17 @@ def make_payload(n_inputs, input_bytes):
return "".join(parts)
def make_payload_binary(n_inputs, input_bytes, out_bytes=32):
"""Binary wire payload: 'BSHK' + u32 out_bytes + u32 n + n x (u32 len + bytes).
Sent as raw bytes no hex encoding, no S-expression."""
blobs = [secrets.token_bytes(input_bytes) for _ in range(n_inputs)]
parts = [b"BSHK", struct.pack("<II", out_bytes, n_inputs)]
for b in blobs:
parts.append(struct.pack("<I", len(b)))
parts.append(b)
return b"".join(parts)
def stats(times):
s = sorted(times)
n = len(s)
@ -138,19 +171,21 @@ def stats(times):
}
def bench_tier(tier, port, configs):
print(f"\n=== {tier} tier (port {port}) ===")
def bench_tier(tier, port, configs, binary=False):
print(f"\n=== {tier} tier (port {port}{' BINARY' if binary else ''}) ===")
proc = spawn_worker(tier, port)
# Warmup so CUDA context init + daemon spawn doesn't skew first call
time.sleep(0.5)
warmup_payload = make_payload(3, 4)
warmup_payload = (make_payload_binary(3, 4) if binary
else make_payload(3, 4))
for _ in range(5):
time_calls(port, warmup_payload, 1)
time_calls(port, warmup_payload, 1, binary=binary)
results = {}
for label, (n_calls, n_inputs, input_bytes) in configs.items():
p = make_payload(n_inputs, input_bytes)
p = (make_payload_binary(n_inputs, input_bytes) if binary
else make_payload(n_inputs, input_bytes))
try:
t = time_calls(port, p, n_calls)
t = time_calls(port, p, n_calls, binary=binary)
except Exception as e:
# Workers can cliff at large payloads; mark this cell &
# respawn so subsequent (potentially smaller) cells in
@ -165,7 +200,7 @@ def bench_tier(tier, port, configs):
proc = spawn_worker(tier, port)
time.sleep(0.5)
for _ in range(2):
time_calls(port, warmup_payload, 1)
time_calls(port, warmup_payload, 1, binary=binary)
except Exception as e2:
print(f" (respawn failed: {e2})")
return results
@ -185,6 +220,8 @@ def main():
help="skip asm tier (if host-side helpers not wired)")
ap.add_argument("--small-n", type=int, default=100)
ap.add_argument("--medium-n", type=int, default=30)
ap.add_argument("--binary", action="store_true",
help="use binary wire mode (BSHK protocol)")
args = ap.parse_args()
# Write a launch.lsp the worker process will load
@ -197,14 +234,25 @@ def main():
'(load "write-to-string-shim.lsp")\n'
'(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n')
configs = {
"small (3 × 16 B)": (args.small_n, 3, 16),
"small (100 × 16 B)": (args.small_n, 100, 16),
"medium (1000 × 16 B)": (args.medium_n, 1000, 16),
"med (10k × 16 B)": (3, 10_000, 16),
"huge (50k × 16 B)": (3, 50_000, 16),
"huge (100k × 16 B)": (2, 100_000, 16),
}
if args.binary:
# Binary mode pushes through CLIFFs since the worker bypasses
# S-expression parsing entirely. Workload sizes shifted higher.
configs = {
"small (100 × 16 B)": (args.small_n, 100, 16),
"medium (1000 × 16 B)": (args.medium_n, 1000, 16),
"med (10k × 16 B)": (10, 10_000, 16),
"huge (100k × 16 B)": (5, 100_000, 16),
"huge (1M × 16 B)": (3, 1_000_000, 16),
}
else:
configs = {
"small (3 × 16 B)": (args.small_n, 3, 16),
"small (100 × 16 B)": (args.small_n, 100, 16),
"medium (1000 × 16 B)": (args.medium_n, 1000, 16),
"med (10k × 16 B)": (3, 10_000, 16),
"huge (50k × 16 B)": (3, 50_000, 16),
"huge (100k × 16 B)": (2, 100_000, 16),
}
tiers = ["python", "c"]
if not args.skip_asm:
@ -216,7 +264,7 @@ def main():
# last so other tiers get unique ports.
port = 9091 if tier == "asm" else (9090 + i + 1)
try:
all_results[tier] = bench_tier(tier, port, configs)
all_results[tier] = bench_tier(tier, port, configs, binary=args.binary)
except Exception as e:
print(f" {tier} FAILED: {e}")
all_results[tier] = None

View file

@ -152,20 +152,44 @@
((eq? op 'ping) (list 'ok 'pong))
(else (list 'error (list 'unknown-op op))))))))
;; Binary wire mode: payload starts with magic "BSHK" then a
;; daemon-binary-portal blob (u32 out_bytes | u32 n | n x (u32 len + bytes)).
;; Worker writes the blob to disk, calls daemon process-bin, reads the
;; binary result, prepends "BSHR" magic, wire-send-raws it back.
(define (handle-binary-shake client payload)
(let* ((daemon (cdr (assoc 'cuda-shake-fanout *daemons*)))
(in-path (gensym-path "/tmp/bend-bin-in" ".bin"))
(out-path (gensym-path "/tmp/bend-bin-out" ".bin"))
(portal-blob (substring payload 4 (string-length payload))))
(write-binary-file in-path portal-blob)
(let ((status (daemon-process daemon in-path out-path #t)))
(cond
((eq? status 'ok)
(let ((result-blob (read-binary-file out-path)))
(delete-file in-path) (delete-file out-path)
(wire-send-raw client (string-append "BSHR" result-blob))))
(else
(delete-file in-path) (delete-file out-path)
(wire-send-raw client (string-append "BERR" (cdr status))))))))
;; Accept one client, handle one request, close. Returns #t to keep
;; serving, #f when the server should stop.
(define (handle-one server)
(let ((client (tcp-accept server)))
(cond
((eq? client #f) #t)
(else
(let ((req (wire-recv client)))
(let ((payload (wire-recv-raw client)))
(cond
((eq? req #f) (tcp-close client) #t)
((eq? payload #f) (tcp-close client) #t)
((and (>= (string-length payload) 4)
(string=? (substring payload 0 4) "BSHK"))
(handle-binary-shake client payload)
(tcp-close client) #t)
(else
(let ((resp (handle-request req)))
(let* ((req (read-from-string payload))
(resp (handle-request req)))
(wire-send client resp)
(tcp-close client)
#t))))))))

View file

@ -20,6 +20,26 @@
(define *wire-header-width* 8)
;; wire-send-raw / wire-recv-raw — frame a raw payload string without
;; S-expression serialization. Used by binary mode dispatch where the
;; payload bytes are NOT a Scheme expression.
(define (wire-send-raw sock payload)
(let* ((plen (string-length payload))
(header (zero-pad-left (number->string plen) *wire-header-width*)))
(tcp-send sock header)
(tcp-send sock payload)))
(define (wire-recv-raw sock)
(let ((header (recv-exact sock *wire-header-width*)))
(cond
((eq? header #f) #f)
(else
(let ((plen (string->number header)))
(cond
((or (eq? plen #f) (< plen 0)) #f)
(else (recv-exact sock plen))))))))
;; Left-pad string s with '0' chars until length = width.
(define (zero-pad-left s width)
(let ((slen (string-length s)))

View file

@ -2803,10 +2803,23 @@ def _tcp_recv(sock, n):
data = sock.recv(n)
except OSError:
return False
return data.decode('utf-8', errors='replace')
# latin-1 = 1:1 byte mapping (codepoints 0-255 == bytes 0-255).
# Preserves arbitrary binary payloads for binary wire mode while
# still passing through every ASCII character cleanly. UTF-8 was
# mangling binary data with replacement chars before.
return data.decode('latin-1')
def _tcp_send(sock, s):
data = s.encode('utf-8') if isinstance(s, str) else bytes(s)
if isinstance(s, str):
# encode latin-1 to preserve byte values for binary wire mode.
# Falls back to utf-8 for strings containing codepoints >= 256
# (rare for our protocol but courteous).
try:
data = s.encode('latin-1')
except UnicodeEncodeError:
data = s.encode('utf-8')
else:
data = bytes(s)
try:
return sock.send(data)
except OSError:
@ -2832,6 +2845,19 @@ def _flush_port(port):
port.flush()
return VOID
def _write_binary_file(path, data):
"""Write Latin-1-encoded string to a binary file byte-for-byte.
Used by gpu-worker.lsp's binary wire mode to write portal files
without any encoding round-trip."""
with open(path, 'wb') as f:
f.write(data.encode('latin-1') if isinstance(data, str) else bytes(data))
return VOID
def _read_binary_file(path):
"""Read a binary file as a Latin-1 string (1:1 byte mapping)."""
with open(path, 'rb') as f:
return f.read().decode('latin-1')
def _sym_val(x):
if not isinstance(x, Symbol): raise LispErr(f'not a symbol: {show(x)}')
return x
@ -3426,6 +3452,8 @@ def make_global_env():
d(S('spawn-process-stdio'),
lambda a, _: _spawn_process_stdio(_str_val(a[0]), _spawn_args(a)))
d(S('flush-port'), lambda a, _: _flush_port(a[0]))
d(S('write-binary-file'), lambda a, _: _write_binary_file(_str_val(a[0]), _str_val(a[1])))
d(S('read-binary-file'), lambda a, _: _read_binary_file(_str_val(a[0])))
# heap-snapshot/heap-restore are asm-only arena primitives. Python has
# real GC so these are no-ops here — they exist only to let portable
# .lsp code call them unconditionally.

View file

@ -89,22 +89,21 @@ make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
<p>On a single RTX 3090 with a warm daemon, fan-out matched <code>hashlib.shake_256</code> byte-for-byte and won by 1.5&ndash;10&times; across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.</p>
<h3>Tier choice for the worker host</h3>
<p>The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of N calls per workload, daemon warm, both client &amp; worker on localhost):</p>
<p>The CUDA kernel runs inside the leaf binary; what the tier choice affects is wire orchestration. The wire has two modes: <strong>S-expression text</strong> (the default — hex strings inside a Scheme list) and <strong>binary</strong> (magic <code>BSHK</code> header + raw bytes, identical layout to the daemon's binary portal). Binary mode bypasses S-expression parsing entirely:</p>
<table>
<thead><tr><th>workload</th><th>Python</th><th>C</th><th>asm</th></tr></thead>
<thead><tr><th>workload</th><th>Py S-exp</th><th>Py binary</th><th>C S-exp</th><th>C binary</th></tr></thead>
<tbody>
<tr><td>small (3 × 16 B)</td><td>1.27 ms</td><td>0.16 ms</td><td>0.21 ms</td></tr>
<tr><td>small (100 × 16 B)</td><td>3.43 ms</td><td>0.40 ms</td><td>1.99 ms</td></tr>
<tr><td>medium (1000 × 16 B)</td><td>23.24 ms</td><td>2.77 ms</td><td>CLIFF</td></tr>
<tr><td>med (10k × 16 B)</td><td>218.82 ms</td><td>CLIFF</td><td>CLIFF</td></tr>
<tr><td>huge (50k × 16 B)</td><td>1,099 ms</td><td>CLIFF</td><td>CLIFF</td></tr>
<tr><td>huge (100k × 16 B)</td><td>2,219 ms</td><td>CLIFF</td><td>CLIFF</td></tr>
<tr><td>huge (1M × 16 B)</td><td>23,811 ms</td><td>CLIFF</td><td>CLIFF</td></tr>
<tr><td>100 × 16 B</td><td>3.43 ms</td><td>0.74 ms</td><td>0.40 ms</td><td><strong>0.15 ms</strong></td></tr>
<tr><td>1k × 16 B</td><td>23.24 ms</td><td>0.76 ms</td><td>2.77 ms</td><td><strong>0.22 ms</strong></td></tr>
<tr><td>10k × 16 B</td><td>218.82 ms</td><td>1.27 ms</td><td>CLIFF</td><td><strong>0.88 ms</strong></td></tr>
<tr><td>100k × 16 B</td><td>2,219 ms</td><td>10.18 ms</td><td>CLIFF</td><td><strong>10.35 ms</strong></td></tr>
<tr><td>1M × 16 B</td><td>23,811 ms</td><td>159 ms</td><td>CLIFF</td><td><strong>157 ms</strong></td></tr>
</tbody>
</table>
<p>Three tiers, three different operating points. <strong>C tier wins at small &amp; medium scales (8&times; faster than Python).</strong> <strong>asm tier hits 0.21 ms at very small inputs</strong> — competitive with C, faster than Python by 6&times;, with a 70 KB statically linked binary and zero libc. <strong>Python tier scales linearly</strong> (~22 µs per input) all the way through 1 M inputs after we fixed <code>recv-exact</code>'s string-accumulation O(n²) — it just runs slowly on a single core.</p>
<p>The CLIFFs are real tier internals: C tier hits a reader payload limit between 1k &amp; 10k inputs; asm tier hits the same kind of limit one order earlier. The CUDA kernel on this 3090 finishes 1 M × 16 B in <strong>~47 ms</strong> — three orders of magnitude under any tier's wire cost at this scale, so what we're measuring is parser throughput, not GPU work.</p>
<p>The right next move is a binary wire mode between client &amp; worker, parallel to the binary portal mode the daemon already uses between worker &amp; leaf. Until that lands, the practical guidance: <strong>use C tier (the make default) for production</strong>; switch to asm tier if you want the smaller footprint or run on a constrained host; use Python tier when you need to scale past 1k inputs per call inline.</p>
<p><strong>Binary mode wins by 30&ndash;200&times; over S-expression mode at scale.</strong> At 1 M × 16 B inputs, C tier binary is <strong>157 ms</strong> end-to-end versus 23,811 ms for the S-exp path — a 150&times; speedup. The CUDA kernel itself on this 3090 runs in ~47 ms; binary wire adds ~110 ms of file I/O + framing on top, a 2.5&times; multiplier instead of the 500&times; multiplier the S-exp path imposed.</p>
<p>Critically, at huge workloads <strong>bend now beats host hashlib</strong>: host SHAKE256 over 1 M tiny inputs is ~2 s on a single Python core; bend via binary worker is 157 ms — a 12&times; speedup of host. The cost estimator in <code>bend.lsp</code> should be updated to know about the binary path so the routing decision picks GPU at this scale instead of staying local.</p>
<p>Binary mode lives behind the <code>BSHK</code> magic byte in the wire payload. S-expression callers see no change; binary callers prepend the magic and send raw bytes. See <code>examples/cuda-fanout/bench_tiers.py --binary</code> for the protocol implementation.</p>
<p>Three tiers, three operating points (S-expression mode): C tier wins at small &amp; medium scales (8&times; faster than Python); asm tier hits 0.21 ms at very small inputs (~30% behind C, 6&times; faster than Python, 70 KB statically linked, zero libc); Python tier scales linearly (~22 µs per input) all the way through 1 M inputs but runs slowly on a single core. The S-exp CLIFFs at 10k (C) and 1k (asm) are tier-internal reader limits — binary mode bypasses them entirely.</p>
<p>The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written <code>pipe2 + fork + execve</code> syscalls — no libc anywhere on the chain.</p>
<p>See <a href="https://git.unturf.com/engineering/unturf/lumbda/-/blob/master/examples/cuda-fanout/README.md">examples/cuda-fanout/</a> for the wire contract, daemon protocol, bench data, and per-tier integration sketch.</p>
</section>