bench + www: real numbers — C tier ~9× faster than Python as worker host
Wrote examples/cuda-fanout/bench_tiers.py — spawns a worker per tier, fires N TCP round-trips at three workload sizes through the warm daemon, reports median + p99. Measured on 3090-ai, daemon warm: workload Python C tier C win small (3 × 16 B) 1.27 ms 0.14 ms 9.1× small (100 × 16 B) 3.46 ms 0.41 ms 8.4× medium (1000 × 16 B) 23.51 ms 2.67 ms 8.8× Ratio stays at ~9× across the grid — the per-byte cost of Python's S-expression reader/printer compared to the C tier's reader. Justifies the LUMBDA=c default landed in the previous commit. asm tier worker starts up & listens (after the launch script predefines *argv* '()), but bench script saw malformed responses on this run — likely a write-to-string format difference between asm & Python/C reader. Leaving for follow-up; published numbers cover the tiers that completed end-to-end. www/index.html bend section gains the measured table under a new 'Tier choice for the worker host' subsection. Replaces the earlier hand-wavy ~10× claim with the actual measured numbers.
This commit is contained in:
parent
14edb57cd4
commit
f24afcc5d9
2 changed files with 229 additions and 0 deletions
218
examples/cuda-fanout/bench_tiers.py
Normal file
218
examples/cuda-fanout/bench_tiers.py
Normal file
|
|
@ -0,0 +1,218 @@
|
|||
"""bench_tiers.py — round-trip latency, Python vs C vs asm worker.
|
||||
|
||||
Spawns one worker per tier (each backed by the same warm daemon
|
||||
binary), times N round-trips at small + medium inputs, reports
|
||||
median + p99 + Σ.
|
||||
|
||||
Usage:
|
||||
python3 bench_tiers.py [--small N] [--medium N]
|
||||
python3 bench_tiers.py --skip-asm ; if asm host-side helpers not wired
|
||||
|
||||
What we're measuring:
|
||||
- WIRE orchestration cost (S-expression parse, portal write,
|
||||
pipe-to-daemon, daemon response, portal read, S-expression format)
|
||||
- NOT the CUDA kernel — that's identical across tiers since each
|
||||
tier shells to the same shake256-fanout --daemon binary
|
||||
|
||||
So the differences come from: TCP recv/send buffering, S-expression
|
||||
reader/printer speed, file I/O wrapping, pipe write/flush handling.
|
||||
"""
|
||||
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||
LUMBDA_ROOT = os.path.abspath(os.path.join(ROOT, "..", ".."))
|
||||
|
||||
TIERS = {
|
||||
"python": [
|
||||
"python3", "-u", os.path.join(LUMBDA_ROOT, "lumbda.py"),
|
||||
"launch.lsp", "--port",
|
||||
],
|
||||
"c": [
|
||||
os.path.join(LUMBDA_ROOT, "c", "lumbda"),
|
||||
"launch.lsp", "--port",
|
||||
],
|
||||
# asm tier launch uses a separate file that pre-defines *argv*
|
||||
# (asm doesn't auto-bind it). Port is fixed to 9091 — asm tier
|
||||
# also doesn't parse --port, so each asm bench uses its own
|
||||
# binary launch.
|
||||
"asm": [
|
||||
os.path.join(LUMBDA_ROOT, "asm", "lumbda-gc"),
|
||||
"launch-asm.lsp",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def wire_send(sock, payload):
|
||||
hdr = f"{len(payload):08d}".encode("ascii")
|
||||
sock.sendall(hdr + payload.encode("utf-8"))
|
||||
|
||||
|
||||
def wire_recv(sock):
|
||||
hdr = recv_exact(sock, 8)
|
||||
if hdr is None:
|
||||
return None
|
||||
plen = int(hdr.decode())
|
||||
return recv_exact(sock, plen).decode("utf-8")
|
||||
|
||||
|
||||
def recv_exact(sock, n):
|
||||
buf = b""
|
||||
while len(buf) < n:
|
||||
c = sock.recv(n - len(buf))
|
||||
if not c:
|
||||
return None
|
||||
buf += c
|
||||
return buf
|
||||
|
||||
|
||||
def spawn_worker(tier, port):
|
||||
cmd = TIERS[tier].copy()
|
||||
if tier != "asm":
|
||||
cmd.append(str(port))
|
||||
proc = subprocess.Popen(
|
||||
cmd, cwd=ROOT,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,
|
||||
)
|
||||
# Wait for "listening on port" or timeout
|
||||
deadline = time.monotonic() + 10
|
||||
while time.monotonic() < deadline:
|
||||
line = proc.stdout.readline()
|
||||
if not line:
|
||||
break
|
||||
if "listening on port" in line:
|
||||
return proc
|
||||
proc.kill()
|
||||
raise RuntimeError(f"{tier}: worker didn't ready ({line!r})")
|
||||
|
||||
|
||||
def stop_worker(proc):
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=3)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
def time_calls(port, payload, n):
|
||||
"""N round-trips, return list of per-call seconds."""
|
||||
times = []
|
||||
for _ in range(n):
|
||||
s = socket.socket()
|
||||
s.connect(("127.0.0.1", port))
|
||||
t0 = time.monotonic()
|
||||
wire_send(s, payload)
|
||||
resp = wire_recv(s)
|
||||
elapsed = time.monotonic() - t0
|
||||
s.close()
|
||||
if resp is None or "(ok" not in resp:
|
||||
raise RuntimeError(f"bad response: {resp!r}")
|
||||
times.append(elapsed)
|
||||
return times
|
||||
|
||||
|
||||
def make_payload(n_inputs, input_bytes):
|
||||
inputs = [secrets.token_hex(input_bytes) for _ in range(n_inputs)]
|
||||
parts = ['(cuda-shake-fanout (quote (']
|
||||
for h in inputs:
|
||||
parts.append(f'"{h}" ')
|
||||
parts.append(f")) 32)")
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def stats(times):
|
||||
s = sorted(times)
|
||||
n = len(s)
|
||||
return {
|
||||
"n": n,
|
||||
"min_ms": s[0] * 1e3,
|
||||
"med_ms": s[n // 2] * 1e3,
|
||||
"p99_ms": s[min(n - 1, int(n * 0.99))] * 1e3,
|
||||
"max_ms": s[-1] * 1e3,
|
||||
"tot_s": sum(s),
|
||||
}
|
||||
|
||||
|
||||
def bench_tier(tier, port, configs):
|
||||
print(f"\n=== {tier} tier (port {port}) ===")
|
||||
proc = spawn_worker(tier, port)
|
||||
# Warmup so CUDA context init + daemon spawn doesn't skew first call
|
||||
time.sleep(0.5)
|
||||
warmup_payload = make_payload(3, 4)
|
||||
for _ in range(5):
|
||||
time_calls(port, warmup_payload, 1)
|
||||
results = {}
|
||||
for label, (n_calls, n_inputs, input_bytes) in configs.items():
|
||||
p = make_payload(n_inputs, input_bytes)
|
||||
t = time_calls(port, p, n_calls)
|
||||
st = stats(t)
|
||||
results[label] = st
|
||||
print(f" {label:20} n={st['n']:>3} med={st['med_ms']:>6.2f} ms "
|
||||
f"p99={st['p99_ms']:>6.2f} ms min={st['min_ms']:>5.2f} ms")
|
||||
stop_worker(proc)
|
||||
return results
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--skip-asm", action="store_true",
|
||||
help="skip asm tier (if host-side helpers not wired)")
|
||||
ap.add_argument("--small-n", type=int, default=100)
|
||||
ap.add_argument("--medium-n", type=int, default=30)
|
||||
args = ap.parse_args()
|
||||
|
||||
# Write a launch.lsp the worker process will load
|
||||
with open(os.path.join(ROOT, "launch.lsp"), "w") as f:
|
||||
f.write('(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n')
|
||||
# asm tier doesn't auto-bind *argv*; pre-define it here so
|
||||
# gpu-worker.lsp's parse-port-arg falls through to *default-port*.
|
||||
with open(os.path.join(ROOT, "launch-asm.lsp"), "w") as f:
|
||||
f.write('(define *argv* (quote ()))\n'
|
||||
'(load "wire.lsp")\n(load "gpu-worker.lsp")\n(main)\n')
|
||||
|
||||
configs = {
|
||||
"small (3×16B)": (args.small_n, 3, 16),
|
||||
"small (100×16B)": (args.small_n, 100, 16),
|
||||
"medium (1000×16B)": (args.medium_n, 1000, 16),
|
||||
}
|
||||
|
||||
tiers = ["python", "c"]
|
||||
if not args.skip_asm:
|
||||
tiers.append("asm")
|
||||
|
||||
all_results = {}
|
||||
for i, tier in enumerate(tiers):
|
||||
# asm tier hard-codes port 9091 (no --port parsing); run it
|
||||
# last so other tiers get unique ports.
|
||||
port = 9091 if tier == "asm" else (9090 + i + 1)
|
||||
try:
|
||||
all_results[tier] = bench_tier(tier, port, configs)
|
||||
except Exception as e:
|
||||
print(f" {tier} FAILED: {e}")
|
||||
all_results[tier] = None
|
||||
|
||||
# Comparative table
|
||||
print("\n=== summary (median ms) ===")
|
||||
print(f"{'workload':<22}", end="")
|
||||
for tier in tiers:
|
||||
print(f" {tier:>10}", end="")
|
||||
print()
|
||||
for label in configs:
|
||||
print(f"{label:<22}", end="")
|
||||
for tier in tiers:
|
||||
r = all_results.get(tier)
|
||||
if r is None:
|
||||
print(f" {'FAIL':>10}", end="")
|
||||
else:
|
||||
print(f" {r[label]['med_ms']:>9.2f}", end="")
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -88,6 +88,17 @@ make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
|
|||
(bend (cuda-shake-fanout one-million-inputs 32))</code></pre>
|
||||
|
||||
<p>On a single RTX 3090 with a warm daemon, fan-out matched <code>hashlib.shake_256</code> byte-for-byte and won by 1.5–10× across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.</p>
|
||||
<h3>Tier choice for the worker host</h3>
|
||||
<p>The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of 20 calls per workload, daemon warm):</p>
|
||||
<table>
|
||||
<thead><tr><th>workload</th><th>Python tier</th><th>C tier</th><th>C win</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>small (3 × 16 B)</td><td>1.27 ms</td><td>0.14 ms</td><td>9.1×</td></tr>
|
||||
<tr><td>small (100 × 16 B)</td><td>3.46 ms</td><td>0.41 ms</td><td>8.4×</td></tr>
|
||||
<tr><td>medium (1000 × 16 B)</td><td>23.51 ms</td><td>2.67 ms</td><td>8.8×</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>C tier wins by ~9× across the grid — consistent with the ratio between Python's S-expression parser and the C tier's reader. At very heavy workloads (where the kernel itself takes seconds) the tier choice becomes noise; at light workloads (where bend stays local anyway) the tier choice doesn't matter either. The middle ground is where C tier earns its default.</p>
|
||||
<p>The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written <code>pipe2 + fork + execve</code> syscalls — no libc anywhere on the chain.</p>
|
||||
<p>See <a href="https://git.unturf.com/engineering/unturf/lumbda/-/blob/master/examples/cuda-fanout/README.md">examples/cuda-fanout/</a> for the wire contract, daemon protocol, bench data, and per-tier integration sketch.</p>
|
||||
</section>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue