Every benchmark in the whitepaper now has a Makefile target and
each in-paper result is tagged with its reproduce command.
New / refactored Make targets:
make bench Python tree-walker vs bytecode (§6.1-6.3)
make bench-3way 3-way Python/C/asm head-to-head (§6.4)
make bench-portal portal save+load timings (§7.5)
make bench-portal-cross 3x3 cross-impl portal matrix (§7.2)
make bench-web HTTP vs busybox / python http.server (§11.3)
make bench-rpc-chain Python → C relay → asm chain (§11.4)
make bench-all runs every bench above
bench-3way is a new script (tests/bench-3way.sh) that drives each
impl in its recommended high-performance mode and prints a clean
best-of-two comparison table matching §6.4.
Every script uses the six-layer safety envelope from CLAUDE.md
(ulimit -v + trap + timeout + explicit kill + pgrep verify).
Documented in the whitepaper's §6 Methodology block.
Whitepaper additions:
- §6 Methodology paragraph adds a "Reproducibility" block listing
every Makefile target alongside the section it backs.
- §12 MOAD Audit now cites the canonical MOAD taxonomy:
https://undefect.com/moad-cheat-sheet/
(MOAD-0001 through MOAD-0005) so readers can look up the defect
classes the paper references.
- §6.4, §7.2, §7.5, §11.3, §11.4 each end with a "Reproduce: make
bench-<name>" pointer tying the number to the script that
produces it.
Ran bench-3way on the i5-8350U:
Python --fast: sum-to(100k)=555ms, sum-to(1M)=5038ms, ack(3,8)=18740ms
C --fast: sum-to(100k)= 27ms, sum-to(1M)= 255ms, ack(3,8)= 1465ms
asm: sum-to(100k)= 67ms, sum-to(1M)= 692ms, ack(3,8)= 2300ms
Matches the table in the paper (best-of-two).
Adds a transparent S-expression relay (examples/rpc-relay.lsp) plus a
sequential load generator (examples/rpc-chain-bench.lsp) and a bench
script (tests/rpc-chain-bench.sh) that wires them into multi-hop
chains across runtimes.
The relay is pure byte-forwarding: tcp-accept, tcp-recv, tcp-connect
to backend, tcp-send, tcp-recv reply, tcp-send back. Never parses.
Which is the point — S-expressions are the envelope.
Same rpc-relay.lsp runs as relay in any impl; chains are arbitrary
combinations of {Py, C, asm} nodes.
Measured (200 requests, ping, same laptop):
(A) Py client → asm backend direct, 1 hop 2061 rps
(B) Py client → C relay → asm 2 hops 1234 rps
(C) Py client → Py → C → asm 3 hops 766 rps
(D) asm client → Py → C → asm 3 hops 796 rps
Per-hop cost ≈ 600-700 µs/request (TCP round-trip + context switch).
Safety: every server spawn used the six-layer pattern from CLAUDE.md
(ulimit -v 512MB + timeout 30 + trap + explicit kill + pgrep verify).
Four benchmark cells × up to 3 servers each = 10+ server spawns.
Zero strays, zero safety-net activations.
Prevent recurrence of 2026-04-16 incident where a leaked asm HTTP
server grew to 19.3 GB RSS and crashed the machine.
examples/http-server.lsp:
- Adds *max-requests* = 50000 hard ceiling. Server self-terminates
before unbounded heap growth reaches dangerous levels.
- Loop tracks request count, exits cleanly + closes server socket.
tests/web-benchmark.sh:
- SPAWNED_PIDS array tracks every background process.
- EXIT/INT/TERM trap kills them all (SIGTERM then SIGKILL).
- stop_server does SIGTERM with 500ms grace period then SIGKILL.
- Final straggler check via pgrep narrows to actual HTTP server
processes (not shell/tmux with "uncommonlisp" in the name).
- pkill -9 fallback as belt-and-suspenders.
CLAUDE.md:
- New "Asm memory discipline" section documents the bump allocator
leak behavior and the required operational discipline.
- Test counts updated (571 py + 83 c + 132 asm + 189 shared = 975).
Added tcp-listen/accept/connect/recv/send/close to Python, C, and asm.
One examples/http-server.lsp runs identically in all three impls and
serves HTTP/1.0 with routing, content-type, and content-length headers.
asm additions:
- SYS_SOCKET/BIND/LISTEN/ACCEPT/CONNECT/SETSOCKOPT syscalls
- 6 tcp-* builtins using the existing port encoding (SPECIAL ≥ 1000)
- bi_tcp_connect: dotted-quad IPv4 parser, no DNS dependency
Defects fixed along the way (surfaced by the HTTP server):
- string-append: was 2-arg only; now variadic (walks arg list twice)
- number->string: was stubbed to VAL_VOID; now correctly writes digits
into a heap-allocated string (incl. negative handling)
- String-literal reader: \r and \0 escape sequences now handled (was
silently dropping backslash, treating them as literal 'r' / '0')
- tcp_accept: sockaddr buffer was 8 bytes, now 16 (was corrupting
caller's stack when accept wrote full struct sockaddr_in)
Pinocchio benchmark (tests/web-benchmark.sh):
At concurrency=20, 1000 requests, serving a 1KB body:
uncommonlisp Python 373 req/s
uncommonlisp C 370 req/s
uncommonlisp asm 370 req/s
python3 http.server 381 req/s (stdlib reference)
busybox httpd 382 req/s (production reference)
All five converge within 3% — the client (curl fork/exec) is the
bottleneck, not the server. Our single-threaded blocking servers
are indistinguishable from battle-tested ones at this load.
Binary sizes:
uncommonlisp asm 45 KB (HTTP + everything else)
busybox httpd 2.1 MB (multi-call binary)
python3 8 MB (interpreter)
The asm HTTP server is 46× smaller than busybox and 176× smaller
than Python, serves from 7 Linux syscalls, and the entire protocol
handler is 70 lines of portable Scheme.
Test counts: 132 asm (up 1), rest unchanged. All green.
Benchmark exercises full save×load matrix across Python/C/asm plus the
mismatch cases (wrong format, truncated input, missing file, corrupt
header). Cases that used to segfault or report wrong paths now degrade
cleanly.
asm (uncommonlisp.s):
- (define var) with no value now binds to VOID instead of segfault
- portal-resume checks sys_read returned full 48-byte header; sanity-
checks heap_size and heap_base before committing r14/r15 restore.
Corrupt/truncated portals now return #f cleanly.
py (uncommonlisp.py):
- file-not-found error inside a nested (load) now reports the actual
missing path (via FileNotFoundError.filename) rather than the outer
script path.
- _load wraps UnicodeDecodeError (binary file loaded as text) into a
LispErr with the path; no more raw Python traceback.
tests/portal-benchmark.sh: 50-iter benchmark, 4 parts
(save / load / cross-process / mismatch-classification).
Representative numbers (this laptop, 2026-04-16):
setup+save: Python 126ms, C 3ms, asm 0.9ms
cross-proc: Py→Py 260ms, C→C 6ms, asm→asm 1.5ms
All three test suites still pass: 571 py unit, 131 asm, 189 shared.
67 new tests covering: any/every/find/count/sort/iota/fold-right,
named let with multi-body, internal defines, letrec mutual recursion,
do loops with results, tail position in cond/when/unless/and/or,
nested closures with mutation, deep TCO at 200k depth, string ops
(contains/split/join/trim), apply, variadic args, quasiquote splicing,
set-car!/set-cdr!, list-tail, make-list, hash-table-delete!.
Both Python and C pass 181/181.
tests/functional.lsp — single .lsp file, runs identically in Python and C.
Covers: arithmetic, comparison, booleans, pairs, lists, strings, characters,
vectors, hash tables, control flow, let/lambda/closures, do loops, define,
recursion, TCO (100k depth), quasiquote, macros, type predicates, call/cc,
error handling, mergesort, higher-order programs.
Fixed C call/cc: proper escape continuations via setjmp/longjmp.
make test-all runs: Python unit (571) + C unit (58) + shared functional (114).
