guardrails: bound http server, trap+cleanup bench, asm-no-GC in CLAUDE.md
Prevent recurrence of 2026-04-16 incident where a leaked asm HTTP server grew to 19.3 GB RSS and crashed the machine. examples/http-server.lsp: - Adds *max-requests* = 50000 hard ceiling. Server self-terminates before unbounded heap growth reaches dangerous levels. - Loop tracks request count, exits cleanly + closes server socket. tests/web-benchmark.sh: - SPAWNED_PIDS array tracks every background process. - EXIT/INT/TERM trap kills them all (SIGTERM then SIGKILL). - stop_server does SIGTERM with 500ms grace period then SIGKILL. - Final straggler check via pgrep narrows to actual HTTP server processes (not shell/tmux with "uncommonlisp" in the name). - pkill -9 fallback as belt-and-suspenders. CLAUDE.md: - New "Asm memory discipline" section documents the bump allocator leak behavior and the required operational discipline. - Test counts updated (571 py + 83 c + 132 asm + 189 shared = 975).
This commit is contained in:
parent
bfd4ec7ec8
commit
477bd5f7cd
3 changed files with 104 additions and 23 deletions
32
CLAUDE.md
32
CLAUDE.md
|
|
@ -48,9 +48,35 @@ Then ask fox what the mission is.
|
|||
|
||||
- Python unit/integration: `tests.py` (571 tests)
|
||||
- C unit/integration/JIT/continuations/portal: `c/test.c` (83 tests)
|
||||
- Assembly unit/integration/functional: `asm/test.sh` (75 tests)
|
||||
- Shared functional: `tests/functional.lsp` (181 tests, runs in Python + C)
|
||||
- Total: 910 verified assertions via `make test-all`
|
||||
- Assembly unit/integration/functional: `asm/test.sh` (132 tests)
|
||||
- Shared functional: `tests/functional.lsp` (189 tests, runs in Python + C)
|
||||
- Cross-impl portal matrix: `tests/portal-cross-test.sh` (9 cells)
|
||||
- Portal benchmark: `tests/portal-benchmark.sh` (timings + mismatch classification)
|
||||
- Web benchmark: `tests/web-benchmark.sh` (all three impls + Python http.server + busybox)
|
||||
- Total: **975 verified assertions** via `make test-all`
|
||||
|
||||
## Asm memory discipline — the heap does not shrink
|
||||
|
||||
The asm implementation uses a bump allocator (r15). Every allocation
|
||||
(`string-append`, `tcp-recv`, `make-pair`, `number->string`, etc.)
|
||||
grows r15 monotonically. When r15 hits r13 (heap limit),
|
||||
`heap_grow` mmaps ANOTHER 64 MB chunk. **Nothing is ever freed.**
|
||||
|
||||
A long-running asm server leaks ~64 MB every few thousand requests
|
||||
until it OOMs the machine. This actually crashed fox's machine on
|
||||
2026-04-16 — a benchmark server grew to 19.3 GB RSS / 30 GB virt.
|
||||
|
||||
**Never leave an asm server running in the background.** Specifically:
|
||||
|
||||
- Every background spawn in a bench script gets an EXIT trap:
|
||||
`trap 'kill ${PIDS[@]} 2>/dev/null' EXIT`
|
||||
- For sustained-throughput benchmarks, bound iterations **inside the
|
||||
.lsp** (e.g., `(if (> n 1000) (exit 0) (loop (+ n 1)))`) — never
|
||||
rely on external kill signals alone.
|
||||
- Before declaring a session done, verify:
|
||||
`ps -u fox -o pid,rss,cmd | grep uncommonlisp | grep -v grep` (no output = clean)
|
||||
- C has Boehm GC via `GC_MALLOC`. Python has Python's GC. Asm has
|
||||
neither. Risk scales with how long the asm process lives.
|
||||
|
||||
## MOAD Scanner
|
||||
|
||||
|
|
|
|||
|
|
@ -15,6 +15,13 @@
|
|||
(define *crlf* "\r\n")
|
||||
(define *crlf-crlf* "\r\n\r\n")
|
||||
|
||||
;;; Hard request ceiling. The asm impl has no GC: every request
|
||||
;;; bump-allocates heap cells that never get reclaimed. An unbounded
|
||||
;;; server leaks ~64 MB per heap growth until OOM. This ceiling makes
|
||||
;;; the server self-terminate long before it starves the machine.
|
||||
;;; Benchmarks that want a different cap should `set!` it after load.
|
||||
(define *max-requests* 50000)
|
||||
|
||||
;;; ── HTTP helpers ─────────────────────────────────────────────
|
||||
|
||||
(define (http-response status ctype body)
|
||||
|
|
@ -77,14 +84,20 @@
|
|||
|
||||
(define (serve)
|
||||
(let ((server (tcp-listen *port*)))
|
||||
(display "uncommonlisp http server on :") (display *port*) (newline)
|
||||
(let loop ()
|
||||
(let ((client (tcp-accept server)))
|
||||
(let ((req (tcp-recv client 4096)))
|
||||
(if (and req (> (string-length req) 0))
|
||||
(tcp-send client (handle-request req))
|
||||
#f))
|
||||
(tcp-close client))
|
||||
(loop))))
|
||||
(display "uncommonlisp http server on :") (display *port*)
|
||||
(display " (max ") (display *max-requests*) (display " requests)") (newline)
|
||||
(let loop ((n 0))
|
||||
(if (>= n *max-requests*)
|
||||
(begin
|
||||
(display "request cap reached, exiting\n")
|
||||
(tcp-close server))
|
||||
(begin
|
||||
(let ((client (tcp-accept server)))
|
||||
(let ((req (tcp-recv client 4096)))
|
||||
(if (and req (> (string-length req) 0))
|
||||
(tcp-send client (handle-request req))
|
||||
#f))
|
||||
(tcp-close client))
|
||||
(loop (+ n 1)))))))
|
||||
|
||||
(serve)
|
||||
|
|
|
|||
|
|
@ -16,6 +16,35 @@ PY="python3 uncommonlisp.py --fast"
|
|||
C="./c/uncommonlisp"
|
||||
ASM="./asm/uncommonlisp"
|
||||
|
||||
# Track every server PID we spawn; the EXIT trap kills them all.
|
||||
# Asm has no GC — a leaked server leaks 64 MB per heap growth
|
||||
# indefinitely. Belt + suspenders: kill the whole process group too.
|
||||
declare -a SPAWNED_PIDS=()
|
||||
|
||||
cleanup() {
|
||||
local pid
|
||||
for pid in "${SPAWNED_PIDS[@]}"; do
|
||||
kill "$pid" 2>/dev/null
|
||||
done
|
||||
sleep 0.2
|
||||
for pid in "${SPAWNED_PIDS[@]}"; do
|
||||
kill -9 "$pid" 2>/dev/null
|
||||
done
|
||||
rm -rf "${STATIC_DIR:-/nonexistent/xyz}"
|
||||
# Final sanity: any stray servers from examples/http-server?
|
||||
# Narrow to processes actually serving the HTTP example, not the
|
||||
# shell running this script or tmux sessions with "uncommonlisp"
|
||||
# in their name.
|
||||
local strays
|
||||
strays=$(pgrep -u "$USER" -f 'examples/http-server\.lsp|http\.server 8080' | wc -l)
|
||||
if [ "$strays" -gt 0 ]; then
|
||||
echo "WARNING: $strays HTTP server process(es) still running:" >&2
|
||||
pgrep -u "$USER" -af 'examples/http-server\.lsp|http\.server 8080' >&2
|
||||
pkill -9 -u "$USER" -f 'examples/http-server\.lsp|http\.server 8080' 2>/dev/null
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
pad() { printf " %-36s " "$1"; }
|
||||
|
||||
bench_one() {
|
||||
|
|
@ -38,18 +67,19 @@ print(f'{rps:8.0f} req/s ({t:.3f}s total, concurrency $CONCURRENCY)')"
|
|||
# Setup a 1KB static file for file-serving benchmarks
|
||||
STATIC_DIR=$(mktemp -d)
|
||||
printf '%.0s0123456789abcdef0123456789abcdef' {1..32} > "$STATIC_DIR/bench"
|
||||
trap "rm -rf $STATIC_DIR" EXIT
|
||||
|
||||
echo "═══════════════════════════════════════════════════════════════"
|
||||
echo "HTTP benchmark — $REQUESTS requests, concurrency $CONCURRENCY"
|
||||
echo "═══════════════════════════════════════════════════════════════"
|
||||
echo
|
||||
|
||||
# Helper to start + wait for port ready
|
||||
# Helper to start + wait for port ready. Adds pid to SPAWNED_PIDS so
|
||||
# the EXIT trap kills it even if the script dies unexpectedly.
|
||||
start_server() {
|
||||
local cmd="$1" port="$2"
|
||||
eval "$cmd" > /dev/null 2>&1 &
|
||||
local pid=$!
|
||||
SPAWNED_PIDS+=("$pid")
|
||||
for _ in $(seq 1 20); do
|
||||
if curl -s -o /dev/null -w "%{http_code}" "http://localhost:$port/" 2>/dev/null | grep -q 200; then
|
||||
echo "$pid"
|
||||
|
|
@ -61,38 +91,50 @@ start_server() {
|
|||
return 1
|
||||
}
|
||||
|
||||
stop_server() {
|
||||
local pid="$1"
|
||||
# SIGTERM first; if the process is blocked in accept() it may not
|
||||
# exit cleanly, so force-kill after a short grace period.
|
||||
kill "$pid" 2>/dev/null
|
||||
for _ in 1 2 3 4 5; do
|
||||
kill -0 "$pid" 2>/dev/null || break
|
||||
sleep 0.1
|
||||
done
|
||||
kill -9 "$pid" 2>/dev/null
|
||||
wait "$pid" 2>/dev/null
|
||||
sleep 0.2
|
||||
}
|
||||
|
||||
# ── uncommonlisp Python ──
|
||||
PID=$(start_server "$PY examples/http-server.lsp" 8080)
|
||||
bench_one "uncommonlisp Python (/bench, 1 KB)" 8080 /bench
|
||||
kill "$PID" 2>/dev/null; wait "$PID" 2>/dev/null
|
||||
sleep 0.3
|
||||
stop_server "$PID"
|
||||
|
||||
# ── uncommonlisp C ──
|
||||
PID=$(start_server "$C examples/http-server.lsp" 8080)
|
||||
bench_one "uncommonlisp C (/bench, 1 KB)" 8080 /bench
|
||||
kill "$PID" 2>/dev/null; wait "$PID" 2>/dev/null
|
||||
sleep 0.3
|
||||
stop_server "$PID"
|
||||
|
||||
# ── uncommonlisp asm ──
|
||||
PID=$(start_server "$ASM < examples/http-server.lsp" 8080)
|
||||
bench_one "uncommonlisp asm (/bench, 1 KB)" 8080 /bench
|
||||
kill "$PID" 2>/dev/null; wait "$PID" 2>/dev/null
|
||||
sleep 0.3
|
||||
stop_server "$PID"
|
||||
|
||||
# ── Python http.server (stdlib) ──
|
||||
(cd "$STATIC_DIR" && python3 -m http.server 8080 > /dev/null 2>&1) &
|
||||
PID=$!
|
||||
SPAWNED_PIDS+=("$PID")
|
||||
sleep 1
|
||||
bench_one "python3 -m http.server (1 KB file)" 8080 /bench
|
||||
kill "$PID" 2>/dev/null; wait "$PID" 2>/dev/null
|
||||
sleep 0.3
|
||||
stop_server "$PID"
|
||||
|
||||
# ── busybox httpd ──
|
||||
busybox httpd -f -p 127.0.0.1:8080 -h "$STATIC_DIR" > /dev/null 2>&1 &
|
||||
PID=$!
|
||||
SPAWNED_PIDS+=("$PID")
|
||||
sleep 0.5
|
||||
bench_one "busybox httpd (1 KB file)" 8080 /bench
|
||||
kill "$PID" 2>/dev/null; wait "$PID" 2>/dev/null
|
||||
stop_server "$PID"
|
||||
|
||||
echo
|
||||
echo "═══════════════════════════════════════════════════════════════"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue