whitepaper: diagrams + stats refresh for GC / meta-GC / hash primitives

Diagrams:
  - asm-architecture.dot: adds GC_NAIVE memory cluster (bump, free
    list, conservative stack scan) + meta-GC (with-arena) cluster
    showing the reset path; updates line count (4968 -> 6645),
    builtin count (91 -> 95+), mentions native hash-table-* /
    hash-set-* and the GC-build primitives (with-arena, gc-collect,
    gc-stats, arena-stats).
  - benchmark-binary-size.dot: adds second asm bar for the GC_NAIVE
    build (27 KB stripped vs 23 KB bump-only); updated asm bump
    size from 22 KB (stale) to actual 23 KB.
  - benchmark-gc.dot (new): side-by-side peak RSS for asm bump-only
    (134 MB), naive GC (1.1 MB), meta-GC arena (1.2 MB, 2000/2000
    resets); embedded in §6.6.
  - meta-gc-policy.dot (new): three-way decision tree at
    (with-arena) exit — implicit-GC-fired / mark-in-arena-range /
    no-mark-in-range -> skip / sweep / bulk-reset; embedded in
    §6.6.1.

Stats:
  - 975 verified assertions -> 980 (asm gained 5 via hash-table &
    hash-set tests; 571 Python + 137 asm + 83 C + 189 shared).
  - asm test count 132 -> 137 in the summary list, intro abstract,
    and §11 tier table. Notes that the optional GC build passes
    the same 137 independently (1,117 assertions total when both
    asm binaries are exercised).
  - Stale 4,968 LOC -> 6,645 already fixed in the prior commit;
    the new asm-architecture diagram now matches.

PDF rebuilt, 2.58 MB (was 2.40 MB). All test suites green.

docs/asm-architecture.dot

@@ -1,4 +1,4 @@
-// Assembly implementation architecture — 2026-04-17 refresh
+// Assembly implementation architecture — 2026-04-18 refresh
 // "A diagram is worth 10,000 words." — russell@unturf.com
 digraph asm_arch {
     rankdir=TB
@@ -6,7 +6,7 @@ digraph asm_arch {
     edge [fontname="Helvetica", fontsize=10]
 
     subgraph cluster_binary {
-        label="Binary: 22 KB stripped, zero dependencies"
+        label="Binary: ~22 KB stripped (bump-only) / ~25 KB (GC_NAIVE), zero dependencies"
         style=rounded
         color="#333333"
         fontcolor="#333333"
@@ -24,23 +24,25 @@ digraph asm_arch {
     }
 
     subgraph cluster_memory {
-        label="Memory Model"
+        label="Memory Model — default bump; optional mark-sweep + meta-GC"
         style=rounded
         color="#666666"
-        bump [label="Bump Allocator\n%r15 = heap ptr\n%r13 = heap limit\nheap_grow mmaps 64 MB" fillcolor="#ffeaa7"]
-        snap [label="heap-snapshot\nheap-restore\narena for per-request\nloops (O(1) memory)" fillcolor="#ffeaa7"]
-        tags [label="Tag-in-Low-3-Bits\n0=int 1=pair 2=sym\n3=closure 4=builtin\n5=special 6=string\n7=vector" fillcolor="#ffeaa7"]
+        bump [label="Bump Allocator\n%r15 = heap ptr\n%r13 = heap limit\nheap_grow mmaps 64 MB chunks\n(1 MB under GC_NAIVE)" fillcolor="#ffeaa7"]
+        snap [label="heap-snapshot / heap-restore\n(portable arena, bump-only)\nper-request loops = O(1)" fillcolor="#ffeaa7"]
+        tags [label="Tag-in-Low-3-Bits\n0=int 1=pair 2=sym\n3=closure 4=builtin\n5=special 6=string\n7=vector / hash-table / hash-set\n(disambiguated by negative sentinel)" fillcolor="#ffeaa7"]
+        gc [label="GC_NAIVE build (opt-in)\n• 8 B header per block\n• stop-the-world mark-sweep\n• first-fit free list\n• conservative stack scan" fillcolor="#fab1a0"]
+        meta [label="Meta-GC (with-arena thunk)\n1. snapshot %r15\n2. run thunk (free list off)\n3. mark phase (+ result root)\n4. NO mark in range → bulk reset O(1)\n5. otherwise → sweep fallback" fillcolor="#fd79a8"]
     }
 
     subgraph cluster_interp {
-        label="Interpreter — 4,968 lines of GNU asm (GAS AT&T syntax)"
+        label="Interpreter — 6,645 lines of GNU assembler (GAS, AT&T syntax); as + ld from GNU binutils"
         style=rounded
         color="#666666"
         tokenize [label="Tokenizer\nchar-by-char\n\\n \\r \\t \\0 escapes" fillcolor="#fff3cd"]
         read_fn [label="Reader\nrecursive descent\n→ tagged cons cells\n(load), read-from-string" fillcolor="#fff3cd"]
         eval_fn [label="Evaluator\nTCO via jmp .eval_top\nall special forms\n+ eval as builtin" fillcolor="#d4edda"]
-        print_fn [label="Printer\nitoa, list, vector\ncustom output_fd\nport-aware" fillcolor="#81ecec"]
-        builtins [label="91 Builtins (was 34)\narith cons car cdr list ...\nload ports write-file\nportal-save/resume\ntcp-listen/accept/send/recv\nread-from-string eval" fillcolor="#a29bfe"]
+        print_fn [label="Printer\nitoa, list, vector\nhash-table / hash-set\ncustom output_fd" fillcolor="#81ecec"]
+        builtins [label="95+ Builtins\narith cons car cdr list ...\nload ports write-file\nportal-save/resume\ntcp-listen/accept/send/recv\nread-from-string eval\nnative hash-table-* / hash-set-*\n(GC build: with-arena, gc-collect,\n gc-stats, arena-stats)" fillcolor="#a29bfe"]
     }
 
     subgraph cluster_env {
@@ -66,8 +68,13 @@ digraph asm_arch {
     env_chain -> sym_intern
     bump -> tags
     bump -> snap [label="arena pattern" style=dashed]
+    bump -> gc [label="GC_NAIVE flag" style=dashed color="#fab1a0"]
+    gc -> meta [label="with-arena" color="#fd79a8"]
     eval_fn -> bump [label="cons/closure alloc"]
+    meta -> bump [label="reset %r15" style=dashed]
 
     {rank=same; fs_io; mem_io; net_io; misc}
     {rank=same; tokenize; read_fn}
+    {rank=same; bump; tags}
+    {rank=same; gc; meta}
 }

docs/benchmark-binary-size.dot

@@ -1,18 +1,19 @@
-// Binary size comparison (2026-04-17 refresh)
+// Binary size comparison (2026-04-18 refresh)
 // "A diagram is worth 10,000 words." — russell@unturf.com
 digraph binary_size {
     rankdir=LR
     node [shape=record, style=filled, fontname="Helvetica", fontsize=11]
     edge [style=invis]
-    label="Binary size (smaller is leaner)"
+    label="Binary size (smaller is leaner) — stripped, i5-8350U"
     labelloc=t
     fontsize=14
     fontname="Helvetica Bold"
 
-    asm [label="{Assembly|22 KB stripped\nzero libs\n14 Linux syscalls}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
+    asm [label="{Assembly (bump-only)|23 KB stripped\nzero libs\n14 Linux syscalls}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
+    asm_gc [label="{Assembly (GC_NAIVE)|27 KB stripped\nmark-sweep + meta-GC arena\n+4 KB over bump-only}" fillcolor="#a29bfe" fontcolor=white width=1.2]
     c [label="{C + JIT|205 KB\nlibc only}" fillcolor="#00b894" fontcolor=white width=3.0]
     busybox [label="{busybox httpd|2.1 MB\nmulti-call}" fillcolor="#fdcb6e" width=10.0]
     py [label="{Python 3 interp|8.0 MB\nstandalone binary}" fillcolor="#e17055" fontcolor=white width=15.0]
 
-    asm -> c -> busybox -> py
+    asm -> asm_gc -> c -> busybox -> py
 }

docs/benchmark-gc.dot

@@ -0,0 +1,17 @@
+// GC control-group comparison — 2026-04-18
+// "A diagram is worth 10,000 words." — russell@unturf.com
+digraph bench_gc {
+    rankdir=LR
+    node [shape=record, style=filled, fontname="Helvetica", fontsize=11]
+    edge [style=invis]
+    label="Peak RSS — 2000 × build-list 200 + sum + discard (i5-8350U)"
+    labelloc=t
+    fontsize=14
+    fontname="Helvetica Bold"
+
+    bump [label="{Asm bump-only|peak RSS: 133.9 MB\nunbounded growth}" fillcolor="#e17055" fontcolor=white width=13.0]
+    gc [label="{Asm naive GC|peak RSS: 1.1 MB\nsteady state\n-30% throughput}" fillcolor="#00b894" fontcolor=white width=1.0]
+    arena [label="{Asm meta-GC arena|peak RSS: 1.2 MB\n2000/2000 resets\n-1% throughput vs naive}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
+
+    bump -> gc -> arena
+}

docs/meta-gc-policy.dot

@@ -0,0 +1,33 @@
+// Meta-GC decision policy — 2026-04-18
+// "A diagram is worth 10,000 words." — russell@unturf.com
+digraph meta_gc {
+    rankdir=LR
+    node [shape=box, style="filled,rounded", fontname="Helvetica", fontsize=10]
+    edge [fontname="Helvetica", fontsize=9]
+    label="Meta-GC decision at (with-arena thunk) exit"
+    labelloc=t
+    fontsize=13
+    fontname="Helvetica Bold"
+
+    enter [label="Arena enter\nsnapshot %r15\nactive ← 1\nfreelist OFF" fillcolor="#fff3cd"]
+    thunk [label="Run thunk\n(apply_proc_raw)\n+ zero volatile regs" fillcolor="#d4edda"]
+    gate1 [label="arena_active\nstill 1?" shape=diamond fillcolor="#fab1a0"]
+    verify [label="Mark phase\n(globals, syms,\nstack, + result)" fillcolor="#81ecec"]
+    gate2 [label="Mark in\n[snap, r15)?" shape=diamond fillcolor="#81ecec"]
+
+    reset [label="BULK RESET O(1)\n%r15 ← snap\nresets++" fillcolor="#00b894" fontcolor=white]
+    escape [label="SWEEP FALLBACK\ngc_sweep full\nescapes++" fillcolor="#e17055" fontcolor=white]
+    abort [label="SKIP reset\n(implicit GC ran)\nescapes++" fillcolor="#fab1a0"]
+
+    ret [label="Return result\n(arena-stats)" fillcolor="#dfe6e9"]
+
+    enter -> thunk -> gate1
+    gate1 -> verify [label="yes"]
+    gate1 -> abort [label="no"]
+    verify -> gate2
+    gate2 -> reset [label="no"]
+    gate2 -> escape [label="yes"]
+    reset -> ret
+    escape -> ret
+    abort -> ret
+}

whitepaper/uncommonlisp-whitepaper.rst

@@ -73,7 +73,7 @@ All three share one interchange format: **Scheme source itself**. An S-expressio
 
 **S-expressions over sockets.** The wire protocol for a 90-line RPC server is one Scheme form per connection. With ``read-from-string`` and ``eval`` added to all three impls (89 bytes of asm for ``eval``, a 20-line asm reader swap for ``read-from-string``), a transparent byte-forwarding relay composes arbitrary chains: a Python client can reach an asm backend through a C relay and a Python relay, four runtimes strung together without any format translation between hops. Each relay adds ~650 µs/request on the same laptop. The language is the envelope.
 
-975 verified assertions pass identically across the three implementations (571 Python unit, 132 asm, 189 shared Python+C functional, 83 C unit). Every implementation consumes every format it can reach; mismatch cases (wrong format, truncated input, missing file, corrupt header) degrade gracefully with ``#f`` or a clean error.
+980 verified assertions pass identically across the three implementations (571 Python unit, 137 asm unit + integration + functional, 189 shared Python+C functional, 83 C unit). The optional GC build passes the same 137 asm tests independently, making it 1,117 assertions with both asm binaries exercised. Every implementation consumes every format it can reach; mismatch cases (wrong format, truncated input, missing file, corrupt header) degrade gracefully with ``#f`` or a clean error.
 
 The paper further presents the EML universality proof: a single operator ``eml(x, y) = exp(x) - ln(y)`` with the constant 1 generates all elementary functions (exp, ln, arithmetic, negation, complex plane access, trigonometry). Verified numerically in Python, verified in Lumbda's own bytecode, & proven formally in Lean 4 with zero ``sorry``.
 
@@ -499,6 +499,12 @@ Same source, same tests (137/137 pass on both). The GC build adds an 8-byte ``(s
 
 **124× less memory at a ~30% throughput cost.** That is the honest GC tax at the naive end of the spectrum — the number we had been guessing at before building the control group. Every future memory-management proposal (generational, incremental, region-based) now has a concrete floor to beat.
 
+.. figure:: diagrams/benchmark-gc.png
+   :width: 88%
+   :align: center
+
+   *Peak RSS across the three asm memory strategies on the same 2,000-iteration build-sum-discard workload. Bump-only grows unboundedly (133.9 MB) because nothing is freed; naive mark-sweep bounds at 1.1 MB (steady state) at a ~30% throughput cost; the meta-GC arena wrapping the same body reaches 1.2 MB at roughly the same time as naive (100% reset rate on this workload means the arena path replaces 199 of the 200 sweeps with O(1) bulk reclaims).*
+
 6.6.1 Meta-GC: Arena Fast Path with Mark-Phase Verifier
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -510,6 +516,12 @@ On top of the naive sweep, the GC build exposes a Lumbda primitive ``(with-arena
 
 Free-list reuse is disabled while an arena is active so the chain stays pristine for a verbatim restore; ``heap_alloc`` enforces this via a global ``arena_active`` flag. One critical correctness detail: after ``apply_proc_raw`` returns from the thunk, volatile registers contain stale tagged pointers into the arena. The conservative stack scan would otherwise treat those residuals as live roots and trigger a false escape on every call. Zero-ing ``%rax, %rcx, %rdx, %rsi, %rdi, %rbp, %r8-%r12`` before the verifier runs removes this hazard.
 
+.. figure:: diagrams/meta-gc-policy.png
+   :width: 88%
+   :align: center
+
+   *Three-way policy the meta-GC runs at every ``(with-arena thunk)`` exit. Green path = O(1) bulk reset (the transient-workload win); red path = escape detected, fall through to naive sweep (correctness preserved when data survives the thunk); orange path = implicit GC fired during the thunk, snapshot is already stale so skip the reset attempt. Every counter the policy updates is observable via ``(arena-stats)``.*
+
 **Meta-GC benchmark** (same workload, same binary, both phases in one process):
 
 .. table::
@@ -925,11 +937,11 @@ All C & Python benchmarks measured in-process (no startup overhead). Assembly ti
 11.1 Test Coverage
 ^^^^^^^^^^^^^^^^^^^^
 
-**975 verified assertions** across all implementations, all green under ``make test-all``:
+**980 verified assertions** across all implementations, all green under ``make test-all``:
 
 - Python unit + integration: **571 tests** (``tests.py``)
 - C unit + integration + JIT + continuations + portal: **83 tests** (``c/test.c``)
-- Assembly unit + integration + functional: **132 tests** (``asm/test.sh``)
+- Assembly unit + integration + functional: **137 tests** (``asm/test.sh``), passing identically under both the bump-only and ``GC_NAIVE`` builds
 - Shared functional (same ``.lsp`` in Python + C): **189 tests** (``tests/functional.lsp``)
 
 The shared functional suite matters: it runs byte-identical Scheme source through two different runtimes & compares output. Python & C agree 189 times per run. When they disagree, that tells us something specific & actionable.
@@ -1159,7 +1171,7 @@ The Python implementation had a similar defect: ``_define_record_type`` used ``l
 - ``c/builtins.c``'s ``bi_string_replace`` scanned the source byte-by-byte, calling ``strncmp(src, from, from_len)`` at every position. O(N·k). Replaced with ``strstr`` (libc-tuned, typically Boyer-Moore-Horspool) called in a loop that skips to the next match. O(N + matches·k).
 - ``uncommonlisp.py``'s ``_tokenize_lines`` called ``src.count('\\n', 0, m.start())`` per token to compute line numbers. O(N·M). Replaced with a single pass that builds a ``line_starts`` array, then bisects per token. O(M + N log M).
 
-A third correctness fix landed after the portal-over-HTTP demo exposed it: ``uncommonlisp.py``'s ``Env.lookup`` used to short-cut from the local frame directly to the global frame before walking intermediate parents. That was fast but wrong — a let-loop parameter named the same as a global builtin (``count``, a SRFI-1 procedure) got shadowed in reverse, the shortcut returned the global builtin instead of walking up to the loop's parameter frame. Fix: walk ``self → self.p → ... → global`` in order, without any shortcut. The inline cache at ``OP_LOOKUP`` was correspondingly tightened to validate the full chain before firing. 975 tests remained green.
+A third correctness fix landed after the portal-over-HTTP demo exposed it: ``uncommonlisp.py``'s ``Env.lookup`` used to short-cut from the local frame directly to the global frame before walking intermediate parents. That was fast but wrong — a let-loop parameter named the same as a global builtin (``count``, a SRFI-1 procedure) got shadowed in reverse, the shortcut returned the global builtin instead of walking up to the loop's parameter frame. Fix: walk ``self → self.p → ... → global`` in order, without any shortcut. The inline cache at ``OP_LOOKUP`` was correspondingly tightened to validate the full chain before firing. 980 tests remained green.
 
 Every release audit surfaces more. Writing new code is writing new sediment, unless the audit runs.