whitepaper: diagrams + stats refresh for GC / meta-GC / hash primitives

Diagrams:
  - asm-architecture.dot: adds GC_NAIVE memory cluster (bump, free
    list, conservative stack scan) + meta-GC (with-arena) cluster
    showing the reset path; updates line count (4968 -> 6645),
    builtin count (91 -> 95+), mentions native hash-table-* /
    hash-set-* and the GC-build primitives (with-arena, gc-collect,
    gc-stats, arena-stats).
  - benchmark-binary-size.dot: adds second asm bar for the GC_NAIVE
    build (27 KB stripped vs 23 KB bump-only); updated asm bump
    size from 22 KB (stale) to actual 23 KB.
  - benchmark-gc.dot (new): side-by-side peak RSS for asm bump-only
    (134 MB), naive GC (1.1 MB), meta-GC arena (1.2 MB, 2000/2000
    resets); embedded in §6.6.
  - meta-gc-policy.dot (new): three-way decision tree at
    (with-arena) exit — implicit-GC-fired / mark-in-arena-range /
    no-mark-in-range -> skip / sweep / bulk-reset; embedded in
    §6.6.1.

Stats:
  - 975 verified assertions -> 980 (asm gained 5 via hash-table &
    hash-set tests; 571 Python + 137 asm + 83 C + 189 shared).
  - asm test count 132 -> 137 in the summary list, intro abstract,
    and §11 tier table. Notes that the optional GC build passes
    the same 137 independently (1,117 assertions total when both
    asm binaries are exercised).
  - Stale 4,968 LOC -> 6,645 already fixed in the prior commit;
    the new asm-architecture diagram now matches.

PDF rebuilt, 2.58 MB (was 2.40 MB). All test suites green.
This commit is contained in:
russell@unturf.com 2026-04-18 10:46:29 -04:00
parent ef77a8023f
commit 9b1a60226d
14 changed files with 1175 additions and 996 deletions

View file

@ -1,4 +1,4 @@
// Assembly implementation architecture — 2026-04-17 refresh
// Assembly implementation architecture — 2026-04-18 refresh
// "A diagram is worth 10,000 words." — russell@unturf.com
digraph asm_arch {
rankdir=TB
@ -6,7 +6,7 @@ digraph asm_arch {
edge [fontname="Helvetica", fontsize=10]
subgraph cluster_binary {
label="Binary: 22 KB stripped, zero dependencies"
label="Binary: ~22 KB stripped (bump-only) / ~25 KB (GC_NAIVE), zero dependencies"
style=rounded
color="#333333"
fontcolor="#333333"
@ -24,23 +24,25 @@ digraph asm_arch {
}
subgraph cluster_memory {
label="Memory Model"
label="Memory Model — default bump; optional mark-sweep + meta-GC"
style=rounded
color="#666666"
bump [label="Bump Allocator\n%r15 = heap ptr\n%r13 = heap limit\nheap_grow mmaps 64 MB" fillcolor="#ffeaa7"]
snap [label="heap-snapshot\nheap-restore\narena for per-request\nloops (O(1) memory)" fillcolor="#ffeaa7"]
tags [label="Tag-in-Low-3-Bits\n0=int 1=pair 2=sym\n3=closure 4=builtin\n5=special 6=string\n7=vector" fillcolor="#ffeaa7"]
bump [label="Bump Allocator\n%r15 = heap ptr\n%r13 = heap limit\nheap_grow mmaps 64 MB chunks\n(1 MB under GC_NAIVE)" fillcolor="#ffeaa7"]
snap [label="heap-snapshot / heap-restore\n(portable arena, bump-only)\nper-request loops = O(1)" fillcolor="#ffeaa7"]
tags [label="Tag-in-Low-3-Bits\n0=int 1=pair 2=sym\n3=closure 4=builtin\n5=special 6=string\n7=vector / hash-table / hash-set\n(disambiguated by negative sentinel)" fillcolor="#ffeaa7"]
gc [label="GC_NAIVE build (opt-in)\n• 8 B header per block\n• stop-the-world mark-sweep\n• first-fit free list\n• conservative stack scan" fillcolor="#fab1a0"]
meta [label="Meta-GC (with-arena thunk)\n1. snapshot %r15\n2. run thunk (free list off)\n3. mark phase (+ result root)\n4. NO mark in range → bulk reset O(1)\n5. otherwise → sweep fallback" fillcolor="#fd79a8"]
}
subgraph cluster_interp {
label="Interpreter — 4,968 lines of GNU asm (GAS AT&T syntax)"
label="Interpreter — 6,645 lines of GNU assembler (GAS, AT&T syntax); as + ld from GNU binutils"
style=rounded
color="#666666"
tokenize [label="Tokenizer\nchar-by-char\n\\n \\r \\t \\0 escapes" fillcolor="#fff3cd"]
read_fn [label="Reader\nrecursive descent\n→ tagged cons cells\n(load), read-from-string" fillcolor="#fff3cd"]
eval_fn [label="Evaluator\nTCO via jmp .eval_top\nall special forms\n+ eval as builtin" fillcolor="#d4edda"]
print_fn [label="Printer\nitoa, list, vector\ncustom output_fd\nport-aware" fillcolor="#81ecec"]
builtins [label="91 Builtins (was 34)\narith cons car cdr list ...\nload ports write-file\nportal-save/resume\ntcp-listen/accept/send/recv\nread-from-string eval" fillcolor="#a29bfe"]
print_fn [label="Printer\nitoa, list, vector\nhash-table / hash-set\ncustom output_fd" fillcolor="#81ecec"]
builtins [label="95+ Builtins\narith cons car cdr list ...\nload ports write-file\nportal-save/resume\ntcp-listen/accept/send/recv\nread-from-string eval\nnative hash-table-* / hash-set-*\n(GC build: with-arena, gc-collect,\n gc-stats, arena-stats)" fillcolor="#a29bfe"]
}
subgraph cluster_env {
@ -66,8 +68,13 @@ digraph asm_arch {
env_chain -> sym_intern
bump -> tags
bump -> snap [label="arena pattern" style=dashed]
bump -> gc [label="GC_NAIVE flag" style=dashed color="#fab1a0"]
gc -> meta [label="with-arena" color="#fd79a8"]
eval_fn -> bump [label="cons/closure alloc"]
meta -> bump [label="reset %r15" style=dashed]
{rank=same; fs_io; mem_io; net_io; misc}
{rank=same; tokenize; read_fn}
{rank=same; bump; tags}
{rank=same; gc; meta}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 276 KiB

Before After
Before After

View file

@ -1,18 +1,19 @@
// Binary size comparison (2026-04-17 refresh)
// Binary size comparison (2026-04-18 refresh)
// "A diagram is worth 10,000 words." — russell@unturf.com
digraph binary_size {
rankdir=LR
node [shape=record, style=filled, fontname="Helvetica", fontsize=11]
edge [style=invis]
label="Binary size (smaller is leaner)"
label="Binary size (smaller is leaner) — stripped, i5-8350U"
labelloc=t
fontsize=14
fontname="Helvetica Bold"
asm [label="{Assembly|22 KB stripped\nzero libs\n14 Linux syscalls}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
asm [label="{Assembly (bump-only)|23 KB stripped\nzero libs\n14 Linux syscalls}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
asm_gc [label="{Assembly (GC_NAIVE)|27 KB stripped\nmark-sweep + meta-GC arena\n+4 KB over bump-only}" fillcolor="#a29bfe" fontcolor=white width=1.2]
c [label="{C + JIT|205 KB\nlibc only}" fillcolor="#00b894" fontcolor=white width=3.0]
busybox [label="{busybox httpd|2.1 MB\nmulti-call}" fillcolor="#fdcb6e" width=10.0]
py [label="{Python 3 interp|8.0 MB\nstandalone binary}" fillcolor="#e17055" fontcolor=white width=15.0]
asm -> c -> busybox -> py
asm -> asm_gc -> c -> busybox -> py
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 34 KiB

Before After
Before After

17
docs/benchmark-gc.dot Normal file
View file

@ -0,0 +1,17 @@
// GC control-group comparison — 2026-04-18
// "A diagram is worth 10,000 words." — russell@unturf.com
digraph bench_gc {
rankdir=LR
node [shape=record, style=filled, fontname="Helvetica", fontsize=11]
edge [style=invis]
label="Peak RSS — 2000 × build-list 200 + sum + discard (i5-8350U)"
labelloc=t
fontsize=14
fontname="Helvetica Bold"
bump [label="{Asm bump-only|peak RSS: 133.9 MB\nunbounded growth}" fillcolor="#e17055" fontcolor=white width=13.0]
gc [label="{Asm naive GC|peak RSS: 1.1 MB\nsteady state\n-30% throughput}" fillcolor="#00b894" fontcolor=white width=1.0]
arena [label="{Asm meta-GC arena|peak RSS: 1.2 MB\n2000/2000 resets\n-1% throughput vs naive}" fillcolor="#6c5ce7" fontcolor=white width=1.0]
bump -> gc -> arena
}

BIN
docs/benchmark-gc.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

33
docs/meta-gc-policy.dot Normal file
View file

@ -0,0 +1,33 @@
// Meta-GC decision policy — 2026-04-18
// "A diagram is worth 10,000 words." — russell@unturf.com
digraph meta_gc {
rankdir=LR
node [shape=box, style="filled,rounded", fontname="Helvetica", fontsize=10]
edge [fontname="Helvetica", fontsize=9]
label="Meta-GC decision at (with-arena thunk) exit"
labelloc=t
fontsize=13
fontname="Helvetica Bold"
enter [label="Arena enter\nsnapshot %r15\nactive ← 1\nfreelist OFF" fillcolor="#fff3cd"]
thunk [label="Run thunk\n(apply_proc_raw)\n+ zero volatile regs" fillcolor="#d4edda"]
gate1 [label="arena_active\nstill 1?" shape=diamond fillcolor="#fab1a0"]
verify [label="Mark phase\n(globals, syms,\nstack, + result)" fillcolor="#81ecec"]
gate2 [label="Mark in\n[snap, r15)?" shape=diamond fillcolor="#81ecec"]
reset [label="BULK RESET O(1)\n%r15 ← snap\nresets++" fillcolor="#00b894" fontcolor=white]
escape [label="SWEEP FALLBACK\ngc_sweep full\nescapes++" fillcolor="#e17055" fontcolor=white]
abort [label="SKIP reset\n(implicit GC ran)\nescapes++" fillcolor="#fab1a0"]
ret [label="Return result\n(arena-stats)" fillcolor="#dfe6e9"]
enter -> thunk -> gate1
gate1 -> verify [label="yes"]
gate1 -> abort [label="no"]
verify -> gate2
gate2 -> reset [label="no"]
gate2 -> escape [label="yes"]
reset -> ret
escape -> ret
abort -> ret
}

BIN
docs/meta-gc-policy.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB