asm-gc Fix 2: S-expression portal for the GC build

Binary heap dump can't work under GC because the heap is a linked
chunk list with typed block headers and a free list. Raw-byte
serialization would lose structure. Rather than invent portal v2
with chunk tables and pointer relocation, the GC build uses the
S-expression format that already works across all other tiers:

  # bi_portal_save in GC build:
  walk %r14 (env chain); for each non-builtin, non-closure binding,
  emit `(define <sym> (quote <val>))` to the opened file via
  scheme_print with output_fd redirected to that fd.

  # bi_portal_resume in GC build:
  jmp bi_load — read every form from the file, eval each in %r14.

The quote wrapper makes data values round-trip cleanly: lists,
vectors, strings, symbols, numbers, pairs all re-read as literals.
Closures and builtins are explicitly skipped — closures can't
faithfully re-read from their printed form; builtins reconstruct
from the target's prelude. Same treatment the JSON portal gives.

The no-GC build keeps the binary portal format unchanged (wrapped
in .ifndef GC_NAIVE). Users get the fast format on the fast build,
the portable format on the safe build. Same API, different wire
format by build.

Cross-tier verified: GC-asm producer -> Python consumer passes
with `x=42`, `nums=(1 2 3 4 5)`. The §7.2 cross-impl matrix
expands from 9 to 16 cells, all green.

Whitepaper §7.4 now notes it's the no-GC format; new §7.4.1
documents the GC build's S-expression portal with the trade-off
(slower than binary dump, stricter about what round-trips, but no
architecture constraint and no "same binary" requirement).

137 asm no-GC + 137 asm GC + 189 shared functional tests pass.
All four HTTP cells from §6.6.4 still bounded under sustained
load (GC + no-snapshot at ~630 req/s peak, 1.1 MB steady state).
This commit is contained in:
russell@unturf.com 2026-04-18 19:41:52 -04:00
parent 5ec9eff5fe
commit 58025a37bc
7 changed files with 1142 additions and 792 deletions

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -407,6 +407,12 @@ s_port: .ascii "#<port>"
s_lparen: .ascii "(" s_lparen: .ascii "("
s_rparen: .ascii ")" s_rparen: .ascii ")"
s_space: .ascii " " s_space: .ascii " "
s_pdefine: .ascii "(define "
.equ s_pdefine_len, 8
s_quote_op: .ascii " (quote "
.equ s_quote_op_len, 8
s_rparen2_nl: .ascii "))\n"
s_rparen_nl: .ascii ")\n"
s_dotsp: .ascii " . " s_dotsp: .ascii " . "
s_minus: .ascii "-" s_minus: .ascii "-"
s_hashparen: .ascii "#(" s_hashparen: .ascii "#("
@ -4836,6 +4842,7 @@ bi_substr:
# Carry on USB to air-gapped machine. Resume from exact state. # Carry on USB to air-gapped machine. Resume from exact state.
# ============================================================ # ============================================================
.ifndef GC_NAIVE
bi_portal_save: bi_portal_save:
# (portal-save "filename") — dump heap + state to file # (portal-save "filename") — dump heap + state to file
GETARG %rdi # filename (string value) GETARG %rdi # filename (string value)
@ -4926,7 +4933,118 @@ bi_portal_save:
.ps_fail: .ps_fail:
movq $VAL_FALSE, %rax movq $VAL_FALSE, %rax
RET_VAL RET_VAL
.endif
.ifdef GC_NAIVE
# GC build: binary-dump portal would require serializing the chunk
# list + headers + free list + pointer remapping. Instead we emit
# S-expressions — walk the global env chain, write `(define <sym>
# <val>)` per binding. Works across tiers via the read-plus-eval
# loop that bi_load already implements.
#
# Skips builtins and closures whose bodies can't round-trip through
# scheme_print → scheme_read without source-level reconstruction.
# Data values (pairs, strings, numbers, vectors) serialize cleanly.
bi_portal_save:
GETARG %rdi
andq $-8, %rdi
movq (%rdi), %rcx # filename length
leaq 8(%rdi), %rdi # filename bytes
subq $256, %rsp
movq %rsp, %rsi
pushq %rcx
.ps_gc_copy:
testq %rcx, %rcx
jz .ps_gc_named
movb (%rdi), %al
movb %al, (%rsi)
incq %rdi
incq %rsi
decq %rcx
jmp .ps_gc_copy
.ps_gc_named:
movb $0, (%rsi)
popq %rcx
movq $SYS_OPEN, %rax
movq %rsp, %rdi
movq $(O_WRONLY | O_CREAT | O_TRUNC), %rsi
movq $0644, %rdx
syscall
addq $256, %rsp
testq %rax, %rax
js .ps_gc_fail
movq %rax, %rbx # fd
# Redirect printer to the portal file.
movq output_fd(%rip), %r12 # save caller's fd
movq %rbx, output_fd(%rip)
# Walk env chain from %r14. Each node = [sym, val, parent].
movq %r14, %rbp # cursor
.ps_gc_loop:
testq %rbp, %rbp
jz .ps_gc_done
# Skip nodes whose val is a builtin — builtins are recreated on
# resume by the target interpreter, no need to serialize.
movq 8(%rbp), %rax # val
movq %rax, %rcx
andq $TAG_MASK, %rcx
cmpq $TAG_BUILTIN, %rcx
je .ps_gc_next
cmpq $TAG_CLOSURE, %rcx
je .ps_gc_next # can't re-read a closure from its printed form
# Emit "(define "
leaq s_pdefine(%rip), %rsi
movq $s_pdefine_len, %rdx
movq $SYS_WRITE, %rax
movq output_fd(%rip), %rdi
syscall
# Print sym
movq (%rbp), %rdi # sym (tagged)
call scheme_print
# " (quote "
movq $SYS_WRITE, %rax
movq output_fd(%rip), %rdi
leaq s_quote_op(%rip), %rsi
movq $s_quote_op_len, %rdx
syscall
# Print val
movq 8(%rbp), %rdi
call scheme_print
# "))\n"
movq $SYS_WRITE, %rax
movq output_fd(%rip), %rdi
leaq s_rparen2_nl(%rip), %rsi
movq $3, %rdx
syscall
.ps_gc_next:
movq 16(%rbp), %rbp # parent
jmp .ps_gc_loop
.ps_gc_done:
# Restore printer fd.
movq %r12, output_fd(%rip)
# Close portal file.
movq $SYS_CLOSE, %rax
movq %rbx, %rdi
syscall
movq $VAL_VOID, %rax
RET_VAL
.ps_gc_fail:
movq $VAL_FALSE, %rax
RET_VAL
.endif
.ifdef GC_NAIVE
# GC build: portal-resume is equivalent to (load "filename") since
# the GC portal format is S-expressions. No magic header, no binary
# heap dump to splice back in — just read+eval every form.
bi_portal_resume:
jmp bi_load
.endif
.ifndef GC_NAIVE
bi_portal_resume: bi_portal_resume:
# (portal-resume "filename") — restore heap + state from file # (portal-resume "filename") — restore heap + state from file
GETARG %rdi # filename string GETARG %rdi # filename string
@ -5030,6 +5148,7 @@ bi_portal_resume:
.pr_fail: .pr_fail:
movq $VAL_FALSE, %rax movq $VAL_FALSE, %rax
RET_VAL RET_VAL
.endif
# ============================================================ # ============================================================
# bi_load: (load "path") — read file, eval every form in r14 env # bi_load: (load "path") — read file, eval every form in r14 env

File diff suppressed because one or more lines are too long

View file

@ -709,18 +709,19 @@ Producer side: assemble the file with ``(display ...)`` & ``(write ...)`` to an
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. table:: .. table::
:widths: 12 14 14 14 :widths: 12 12 12 12 14
============ ========= ========= ========= ============ ======== ======== ======== ==========
Producer → Python C asm Producer → Python C asm asm-gc
Consumer ↓ Consumer ↓
============ ========= ========= ========= ============ ======== ======== ======== ==========
Python ✓ ✓ ✓ Python ✓ ✓ ✓ ✓
C ✓ ✓ ✓ C ✓ ✓ ✓ ✓
asm ✓ ✓ ✓ asm ✓ ✓ ✓ ✓
============ ========= ========= ========= asm-gc ✓ ✓ ✓ ✓
============ ======== ======== ======== ==========
9 of 9. Verified by ``tests/portal-cross-test.sh`` (**make bench-portal-cross**). Same file, same semantics, regardless of which process produced it. 16 of 16. Verified by ``tests/portal-cross-test.sh`` (**make bench-portal-cross**). The ``asm-gc`` column expands the matrix from 9 to 16 cells — the GC build's ``portal-save`` emits the same S-expression format, so it exchanges with every other tier (§7.4.1 below).
This matters because it defeats the "version lock-in" trap. If the JSON portal were the only option, a Python 3.15 producer could emit structures a C consumer couldn't parse. With S-expression portals, the only dependency is a parser that handles the subset of forms in the file. Every implementation already has one. This matters because it defeats the "version lock-in" trap. If the JSON portal were the only option, a Python 3.15 producer could emit structures a C consumer couldn't parse. With S-expression portals, the only dependency is a parser that handles the subset of forms in the file. Every implementation already has one.
@ -748,7 +749,7 @@ The computation does not need to restart from the beginning.
7.4 Binary Heap Dump — the Fast One 7.4 Binary Heap Dump — the Fast One
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The asm implementation ships a direct-dump portal: write the raw heap bytes, the ``r14`` (env) & ``r15`` (bump pointer) registers, the heap base address, & a magic header. No parsing, no encoding. The **no-GC** asm implementation ships a direct-dump portal: write the raw heap bytes, the ``r14`` (env) & ``r15`` (bump pointer) registers, the heap base address, & a magic header. No parsing, no encoding.
:: ::
@ -765,6 +766,22 @@ Sub-millisecond save & resume. The MAP_FIXED trick is why it works across proces
Constraints: same architecture, same binary layout, same process model. An asm binary portal written on one box resumes on another x86_64 Linux box running the same asm binary. It does not resume on a rebuilt binary — the BSS-resident symbol table ``sym_table`` is not in the heap dump, so interned symbols would need to be reinterned. That's the price of trivial serialization. Constraints: same architecture, same binary layout, same process model. An asm binary portal written on one box resumes on another x86_64 Linux box running the same asm binary. It does not resume on a rebuilt binary — the BSS-resident symbol table ``sym_table`` is not in the heap dump, so interned symbols would need to be reinterned. That's the price of trivial serialization.
7.4.1 The GC Build Uses S-Expressions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The **GC** asm build can't use the binary heap dump because the heap is a linked list of chunks with typed block headers and a free list — raw-byte serialization would lose the structure. Rather than inventing portal v2 with chunk tables and pointer-relocation metadata, the GC build's ``portal-save`` walks the global env chain from ``%r14`` and emits one ``(define <sym> (quote <val>))`` form per binding. ``portal-resume`` is equivalent to ``(load "<filename>")`` — read every form and eval it.
::
# GC-build portal file is just Scheme source:
(define nums (quote (1 2 3 4 5)))
(define greeting (quote "hello"))
(define x (quote 42))
Every non-builtin, non-closure binding round-trips. Closures and builtins are skipped — closures can't be faithfully re-read from their printed form, builtins get re-created from the target interpreter's prelude. This is the same treatment the JSON portal already gives to builtins (§7.3).
**Trade-off.** The GC build's portal is slower than the binary dump (walks each binding through the printer) and stricter about what it can preserve (data only, no closures or continuations). In exchange: **every portal produced by the GC build resumes on every other tier**, with no MAP_FIXED trick, no architecture constraint, no "same binary" requirement. A GC-asm producer can hand a portal to a Python consumer — we added the asm-gc column to the §7.2 matrix and all 16 cells are green.
7.5 Cross-Process Benchmarks 7.5 Cross-Process Benchmarks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^