asm-gc Fix 2: S-expression portal for the GC build
Binary heap dump can't work under GC because the heap is a linked chunk list with typed block headers and a free list. Raw-byte serialization would lose structure. Rather than invent portal v2 with chunk tables and pointer relocation, the GC build uses the S-expression format that already works across all other tiers: # bi_portal_save in GC build: walk %r14 (env chain); for each non-builtin, non-closure binding, emit `(define <sym> (quote <val>))` to the opened file via scheme_print with output_fd redirected to that fd. # bi_portal_resume in GC build: jmp bi_load — read every form from the file, eval each in %r14. The quote wrapper makes data values round-trip cleanly: lists, vectors, strings, symbols, numbers, pairs all re-read as literals. Closures and builtins are explicitly skipped — closures can't faithfully re-read from their printed form; builtins reconstruct from the target's prelude. Same treatment the JSON portal gives. The no-GC build keeps the binary portal format unchanged (wrapped in .ifndef GC_NAIVE). Users get the fast format on the fast build, the portable format on the safe build. Same API, different wire format by build. Cross-tier verified: GC-asm producer -> Python consumer passes with `x=42`, `nums=(1 2 3 4 5)`. The §7.2 cross-impl matrix expands from 9 to 16 cells, all green. Whitepaper §7.4 now notes it's the no-GC format; new §7.4.1 documents the GC build's S-expression portal with the trade-off (slower than binary dump, stricter about what round-trips, but no architecture constraint and no "same binary" requirement). 137 asm no-GC + 137 asm GC + 189 shared functional tests pass. All four HTTP cells from §6.6.4 still bounded under sustained load (GC + no-snapshot at ~630 req/s peak, 1.1 MB steady state).
This commit is contained in:
parent
5ec9eff5fe
commit
58025a37bc
7 changed files with 1142 additions and 792 deletions
BIN
asm/uncommonlisp
BIN
asm/uncommonlisp
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -407,6 +407,12 @@ s_port: .ascii "#<port>"
|
||||||
s_lparen: .ascii "("
|
s_lparen: .ascii "("
|
||||||
s_rparen: .ascii ")"
|
s_rparen: .ascii ")"
|
||||||
s_space: .ascii " "
|
s_space: .ascii " "
|
||||||
|
s_pdefine: .ascii "(define "
|
||||||
|
.equ s_pdefine_len, 8
|
||||||
|
s_quote_op: .ascii " (quote "
|
||||||
|
.equ s_quote_op_len, 8
|
||||||
|
s_rparen2_nl: .ascii "))\n"
|
||||||
|
s_rparen_nl: .ascii ")\n"
|
||||||
s_dotsp: .ascii " . "
|
s_dotsp: .ascii " . "
|
||||||
s_minus: .ascii "-"
|
s_minus: .ascii "-"
|
||||||
s_hashparen: .ascii "#("
|
s_hashparen: .ascii "#("
|
||||||
|
|
@ -4836,6 +4842,7 @@ bi_substr:
|
||||||
# Carry on USB to air-gapped machine. Resume from exact state.
|
# Carry on USB to air-gapped machine. Resume from exact state.
|
||||||
# ============================================================
|
# ============================================================
|
||||||
|
|
||||||
|
.ifndef GC_NAIVE
|
||||||
bi_portal_save:
|
bi_portal_save:
|
||||||
# (portal-save "filename") — dump heap + state to file
|
# (portal-save "filename") — dump heap + state to file
|
||||||
GETARG %rdi # filename (string value)
|
GETARG %rdi # filename (string value)
|
||||||
|
|
@ -4926,7 +4933,118 @@ bi_portal_save:
|
||||||
.ps_fail:
|
.ps_fail:
|
||||||
movq $VAL_FALSE, %rax
|
movq $VAL_FALSE, %rax
|
||||||
RET_VAL
|
RET_VAL
|
||||||
|
.endif
|
||||||
|
|
||||||
|
.ifdef GC_NAIVE
|
||||||
|
# GC build: binary-dump portal would require serializing the chunk
|
||||||
|
# list + headers + free list + pointer remapping. Instead we emit
|
||||||
|
# S-expressions — walk the global env chain, write `(define <sym>
|
||||||
|
# <val>)` per binding. Works across tiers via the read-plus-eval
|
||||||
|
# loop that bi_load already implements.
|
||||||
|
#
|
||||||
|
# Skips builtins and closures whose bodies can't round-trip through
|
||||||
|
# scheme_print → scheme_read without source-level reconstruction.
|
||||||
|
# Data values (pairs, strings, numbers, vectors) serialize cleanly.
|
||||||
|
bi_portal_save:
|
||||||
|
GETARG %rdi
|
||||||
|
andq $-8, %rdi
|
||||||
|
movq (%rdi), %rcx # filename length
|
||||||
|
leaq 8(%rdi), %rdi # filename bytes
|
||||||
|
|
||||||
|
subq $256, %rsp
|
||||||
|
movq %rsp, %rsi
|
||||||
|
pushq %rcx
|
||||||
|
.ps_gc_copy:
|
||||||
|
testq %rcx, %rcx
|
||||||
|
jz .ps_gc_named
|
||||||
|
movb (%rdi), %al
|
||||||
|
movb %al, (%rsi)
|
||||||
|
incq %rdi
|
||||||
|
incq %rsi
|
||||||
|
decq %rcx
|
||||||
|
jmp .ps_gc_copy
|
||||||
|
.ps_gc_named:
|
||||||
|
movb $0, (%rsi)
|
||||||
|
popq %rcx
|
||||||
|
|
||||||
|
movq $SYS_OPEN, %rax
|
||||||
|
movq %rsp, %rdi
|
||||||
|
movq $(O_WRONLY | O_CREAT | O_TRUNC), %rsi
|
||||||
|
movq $0644, %rdx
|
||||||
|
syscall
|
||||||
|
addq $256, %rsp
|
||||||
|
testq %rax, %rax
|
||||||
|
js .ps_gc_fail
|
||||||
|
movq %rax, %rbx # fd
|
||||||
|
|
||||||
|
# Redirect printer to the portal file.
|
||||||
|
movq output_fd(%rip), %r12 # save caller's fd
|
||||||
|
movq %rbx, output_fd(%rip)
|
||||||
|
|
||||||
|
# Walk env chain from %r14. Each node = [sym, val, parent].
|
||||||
|
movq %r14, %rbp # cursor
|
||||||
|
.ps_gc_loop:
|
||||||
|
testq %rbp, %rbp
|
||||||
|
jz .ps_gc_done
|
||||||
|
# Skip nodes whose val is a builtin — builtins are recreated on
|
||||||
|
# resume by the target interpreter, no need to serialize.
|
||||||
|
movq 8(%rbp), %rax # val
|
||||||
|
movq %rax, %rcx
|
||||||
|
andq $TAG_MASK, %rcx
|
||||||
|
cmpq $TAG_BUILTIN, %rcx
|
||||||
|
je .ps_gc_next
|
||||||
|
cmpq $TAG_CLOSURE, %rcx
|
||||||
|
je .ps_gc_next # can't re-read a closure from its printed form
|
||||||
|
# Emit "(define "
|
||||||
|
leaq s_pdefine(%rip), %rsi
|
||||||
|
movq $s_pdefine_len, %rdx
|
||||||
|
movq $SYS_WRITE, %rax
|
||||||
|
movq output_fd(%rip), %rdi
|
||||||
|
syscall
|
||||||
|
# Print sym
|
||||||
|
movq (%rbp), %rdi # sym (tagged)
|
||||||
|
call scheme_print
|
||||||
|
# " (quote "
|
||||||
|
movq $SYS_WRITE, %rax
|
||||||
|
movq output_fd(%rip), %rdi
|
||||||
|
leaq s_quote_op(%rip), %rsi
|
||||||
|
movq $s_quote_op_len, %rdx
|
||||||
|
syscall
|
||||||
|
# Print val
|
||||||
|
movq 8(%rbp), %rdi
|
||||||
|
call scheme_print
|
||||||
|
# "))\n"
|
||||||
|
movq $SYS_WRITE, %rax
|
||||||
|
movq output_fd(%rip), %rdi
|
||||||
|
leaq s_rparen2_nl(%rip), %rsi
|
||||||
|
movq $3, %rdx
|
||||||
|
syscall
|
||||||
|
.ps_gc_next:
|
||||||
|
movq 16(%rbp), %rbp # parent
|
||||||
|
jmp .ps_gc_loop
|
||||||
|
.ps_gc_done:
|
||||||
|
# Restore printer fd.
|
||||||
|
movq %r12, output_fd(%rip)
|
||||||
|
# Close portal file.
|
||||||
|
movq $SYS_CLOSE, %rax
|
||||||
|
movq %rbx, %rdi
|
||||||
|
syscall
|
||||||
|
movq $VAL_VOID, %rax
|
||||||
|
RET_VAL
|
||||||
|
.ps_gc_fail:
|
||||||
|
movq $VAL_FALSE, %rax
|
||||||
|
RET_VAL
|
||||||
|
.endif
|
||||||
|
|
||||||
|
.ifdef GC_NAIVE
|
||||||
|
# GC build: portal-resume is equivalent to (load "filename") since
|
||||||
|
# the GC portal format is S-expressions. No magic header, no binary
|
||||||
|
# heap dump to splice back in — just read+eval every form.
|
||||||
|
bi_portal_resume:
|
||||||
|
jmp bi_load
|
||||||
|
.endif
|
||||||
|
|
||||||
|
.ifndef GC_NAIVE
|
||||||
bi_portal_resume:
|
bi_portal_resume:
|
||||||
# (portal-resume "filename") — restore heap + state from file
|
# (portal-resume "filename") — restore heap + state from file
|
||||||
GETARG %rdi # filename string
|
GETARG %rdi # filename string
|
||||||
|
|
@ -5030,6 +5148,7 @@ bi_portal_resume:
|
||||||
.pr_fail:
|
.pr_fail:
|
||||||
movq $VAL_FALSE, %rax
|
movq $VAL_FALSE, %rax
|
||||||
RET_VAL
|
RET_VAL
|
||||||
|
.endif
|
||||||
|
|
||||||
# ============================================================
|
# ============================================================
|
||||||
# bi_load: (load "path") — read file, eval every form in r14 env
|
# bi_load: (load "path") — read file, eval every form in r14 env
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -709,18 +709,19 @@ Producer side: assemble the file with ``(display ...)`` & ``(write ...)`` to an
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
.. table::
|
.. table::
|
||||||
:widths: 12 14 14 14
|
:widths: 12 12 12 12 14
|
||||||
|
|
||||||
============ ========= ========= =========
|
============ ======== ======== ======== ==========
|
||||||
Producer → Python C asm
|
Producer → Python C asm asm-gc
|
||||||
Consumer ↓
|
Consumer ↓
|
||||||
============ ========= ========= =========
|
============ ======== ======== ======== ==========
|
||||||
Python ✓ ✓ ✓
|
Python ✓ ✓ ✓ ✓
|
||||||
C ✓ ✓ ✓
|
C ✓ ✓ ✓ ✓
|
||||||
asm ✓ ✓ ✓
|
asm ✓ ✓ ✓ ✓
|
||||||
============ ========= ========= =========
|
asm-gc ✓ ✓ ✓ ✓
|
||||||
|
============ ======== ======== ======== ==========
|
||||||
|
|
||||||
9 of 9. Verified by ``tests/portal-cross-test.sh`` (**make bench-portal-cross**). Same file, same semantics, regardless of which process produced it.
|
16 of 16. Verified by ``tests/portal-cross-test.sh`` (**make bench-portal-cross**). The ``asm-gc`` column expands the matrix from 9 to 16 cells — the GC build's ``portal-save`` emits the same S-expression format, so it exchanges with every other tier (§7.4.1 below).
|
||||||
|
|
||||||
This matters because it defeats the "version lock-in" trap. If the JSON portal were the only option, a Python 3.15 producer could emit structures a C consumer couldn't parse. With S-expression portals, the only dependency is a parser that handles the subset of forms in the file. Every implementation already has one.
|
This matters because it defeats the "version lock-in" trap. If the JSON portal were the only option, a Python 3.15 producer could emit structures a C consumer couldn't parse. With S-expression portals, the only dependency is a parser that handles the subset of forms in the file. Every implementation already has one.
|
||||||
|
|
||||||
|
|
@ -748,7 +749,7 @@ The computation does not need to restart from the beginning.
|
||||||
7.4 Binary Heap Dump — the Fast One
|
7.4 Binary Heap Dump — the Fast One
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
The asm implementation ships a direct-dump portal: write the raw heap bytes, the ``r14`` (env) & ``r15`` (bump pointer) registers, the heap base address, & a magic header. No parsing, no encoding.
|
The **no-GC** asm implementation ships a direct-dump portal: write the raw heap bytes, the ``r14`` (env) & ``r15`` (bump pointer) registers, the heap base address, & a magic header. No parsing, no encoding.
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
|
|
@ -765,6 +766,22 @@ Sub-millisecond save & resume. The MAP_FIXED trick is why it works across proces
|
||||||
|
|
||||||
Constraints: same architecture, same binary layout, same process model. An asm binary portal written on one box resumes on another x86_64 Linux box running the same asm binary. It does not resume on a rebuilt binary — the BSS-resident symbol table ``sym_table`` is not in the heap dump, so interned symbols would need to be reinterned. That's the price of trivial serialization.
|
Constraints: same architecture, same binary layout, same process model. An asm binary portal written on one box resumes on another x86_64 Linux box running the same asm binary. It does not resume on a rebuilt binary — the BSS-resident symbol table ``sym_table`` is not in the heap dump, so interned symbols would need to be reinterned. That's the price of trivial serialization.
|
||||||
|
|
||||||
|
7.4.1 The GC Build Uses S-Expressions
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The **GC** asm build can't use the binary heap dump because the heap is a linked list of chunks with typed block headers and a free list — raw-byte serialization would lose the structure. Rather than inventing portal v2 with chunk tables and pointer-relocation metadata, the GC build's ``portal-save`` walks the global env chain from ``%r14`` and emits one ``(define <sym> (quote <val>))`` form per binding. ``portal-resume`` is equivalent to ``(load "<filename>")`` — read every form and eval it.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
# GC-build portal file is just Scheme source:
|
||||||
|
(define nums (quote (1 2 3 4 5)))
|
||||||
|
(define greeting (quote "hello"))
|
||||||
|
(define x (quote 42))
|
||||||
|
|
||||||
|
Every non-builtin, non-closure binding round-trips. Closures and builtins are skipped — closures can't be faithfully re-read from their printed form, builtins get re-created from the target interpreter's prelude. This is the same treatment the JSON portal already gives to builtins (§7.3).
|
||||||
|
|
||||||
|
**Trade-off.** The GC build's portal is slower than the binary dump (walks each binding through the printer) and stricter about what it can preserve (data only, no closures or continuations). In exchange: **every portal produced by the GC build resumes on every other tier**, with no MAP_FIXED trick, no architecture constraint, no "same binary" requirement. A GC-asm producer can hand a portal to a Python consumer — we added the asm-gc column to the §7.2 matrix and all 16 cells are green.
|
||||||
|
|
||||||
7.5 Cross-Process Benchmarks
|
7.5 Cross-Process Benchmarks
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue