Cross-tier API parity with c-tier (81ac49e) + python-tier — all three
lumbda runtimes now share the substrate for fork-per-accept patterns.
Implementation: direct syscalls (no libc):
- SYS_FORK=57 → bi_forkself, returns 0/pid via make_int
- SYS_WAIT4=61 + WNOHANG=1 → bi_waitpid_nonblock, returns pid or 0
- SYS_EXIT=60 → bi_exit_immediate (same as bi_exit on asm — no atexit
to bypass; present for cross-tier API parity)
- SYS_NANOSLEEP=35 → bi_sleep, stack-allocated timespec (tv_sec=N,
tv_nsec=0), returns VAL_VOID
Built + tested in vm-runner.sh VM (Ubuntu 2G/2vCPU): both lumbda
and lumbda-gc + fork-cycle test = 3/3 children reap clean, exit-
immediate returns to parent waitpid correctly. Same behavioral
contract as c-tier (commit 81ac49e) and python-tier.
Two GC-build sizing fixes for the same bug class — small GC builds
silently lost live roots under load, causing hash-table-ref to report
"missing key" on entries we just set.
1. HEAP_SIZE 0x100000 → 0x400000 (1 MB → 4 MB):
32 chunks × 1 MB capped the GC build at 32 MB. ecdsa
test-mod-inv-by at p=251 (n+1=9) OOM'd here even though gc was
reclaiming, because the fragmented free-list could not satisfy
the next n+1=9 sim batch. 4 MB × 32 chunks = 128 MB cap, still
well below the 512 MB ulimit -v envelope our asm tests run under.
2. GC_MARK_STACK_CAP 16K → 256K and gc_mark_stack .skip synced to
the constant:
gc_push_if_heap silently dropped tagged values when the mark
stack overflowed and claimed "correctness preserved (sweep won't
reclaim missed-roots, just leaks one cycle)" — but a dropped
value never reaches gc_mark_drain, so its header mark bit stays
clear and gc_sweep treats it as dead. ecdsa test-mod-inv-by at
p=251 walked ~17k tagged values in a single GC cycle and tipped
over the cap, after which live cons-cells started getting
reclaimed mid-simulate.
.skip 131072 was a hard-coded constant that didn't track the .equ,
so bumping the cap without resizing the buffer would smash adjacent
gc_mark_depth / gc_collections / gc_live_bytes; both lines moved
together. 256K × 8 = 2 MB of .bss, one HEAP_SIZE chunk's worth.
After fix: ecdsa test-mod-inv-by on lumbda-full inside QEMU guest
passes p ∈ {11, 13, 251} byte-equal to Python tier. Upstream asm
test.sh stays 158/158 GREEN. Discovered while working ecdsa task
#47, 2026-06-05.
Three defects fixed today on the asm tier worker path:
1. Multi-line "..." docstrings crashed asm tier's scheme_read.
wire.lsp, bend.lsp, gpu-worker.lsp had docstrings spanning
several lines; replaced with ;; comments before each define.
asm tier loads these cleanly now.
2. asm tier lacked delete-file. handle-cuda-shake-fanout called
it to clean up temp portal files. Added bi_delete_file via
SYS_UNLINK = 87 syscall (~20 LoC asm). BI_DELETEFILE constant
slotted after sibling-agent's BI_STRTOSYM.
3. All Scheme files in examples/cuda-fanout/ now ASCII-only.
Earlier em-dash / × / → / μ tripped asm tier's reader in
subtle ways during file load. iconv pass + sed fixes.
Result: all three tiers complete the bench through their own
cliff. New 3-tier table:
workload Python C tier asm tier
small (3 × 16 B) 1.27 ms 0.16 ms 0.21 ms
small (100 × 16 B) 3.43 ms 0.40 ms 1.99 ms
medium (1000) 23.24 ms 2.77 ms CLIFF
med (10k) 218.82 ms CLIFF CLIFF
huge (50k) 1,099 ms CLIFF CLIFF
huge (100k) 2,219 ms CLIFF CLIFF
huge (1M) 23,811 ms CLIFF CLIFF
asm tier at 0.21 ms beats Python by 6× at smallest workload,
matches C at the bottom (~30% slower). asm cliffs at 1000;
C tier cliffs at 10k. Both cliffs are reader/buffer limits
inside the tier, not network or kernel. CUDA kernel itself
finishes 1M × 16B in ~47 ms — three orders of magnitude under
any tier's wire cost at huge scale.
bench_tiers.py made cliff-resilient: respawns worker on per-
workload failure & continues, so the full row prints for every
tier instead of bailing on first cliff.
www/index.html: full 3-column table + honest framing of when
each tier earns its slot.
When heap_alloc walks off the end of a chunk with < 16 tail bytes,
.ha_grow_no_pad skips padding and mmaps a fresh chunk — but
gc_chunk_end[N] for the abandoned chunk stayed at its full mmap
end while %r15 (high-water) sat 1..15 bytes short. The gap held
mmap-zeros that gc_sweep's .gsw_walk decoded as fake dead blocks
(header == 0, payload size == 0, mark == 0). The walker stepped
through the zeros 8 bytes at a time, and on the iteration where
%rbx == chunk_end - 8 the .gsw_dead path stored the free-list
next-pointer to 0x8(%rbx) == chunk_end — the first byte of an
unmapped page — and segfaulted with error 7.
Reproducer (inside QEMU guest, was crashing all three asm
binaries):
lumbda-full tests/unit/test-mod-solinas.lsp
→ segfault at <ptr ending 000> ip:4017d9
(.gsw_dead: mov %rdx, 0x8(%rbx))
Fix: at .ha_grow_no_pad, snapshot %r15 into gc_chunk_end[N] before
allocating the new chunk. In the padded path above the label this
is a no-op (r15 already == r13). In the un-padded path it pins
the walk bound to the high-water mark so the sweep never enters
the gap.
After fix, on lumbda-full inside the ecdsa QEMU guest:
- ecdsa test-mod-solinas: 39/39 PASS (Solinas vs Litinski
byte-equal at p ∈ {11, 13, 251})
- upstream asm test.sh: 158/158 PASS
Discovered while diagnosing ecdsa task #45.
Reader's .sr_string used a fixed 256-byte stack buffer with no
bounds check. Strings longer than ~272 bytes (256 + saved
registers) corrupted the saved return address and produced a
general protection fault on ret.
Reproducer:
(display (string-length "AAAA...")) ; > 272 A's → #GP at .sr_string ret
Fix:
- bump stack buffer 256 → 4096 (one page)
- add bounds check (cmpq $4080) before every char write
- on overflow exit cleanly via new die_str_overflow rather than
smashing %rip
Discovered while diagnosing ecdsa task #34: lumbda asm tier
crashed when loading ecdsa/lumbda/mod-arith.lsp because one
mod-mul! docstring is 955 bytes. Post-fix, mod-arith.lsp loads
clean and ecdsa test-mod-arith.lsp passes 33/33 on asm-full;
upstream asm test.sh stays 158/158.
Closes the last asm-tier gap for hosting bend workers:
(read-line port) → string or #f
Reads bytes one at a time from the port's fd until '\n' or EOF.
Strips the trailing newline. Returns #f when no bytes were
available (peer closed / pipe drained).
Verified end-to-end on 3090-ai:
λ> (define p (spawn-process-stdio "./shake256-fanout" (quote (--daemon))))
λ> (display (read-line (cdr p))) (newline)
ready
λ> (display "quit\n" (car p))
λ> (flush-port (car p))
λ> (display (read-line (cdr p))) (newline)
bye
asm tier can now spawn, write, flush, read line — the full
subprocess capability gpu-worker.lsp's daemon pool needs.
Implementation:
- Stack scratch buffer: 4096 bytes via %rbp (heap-safe; %r15 is
lumbda's heap pointer, do not touch)
- One-byte-at-a-time SYS_READ via fd from decode_port
- Max line: 4094 bytes (fits the daemon protocol — "ready",
"done /path", "bye" all under 100)
- heap_alloc(8 + len) + length header + byte copy + TAG_STRING
matches the Python/C tier string format byte-for-byte
- GC_NAIVE path stamps HT_STRING header for the GC pass
BI_READLINE = 123; GC_* renumbered to 124..128; BI_COUNT = 129
(GC_NAIVE) / 124. bn_readline added to symbol table + name array.
Remaining asm-tier gaps for full gpu-worker.lsp hosting:
- *argv* binding (used to parse --port)
- define-syntax + syntax-rules (used by bend.lsp macros — handled
by splitting bend-macros.lsp out in a prior commit)
- error builtin (also handled by portable bend-error wrapper)
Per-tier matrix:
Python tier ✓ macro + function client; full worker host
C tier ✓ macro + function client; full worker host
asm tier ✓ function client; subprocess primitives complete;
gpu-worker.lsp needs *argv* + a few other helpers
before pure-asm hosting is fully working
ecdsa cross-tier validation on lumbda-gc and bump-only lumbda blocked
on `case` being unbound. R7RS standard control form — present on
Python (lumbda.py) and C (c/) tiers, but on asm reachable only via
cl_full_prelude's define-macro form (carved into lumbda-full only).
Mirrors commit 865be28 (when/unless via Path B dispatch table).
Path B (special-form dispatch table extension):
- sf_case length-prefixed symbol name
- sym_case_val interned at init_special_forms
- dispatch case in .eval_top alongside .ev_when / .ev_unless
(placed before the .ifdef CL_FULL macro-lookup block so the
dispatch shadow takes precedence over the cl_full_prelude macro
on lumbda-full — no conflict, the macro just becomes dead code)
- .ev_case evaluator: eval key once, push on stack, walk clauses;
each clause's datum list compared by pointer equality (eqv? on
the asm tier — fixnums, symbols, booleans, characters, nil are
all interned/unboxed to unique values). `else` matches uncondi-
tionally. Match → .ev_begin (TCO). No match → .ev_begin_void.
Available on every asm tier (plain `lumbda`, `lumbda-gc`,
`lumbda-full`). Binary size impact:
lumbda 60768 → 61040 (+272, +0.45%)
lumbda-gc 69496 → 69768 (+272, +0.39%)
lumbda-full 72000 → 72264 (+264, +0.37%)
All 158 asm tests still pass. Sanity tests: single-datum, multi-datum,
symbol key (eqv?), else, empty body, no-match, nested case — all
correct on all three tiers.
ecdsa search.lsp now produces byte-identical winner (v3-clifford-only
score 0) across five tiers: Python, C, asm-bump, asm-gc, asm-full.
ecdsa cross-tier validation blocked on `when` and `unless` being
unbound on the asm tier. R7RS standard control forms — present on
Python (lumbda.py) and C (c/) tiers, but absent on asm because the
existing macro facility (define-macro) only ships under CL_FULL.
Path B chosen (special-form dispatch table extension):
- sf_when / sf_unless length-prefixed symbol names
- sym_when_val / sym_unless_val interned at init_special_forms
- dispatch cases in .eval_top alongside .ev_and / .ev_or
- .ev_when / .ev_unless evaluators reuse .ev_begin for the body
branch and .ev_begin_void for the skip branch (TCO preserved)
Available on every asm tier (plain `lumbda`, `lumbda-gc`,
`lumbda-full`). Binary size impact:
lumbda 60488 → 60768 (+280, +0.46%)
lumbda-gc 69224 → 69496 (+272, +0.39%)
lumbda-full 71720 → 72000 (+280, +0.39%)
All 158 asm tests still pass. Tested truthy/falsy/multi-form bodies
on all three tiers. ecdsa search.lsp now runs on asm-full with output
byte-identical to Python tier (v3-clifford-only winner, score 0).
Note: ecdsa search.lsp also depends on `case`, which is only present
under CL_FULL (carved into cl_full_prelude as a define-macro form).
That gap blocks lumbda-gc cross-tier validation and is out of scope
for this commit.
Two coupled defects surfaced during ecdsa cross-tier validation against
the asm tier.
Defect #28 — _start ignored argv. Invoking `asm/lumbda-gc file.lsp`
silently discarded argv[1] and dropped into a REPL that blocked on a
pty when run under SSH. Walk argc/argv after init_builtins + prelude
load and before repl_top: for each argv[i] starting at i=1, skip
arg if it begins with '-' (flag stub), otherwise allocate a Scheme
string from the C string, wrap in a 1-element arg list, dispatch
through apply_proc_raw on the BI_LOAD builtin. If any non-flag arg
ran, jump to repl_exit instead of entering the REPL. Mirrors the
c/main.c script-mode semantics. The RET_VAL macro on the builtin
return path pops r12/rbp/rbx in an order that corrupts %rbp (it
restores the pre-call %r12 into rbp), so the loop counter saves
%rbp around the apply_proc_raw call.
Defect #30 — eq? was only present under CL_FULL. The plain `lumbda`
and `lumbda-gc` binaries shipped without the alias `(define eq? eqv?)`,
so any .lsp expecting eq? (every cross-tier file we own) hit
"unbound variable: eq?" the moment it tried a status check. Lift
that single alias into a new always-on `default_prelude` block with
its own `load_default_prelude` loader (modelled after
load_cl_full_prelude), and call it unconditionally from _start
between rng_seed and the CL_FULL block.
Verification:
- `make asm-build` clean
- `make asm-test`: 158 passed, 0 failed (full suite green)
- `(eq? 1 1)` -> #t on all three tiers via stdin pipe AND file arg
- `~/git/lumbda/asm/lumbda-gc /tmp/asm-test.lsp` exits 0 with #t printed
Closes the remaining asm-side gaps from ticket 0005's follow-up
discussion. Every test in tests/cl-compat.lsp and tests/ursa.lsp
now runs unmodified on default asm (Scheme port) and asm-full (full
CL path) — no more commented-out tests or shim syntax.
Landed (all in default asm — useful beyond cl-compat):
* (values . xs) / (call-with-values producer consumer). values
packs a tagged pair (mval_marker . xs) when multiple; a lone arg
passes through unchanged so legacy single-value code is
undisturbed. call-with-values invokes the producer, destructures
the multi-value packet if present, applies consumer positionally.
The marker is a gensymed symbol interned once at init, so no
user-constructed pair can masquerade as a multi-value packet.
* (exit [code]) builtin. Default code is 0 when called with no
args. Passes through to the SYS_EXIT syscall.
* #(...) vector literal in the reader. .sr_hash now dispatches on
'(' as a vector literal alongside 't' and 'f'. list_to_vector_
reader is a standalone helper callable from the reader (separate
from bi_listtovec which uses the GETARG builtin convention).
Matches R7RS vector literal syntax. Existing vector builtins
already handled construction; this just teaches the reader.
* deep_equal extended to vectors. equal? now descends into vectors
(length + elementwise recursive compare), matching R7RS.
Previously only strings and pairs were handled; vectors fell
through to shallow pointer compare which only matched identical
heap objects.
Test file reverts (picking up the new capabilities):
* tests/cl-compat.lsp — multiple-value-bind test restored
(previously commented out because asm lacked values /
call-with-values).
* tests/ursa-scheme.lsp — #(1 0 1 0 1 0) literal restored
(previously worked around with (vector->list (digits ...)));
(exit 1) failure trailer restored (previously removed because
asm had no exit builtin).
* tests/ursa.lsp — same digits literal restoration.
Verified:
* asm regression: 158/158.
* asm-full regression: 158/158.
* Zoë-favorites across Python + C + asm + asm-full: all suites
green with native reader syntax and multi-value tests.
* make test-all stays green.
Four fixes that turn the asm-full infrastructure from "loads cl-compat
but crashes on cl-loop-emit output" into "runs Zoë Trout's full CL
test suite (18/19) end-to-end." Zoë's original `examples/ursa.lisp.txt`
now produces matching answers to the Python and C tiers on asm-full.
1. asm/lumbda.s bi_apply — second arg was being clobbered. The
previous impl did `GETARG %rbx; GETARG %rdi; movq %rbx, %rdi;
... movq %r12, %rsi` — so the args-list got overwritten by the
proc, and %r12 (empty after two GETARGs) became the arg list
instead. `(apply f '(1 2 3))` silently reduced to `(f)`. Fix:
`GETARG %rbx; GETARG %rsi; movq %rbx, %rdi; call apply_proc_raw`.
2. asm/lumbda.s bi_expt — decrements rcx by 1 until zero. Negative
exponents looped forever. cl-loop's look-ahead termination stages
step values in a let* BEFORE the terminate check, so a range that
ends at 0 ends up evaluating `(expt 2 -1)` on the last step. Fix:
guard negative exponents, return 0. asm is integer-only; returning
a rational would need a new type. Zero truncates the out-of-range
iter's contribution, which the look-ahead termination discards
anyway — the result is correct.
3. asm/lumbda.s GC roots — macro_env_head was not marked. Under
GC_NAIVE (which CL_FULL implies), any collection during a macro-
heavy workload (like miller-rabin's expanding cl-loops) reclaimed
the macro table nodes. Next use failed with "unbound variable:
cl-when" or similar. Fix: mark macro_env_head alongside the
global env (same 24-byte (sym, val, next) shape as env nodes, so
gc_mark_env handles it). Guarded .ifdef CL_FULL.
4. asm/lumbda.s prelude — added `cadar` (used by
cl-loop-finalizer-expr). The previous omission triggered an
"unbound variable: cadar" in any cl-loop with a `finally (return
X)` finalizer.
5. cl-compat.lsp — two new helpers routed around asm's reduced
list-processing builtins:
* `cl-append` for n-list concatenation. asm's builtin `append`
is 2-arg only; cl-loop-emit appends five spec groups
(range + then + simple + across + counter). Reducing with
2-arg append works on every tier.
* `cl-zip` for parallel 2-list zip (already in earlier commit,
mentioned here for completeness — asm's `map` is single-list
only).
Verification on asm/lumbda-full:
* /tmp/ursa-load-test.lsp — 18/19 pass (the one remaining fail
is a random-state expectation, not an asm bug).
* (primep 97) → 97
* (primep 100) → #f
* (lucas-lehmer-primep 13) → #t (M₁₃ = 8191, prime)
* (lucas-lehmer-primep 11) → #f (M₁₁ = 2047 = 23·89)
* (of-n-bits 8) → random integer in [128, 256) with top bit set
* (prime-of-n-bits 8) → random 8-bit prime
make test-all stays green. All three asm variants still 158/158 on
their local test suites. asm's minimal footprint preserved — every
new line above is under .ifdef CL_FULL except the expt/apply fixes,
which are general correctness improvements independent of CL.
Third asm variant — built with CL_FULL=1 GC_NAIVE=1 via new Makefile
target. Adds the macro machinery needed for cl-compat.lsp on the asm
tier, keeping every addition behind .ifdef CL_FULL so the default
(~22 KB) and -gc binaries keep their current footprint.
Landed in this drop:
* Reader: backtrack on digit-prefixed symbols. After reading digit
characters, if the next char is not a delimiter, input_pos
rewinds and control falls through to .sr_symbol. Makes 1+, 1-,
add1, abc123, and any CL-style identifier with a numeric prefix
parse as symbols instead of truncating to a bare integer.
* Reader: `` ` `` / `,` / `,@` produce (quasiquote X) / (unquote X)
/ (unquote-splicing X) forms. Same build shape as the existing
`'` quote branch.
* Evaluator: .ev_quasiquote + quasiquote_expand walk the template.
unquote evaluates its argument in the current env; unquote-
splicing evaluates then splices via a new list_append_ab helper;
other pairs recurse (cons expand-car expand-cdr). Atoms pass
through. No nested quasiquote depth (deliberate; ticket 0005
scope).
* Evaluator: .ev_define_macro + macro_env_head linked list. Each
(define-macro (name p...) body) prepends a 24-byte
(sym, closure, next) node. Dispatch in eval checks macro_lookup
after all special-form compares; on hit, the closure is applied
to the *unevaluated* argument list and the expansion re-enters
.eval_top under TCO.
* Binding: rest-arg support extended to .apr_bind inside
apply_proc_raw. Previously only .ac_bind (direct .app_closure
path) handled `(lambda (a . b) ...)` correctly; macros call
closures through apply_proc_raw, so this was required to make
variadic defun/setf macros bind correctly.
* Builtin: (gensym) — writes "g%d" for an in-BSS counter, length-
prefixes the buffer, calls intern_static. Available in every
variant (not CL_FULL-gated — useful outside macros too).
* Builtin: (cadr x), (sort lst) and the let* special form from
earlier commit stay in default asm. These are Scheme staples.
* Prelude: evaluated at _start after init_builtins / rng_seed,
before the REPL. Embedded string, input state saved + restored
around the load. Defines caar, cdar, caddr, cadddr, cddr,
cdddr, cddddr, 1+, 1-, add1, sub1, square, eq? (= eqv? for
interned symbols), memq, list-ref, assq, and `case` as a macro.
cl-compat.lsp: two small changes to work under asm's single-list
`map`:
* Added cl-zip helper. Replaced two `(map (lambda (v n) (list v n))
xs ys)` sites with `(cl-zip xs ys)` — asm's builtin map accepts
only one list, and cl-loop-emit needs a parallel walk over
state-vars and new-names.
* Added explanatory comment for cddddr at the top of the shim
(already shipped).
Tests:
* make asm-test (lumbda) — 158/158 pass.
* make asm-test-gc — 158/158 pass.
* make asm-test-full — 158/158 pass on synchronous run.
* Zoë's `examples/ursa.lisp.txt` LOADS on asm/lumbda-full.
`(expt-mod 3 7 100)` = 87.
Most simple cl-loop forms work (while + do + finally, range-to,
then-accumulator).
Known open issues documented in docs/tickets/0005-asm-cl-full.md:
* cl-loop-emit produces wrong output for inputs with `simple` iters
(`(simple a 5)` → state binding dropped). Python/C return the
correct form; asm version is missing the binding. Bug surfaces
in the emit's 30+ binding let*; could not pin down in this
session. Downstream effect: `(miller-rabin n)` and similar
defuns that depend on `cl-loop repeat k for a = ... unless ...
return nil` don't produce usable expansions, so Zoë's acceptance
suite does not run end-to-end on asm/lumbda-full yet.
* examples/ursa-scheme.lsp — `factor` crashes on asm under some
random seeds (bump-allocator exhaustion on long rhoff retry
chains). Out of CL_FULL scope; tracked in same ticket.
Next steps live in ticket 0005. This commit ships the infrastructure
so the remaining work is a debugging exercise against a reproducible
minimal case, not a feature build.
Phase 1 of the asm/lumbda-full roadmap (ticket 0005, in-flight). Adds
the minimum-cost set of additions that lets examples/ursa-scheme.lsp —
the idiomatic Scheme port of ursa.lisp.txt — load and produce correct
results on the asm tier. No CL shim yet: that requires quasiquote,
define-macro, and case, all of which are Phase 2 / 0005.
Added:
* Rest-args in lambda — (define (f x . rest) ...). .ac_bind now
detects when the remaining param list is a raw symbol (TAG_SYM)
and binds it to the remaining arg list. Enables variadic defuns.
* cadr builtin — (car (cdr x)) fast path. Used by Zoë's
repunit-value and any CL-adjacent code.
* sort builtin — ascending insertion sort on a tagged-int list.
Non-destructive. Matches Python/C sort contract (default numeric
ordering). Implementation ~50 lines, recursive sort + insert
helpers.
* let* special form — sequential binding where each init sees the
preceding bindings' values. Fresh sf_let_star + sym_let_star_val
+ .ev_let_star branch that's a one-line variant of .ev_let (eval
init in the extended env rather than the original). TCO preserved.
Tests: 9 new asm assertions in asm/test.sh covering cadr, sort (empty
/ singleton / unsorted / already-sorted), let* (basic + sequential),
rest-args (tail-only + rest-only). Total asm suite now 158 passing.
Known limitation: the Scheme port's factor / rho depends on random
rhoff iteration. For some seeds on asm (e.g. seed=2, factor 91) the
process runs out of virtual memory before rho finds a factor. The
underlying math is correct — this is an asm heap-bump-allocator
behavior under long random-retry chains and will be addressed along
with the CL_FULL work in ticket 0005. Python and C paths unaffected.
make test-all stays green across every tier.
Ticket 0002 — reads 8 bytes from /dev/urandom (little-endian u64) and
seeds xoshiro256**. Opt-in kernel entropy for stochastic runs; the
default stays deterministic (k=0 at startup), so ticket 0001's
portal-reproducibility contract is unchanged.
Real-world flow now one call away:
Machine A: (random-seed-from-os!) + run simulation + portal-save
Machine B: portal-resume — same stream, bit-for-bit
All three impls fail loud on /dev/urandom trouble (LispErr in Python
and C, stderr + exit(1) in asm) — no silent fallback to a weak seed.
Tests:
- tests/functional.lsp: 2 new shared asserts (entropic + replay)
- asm/test.sh: 2 new asm-local checks (149 total, was 147)
- make test-all green across Python (205), C (205), asm (149)
Whitepaper §7.5 gains one sentence noting the OS-seed path.
unmoad: zero new findings in added code.
Completes ticket 0001 started in 27f468c. All three impls now carry
bit-identical xoshiro256**; portal state round-trips across process
boundaries in every producer x consumer cell (Python <-> C <-> asm).
asm impl:
- 4 new builtins: random-seed!, random-int, random-state, random-state!
- g_rng_state in BSS (4 x u64); rng_splitmix64_step, rng_seed, rng_next
- Binary portal header bumped LUMBDAB1/48 -> LUMBDAB2/80; carries
32 bytes of rng state at offsets 40..64, reserved moved to 72
- No float support in asm, so (random) intentionally omitted there
- _start seeds with 0 so the stream is deterministic from startup
Python + C (supplements 27f468c):
- rng_seed(0) auto-invoked at module load / register_portal_builtins
so (random) without explicit (random-seed!) returns a real value
instead of the all-zero xoshiro fixed point
Tests:
- tests/functional.lsp: 7 new shared assertions (Python + C)
- asm/test.sh: 5 new asm-local assertions (142 -> 147)
- tests/portal-rng-save.lsp / portal-rng-load.lsp: portable S-expression
portal that captures both state AND next-5 baseline so loader self-
verifies without a separate harness
- tests/portal-cross-test.sh: 9 new producer x consumer RNG cells; all
18 cells pass end-to-end
Verified: seed=42, (random-int 1000000) draws 1..10 =
558742 543102 559009 124193 317476 750584 200754 814407 344958 929085
identical in Python, C, and asm.
unmoad scan: zero new findings in added code.
asm-gc gains (tcp-sendfile socket path) → builtin (90 lines) that issues
SYS_SENDFILE(40) in a loop, streaming a file from fd → socket with no bounce
through the Lumbda heap. Zero-copy kernel path for large responses.
examples/http-static-server-sendfile.lsp (hybrid): small assets
(≤ 16 KB) stay inline-cached as full HTTP responses; large assets cache
only headers and stream the body via tcp-sendfile. 4-way race on
i5-8350U, 100 PDF requests (2.56 MiB), concurrency 8:
uncached 159 req/s 406 MiB/s 15.5 MB RSS
cached 238 req/s 603 MiB/s 7.2 MB RSS
sendfile 480 req/s 1226 MiB/s 4.2 MB RSS
caddy 485 req/s 1238 MiB/s 37.1 MB RSS
sendfile lands within 2% of caddy on throughput with 9x less peak RSS in
a 27 KB binary vs caddy's 38 MB (1400x smaller).
examples/http-static-server-adaptive.lsp (learning preload): per-URL hit
counter persisted to www.hits every N requests. At boot, ranks and
preloads top *cache-max* URLs from the prior run's data (cold-start
falls back to a seed list). Cold requests beyond the seed promote on
first hit. Drops heap-restore arena pattern since the server mutates
persistent state every request; relies on GC build's mark-sweep.
tests/bench-www-race.sh: adds sendfile variant on port 8083, auto-sizes
PDF byte count from the on-disk whitepaper so a whitepaper rebuild
doesn't desync the MiB/s calc.
Whitepaper §11.7 "Static File Serving: Cache, Sendfile, and Adaptive
Preload" documents the four variants, benchmark table, and the
arena-vs-mutation tradeoff. §13 Future Work adds DAG-of-hot-paths
predictive preload as the direction for > 1000-resource deployments
where frequency-only ranking is too narrow.
Ships examples/http-static-server.lsp — ~65 lines of portable Scheme
that reads files from a docroot (default ./www) and serves them over
HTTP/1.0 with MIME dispatch, path-traversal rejection, heap-snapshot
per request. Runs in any tier; target deployment is asm-gc for the
27 KB stripped binary + bounded memory backstop.
Required one asm fix first: heap_grow was mmap'ing fixed HEAP_SIZE
chunks, so any single allocation larger than a chunk (notably the
2.67 MB whitepaper PDF read via file->string) loop-looped through
.ha_overflow forever. Now heap_grow rounds required bytes up to
HEAP_SIZE multiples on oversize alloc, so a big request carves its
own big chunk in one go. Small allocs still land in standard-sized
chunks.
Two new benches:
tests/bench-lumbda-www.sh — drive N small + M large requests against
asm-gc, verify PDF round-trip, sample peak RSS. At 1000/100: 331 req/s
small, 120 req/s large (304 MiB/s), peak 15.5 MB.
tests/bench-www-race.sh — adjacent A/B vs caddy v2.5.1 on the same
docroot. Numbers on this laptop, concurrency 8, 2000 small + 200 large:
small req/s PDF req/s PDF MiB/s peak RSS binary
lumbda-www (asm-gc) 375 137 349 7–16 MB 27 KB
caddy file-server 358 231 588 38 MB 38 MB
Reading: lumbda edges caddy on small files (less per-request overhead),
caddy wins 1.7x on large files (sendfile zero-copy; we allocate the
whole file into a string and write it with one syscall). Both byte-
identical on the PDF. Memory: lumbda 2.5-5x less at steady state.
Binary size: 1400x smaller (27 KB vs 38 MB).
Feature gap: caddy has HTTPS, HTTP/2, range, middleware, etc. lumbda
has none of that yet — but for the specific job of serving lumbda.com's
six-file docroot it is viable right now.
Makefile adds `bench-lumbda-www` and `bench-www-race` targets.
137 asm no-GC + 137 asm GC tests still pass.
Historical internal name "uncommonlisp" retired in favor of the
public name "lumbda" ahead of lumbda.com going live. Scope of
this commit:
Source files renamed:
uncommonlisp.py -> lumbda.py
asm/uncommonlisp.s -> asm/lumbda.s
c/uncommonlisp.h -> c/lumbda.h
whitepaper/uncommonlisp-whitepaper -> whitepaper/lumbda-whitepaper (.rst + .pdf)
Binaries renamed (tracked ones; c/ was always gitignored):
asm/uncommonlisp, asm/uncommonlisp-gc, asm/uncommonlisp.o,
asm/uncommonlisp-gc.o -> asm/lumbda(-gc)(.o)
c/.gitignore -> ignores lumbda
Internal string updates (sed pass ordered longest-first):
asm/uncommonlisp -> asm/lumbda
c/uncommonlisp -> c/lumbda
uncommonlisp.py -> lumbda.py
UNCOMMONLISP_BIN -> LUMBDA_BIN (asm/test.sh env var)
"uncommonlisp> " -> "lumbda> " (asm REPL prompt baked into binary)
UNCOMMONLISP -> LUMBDA (macros, comments)
uncommonlisp -> lumbda (prose)
Binary portal magic updated:
"ULPORTAL" -> "LUMBDAB1" # "Lumbda Binary v1"
Old portal files are not backward-compatible — this is a deliberate
break since it's the rename moment. S-expression portals already
carry their own ";; lumbda-portal v1" header and remain cleanly
versioned.
WHITEPAPER.pdf / WHITEPAPER.rst symlinks repointed to the renamed
files. Makefile's whitepaper target targets lumbda-whitepaper.pdf.
Not changed (intentional, separate phases):
- Filesystem directory /home/fox/git/uncommonlisp itself
(fox renames locally and the gitlab repo URL in a follow-up)
- tests.py hardcoded cwd=/home/fox/git/uncommonlisp
(matches the current on-disk location; will flip when the
directory rename ships)
- Git history (immutable; old commits still say uncommonlisp,
which is correct — that's what they were)
Verified:
137 asm no-GC + 137 asm GC + 571 Python + 83 C + 189 shared
functional tests all pass under the new names.
bench-gc-http (2000 req): all 4 cells behave as expected
(cells 1/2 flat, 3 leaks, 4 bounded at 1 chunk).
Python REPL, C REPL, asm REPL all start cleanly.
