Two coupled defects surfaced during ecdsa cross-tier validation against
the asm tier.
Defect #28 — _start ignored argv. Invoking `asm/lumbda-gc file.lsp`
silently discarded argv[1] and dropped into a REPL that blocked on a
pty when run under SSH. Walk argc/argv after init_builtins + prelude
load and before repl_top: for each argv[i] starting at i=1, skip
arg if it begins with '-' (flag stub), otherwise allocate a Scheme
string from the C string, wrap in a 1-element arg list, dispatch
through apply_proc_raw on the BI_LOAD builtin. If any non-flag arg
ran, jump to repl_exit instead of entering the REPL. Mirrors the
c/main.c script-mode semantics. The RET_VAL macro on the builtin
return path pops r12/rbp/rbx in an order that corrupts %rbp (it
restores the pre-call %r12 into rbp), so the loop counter saves
%rbp around the apply_proc_raw call.
Defect #30 — eq? was only present under CL_FULL. The plain `lumbda`
and `lumbda-gc` binaries shipped without the alias `(define eq? eqv?)`,
so any .lsp expecting eq? (every cross-tier file we own) hit
"unbound variable: eq?" the moment it tried a status check. Lift
that single alias into a new always-on `default_prelude` block with
its own `load_default_prelude` loader (modelled after
load_cl_full_prelude), and call it unconditionally from _start
between rng_seed and the CL_FULL block.
Verification:
- `make asm-build` clean
- `make asm-test`: 158 passed, 0 failed (full suite green)
- `(eq? 1 1)` -> #t on all three tiers via stdin pipe AND file arg
- `~/git/lumbda/asm/lumbda-gc /tmp/asm-test.lsp` exits 0 with #t printed
Closes the remaining asm-side gaps from ticket 0005's follow-up
discussion. Every test in tests/cl-compat.lsp and tests/ursa.lsp
now runs unmodified on default asm (Scheme port) and asm-full (full
CL path) — no more commented-out tests or shim syntax.
Landed (all in default asm — useful beyond cl-compat):
* (values . xs) / (call-with-values producer consumer). values
packs a tagged pair (mval_marker . xs) when multiple; a lone arg
passes through unchanged so legacy single-value code is
undisturbed. call-with-values invokes the producer, destructures
the multi-value packet if present, applies consumer positionally.
The marker is a gensymed symbol interned once at init, so no
user-constructed pair can masquerade as a multi-value packet.
* (exit [code]) builtin. Default code is 0 when called with no
args. Passes through to the SYS_EXIT syscall.
* #(...) vector literal in the reader. .sr_hash now dispatches on
'(' as a vector literal alongside 't' and 'f'. list_to_vector_
reader is a standalone helper callable from the reader (separate
from bi_listtovec which uses the GETARG builtin convention).
Matches R7RS vector literal syntax. Existing vector builtins
already handled construction; this just teaches the reader.
* deep_equal extended to vectors. equal? now descends into vectors
(length + elementwise recursive compare), matching R7RS.
Previously only strings and pairs were handled; vectors fell
through to shallow pointer compare which only matched identical
heap objects.
Test file reverts (picking up the new capabilities):
* tests/cl-compat.lsp — multiple-value-bind test restored
(previously commented out because asm lacked values /
call-with-values).
* tests/ursa-scheme.lsp — #(1 0 1 0 1 0) literal restored
(previously worked around with (vector->list (digits ...)));
(exit 1) failure trailer restored (previously removed because
asm had no exit builtin).
* tests/ursa.lsp — same digits literal restoration.
Verified:
* asm regression: 158/158.
* asm-full regression: 158/158.
* Zoë-favorites across Python + C + asm + asm-full: all suites
green with native reader syntax and multi-value tests.
* make test-all stays green.
Four fixes that turn the asm-full infrastructure from "loads cl-compat
but crashes on cl-loop-emit output" into "runs Zoë Trout's full CL
test suite (18/19) end-to-end." Zoë's original `examples/ursa.lisp.txt`
now produces matching answers to the Python and C tiers on asm-full.
1. asm/lumbda.s bi_apply — second arg was being clobbered. The
previous impl did `GETARG %rbx; GETARG %rdi; movq %rbx, %rdi;
... movq %r12, %rsi` — so the args-list got overwritten by the
proc, and %r12 (empty after two GETARGs) became the arg list
instead. `(apply f '(1 2 3))` silently reduced to `(f)`. Fix:
`GETARG %rbx; GETARG %rsi; movq %rbx, %rdi; call apply_proc_raw`.
2. asm/lumbda.s bi_expt — decrements rcx by 1 until zero. Negative
exponents looped forever. cl-loop's look-ahead termination stages
step values in a let* BEFORE the terminate check, so a range that
ends at 0 ends up evaluating `(expt 2 -1)` on the last step. Fix:
guard negative exponents, return 0. asm is integer-only; returning
a rational would need a new type. Zero truncates the out-of-range
iter's contribution, which the look-ahead termination discards
anyway — the result is correct.
3. asm/lumbda.s GC roots — macro_env_head was not marked. Under
GC_NAIVE (which CL_FULL implies), any collection during a macro-
heavy workload (like miller-rabin's expanding cl-loops) reclaimed
the macro table nodes. Next use failed with "unbound variable:
cl-when" or similar. Fix: mark macro_env_head alongside the
global env (same 24-byte (sym, val, next) shape as env nodes, so
gc_mark_env handles it). Guarded .ifdef CL_FULL.
4. asm/lumbda.s prelude — added `cadar` (used by
cl-loop-finalizer-expr). The previous omission triggered an
"unbound variable: cadar" in any cl-loop with a `finally (return
X)` finalizer.
5. cl-compat.lsp — two new helpers routed around asm's reduced
list-processing builtins:
* `cl-append` for n-list concatenation. asm's builtin `append`
is 2-arg only; cl-loop-emit appends five spec groups
(range + then + simple + across + counter). Reducing with
2-arg append works on every tier.
* `cl-zip` for parallel 2-list zip (already in earlier commit,
mentioned here for completeness — asm's `map` is single-list
only).
Verification on asm/lumbda-full:
* /tmp/ursa-load-test.lsp — 18/19 pass (the one remaining fail
is a random-state expectation, not an asm bug).
* (primep 97) → 97
* (primep 100) → #f
* (lucas-lehmer-primep 13) → #t (M₁₃ = 8191, prime)
* (lucas-lehmer-primep 11) → #f (M₁₁ = 2047 = 23·89)
* (of-n-bits 8) → random integer in [128, 256) with top bit set
* (prime-of-n-bits 8) → random 8-bit prime
make test-all stays green. All three asm variants still 158/158 on
their local test suites. asm's minimal footprint preserved — every
new line above is under .ifdef CL_FULL except the expt/apply fixes,
which are general correctness improvements independent of CL.
Third asm variant — built with CL_FULL=1 GC_NAIVE=1 via new Makefile
target. Adds the macro machinery needed for cl-compat.lsp on the asm
tier, keeping every addition behind .ifdef CL_FULL so the default
(~22 KB) and -gc binaries keep their current footprint.
Landed in this drop:
* Reader: backtrack on digit-prefixed symbols. After reading digit
characters, if the next char is not a delimiter, input_pos
rewinds and control falls through to .sr_symbol. Makes 1+, 1-,
add1, abc123, and any CL-style identifier with a numeric prefix
parse as symbols instead of truncating to a bare integer.
* Reader: `` ` `` / `,` / `,@` produce (quasiquote X) / (unquote X)
/ (unquote-splicing X) forms. Same build shape as the existing
`'` quote branch.
* Evaluator: .ev_quasiquote + quasiquote_expand walk the template.
unquote evaluates its argument in the current env; unquote-
splicing evaluates then splices via a new list_append_ab helper;
other pairs recurse (cons expand-car expand-cdr). Atoms pass
through. No nested quasiquote depth (deliberate; ticket 0005
scope).
* Evaluator: .ev_define_macro + macro_env_head linked list. Each
(define-macro (name p...) body) prepends a 24-byte
(sym, closure, next) node. Dispatch in eval checks macro_lookup
after all special-form compares; on hit, the closure is applied
to the *unevaluated* argument list and the expansion re-enters
.eval_top under TCO.
* Binding: rest-arg support extended to .apr_bind inside
apply_proc_raw. Previously only .ac_bind (direct .app_closure
path) handled `(lambda (a . b) ...)` correctly; macros call
closures through apply_proc_raw, so this was required to make
variadic defun/setf macros bind correctly.
* Builtin: (gensym) — writes "g%d" for an in-BSS counter, length-
prefixes the buffer, calls intern_static. Available in every
variant (not CL_FULL-gated — useful outside macros too).
* Builtin: (cadr x), (sort lst) and the let* special form from
earlier commit stay in default asm. These are Scheme staples.
* Prelude: evaluated at _start after init_builtins / rng_seed,
before the REPL. Embedded string, input state saved + restored
around the load. Defines caar, cdar, caddr, cadddr, cddr,
cdddr, cddddr, 1+, 1-, add1, sub1, square, eq? (= eqv? for
interned symbols), memq, list-ref, assq, and `case` as a macro.
cl-compat.lsp: two small changes to work under asm's single-list
`map`:
* Added cl-zip helper. Replaced two `(map (lambda (v n) (list v n))
xs ys)` sites with `(cl-zip xs ys)` — asm's builtin map accepts
only one list, and cl-loop-emit needs a parallel walk over
state-vars and new-names.
* Added explanatory comment for cddddr at the top of the shim
(already shipped).
Tests:
* make asm-test (lumbda) — 158/158 pass.
* make asm-test-gc — 158/158 pass.
* make asm-test-full — 158/158 pass on synchronous run.
* Zoë's `examples/ursa.lisp.txt` LOADS on asm/lumbda-full.
`(expt-mod 3 7 100)` = 87.
Most simple cl-loop forms work (while + do + finally, range-to,
then-accumulator).
Known open issues documented in docs/tickets/0005-asm-cl-full.md:
* cl-loop-emit produces wrong output for inputs with `simple` iters
(`(simple a 5)` → state binding dropped). Python/C return the
correct form; asm version is missing the binding. Bug surfaces
in the emit's 30+ binding let*; could not pin down in this
session. Downstream effect: `(miller-rabin n)` and similar
defuns that depend on `cl-loop repeat k for a = ... unless ...
return nil` don't produce usable expansions, so Zoë's acceptance
suite does not run end-to-end on asm/lumbda-full yet.
* examples/ursa-scheme.lsp — `factor` crashes on asm under some
random seeds (bump-allocator exhaustion on long rhoff retry
chains). Out of CL_FULL scope; tracked in same ticket.
Next steps live in ticket 0005. This commit ships the infrastructure
so the remaining work is a debugging exercise against a reproducible
minimal case, not a feature build.
Phase 1 of the asm/lumbda-full roadmap (ticket 0005, in-flight). Adds
the minimum-cost set of additions that lets examples/ursa-scheme.lsp —
the idiomatic Scheme port of ursa.lisp.txt — load and produce correct
results on the asm tier. No CL shim yet: that requires quasiquote,
define-macro, and case, all of which are Phase 2 / 0005.
Added:
* Rest-args in lambda — (define (f x . rest) ...). .ac_bind now
detects when the remaining param list is a raw symbol (TAG_SYM)
and binds it to the remaining arg list. Enables variadic defuns.
* cadr builtin — (car (cdr x)) fast path. Used by Zoë's
repunit-value and any CL-adjacent code.
* sort builtin — ascending insertion sort on a tagged-int list.
Non-destructive. Matches Python/C sort contract (default numeric
ordering). Implementation ~50 lines, recursive sort + insert
helpers.
* let* special form — sequential binding where each init sees the
preceding bindings' values. Fresh sf_let_star + sym_let_star_val
+ .ev_let_star branch that's a one-line variant of .ev_let (eval
init in the extended env rather than the original). TCO preserved.
Tests: 9 new asm assertions in asm/test.sh covering cadr, sort (empty
/ singleton / unsorted / already-sorted), let* (basic + sequential),
rest-args (tail-only + rest-only). Total asm suite now 158 passing.
Known limitation: the Scheme port's factor / rho depends on random
rhoff iteration. For some seeds on asm (e.g. seed=2, factor 91) the
process runs out of virtual memory before rho finds a factor. The
underlying math is correct — this is an asm heap-bump-allocator
behavior under long random-retry chains and will be addressed along
with the CL_FULL work in ticket 0005. Python and C paths unaffected.
make test-all stays green across every tier.
Ticket 0002 — reads 8 bytes from /dev/urandom (little-endian u64) and
seeds xoshiro256**. Opt-in kernel entropy for stochastic runs; the
default stays deterministic (k=0 at startup), so ticket 0001's
portal-reproducibility contract is unchanged.
Real-world flow now one call away:
Machine A: (random-seed-from-os!) + run simulation + portal-save
Machine B: portal-resume — same stream, bit-for-bit
All three impls fail loud on /dev/urandom trouble (LispErr in Python
and C, stderr + exit(1) in asm) — no silent fallback to a weak seed.
Tests:
- tests/functional.lsp: 2 new shared asserts (entropic + replay)
- asm/test.sh: 2 new asm-local checks (149 total, was 147)
- make test-all green across Python (205), C (205), asm (149)
Whitepaper §7.5 gains one sentence noting the OS-seed path.
unmoad: zero new findings in added code.
Completes ticket 0001 started in 27f468c. All three impls now carry
bit-identical xoshiro256**; portal state round-trips across process
boundaries in every producer x consumer cell (Python <-> C <-> asm).
asm impl:
- 4 new builtins: random-seed!, random-int, random-state, random-state!
- g_rng_state in BSS (4 x u64); rng_splitmix64_step, rng_seed, rng_next
- Binary portal header bumped LUMBDAB1/48 -> LUMBDAB2/80; carries
32 bytes of rng state at offsets 40..64, reserved moved to 72
- No float support in asm, so (random) intentionally omitted there
- _start seeds with 0 so the stream is deterministic from startup
Python + C (supplements 27f468c):
- rng_seed(0) auto-invoked at module load / register_portal_builtins
so (random) without explicit (random-seed!) returns a real value
instead of the all-zero xoshiro fixed point
Tests:
- tests/functional.lsp: 7 new shared assertions (Python + C)
- asm/test.sh: 5 new asm-local assertions (142 -> 147)
- tests/portal-rng-save.lsp / portal-rng-load.lsp: portable S-expression
portal that captures both state AND next-5 baseline so loader self-
verifies without a separate harness
- tests/portal-cross-test.sh: 9 new producer x consumer RNG cells; all
18 cells pass end-to-end
Verified: seed=42, (random-int 1000000) draws 1..10 =
558742 543102 559009 124193 317476 750584 200754 814407 344958 929085
identical in Python, C, and asm.
unmoad scan: zero new findings in added code.
asm-gc gains (tcp-sendfile socket path) → builtin (90 lines) that issues
SYS_SENDFILE(40) in a loop, streaming a file from fd → socket with no bounce
through the Lumbda heap. Zero-copy kernel path for large responses.
examples/http-static-server-sendfile.lsp (hybrid): small assets
(≤ 16 KB) stay inline-cached as full HTTP responses; large assets cache
only headers and stream the body via tcp-sendfile. 4-way race on
i5-8350U, 100 PDF requests (2.56 MiB), concurrency 8:
uncached 159 req/s 406 MiB/s 15.5 MB RSS
cached 238 req/s 603 MiB/s 7.2 MB RSS
sendfile 480 req/s 1226 MiB/s 4.2 MB RSS
caddy 485 req/s 1238 MiB/s 37.1 MB RSS
sendfile lands within 2% of caddy on throughput with 9x less peak RSS in
a 27 KB binary vs caddy's 38 MB (1400x smaller).
examples/http-static-server-adaptive.lsp (learning preload): per-URL hit
counter persisted to www.hits every N requests. At boot, ranks and
preloads top *cache-max* URLs from the prior run's data (cold-start
falls back to a seed list). Cold requests beyond the seed promote on
first hit. Drops heap-restore arena pattern since the server mutates
persistent state every request; relies on GC build's mark-sweep.
tests/bench-www-race.sh: adds sendfile variant on port 8083, auto-sizes
PDF byte count from the on-disk whitepaper so a whitepaper rebuild
doesn't desync the MiB/s calc.
Whitepaper §11.7 "Static File Serving: Cache, Sendfile, and Adaptive
Preload" documents the four variants, benchmark table, and the
arena-vs-mutation tradeoff. §13 Future Work adds DAG-of-hot-paths
predictive preload as the direction for > 1000-resource deployments
where frequency-only ranking is too narrow.
Historical internal name "uncommonlisp" retired in favor of the
public name "lumbda" ahead of lumbda.com going live. Scope of
this commit:
Source files renamed:
uncommonlisp.py -> lumbda.py
asm/uncommonlisp.s -> asm/lumbda.s
c/uncommonlisp.h -> c/lumbda.h
whitepaper/uncommonlisp-whitepaper -> whitepaper/lumbda-whitepaper (.rst + .pdf)
Binaries renamed (tracked ones; c/ was always gitignored):
asm/uncommonlisp, asm/uncommonlisp-gc, asm/uncommonlisp.o,
asm/uncommonlisp-gc.o -> asm/lumbda(-gc)(.o)
c/.gitignore -> ignores lumbda
Internal string updates (sed pass ordered longest-first):
asm/uncommonlisp -> asm/lumbda
c/uncommonlisp -> c/lumbda
uncommonlisp.py -> lumbda.py
UNCOMMONLISP_BIN -> LUMBDA_BIN (asm/test.sh env var)
"uncommonlisp> " -> "lumbda> " (asm REPL prompt baked into binary)
UNCOMMONLISP -> LUMBDA (macros, comments)
uncommonlisp -> lumbda (prose)
Binary portal magic updated:
"ULPORTAL" -> "LUMBDAB1" # "Lumbda Binary v1"
Old portal files are not backward-compatible — this is a deliberate
break since it's the rename moment. S-expression portals already
carry their own ";; lumbda-portal v1" header and remain cleanly
versioned.
WHITEPAPER.pdf / WHITEPAPER.rst symlinks repointed to the renamed
files. Makefile's whitepaper target targets lumbda-whitepaper.pdf.
Not changed (intentional, separate phases):
- Filesystem directory /home/fox/git/uncommonlisp itself
(fox renames locally and the gitlab repo URL in a follow-up)
- tests.py hardcoded cwd=/home/fox/git/uncommonlisp
(matches the current on-disk location; will flip when the
directory rename ships)
- Git history (immutable; old commits still say uncommonlisp,
which is correct — that's what they were)
Verified:
137 asm no-GC + 137 asm GC + 571 Python + 83 C + 189 shared
functional tests all pass under the new names.
bench-gc-http (2000 req): all 4 cells behave as expected
(cells 1/2 flat, 3 leaks, 4 bounded at 1 chunk).
Python REPL, C REPL, asm REPL all start cleanly.
