russell@unturf.com
192118388f
Zoë Trout's favorites at wedgewack.org/ursa.lisp.txt are Common Lisp:
iterative LOOP macros, setf cascades, defun with &optional, image-
based stone-lisp culture. Her first contribution to lumbda was a
question — "do we care for our programs, and how long are they alive
for?" — and the answer now extends beyond the RNG portal (§7.5) to
iteration style itself.
Four-phase delivery, all under ticket 0004:
Phase A — idiomatic Scheme ports at examples/ursa-scheme.lsp.
Every Zoë defun rewritten as named-let + tail recursion + list-
backed work queue + type-predicate dispatch.
Phase B — CL compat shim at cl-compat.lsp.
defun (with &optional), setf (simple vars, multi-pair), flet,
multiple-value-bind, t / nil (nil=#f so cond/if compose),
evenp/oddp/plusp/minusp/zerop, mod/ash/logbitp/nreverse,
cl-when/cl-unless (plain when is a void-returning lumbda special
form), declare (no-op), cddddr (missing accessor).
Phase C — cl-loop macro covering 14 patterns.
while/until/repeat, for VAR from A to/below/downto B, for VAR =
INIT [then STEP], for VAR across VEC, of-type T, do, when/unless
return, finally (return VAL). Sequential do*-style stepping via
gensym + cl-subst. Look-ahead termination so `repeat 4 for s = 4
then (- (* s s) 2) finally (return s)` returns 37634 (pre-step)
rather than 1416317954 (post-step). Every expansion ends in a
named-let tail call — TCO holds for loops of any length.
Phase D — load examples/ursa.lisp.txt with minimal annotation.
Preserves Zoë's CL. Minimal edits documented in file header:
load cl-compat.lsp, loop→cl-loop, when→cl-when, random→random-int,
&key→&optional. rho/digits omitted (need make-array/CLOS — see
ticket 0004 for scope boundary).
Defect uncovered along the way (c/types.c env_lookup): a "global
shortcut" checked global env immediately after missing the local
frame, SKIPPING intermediate parent scopes. Broke lexical scoping
whenever a parent scope shadowed a global. Reproduced with
(define s 4)
(let ((s 100)) (let ((m 0)) s)) ; returned 4, should return 100
Any nested let whose body referenced a shadowed name silently read
the global. Fix: remove the shortcut, walk the parent chain end-to-
end. 1255 assertions across five suites pass unchanged after fix —
surfaced only because cl-loop iterator names routinely collide with
globals accumulated in a stone-lisp image.
Whitepaper §9.2 documents the CL-in-Scheme design and the guarantees
that survive (TCO, portal determinism, cross-impl reproducibility).
Zoë added to authors + acknowledgments; reacknowledgment reframes
her first contribution as the deeper program-lifetime question, with
RNG portal as a derivative (§7.5) and cl-loop as the follow-up.
Tests: tests/cl-compat.lsp (44 assertions) and tests/ursa.lsp (28
assertions) exercise both paths under Python + C via tests/zoe-
favorites-test.sh, wired into make test-all.
MOAD notes: unmoad flags memq/assq in cl-compat.lsp over cl-loop-
keywords (~30 elements, constant) and var->new (≤4 state vars per
loop). Both are macro-expansion-time, bounded-small-N — not runtime
hot paths. Pre-existing c/types.c findings (strcmp-in-loop for
record-type lookup) are not from this change.
|
||
|---|---|---|
| .. | ||
| tickets | ||
| asm-architecture.dot | ||
| asm-architecture.png | ||
| benchmark-ack.dot | ||
| benchmark-ack.png | ||
| benchmark-binary-size.dot | ||
| benchmark-binary-size.png | ||
| benchmark-fib.dot | ||
| benchmark-fib.png | ||
| benchmark-gc.dot | ||
| benchmark-gc.png | ||
| benchmark-speedup.dot | ||
| benchmark-speedup.png | ||
| benchmark-sumto.dot | ||
| benchmark-sumto.png | ||
| c-architecture.dot | ||
| c-architecture.png | ||
| gpu-architecture.md | ||
| jit-pipeline.dot | ||
| jit-pipeline.png | ||
| meta-gc-policy.dot | ||
| meta-gc-policy.png | ||
| python-architecture.dot | ||
| python-architecture.png | ||
| README.md | ||
lumbda Architecture Documentation
"A diagram is worth 10,000 words." — russell@unturf.com
Three implementations of the same Scheme language, sharing the same .lsp test files.
Python Implementation (lumbda.py)
3,324 lines. Bytecode compiler + stack VM + full continuations + portal.
Execution tiers:
- Tree-walker (
leval): default, handles all forms including macros - Bytecode VM (
--fast): 40 opcodes + superinstructions, 7-19x faster - Python JIT prototype: exec()-based transpilation (labeled as prototype)
Key features:
- Full multi-shot continuations via explicit frame stack
- Portal: serialize VM state to JSON, resume on another machine
- Inline cache, constant folding, peephole optimizer
- Source maps for error reporting with line numbers
- Bytecode serialization (.lspc files)
Tests: 571 unit + integration tests (tests.py)
C Implementation (c/)
8,120 lines. Tree-walker + bytecode VM + x86_64 JIT.
Execution tiers:
- Tree-walker: default, full special form support
- Bytecode VM (
--fast): matching Python's opcodes - x86_64 JIT (
--jit): 10-24x faster than CPython
Key features:
- NaN-boxed 64-bit values (zero-alloc numbers)
- Hash-map environments with parent chain + global shortcut
- Interned symbols
- Real JIT: mmap(PROT_EXEC) + raw x86_64 bytes
Tests: 76 unit + integration + JIT tests (test.c)
Assembly Implementation (asm/)
2,592 lines of GNU assembler. 13KB binary. Zero dependencies.
Design:
- No C. No libc. Only Linux syscalls (read, write, mmap, exit)
- Tag-in-low-3-bits value representation
- Bump allocator on 64MB mmap'd page
- TCO via
jmp .eval_top(never grows the stack) - 34 builtins, all special forms
Tests: 75 unit + integration + functional tests (test.sh)
JIT Pipeline (c/jit.c)
1,309 lines. Compiles Scheme AST directly to x86_64 machine code.
What gets JIT'd:
- if, cond, and, or (conditional jumps)
- +, -, *, =, <, >, <=, >= (native integer ops)
- let, let* (stack-allocated locals)
- Named-let loops (native jmp, zero call overhead)
- car, cdr, cons, null?, pair? (NaN-box pointer ops)
- Self-recursive calls (call/ret) and tail calls (jmp)
What falls back to interpreter:
- call/cc, macros, syntax-rules, quasiquote, modules
- String/vector/hash-table operations
- Any form the AST analyzer can't verify as integer-safe
Performance Summary
All benchmarks measured in-process (no startup overhead) on the same machine.
| Implementation | ack(3,4) | fib(35) | sum-to(50k) | Binary |
|---|---|---|---|---|
| C + x86_64 JIT | 0.19ms | 0.09ms | 0.55ms | 171KB |
| CPython (native) | 1.3ms | 0.006ms | 5.5ms | ~5MB |
| C interpreter | 20ms | 0.06ms | 109ms | 171KB |
| Python bytecode VM | 149ms | 0.75ms | 437ms | 3,324 lines |
| Assembly (13KB) | ~8ms* | ~0.6ms* | ~43ms* | 13KB |
*Assembly times include process startup + tokenizer + parser.
The JIT runs Scheme faster than CPython runs Python on recursive workloads: ack(3,4) is 7x faster, sum-to(50k) is 10x faster. The JIT compiles Scheme AST directly to x86_64 machine code via mmap(PROT_EXEC).
Test Coverage
943 verified assertions across all implementations:
make test-all
Python unit/integration: 571 tests
C unit/integration/JIT: 83 tests
Assembly unit/int/func: 108 tests
Shared functional: 181 tests (Python + C)
Total: 943 assertions
The language is R7RS Scheme. lumbda is the project name — a play on Common Lisp, since this is decidedly uncommon.