Brings the WAT tier to parity with c-emcc and pyodide for streaming
output during evalLisp. Previously the asm tier buffered everything
in the 0x10000 output region and the JS loader only read the bytes
AFTER lumbda_eval returned — fox saw the bend demo's pre-call
displays sit invisible for 18 s and then appear all at once.
WAT-side changes:
- New env.emit_chunk(ptr, len) import. Host function forwards the
slice to the current onChunk callback so the worker can postMessage
a chunk to the playground panel as work happens.
- New $flush_start global tracks the offset (relative to 0x10000)
where the next emit_chunk slice begins. Reset to 0 at the top
of lumbda_eval alongside $output_len so successive evals don't
re-emit stale bytes.
- $out_char now checks for newline (i32.const 10) after the store.
A newline emits the slice [flush_start, output_len) and advances
flush_start to the end. Every display call ends up flushing on
its trailing newline; per-char displays without a newline get
buffered until the next newline or the eval-end trailing flush.
- $lumbda_eval ends with a trailing-flush guard so any non-newline-
terminated content (e.g. print_value's final repr) reaches the
stream instead of only landing through the final lumbda_output_*
read.
JS loader:
- importObj.env.emit_chunk decodes the slice from wasm memory and
forwards to refs.currentOnChunk.
- evalLisp(src, onChunk) parameter; sets/clears currentOnChunk
around the lumbda_eval call. Same shape as c-emcc + pyodide.
Tests: every node test that instantiates the asm wasm directly now
declares a stub emit_chunk() {} alongside its bend_call stub —
unit, integration, functional-cross, parity-cross-tier. Node test
suite passes 23/23 unit; parity probe times out in its full sweep
under our 30s ceiling so it gets run separately.
Quick smoke: (display "line 1") (newline) (display "line 2") (newline)
(display "line 3") emits three chunks via onChunk — "line 1\n",
"line 2\n", "line 3" — and lumbda_output_* still has the full
"line 1\nline 2\nline 3" as before.
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Variable-length signed bignums on the asm-wasm tier. Layout:
[tag=10, sign:i32, n_limbs:i32, limbs[]:u32]
Little-endian u32 limbs (base 2^32). i64 used for limb-pair products
in bn_mul and for the (rem << 32) | limb shift in bn_divmod_small.
Promotion: num_add/sub/mul/cmp inspect operands and pick the right
representation (fixnum, rational, bignum). Fixnum overflow in +/-/* is
detected by computing in i64 and checking against the 30-bit fixnum
range — outside that, operands lift to bignums.
(expt 2 1024) uses exponentiation-by-squaring through num_mul so
intermediate products auto-promote, returning the exact 309-digit value.
Reader: digit parsing accumulates via num_add/num_mul, so a literal of
any length reads as the narrowest representation that holds it.
Parity corpus: KNOWN_DIVERGE is now empty. 237/237 passing across
python (ref), c-wasm, and asm-wasm. New asserts pin the bignum surface
so a regression breaks make wasm-test immediately.
New: wasm/tests/parity-cross-tier.mjs runs the parity-corpus.mjs (216
test cases tagged by whitepaper section / R7RS concept) against three
tiers — native python (reference), c-wasm, asm-wasm — and fails on any
unknown divergence. Known gaps live in KNOWN_DIVERGE so the table stays
green while the bignum / call/cc / etc. work proceeds.
Wired into `make wasm-test` so a regression against any spec claim gets
caught before merge.
Bugs caught and fixed:
- python remainder: was `signed_a % signed_b * sign(a)`, which double-
applied the sign of a (python's % floors) — gave -3 for (-17, 5)
instead of the R7RS-correct -2. Now uses abs() on both sides.
- asm-wasm modulo: was i32.rem_s (truncated, remainder semantics)
where R7RS modulo wants sign of divisor. Added the "if rem and
divisor disagree on sign, add divisor" branch.
Cross-tier numbers after fix:
216 passing
3 known diverge: expt-2-100, expt-3-50, big-arith — all asm-wasm
(no bignums on the asm tier yet; whitepaper §2.1 claim still open)
0 fail
REPL layout: body is now the scroll container, prompt-bar is
position:fixed at the viewport bottom so it doesn't get pushed off
screen by a long transcript. Empty space above the prompt on a fresh
session reads like a terminal.
All other tests still pass: 20 unit, 8 integration, 11 functional.
Adds tag-9 rational type to the asm tier. Layout [tag=9, num:i32, den:i32].
make_rational normalizes via gcd and collapses to a fixnum when den
reduces to 1, so 14/2 stays as 7.
Arithmetic (+, -, *, /, =, <, >, <=, >=) now promotes to rational when
any argument is rational. Mixed fixnum/rational lifts the fixnum
accumulator into a rational mid-loop so (+ 1 1/2) returns 3/2, not 1/2.
Reader parses "67/7" literals via the existing atom path: after the
numerator's digits, if '/' follows we keep reading the denominator and
hand back a normalized rational. Falls through to symbol if either side
isn't all digits.
Printer renders rationals as "n/d". equal_p compares numbers by value
(1/2 = 2/4, 3 = 6/2). is_number / number? cover both fixnums and
rationals.
eval now treats rationals as self-evaluating — without this, '1/3'
parsed correctly but evaluated to VOID.
Mandelbrot demo: switched from (/ a b) to (quotient a b) for the
fixed-point math. The demo had been relying on integer truncation
that '/' no longer provides on tiers with R7RS-correct rationals.
Bignums still pending: 31-bit num/den overflows with huge denominators.
Real lift comes with the bignum task in the C tier (which has them) or
a new bignum module in the WAT.
Cross-tier check still hangs on the bigger TCO-heavy sections of
functional.lsp — separate from rationals. Will keep grinding.
Tests: unit 20/20, integration 8/8, functional 11/11.
bend!-call from the WAT tier
- New WAT import: (import "env" "bend_call"). The host loader supplies
a sync XMLHttpRequest that POSTs the payload to a configured URL
(workers only — sync XHR isn't allowed on main thread).
- New primitive (bend!-call "<payload>") returns the response as a
lumbda string. Works in playground and REPL once the bend URL is
saved in the new top bar.
- bendUrl persists in plain localStorage (not encrypted — it's a
server address, not a secret).
- Tests stub the import with a no-op so unit / integration / functional
suites keep instantiating cleanly.
Playground + REPL zoom 133% by default
- html { zoom: 1.33 } so the styleguide sizes read comfortably without
requiring browser-level zoom.
Free-form default = cross-tier assertion runner in Lisp
- The default editor content for "free form" is now a small assertion
framework matching tests/functional.lsp's PASS/FAIL convention. A
starter the user can extend, runs identically on the three tiers.
C tier + Python tier (bend) are wired through the runner stub; full
bend integration in those tiers comes next once their loaders learn
about setBendUrl.
Adds (write x) and (write-string s) primitives. write quotes strings
and #\-prefixes chars — what tests/functional.lsp's assert-equal
uses to print failures.
New target: make wasm-test-functional-cross (also rolled into wasm-test)
runs the 205-assertion tests/functional.lsp against each WASM tier and
reports pass counts:
c-wasm: 205 / 205 (full parity with native c)
asm-wasm: 93 / 111 reaches mid-suite before stack overflow on
a deeply recursive test; the 84% it reaches
passes. Documented progress toward full parity
with asm/lumbda.s.
The runner exit-soft on the asm tier — it's a measurement, not a gate.
User-visible changes
- Cancel button — terminates the running worker. Pyodide's slow mandelbrot
no longer freezes the UI; click cancel and the elapsed counter freezes
at "(cancelled @ NNNN ms)".
- Live ms counter ticks per animation frame while a tier is busy, so the
Pyodide tier's ~5-15 s wait is visible instead of looking hung.
- Restyled to match lumbda.com: chunkfive wordmark, --green: #227842
(light) / #5ec07a (dark), lumbda-logo-green.png, lowercase "lumbda"
everywhere. Pulls fonts/chunkfive locally so the playground stays
self-contained.
Architecture
- All tier evaluations now run inside a Web Worker (wasm/app/worker.mjs)
so the main thread stays responsive. Cancel = worker.terminate(); next
eval respawns a fresh worker.
- Loaders use new URL("./...", import.meta.url) so paths resolve against
the loader file's own location — works identically in window and
worker contexts, no baseURL argument needed.
- C tier Emscripten build flipped to EXPORT_ES6=1; loader uses dynamic
`import()` of the factory module. Integration test updated accordingly.
- Python loader uses `import("pyodide.mjs")` (ES module) instead of
document.createElement, which doesn't exist in workers.
Bug fixes
- Asm tier state leak: running the same demo twice on a cached WASM
instance produced corrupted output (every other cell on row 2+ rendered
as " " instead of the expected shade char). Root cause: top-level eval
passed `global_env` as the env, so closures captured stale globals;
fixed by passing NIL — env_lookup falls back to the CURRENT global_env
via its existing two-pass walk. Multi-run regression added to the
functional test suite.
- fib-ack demo: (ack 3 4) was too heavy for Pyodide (minutes). Cut to
(ack 3 3) + (fib 20) max so every tier finishes in seconds.
Test discipline
- Root `make test-all` now includes `wasm-test`. Adding a language
feature without exercising it on all six implementations is no longer
possible by accident.
- Functional suite: 11 assertions (was 10) — adds asm multi-run stability.
- Integration + unit: still 20 + 8.
Adds a parallel build of all three Lumbda implementations to WASM, a
single-page playground at www/playground/, and a verified test suite.
Tiers
- Python: Pyodide (CPython-in-WASM) hosting lumbda.py
- C: Emscripten build of c/ (tree-walker + bytecode VM; jit.c
stubbed, gc.c uses its existing no-Boehm fallback)
- Asm: hand-written asm/lumbda.wat — parallel impl to asm/lumbda.s.
Reader, eval (lambda/define/if/cond/let/and/or/quote/set!),
recursion across mutated top-level env, bump allocator with
memory.grow, 24 primitives. ~1200 lines of raw WAT.
SPA (wasm/app/, deployed to www/playground/)
- CodeMirror 6 editor (Scheme highlighting) on left, output on right
- Radios: 4 demos (Mandelbrot, Fib+Ack, Sieve, self-interp meta-eval)
x 4 tiers (Python | C | Asm | All three)
- All-three mode renders the three tier outputs side by side with
per-tier elapsed timing
Tests (38 verified assertions)
- 20 unit (Node): per-tier module loads, eval smoke
- 8 integration (Node): each demo on c+asm WASM byte-matches the
canonical native Python run
- 10 functional (Playwright headless Chromium): page mounts, every
demo runs on every tier, all-three renders
Makefile
- Root targets: wasm-build, wasm-test, wasm-test-fn, wasm-serve,
wasm-deploy, wasm-clean
- wasm/Makefile orchestrates the three tier builds; deploy copies
dist/ into www/playground/
Asm tier notes
- WAT linear symbol intern + linear env lookup is MOAD-0001 at scale;
documented in the asm/lumbda.wat header and in the SPA footer. The
demos hit ~30 globals so the linear walks are cheap enough.
- Bump allocator never frees (matches asm/lumbda.s heap discipline);
memory.grow expands by 1 MB chunks. Browser tab tears down at unload.
Toolchain (developer prerequisites)
- Emscripten 6.0.0 via emsdk at ~/git/emsdk
- wabt 1.0.36 at ~/git/wabt
- Playwright for functional tests (symlinked from ~/git/agnt)
