A throttled forever-counter on the asm tier ran for ~5.7 s and
then trapped 'index out of bounds' — the WAT writes output to a
fixed 64 KB region at 0x10000–0x1FFFF, and a fast (display X)
(newline) loop accumulates faster than the buffer can drain. After
about 6,400 ticks at ~10 chars each, overflowed the
region into the source buffer at 0x20000 and the next i32.store8
fell off linear memory.
Fix is a contract change on emit_chunk: its signature picks up an
i32 return — 1 tells the WAT to recycle (zero output_len AND
flush_start), 0 keeps the original 'just advance flush_start'
semantics so callers that read lumbda_output_ptr/len after eval
still see the full buffer.
The asm loader returns 1 from emit_chunk, accumulating every
flushed slice into refs.accumulated. evalLisp's return value is now
refs.accumulated + the trailing (still-unflushed) buffer slice
rather than just the lumbda_output_ptr/len slice — caller still
gets the complete output, the WAT-side buffer just keeps recycling.
Test stubs already declared emit_chunk() {} which returns
undefined — JS->wasm i32 coercion turns that into 0, preserving
the no-recycle behavior they expected. unit/integration suites
(31 tests total) still pass.
Trailing flush in lumbda_eval also honors the return value: if the
host consumed, zero both offsets so the final lumbda_output_ptr/len
read returns 0 bytes (loader already accumulated the trailing
slice — no need to re-deliver). Earlier draft of this patch
double-emitted the trailing slice because we read it through both
the emit_chunk path and the final-buffer path.
Brings the WAT tier to parity with c-emcc and pyodide for streaming
output during evalLisp. Previously the asm tier buffered everything
in the 0x10000 output region and the JS loader only read the bytes
AFTER lumbda_eval returned — fox saw the bend demo's pre-call
displays sit invisible for 18 s and then appear all at once.
WAT-side changes:
- New env.emit_chunk(ptr, len) import. Host function forwards the
slice to the current onChunk callback so the worker can postMessage
a chunk to the playground panel as work happens.
- New $flush_start global tracks the offset (relative to 0x10000)
where the next emit_chunk slice begins. Reset to 0 at the top
of lumbda_eval alongside $output_len so successive evals don't
re-emit stale bytes.
- $out_char now checks for newline (i32.const 10) after the store.
A newline emits the slice [flush_start, output_len) and advances
flush_start to the end. Every display call ends up flushing on
its trailing newline; per-char displays without a newline get
buffered until the next newline or the eval-end trailing flush.
- $lumbda_eval ends with a trailing-flush guard so any non-newline-
terminated content (e.g. print_value's final repr) reaches the
stream instead of only landing through the final lumbda_output_*
read.
JS loader:
- importObj.env.emit_chunk decodes the slice from wasm memory and
forwards to refs.currentOnChunk.
- evalLisp(src, onChunk) parameter; sets/clears currentOnChunk
around the lumbda_eval call. Same shape as c-emcc + pyodide.
Tests: every node test that instantiates the asm wasm directly now
declares a stub emit_chunk() {} alongside its bend_call stub —
unit, integration, functional-cross, parity-cross-tier. Node test
suite passes 23/23 unit; parity probe times out in its full sweep
under our 30s ceiling so it gets run separately.
Quick smoke: (display "line 1") (newline) (display "line 2") (newline)
(display "line 3") emits three chunks via onChunk — "line 1\n",
"line 2\n", "line 3" — and lumbda_output_* still has the full
"line 1\nline 2\nline 3" as before.
The WAT $read_string function scanned bytes until the first " and copied
raw, with NO escape handling. So a source string like
"(\"00\" \"01\")" got chopped at the first \" — the asm tier read only
"(\\" before terminating, producing a mangled payload that the worker
couldn't parse. The bend-gpu demo's payload built via string-append of
escaped-quote strings came out as "(cuda-shake-fanout (\ \ \) 32)" in
the asm tier, sent garbage to bend, and got nothing useful back.
Two-pass fix to match Python/C tier behavior:
1. Scan-and-count pass: walks source-ptr to the closing ", but
when it sees \ it skips the next byte so embedded \" doesn't
terminate the string. Counts decoded output bytes (each \X
contributes one byte, not two).
2. Allocate + copy-and-decode pass: walks the same range, converts
\n → 0x0A, \t → 0x09, and any other \X (including \" and \\)
→ X. Matches the lenient fallback the desktop tiers use.
Verified via cross-tier parity probe — 255/255 still passing.
Demo payload now constructs as
"(cuda-shake-fanout (\"00\" \"01\" \"deadbeef\") 32)" (45 bytes,
identical to Python/C reads) and the asm playground returns
(ok (HEX0 HEX1 HEX2)) from the live 3090-ai bend worker.
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Step 1 of the GC effort. Each tier loader now exposes heapStats():
- asm-wasm — lumbda_heap_used / lumbda_heap_total wat exports
- c-wasm — emscripten linear memory size (no free path right now,
so used = total; documented in the loader)
- python — pyodide module linear memory size; CPython GC cycles
this naturally
Worker handles a "heap" message kind that round-trips the active tab's
loaded tiers; repl tabbar shows a compact "py 12M · c 32M · asm 4M"
strip next to the buttons. Polls every 2s.
Doesn't solve the leak — just makes pressure visible so the user knows
when to use "reboot tier". Real GC (Cheney over the WAT bump allocator,
Boehm-em or custom mark-sweep for c-wasm) coming next.
Variable-length signed bignums on the asm-wasm tier. Layout:
[tag=10, sign:i32, n_limbs:i32, limbs[]:u32]
Little-endian u32 limbs (base 2^32). i64 used for limb-pair products
in bn_mul and for the (rem << 32) | limb shift in bn_divmod_small.
Promotion: num_add/sub/mul/cmp inspect operands and pick the right
representation (fixnum, rational, bignum). Fixnum overflow in +/-/* is
detected by computing in i64 and checking against the 30-bit fixnum
range — outside that, operands lift to bignums.
(expt 2 1024) uses exponentiation-by-squaring through num_mul so
intermediate products auto-promote, returning the exact 309-digit value.
Reader: digit parsing accumulates via num_add/num_mul, so a literal of
any length reads as the narrowest representation that holds it.
Parity corpus: KNOWN_DIVERGE is now empty. 237/237 passing across
python (ref), c-wasm, and asm-wasm. New asserts pin the bignum surface
so a regression breaks make wasm-test immediately.
New: wasm/tests/parity-cross-tier.mjs runs the parity-corpus.mjs (216
test cases tagged by whitepaper section / R7RS concept) against three
tiers — native python (reference), c-wasm, asm-wasm — and fails on any
unknown divergence. Known gaps live in KNOWN_DIVERGE so the table stays
green while the bignum / call/cc / etc. work proceeds.
Wired into `make wasm-test` so a regression against any spec claim gets
caught before merge.
Bugs caught and fixed:
- python remainder: was `signed_a % signed_b * sign(a)`, which double-
applied the sign of a (python's % floors) — gave -3 for (-17, 5)
instead of the R7RS-correct -2. Now uses abs() on both sides.
- asm-wasm modulo: was i32.rem_s (truncated, remainder semantics)
where R7RS modulo wants sign of divisor. Added the "if rem and
divisor disagree on sign, add divisor" branch.
Cross-tier numbers after fix:
216 passing
3 known diverge: expt-2-100, expt-3-50, big-arith — all asm-wasm
(no bignums on the asm tier yet; whitepaper §2.1 claim still open)
0 fail
REPL layout: body is now the scroll container, prompt-bar is
position:fixed at the viewport bottom so it doesn't get pushed off
screen by a long transcript. Empty space above the prompt on a fresh
session reads like a terminal.
All other tests still pass: 20 unit, 8 integration, 11 functional.
Adds tag-9 rational type to the asm tier. Layout [tag=9, num:i32, den:i32].
make_rational normalizes via gcd and collapses to a fixnum when den
reduces to 1, so 14/2 stays as 7.
Arithmetic (+, -, *, /, =, <, >, <=, >=) now promotes to rational when
any argument is rational. Mixed fixnum/rational lifts the fixnum
accumulator into a rational mid-loop so (+ 1 1/2) returns 3/2, not 1/2.
Reader parses "67/7" literals via the existing atom path: after the
numerator's digits, if '/' follows we keep reading the denominator and
hand back a normalized rational. Falls through to symbol if either side
isn't all digits.
Printer renders rationals as "n/d". equal_p compares numbers by value
(1/2 = 2/4, 3 = 6/2). is_number / number? cover both fixnums and
rationals.
eval now treats rationals as self-evaluating — without this, '1/3'
parsed correctly but evaluated to VOID.
Mandelbrot demo: switched from (/ a b) to (quotient a b) for the
fixed-point math. The demo had been relying on integer truncation
that '/' no longer provides on tiers with R7RS-correct rationals.
Bignums still pending: 31-bit num/den overflows with huge denominators.
Real lift comes with the bignum task in the C tier (which has them) or
a new bignum module in the WAT.
Cross-tier check still hangs on the bigger TCO-heavy sections of
functional.lsp — separate from rationals. Will keep grinding.
Tests: unit 20/20, integration 8/8, functional 11/11.
WASM tail-call instruction wired into every tail position:
if branches, eval_begin last expression, all special-form dispatchers
(cond/when/unless/case/let/let*/letrec/begin/and/or), function
application's $apply, and apply's closure-body $eval_begin.
wat2wasm + wasm-validate now use --enable-tail-call.
Reader fixes:
- Dotted pair syntax: (a b . rest) parses as a real dotted list.
Without this, variadic params and other dotted-cdr forms parsed
as 4-element proper lists.
- Stray ) at top level advances source_ptr instead of spinning
forever. Found via bisection of functional.lsp under TCO.
- eval_args guards against non-pair tails so a misplaced dotted
argument (e.g. an unexpanded macro template) can't deref garbage.
Cross-tier numbers (wasm-test-functional-cross):
before: 93/111 reached, stack overflow on (ack 3 4)
now: progresses through the full TCO section, deep let, named-let
to 100k, mutual recursion to 200k. Still climbing.
Bend gpu demo:
6th playground option ("bend (gpu dispatch) ⚡") ships a SHAKE256
fan-out at 1M inputs via (bend!-call ...). Run button is GUARDED:
if no bend URL is configured, refuses with
"set a bend URL first — this demo is GPU-only by design".
Protects customer machines from burning minutes on a workload the
local tiers cannot finish in reasonable time.
bend!-call from the WAT tier
- New WAT import: (import "env" "bend_call"). The host loader supplies
a sync XMLHttpRequest that POSTs the payload to a configured URL
(workers only — sync XHR isn't allowed on main thread).
- New primitive (bend!-call "<payload>") returns the response as a
lumbda string. Works in playground and REPL once the bend URL is
saved in the new top bar.
- bendUrl persists in plain localStorage (not encrypted — it's a
server address, not a secret).
- Tests stub the import with a no-op so unit / integration / functional
suites keep instantiating cleanly.
Playground + REPL zoom 133% by default
- html { zoom: 1.33 } so the styleguide sizes read comfortably without
requiring browser-level zoom.
Free-form default = cross-tier assertion runner in Lisp
- The default editor content for "free form" is now a small assertion
framework matching tests/functional.lsp's PASS/FAIL convention. A
starter the user can extend, runs identically on the three tiers.
C tier + Python tier (bend) are wired through the runner stub; full
bend integration in those tiers comes next once their loaders learn
about setBendUrl.
Adds (write x) and (write-string s) primitives. write quotes strings
and #\-prefixes chars — what tests/functional.lsp's assert-equal
uses to print failures.
New target: make wasm-test-functional-cross (also rolled into wasm-test)
runs the 205-assertion tests/functional.lsp against each WASM tier and
reports pass counts:
c-wasm: 205 / 205 (full parity with native c)
asm-wasm: 93 / 111 reaches mid-suite before stack overflow on
a deeply recursive test; the 84% it reaches
passes. Documented progress toward full parity
with asm/lumbda.s.
The runner exit-soft on the asm tier — it's a measurement, not a gate.
WAT prelude (evaluated after primitive binding at init) adds:
map, filter, fold-left, fold-right, for-each, any, every,
count, find, sort (quicksort), vector-map, vector-for-each,
vector-fill!, string-split, string-trim, string->list,
random-state, assert-equal/true/false.
Higher-order ops are now Lisp-defined, not primitive bloat. Eval-time
parse + bind happens once per WASM instance startup.
Playground output: per fox, single append-only column instead of
3-up grid. Tiers still race in parallel workers; whichever finishes
first appears first in the output. Live ms counters move to the status
bar (python 312ms · c 47ms · asm 89ms).
Two new heap tags:
7 = vector [tag, len, elem_0, elem_1, ...] 8 + 4*len bytes
8 = hashtable [tag, count, alist_ptr] 12 bytes
Vector primitives (88-95):
vector, vector?, make-vector, vector-length,
vector-ref, vector-set!, vector->list, list->vector
Hash table primitives (96-106):
make-hash-table, hash-table?, hash-table-set!,
hash-table-ref, hash-table-ref/default,
hash-table-delete!, hash-table-exists?, hash-table-size,
hash-table-keys, hash-table-values, hash-table->alist
Hash table lookup is linear (equal? on each key) — fine for browser-scale
demos. Same linear-scan caveat as the symbol intern; would warrant a real
hash function at scale.
print_value now renders vectors as #(a b c) and hashtables as #<hashtable>.
User-visible changes
- Cancel button — terminates the running worker. Pyodide's slow mandelbrot
no longer freezes the UI; click cancel and the elapsed counter freezes
at "(cancelled @ NNNN ms)".
- Live ms counter ticks per animation frame while a tier is busy, so the
Pyodide tier's ~5-15 s wait is visible instead of looking hung.
- Restyled to match lumbda.com: chunkfive wordmark, --green: #227842
(light) / #5ec07a (dark), lumbda-logo-green.png, lowercase "lumbda"
everywhere. Pulls fonts/chunkfive locally so the playground stays
self-contained.
Architecture
- All tier evaluations now run inside a Web Worker (wasm/app/worker.mjs)
so the main thread stays responsive. Cancel = worker.terminate(); next
eval respawns a fresh worker.
- Loaders use new URL("./...", import.meta.url) so paths resolve against
the loader file's own location — works identically in window and
worker contexts, no baseURL argument needed.
- C tier Emscripten build flipped to EXPORT_ES6=1; loader uses dynamic
`import()` of the factory module. Integration test updated accordingly.
- Python loader uses `import("pyodide.mjs")` (ES module) instead of
document.createElement, which doesn't exist in workers.
Bug fixes
- Asm tier state leak: running the same demo twice on a cached WASM
instance produced corrupted output (every other cell on row 2+ rendered
as " " instead of the expected shade char). Root cause: top-level eval
passed `global_env` as the env, so closures captured stale globals;
fixed by passing NIL — env_lookup falls back to the CURRENT global_env
via its existing two-pass walk. Multi-run regression added to the
functional test suite.
- fib-ack demo: (ack 3 4) was too heavy for Pyodide (minutes). Cut to
(ack 3 3) + (fib 20) max so every tier finishes in seconds.
Test discipline
- Root `make test-all` now includes `wasm-test`. Adding a language
feature without exercising it on all six implementations is no longer
possible by accident.
- Functional suite: 11 assertions (was 10) — adds asm multi-run stability.
- Integration + unit: still 20 + 8.
Adds a parallel build of all three Lumbda implementations to WASM, a
single-page playground at www/playground/, and a verified test suite.
Tiers
- Python: Pyodide (CPython-in-WASM) hosting lumbda.py
- C: Emscripten build of c/ (tree-walker + bytecode VM; jit.c
stubbed, gc.c uses its existing no-Boehm fallback)
- Asm: hand-written asm/lumbda.wat — parallel impl to asm/lumbda.s.
Reader, eval (lambda/define/if/cond/let/and/or/quote/set!),
recursion across mutated top-level env, bump allocator with
memory.grow, 24 primitives. ~1200 lines of raw WAT.
SPA (wasm/app/, deployed to www/playground/)
- CodeMirror 6 editor (Scheme highlighting) on left, output on right
- Radios: 4 demos (Mandelbrot, Fib+Ack, Sieve, self-interp meta-eval)
x 4 tiers (Python | C | Asm | All three)
- All-three mode renders the three tier outputs side by side with
per-tier elapsed timing
Tests (38 verified assertions)
- 20 unit (Node): per-tier module loads, eval smoke
- 8 integration (Node): each demo on c+asm WASM byte-matches the
canonical native Python run
- 10 functional (Playwright headless Chromium): page mounts, every
demo runs on every tier, all-three renders
Makefile
- Root targets: wasm-build, wasm-test, wasm-test-fn, wasm-serve,
wasm-deploy, wasm-clean
- wasm/Makefile orchestrates the three tier builds; deploy copies
dist/ into www/playground/
Asm tier notes
- WAT linear symbol intern + linear env lookup is MOAD-0001 at scale;
documented in the asm/lumbda.wat header and in the SPA footer. The
demos hit ~30 globals so the linear walks are cheap enough.
- Bump allocator never frees (matches asm/lumbda.s heap discipline);
memory.grow expands by 1 MB chunks. Browser tab tears down at unload.
Toolchain (developer prerequisites)
- Emscripten 6.0.0 via emsdk at ~/git/emsdk
- wabt 1.0.36 at ~/git/wabt
- Playwright for functional tests (symlinked from ~/git/agnt)
