Tab switching during a long-running eval used to silently abandon
the calc — output stopped streaming, no snapshot, nothing to come
back to. Now setActiveTab pauses the outgoing tab's eval (and
optionally portal-saves the env), terminates the worker, and on
re-entry hydrates + re-fires the original input.
Pieces:
* serve-coop.py + make serve-repl — dev server that emits
Cross-Origin-Opener-Policy: same-origin and
Cross-Origin-Embedder-Policy: require-corp so SharedArrayBuffer
is constructable in the browser. Same headers production needs.
* C tier eval-loop pause poll — c/eval.c grows lumbda_check_pause(),
guarded by #ifdef LUMBDA_WASM. Called at the top of leval()'s
while(1); masked to every 1024th iteration so the polling cost
stays under noise floor. When the JS-library import
js_lumbda_pause_requested returns 1, lisp_error("paused")
longjmps out so module-global env survives intact for the
portal-snapshot that follows.
* SAB plumbing — main thread allocates new SharedArrayBuffer(4),
hands it through worker config → runner.setPauseFlag →
lumbda-c.loader.setPauseFlag → globalThis._lumbdaCPauseFlag.
Atomics.store / Atomics.load on index 0 is the signalling
channel. Falls back to null when COOP/COEP isn't isolated, in
which case pause degrades to a hard worker.terminate().
* autoPauseTab() — on setActiveTab away, snapshots the tier
(C tier with SAB) or hard-cancels (other tiers / no SAB),
stashes tab.autoPause = {tier, blob, inputSrc, savedAt},
terminates the workers so the heap is reclaimed.
* autoResumeTab() — on setActiveTab into a tab with autoPause,
reboots the tier, hydrates MEMFS, runs (portal-load! ...), then
re-fires the original input via sendInput so the eval restarts
from the saved state. Asm + Python paths re-run from scratch
until their poll sites land.
Also closes two UX papercuts from fox: chip ⇣ export icon bumped
from 0.85em muted to 1em green so it's actually discoverable; the
scope toggle now reads "scope: this tab" / "scope: all tabs" so the
button label describes the state rather than a target.
Three UX gaps on the portal-bar closed in one pass plus a defensive
fix on the C tier's heap probe:
* Overwrite guard — saving with an existing name asks 'overwrite?'
with the existing entry's tier + savedAt. Rename via dbl-click on
the chip label; same overwrite guard applies on rename.
* Export / import — a ⇣ icon on each chip downloads it as
<name>.portal.json (opaque blob for c/python, replay-inputs for
asm). A 📁 import button on the portal-bar accepts a .portal.json
file via hidden <input type="file">; collisions prompt overwrite,
decline auto-suffixes (baseName-2, -3, …) so importing a 2nd copy
always lands somewhere.
* Cross-tab restore — a 📂 this tab / 🌐 all tabs toggle switches
the chip strip between the active tab's checkpoints and every
tab's. Global chips render as 'name · tabName' with a dashed
border; click restores the snapshot into the active tab (the
saved cp is passed through restoreCheckpoint's new sourceCp
argument so the chip doesn't need a checkpoints[name] match on
the active tab). Edit/delete are hidden in global mode — the
user switches to the owning tab to manage chips.
* heapStats defensive — wasm/c/lumbda-c.loader.js now returns null
when module.HEAPU8 isn't live yet (caught by the heap poll firing
during the tiny window between tier reboot and Module init), so a
restore no longer surfaces 'Cannot read properties of undefined
(reading byteLength)' as a TypeError.
Smoke-tested headlessly: overwrite confirm fires with the expected
message; rename via dblclick swaps the label; ⇣ produces a download
named '<name>.portal.json'; toggle shows both tabs' chips with the
'· tabName' annotation; import round-trips back into the receiving
tab. Zero page errors across the full flow.
Adds a portal-bar to the REPL between tabbar and transcript: a save
button + chip strip showing all saved checkpoints for the active tab.
Click a chip to restore, click × to delete.
Per-tier strategy:
* c, python — call the tier's (portal-snapshot! NAME), then read the
JSON blob out of MEMFS (Emscripten/Pyodide FS) and stash it in the
encrypted vault entry. Restore reverses: hydrate MEMFS, then
(portal-load! NAME) merges the bindings into the live env.
* asm — no portal serializer in the WAT tier yet (would need a
Cheney-aware walk). Falls back to transcript replay: save snapshots
every successful prior input, restore reboots the tier and re-evals
them in order.
Plumbing:
* Worker bridge: new portal-save / portal-load message kinds wire
MEMFS reads/writes to the main thread.
* runner.js exposes portalSave / portalLoad — null when a tier
hasn't implemented portals (asm stays grey).
* C tier: replace EM_JS with extern + --js-library for js_lumbda_bend_call
(EM_JS-generated declaration was unreachable from wasmImports at
instantiate time, browsers threw "import object field ... not a
Function"). FS added to EXPORTED_RUNTIME_METHODS so JS can reach
pyodide.FS / Module.FS for MEMFS I/O.
Smoke-tested all three tiers headlessly: save → chip render → restore
round-trips clean on c / python / asm, zero console errors.
Two fixes in one push.
(1) Streaming horizontal-output bug, take three. textContent += chunk
then appendChild(createTextNode(chunk)) both still rendered chunks
horizontally in fox's Firefox tab even though the chunks clearly
contained \\n (node test confirmed; deployed loader hash matched
local; curl-fetched lumbda-c.loader.js carried the line + '\\n' fix).
Whatever the browser was doing with sibling text nodes inside a
<pre> wasn't honoring the embedded newlines.
Switch to one <div class=\"stream-line\"> per logical line. liveBlock.append
walks the incoming chunk byte by byte, every \\n closes the current
pending div and spawns a fresh empty one for the next line. CSS
adds .stream-line { display: block; white-space: pre; } so each
finished line stacks vertically no matter what the parent
white-space rule was doing. Empty lines get a single space so they
take a row instead of collapsing. Reconcile path inside finalize()
compares the joined per-line text to the full output and rebuilds
the div column if they diverge — for the asm-tier fallback we
already had and now also for any future browser/wasm combo where
a flush silently drops a chunk.
Also drops the temporary [c-tier print] console.log debug we added
in the last commit — diagnosis arrived from elsewhere, no point
keeping the spam.
(2) Per-program autosave drafts. scheduleFreeFormSave previously
returned early if the selected program wasn't \"free-form\", so a
user who unlocked the vault, edited the bend-gpu demo, and came
back later found their edits gone — only free-form persisted.
Now drafts live as { [programName]: text } in the vault payload;
every edit, regardless of which radio is selected, debounces a
save into drafts[currentProgram]. unlockVault loads any saved
draft for the current program (and lifts the legacy
top-level freeForm field into drafts['free-form'] so existing
users don't lose their work). loadCurrentDemo shows a saved
draft instead of the ship default whenever one exists; vault bar
stays visible across all programs once the vault is engaged so
the save status note is always reachable.
The deployed loader emits chunks with currentOnChunk(line + '\n'),
which my node tests confirm produces clean per-line chunks
('tick 0\n', 'tick 1\n', ...). But fox still reports horizontal
output in his playground tab — same tier, same loop, same loader
hash on the wire (verified via curl). Browser caches and module
loaders are aggressive enough that hard-refresh isn't always
enough.
Adding a temporary console.log inside Module.print so when fox
opens DevTools and re-runs the throttled loop he can see (a) is
the print callback firing at all? and (b) what does Emscripten
hand us — is it actually 'tick 0' or something stranger like
'tick 0\n' or 'tick' + ' 0' as two calls?
Will revert this once the bug is resolved.
EOF
)
Emscripten's Module.print fires once per line and hands the loader
just the line content — no trailing newline; the caller is expected
to add it back when joining (outBuf.join('\n') does that for the
final return). Streaming path was forwarding the line straight to
onChunk without the newline, so successive chunks concatenated
horizontally in the playground panel — the loop
(let loop ((i 0))
(cond ((= (modulo i 10000) 0) (display \"tick \") (display i) (newline)))
(loop (+ i 1)))
was running fine on c tier (each (newline) ended a printf line) but
arriving in the UI as 'tick 0tick 10000tick 20000...' with no
breaks.
Loader's print/printErr callbacks now pass `line + \"\\n\"` to
currentOnChunk. Final buffered join (which already adds the \\n)
is untouched, so the post-eval output string keeps the same shape.
Pyodide _StreamingStdout sees Python's raw write() bytes including
the \\n already; asm's emit_chunk fires AFTER output_len was
incremented past the \\n, so the slice [flush_start, output_len)
includes it. Neither needed a change.
Previously every (display ...) call buffered into the tier's outBuf /
sys.stdout / WAT output-buffer and only landed in the playground panel
after evalLisp returned. During the 18-second cuda-secp256k1-bench
dispatch the panel stayed blank, then everything (probe + telemetry +
heavy result) appeared at once. Fox: 'demo waits for the full program
to return before emitting instead of line by line as it finishes'.
Each tier's print sink now ALSO calls a per-eval onChunk callback
that the worker forwards to the main thread as a {kind:"chunk"}
postMessage. The main thread spawns an in-progress tier-block on
the first chunk and appends each subsequent chunk to its <pre>,
auto-scrolling. On done/error the timing header finalizes in place;
the block looks identical to a non-streamed appendBlock at the end.
C tier (Emscripten): the Module.print + Module.printErr callbacks
that already fired per-line now invoke currentOnChunk in addition
to buffering. evalLisp accepts an onChunk arg and threads it.
Python tier (pyodide): _lumbda_eval swaps sys.stdout for a
_StreamingStdout subclass of io.StringIO whose write() also calls
into globalThis._lumbdaPyEmitChunk on the JS side. The final
auto-printed value also emits.
Asm tier (WAT): output stays buffered until evalLisp returns —
streaming there needs a new wasm import (out_line) and a wasm
rebuild. Caller-side fallback: app.js's appendBlock path still
runs for asm so its block appears at the end as before.
worker.mjs forwards each chunk via postMessage; app.js's
runOnTierInWorker handles 'chunk' messages by lazily creating
a startLiveBlock on first chunk, appending text on each, and
finalizing the timing in finalize() once done arrives.
Browser instantiation of lumbda-c.wasm failed with:
Aborted(LinkError: import object field 'js_lumbda_bend_call'
is not a Function)
The wasm correctly required env.js_lumbda_bend_call as a function
import, and the EM_JS-generated function existed in the glue —
emcc placed the function declaration at depth 0 of createLumbdaC
where wasmImports lives, so it SHOULD have been hoisted into scope
at instantiate time. It worked in node test runs but threw on
browser load. The exact emcc-version cause is fuzzy; the fix is
to use the textbook mechanism instead of guessing.
New file wasm/c/bend-call-library.js — Emscripten JS library with
mergeInto(LibraryManager.library, { js_lumbda_bend_call: function(...) }).
mergeInto lands the function directly inside wasmImports under the
mangled name _js_lumbda_bend_call (auto-prefix), wired to the
js_lumbda_bend_call import. No scope guessing.
lumbda_wasm_entry.c — drops EM_JS, keeps the C wrapper bi_bend_call_wasm
and declares extern int js_lumbda_bend_call(...) so emcc emits the
env import that the library fills.
wasm/Makefile — adds --js-library c/bend-call-library.js to C_LDFLAGS
and lists the library as a dependency so changes trigger a rebuild.
Verified: deployed glue at lumbda.com now contains the mergeInto
binding (js_lumbda_bend_call:_js_lumbda_bend_call inside wasmImports).
End-to-end smoke via mocked XHR in node returns the expected
(ok pong) round-trip from the demo's first probe step.
lumbda_wasm_entry.c — adds an EM_JS bridge js_lumbda_bend_call that
does sync XHR POST (legal inside Web Workers, the playground's tier
host) and writes the response bytes back into the wasm heap. C
wrapper bi_bend_call_wasm allocates a 256 KiB response buffer, calls
the EM_JS function, wraps the bytes as a lumbda string Value, and
gets registered as the `bend!-call` builtin in lumbda_wasm_init —
not in c/builtins.c, so the native CLI build doesn't acquire a
wasm-flavored binding it can't satisfy.
lumbda-c.loader.js — setBendUrl(url) mutates globalThis._lumbdaCBendUrl
which the EM_JS reads on every call. runner.js plumbing already
propagates a saved playground URL to every tier's setBendUrl on
each save.
C tier now matches asm + pyodide for HTTP-mode bend. End-to-end
verified in node + wasm directly: bend!-call returns the configured
URL response text; "no bend URL configured" when unset; clean
"bend error: ..." string when the XHR throws (e.g., CORS, network).
Tested against the live bend.unturf.com chain via the playground.
Three tiers in parallel now show the same (ok (HEX0 HEX1 HEX2))
result from (cuda-shake-fanout ("00" "01" "deadbeef") 32).
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Step 1 of the GC effort. Each tier loader now exposes heapStats():
- asm-wasm — lumbda_heap_used / lumbda_heap_total wat exports
- c-wasm — emscripten linear memory size (no free path right now,
so used = total; documented in the loader)
- python — pyodide module linear memory size; CPython GC cycles
this naturally
Worker handles a "heap" message kind that round-trips the active tab's
loaded tiers; repl tabbar shows a compact "py 12M · c 32M · asm 4M"
strip next to the buttons. Polls every 2s.
Doesn't solve the leak — just makes pressure visible so the user knows
when to use "reboot tier". Real GC (Cheney over the WAT bump allocator,
Boehm-em or custom mark-sweep for c-wasm) coming next.
User-visible changes
- Cancel button — terminates the running worker. Pyodide's slow mandelbrot
no longer freezes the UI; click cancel and the elapsed counter freezes
at "(cancelled @ NNNN ms)".
- Live ms counter ticks per animation frame while a tier is busy, so the
Pyodide tier's ~5-15 s wait is visible instead of looking hung.
- Restyled to match lumbda.com: chunkfive wordmark, --green: #227842
(light) / #5ec07a (dark), lumbda-logo-green.png, lowercase "lumbda"
everywhere. Pulls fonts/chunkfive locally so the playground stays
self-contained.
Architecture
- All tier evaluations now run inside a Web Worker (wasm/app/worker.mjs)
so the main thread stays responsive. Cancel = worker.terminate(); next
eval respawns a fresh worker.
- Loaders use new URL("./...", import.meta.url) so paths resolve against
the loader file's own location — works identically in window and
worker contexts, no baseURL argument needed.
- C tier Emscripten build flipped to EXPORT_ES6=1; loader uses dynamic
`import()` of the factory module. Integration test updated accordingly.
- Python loader uses `import("pyodide.mjs")` (ES module) instead of
document.createElement, which doesn't exist in workers.
Bug fixes
- Asm tier state leak: running the same demo twice on a cached WASM
instance produced corrupted output (every other cell on row 2+ rendered
as " " instead of the expected shade char). Root cause: top-level eval
passed `global_env` as the env, so closures captured stale globals;
fixed by passing NIL — env_lookup falls back to the CURRENT global_env
via its existing two-pass walk. Multi-run regression added to the
functional test suite.
- fib-ack demo: (ack 3 4) was too heavy for Pyodide (minutes). Cut to
(ack 3 3) + (fib 20) max so every tier finishes in seconds.
Test discipline
- Root `make test-all` now includes `wasm-test`. Adding a language
feature without exercising it on all six implementations is no longer
possible by accident.
- Functional suite: 11 assertions (was 10) — adds asm multi-run stability.
- Integration + unit: still 20 + 8.
Adds a parallel build of all three Lumbda implementations to WASM, a
single-page playground at www/playground/, and a verified test suite.
Tiers
- Python: Pyodide (CPython-in-WASM) hosting lumbda.py
- C: Emscripten build of c/ (tree-walker + bytecode VM; jit.c
stubbed, gc.c uses its existing no-Boehm fallback)
- Asm: hand-written asm/lumbda.wat — parallel impl to asm/lumbda.s.
Reader, eval (lambda/define/if/cond/let/and/or/quote/set!),
recursion across mutated top-level env, bump allocator with
memory.grow, 24 primitives. ~1200 lines of raw WAT.
SPA (wasm/app/, deployed to www/playground/)
- CodeMirror 6 editor (Scheme highlighting) on left, output on right
- Radios: 4 demos (Mandelbrot, Fib+Ack, Sieve, self-interp meta-eval)
x 4 tiers (Python | C | Asm | All three)
- All-three mode renders the three tier outputs side by side with
per-tier elapsed timing
Tests (38 verified assertions)
- 20 unit (Node): per-tier module loads, eval smoke
- 8 integration (Node): each demo on c+asm WASM byte-matches the
canonical native Python run
- 10 functional (Playwright headless Chromium): page mounts, every
demo runs on every tier, all-three renders
Makefile
- Root targets: wasm-build, wasm-test, wasm-test-fn, wasm-serve,
wasm-deploy, wasm-clean
- wasm/Makefile orchestrates the three tier builds; deploy copies
dist/ into www/playground/
Asm tier notes
- WAT linear symbol intern + linear env lookup is MOAD-0001 at scale;
documented in the asm/lumbda.wat header and in the SPA footer. The
demos hit ~30 globals so the linear walks are cheap enough.
- Bump allocator never frees (matches asm/lumbda.s heap discipline);
memory.grow expands by 1 MB chunks. Browser tab tears down at unload.
Toolchain (developer prerequisites)
- Emscripten 6.0.0 via emsdk at ~/git/emsdk
- wabt 1.0.36 at ~/git/wabt
- Playwright for functional tests (symlinked from ~/git/agnt)
