Worker batching used to wait for 64KB / 4096 newlines before
flushing. A program like
(let loop ((n 0))
(when (zero? (modulo n 10000))
(display "Emitted at iteration: ") (display n) (newline))
(loop (+ n 1)))
emits ~25 bytes every 10K iterations, so the user saw nothing for
several seconds — fox reported "this used to work and doesn't seem
to anymore" because the first emission was buried in the batch
buffer.
Added a 100ms wall-clock check alongside the existing size /
newline thresholds. The first chunk to arrive triggers a flush
(lastFlushTime starts at 0), and low-rate streams cap at ~10
flushes/sec. High-rate streams still hit the byte / newline ceilings
first so chunk batching for tight (display) loops is unaffected.
Verified headlessly: the infinite-emitter program now shows the
first emission within a frame, subsequent ones streaming live as
the loop iterates.
Three converging bugs let a cancel-then-send sequence land in a
broken state:
1) cancelAllPendingInActiveTab didn't touch the toolbar — the user
had to wait for the cancelled promise's .catch.then to drain
before send re-enabled. fox saw clicks register as noops because
the button was still disabled.
2) Even after the rejection settled, the stale .then path
re-enabled sendBtn from inside the FIRST run's closure — but by
then the user had already submitted a SECOND run that set
sendBtn disabled. The stale .then clobbered the in-flight state.
3) The OLD worker's last-queued "done" or "error" message could be
delivered after a fresh run had already taken state.pending[k];
the OLD handler would then delete the NEW run's pending entry.
Fixes:
* cancel handler now re-enables send / disables cancel + pause
synchronously and clears tab.activeInput.
* The completion .then re-checks isEvalInFlight(tab) before resetting
toolbar state, so a stale settle from a cancelled run can't
override the fresh run's UI.
* evalInTier's done/error handler only deletes state.pending[k] when
the entry still has the runId we started with.
Verified headlessly: long loop → cancel → submit (+ 1 2) immediately
→ first entry shows error: cancelled, second entry shows result 3,
send re-enabled.
Creating one <div> per streamed line ate the main-thread budget on
high-volume display loops — a 500K-line program took ~12s of DOM
mutation before the user's tab-switch click could even register.
fox's reported "tab keeps looping when I leave it" was the click
sitting in the task queue behind that backlog.
Now the live tier-result holds a single Text node and we append to
its nodeValue. The parent already has white-space: pre so embedded
\n characters render as actual line breaks without any per-line
elements. ~1000x faster on the high-volume path; identical visually
for normal output.
Error path still uses a styled <div> for the trailing 'error: …'
line so just the error is tinted red; the streamed text node keeps
the original colour. finalize's success-with-streaming check now
reads textNode.nodeValue instead of iterating children since there
aren't any anymore.
Playground (wasm/app/app.js) still uses per-line divs — those
follow in a separate change once this proves out in the REPL.
A tight (display ...) loop used to fire one postMessage per display
call — main thread couldn't even register click events because the
message queue grew faster than it could drain. Fox saw this as
"the tab just keeps looping when I leave it" — the tab-switch click
never reached setActiveTab so autoPauseTab never fired, and the
worker kept running until it finished on its own.
Now the worker accumulates chunks into a local string buffer and
postMessages a single chunk-batch message when the buffer hits 64KB
or 4096 newlines. The 'done'/'error' path drains whatever's left
before signalling so the last lines still reach the UI. Main thread
handlers (repl + playground) split the batch back into the same
text+eol sequence the live streaming row expects.
Adds RAF coalescing on the receive side too: attachStreaming now
batches DOM textContent / appendChild updates into one
requestAnimationFrame tick so the 60 Hz repaint budget is shared
across all chunks that landed in that window. finalize() drains
the RAF buffer synchronously before the streamed-vs-expected
match check so error-on-cancel keeps the most recent lines.
If the C/Python eval was stuck in code without a poll site (sync XHR
inside bend!-call, a builtin that doesn't pass through leval), the
SAB pause flag would set, waitForPending would time out after 5s,
and then portal-snapshot would queue behind the still-busy worker
and never resolve. autoPauseTab hung on that await and never
reached its terminate loop — leaving the worker running while the
user sat on a new tab watching the old loop keep streaming.
Now:
* SAB pause wait drops from 5s to 2s.
* If the eval is still in flight after the pause wait, skip the
snapshot attempt entirely — the worker isn't yielding and the
snapshot would just hang the UI.
* Snapshot eval + portal-save bridge are wrapped in withTimeout
(2s + 1.5s) so even if pause partially worked but snapshot
stalled, we fall through to the unconditional worker.terminate
at the bottom of autoPauseTab.
* The hard-cancel branch and the SAB-timeout-fallback now share the
same pending-rejection loop instead of duplicating; the
unconditional terminate loop at the end kills the worker.
pauseAndSnapshot (manual ⏸ button) also surfaces the timeout case
with an alert pointing at cancel + reboot instead of locking the UI.
newTab() used to mutate state.activeTabId directly, skipping
setActiveTab's auto-pause hook AND the toolbar button-state sync.
Opening a new tab during a running eval left the send button
disabled on the brand-new (idle) tab — fox would land on a tab he
couldn't type into.
Now newTab pushes the tab onto state.tabs, then awaits setActiveTab
which handles both the outgoing-tab auto-pause (if applicable) and
the incoming-tab button sync.
Verified end-to-end headless: with COOP/COEP enabled, switching
from a c-tier eval to a new tab now correctly snapshots the source
tab's state, leaves the send button enabled on tab 2, and on
switch-back the resumed eval re-fires from the snapshot.
Rapid tab clicks could fire setActiveTab a second time before the
previous autoPause + autoResume cycle finished, so two concurrent
flows would race on the same outgoing tab — second one snapshotting
torn state. state.switching short-circuits re-entry; the user clicks
again once the active tab settles.
Adds a tabbar button that signals the SAB pause atomic, waits for
the eval to unwind via lisp_error/LispErr("paused"), then drops
into saveCheckpoint so the resulting snapshot lands in the visible
chip strip with a user-chosen name. Workers stay alive so the next
input runs immediately — this is the "I want to inspect/save state
but keep working" path, distinct from autoPauseTab's
"I'm leaving this tab" flow.
The button enables alongside cancel whenever an eval is running and
SAB is available; on plain http.server (no COOP/COEP) it stays
disabled and an alert points at make serve-repl. Asm tier gets a
"not supported on asm tier yet" alert since the WAT interpreter
has no in-eval poll site.
Tabbar grid expanded from 8 to 9 columns; comment updated to track.
Asm tier has no in-eval poll site (the WAT interpreter doesn't read
the SAB atomic), so SAB-pause + portal-snapshot don't apply. Until
the asm interpreter grows its own poll, fall back to the same
replay pattern the manual asm portal-save already uses: stash every
successful prior input from the transcript on tab.autoPause.replayInputs.
autoResumeTab now always reboots the tier first, then:
* replayInputs set — re-eval each in order so env rebuilds,
log a "; resumed asm tier — replayed N prior inputs" entry, then
sendInput re-fires the active input.
* blob set (c / python) — round-trip through portal-load! as before.
* neither (no SAB, lost) — bare sendInput, env starts cold.
Now switching away from a tab mid-asm-eval and switching back
re-establishes every (define …) the user had before, then re-fires
the loop. Same UX shape as the C/Python path.
leval() in lumbda.py grows a counter-gated check (every 1024th
iteration) that calls a module-level _lumbda_pause_hook. Native
Python users leave the hook None and the check short-circuits to a
single bitwise AND. The pyodide loader installs a hook that reads
_lumbdaPyPauseRequested (a JS callback over Atomics.load on the
SAB) so the REPL's auto-pause-on-tab-switch flow drops into the
same path on python that it already uses on C.
Also adds setPauseFlag to the python tier's returned object so
runner.setPauseFlag propagates the SAB through worker config.
repl.js's autoPauseTab no longer falls back to hard-cancel when
the active tier is python — the SAB-poll path covers both. Asm
remains on hard-cancel since the WAT tier has no in-eval poll
site yet.
Test suite (tests.py, 571 tests) passes — verified the no-op
hook path doesn't change native eval semantics.
Tab switching during a long-running eval used to silently abandon
the calc — output stopped streaming, no snapshot, nothing to come
back to. Now setActiveTab pauses the outgoing tab's eval (and
optionally portal-saves the env), terminates the worker, and on
re-entry hydrates + re-fires the original input.
Pieces:
* serve-coop.py + make serve-repl — dev server that emits
Cross-Origin-Opener-Policy: same-origin and
Cross-Origin-Embedder-Policy: require-corp so SharedArrayBuffer
is constructable in the browser. Same headers production needs.
* C tier eval-loop pause poll — c/eval.c grows lumbda_check_pause(),
guarded by #ifdef LUMBDA_WASM. Called at the top of leval()'s
while(1); masked to every 1024th iteration so the polling cost
stays under noise floor. When the JS-library import
js_lumbda_pause_requested returns 1, lisp_error("paused")
longjmps out so module-global env survives intact for the
portal-snapshot that follows.
* SAB plumbing — main thread allocates new SharedArrayBuffer(4),
hands it through worker config → runner.setPauseFlag →
lumbda-c.loader.setPauseFlag → globalThis._lumbdaCPauseFlag.
Atomics.store / Atomics.load on index 0 is the signalling
channel. Falls back to null when COOP/COEP isn't isolated, in
which case pause degrades to a hard worker.terminate().
* autoPauseTab() — on setActiveTab away, snapshots the tier
(C tier with SAB) or hard-cancels (other tiers / no SAB),
stashes tab.autoPause = {tier, blob, inputSrc, savedAt},
terminates the workers so the heap is reclaimed.
* autoResumeTab() — on setActiveTab into a tab with autoPause,
reboots the tier, hydrates MEMFS, runs (portal-load! ...), then
re-fires the original input via sendInput so the eval restarts
from the saved state. Asm + Python paths re-run from scratch
until their poll sites land.
Also closes two UX papercuts from fox: chip ⇣ export icon bumped
from 0.85em muted to 1em green so it's actually discoverable; the
scope toggle now reads "scope: this tab" / "scope: all tabs" so the
button label describes the state rather than a target.
attachStreaming.finalize() used to nuke every streamed line and
rebuild from the eval's final text whenever the streamed buffer
didn't match — which always happened on cancel (no final text)
and on tier errors fired mid-stream (asm 'index out of bounds'
5000 lines into a sieve, say). The user watched output scroll for
30 seconds, hit cancel, and saw it all disappear replaced by
'error: cancelled'.
New finalize logic:
* success + streamed matches full output → keep DOM
* success + no streaming happened → rebuild from text
* error + prior streaming → KEEP streamed divs,
append the error as a single trailing line
* error + no streaming → show error only
Trailing error line gets an .err-line class so just the error is
tinted with --err, the preserved output stays the original colour.
Three UX gaps on the portal-bar closed in one pass plus a defensive
fix on the C tier's heap probe:
* Overwrite guard — saving with an existing name asks 'overwrite?'
with the existing entry's tier + savedAt. Rename via dbl-click on
the chip label; same overwrite guard applies on rename.
* Export / import — a ⇣ icon on each chip downloads it as
<name>.portal.json (opaque blob for c/python, replay-inputs for
asm). A 📁 import button on the portal-bar accepts a .portal.json
file via hidden <input type="file">; collisions prompt overwrite,
decline auto-suffixes (baseName-2, -3, …) so importing a 2nd copy
always lands somewhere.
* Cross-tab restore — a 📂 this tab / 🌐 all tabs toggle switches
the chip strip between the active tab's checkpoints and every
tab's. Global chips render as 'name · tabName' with a dashed
border; click restores the snapshot into the active tab (the
saved cp is passed through restoreCheckpoint's new sourceCp
argument so the chip doesn't need a checkpoints[name] match on
the active tab). Edit/delete are hidden in global mode — the
user switches to the owning tab to manage chips.
* heapStats defensive — wasm/c/lumbda-c.loader.js now returns null
when module.HEAPU8 isn't live yet (caught by the heap poll firing
during the tiny window between tier reboot and Module init), so a
restore no longer surfaces 'Cannot read properties of undefined
(reading byteLength)' as a TypeError.
Smoke-tested headlessly: overwrite confirm fires with the expected
message; rename via dblclick swaps the label; ⇣ produces a download
named '<name>.portal.json'; toggle shows both tabs' chips with the
'· tabName' annotation; import round-trips back into the receiving
tab. Zero page errors across the full flow.
Streaming output now follows the bottom of the page when the user is
already at the bottom, and stops doing so the moment they scroll up.
Scrolling back down re-engages autoscroll. A 64px tolerance covers
sub-pixel scroll positions and the sticky prompt-bar's offset.
Mechanics:
* Module-level scrollPinned flag, recomputed on every window scroll
event (passive).
* maybeAutoscroll() jumps to the bottom only when pinned. Called
from attachStreaming's appendText / appendNewline so every
chunk-text / chunk-eol arriving from the worker tracks.
* renderAll still does an unconditional jump (it fires on
user-initiated actions — send, tab switch, etc. — where the
most-recent content is what they want) and re-pins scrollPinned
afterward so the streaming follow-ups continue to track.
Smoke-tested headlessly: streaming 30+ lines while at bottom keeps
viewport at bottom; scrolling to top during streaming leaves scrollY=0
even as the doc grows to 2KB tall.
Previously align-items: center centered the textarea while the sigil
sat top-aligned with padding-top — single-line input rendered the
sigil visibly higher than the input baseline (see screenshots from
07-52-55 and 07-53-20). Now:
* .prompt-bar uses align-items: baseline so the sigil's text
baseline tracks the textarea's first-line baseline in both 1-line
and N-line cases.
* .prompt-sigil shrinks font-size from 0.95em to 0.92em (matching
the textarea) so cap-height differences don't push baselines
apart.
* line-height: 1.4 pinned on both so baseline geometry stays
predictable (browser default for textarea is 'normal' which
varies per-font).
Adds an inputDraft field to each tab and writes the textarea contents
through saveSoon on every input event (paste, type, autocomplete).
renderAll restores the draft into the textarea when a tab becomes
active, so refresh-after-half-typing or switching between tabs and
back lands the user back on what they were composing.
Plumbing:
* newTab() seeds inputDraft: ""
* setActiveTab() snapshots the outgoing tab's textarea value before
swapping (covers the gap between last input event and next
saveSoon flush)
* renderAll() writes the active tab's draft back into the textarea
(+ autosizeInput so a 40-line recovered draft expands to fit)
* sendInput() clears tab.inputDraft alongside inputEl.value so the
vault next snapshot doesn't preserve an already-committed entry
Smoke-tested: type a 5-line draft, fresh browser context with same
vault password reopens to the same 5 lines + same 82px height. Per-tab
isolation verified — switching tabs preserves each tab's own draft.
Send empties the draft; subsequent reload shows empty input.
Until now the input was pinned at rows="1" — paste a 40-line program
and you edited it through a one-line keyhole. autosizeInput() now sets
height to scrollHeight on every input event, history navigation, and
post-send clear. CSS min/max-height (1.5em / 30vh) clamp the bounds;
once the textarea hits 30vh it scrolls internally instead of pushing
the prompt-bar off screen.
Prompt sigil top-aligns (align-self: start + matching padding-top) so
the λ> stays on the first line of input instead of drifting to
vertical-center as the textarea grows.
Smoke-tested headless: 1-line stays 22px, 6-line grows to 97px, 40-line
caps at 216px (30vh of 720px) with scrollHeight 608 (internal scroll
engaged), send empties back to 22px. Zero page errors.
Adds a portal-bar to the REPL between tabbar and transcript: a save
button + chip strip showing all saved checkpoints for the active tab.
Click a chip to restore, click × to delete.
Per-tier strategy:
* c, python — call the tier's (portal-snapshot! NAME), then read the
JSON blob out of MEMFS (Emscripten/Pyodide FS) and stash it in the
encrypted vault entry. Restore reverses: hydrate MEMFS, then
(portal-load! NAME) merges the bindings into the live env.
* asm — no portal serializer in the WAT tier yet (would need a
Cheney-aware walk). Falls back to transcript replay: save snapshots
every successful prior input, restore reboots the tier and re-evals
them in order.
Plumbing:
* Worker bridge: new portal-save / portal-load message kinds wire
MEMFS reads/writes to the main thread.
* runner.js exposes portalSave / portalLoad — null when a tier
hasn't implemented portals (asm stays grey).
* C tier: replace EM_JS with extern + --js-library for js_lumbda_bend_call
(EM_JS-generated declaration was unreachable from wasmImports at
instantiate time, browsers threw "import object field ... not a
Function"). FS added to EXPORTED_RUNTIME_METHODS so JS can reach
pyodide.FS / Module.FS for MEMFS I/O.
Smoke-tested all three tiers headlessly: save → chip render → restore
round-trips clean on c / python / asm, zero console errors.
evalInTier picks up onChunkText + onChunkEol callback parameters
and routes the new typed-event chunks from worker.mjs (already
posting chunk-text + chunk-eol per 25d2765 / 89d4877) to per-tier
streaming sinks. The REPL's own worker.mjs was already the same
build as the playground's (md5-matched) so no worker changes.
sendInput pre-populates entry.results with one placeholder per tier
(output:"", streaming:true), runs the initial renderAll, then walks
the freshly-rendered DOM rows and attaches streaming refs (one per
tier) directly to each tier-result span. Chunks land in per-line
<div display:block> children inside that span — same shape the
playground uses. metaSpan flips to "<tier> · running…" until done.
On each tier's promise resolve, the placeholder gets mutated in
place (NOT push'd a second time) and the corresponding liveRow's
finalize() reconciles the streamed divs against the full output
(rebuilds from canonical text only if they diverge or there was
an error) and stamps the elapsed-ms meta. renderAll is NOT called
between tier completions any more — that was clobbering sibling
tiers still streaming in 'all three' mode.
Net effect: paste (let loop ((i 0)) (display i) (newline) (loop
(+ i 1))) in the REPL, hit run on 'all three', and you see each
tier's count tick up in its own row in real time — not a frozen
panel followed by a wall of output at the end.
A throttled forever-counter on the asm tier ran for ~5.7 s and
then trapped 'index out of bounds' — the WAT writes output to a
fixed 64 KB region at 0x10000–0x1FFFF, and a fast (display X)
(newline) loop accumulates faster than the buffer can drain. After
about 6,400 ticks at ~10 chars each, overflowed the
region into the source buffer at 0x20000 and the next i32.store8
fell off linear memory.
Fix is a contract change on emit_chunk: its signature picks up an
i32 return — 1 tells the WAT to recycle (zero output_len AND
flush_start), 0 keeps the original 'just advance flush_start'
semantics so callers that read lumbda_output_ptr/len after eval
still see the full buffer.
The asm loader returns 1 from emit_chunk, accumulating every
flushed slice into refs.accumulated. evalLisp's return value is now
refs.accumulated + the trailing (still-unflushed) buffer slice
rather than just the lumbda_output_ptr/len slice — caller still
gets the complete output, the WAT-side buffer just keeps recycling.
Test stubs already declared emit_chunk() {} which returns
undefined — JS->wasm i32 coercion turns that into 0, preserving
the no-recycle behavior they expected. unit/integration suites
(31 tests total) still pass.
Trailing flush in lumbda_eval also honors the return value: if the
host consumed, zero both offsets so the final lumbda_output_ptr/len
read returns 0 bytes (loader already accumulated the trailing
slice — no need to re-deliver). Earlier draft of this patch
double-emitted the trailing slice because we read it through both
the emit_chunk path and the final-buffer path.
Two fixes in one push.
(1) Streaming horizontal-output bug, take three. textContent += chunk
then appendChild(createTextNode(chunk)) both still rendered chunks
horizontally in fox's Firefox tab even though the chunks clearly
contained \\n (node test confirmed; deployed loader hash matched
local; curl-fetched lumbda-c.loader.js carried the line + '\\n' fix).
Whatever the browser was doing with sibling text nodes inside a
<pre> wasn't honoring the embedded newlines.
Switch to one <div class=\"stream-line\"> per logical line. liveBlock.append
walks the incoming chunk byte by byte, every \\n closes the current
pending div and spawns a fresh empty one for the next line. CSS
adds .stream-line { display: block; white-space: pre; } so each
finished line stacks vertically no matter what the parent
white-space rule was doing. Empty lines get a single space so they
take a row instead of collapsing. Reconcile path inside finalize()
compares the joined per-line text to the full output and rebuilds
the div column if they diverge — for the asm-tier fallback we
already had and now also for any future browser/wasm combo where
a flush silently drops a chunk.
Also drops the temporary [c-tier print] console.log debug we added
in the last commit — diagnosis arrived from elsewhere, no point
keeping the spam.
(2) Per-program autosave drafts. scheduleFreeFormSave previously
returned early if the selected program wasn't \"free-form\", so a
user who unlocked the vault, edited the bend-gpu demo, and came
back later found their edits gone — only free-form persisted.
Now drafts live as { [programName]: text } in the vault payload;
every edit, regardless of which radio is selected, debounces a
save into drafts[currentProgram]. unlockVault loads any saved
draft for the current program (and lifts the legacy
top-level freeForm field into drafts['free-form'] so existing
users don't lose their work). loadCurrentDemo shows a saved
draft instead of the ship default whenever one exists; vault bar
stays visible across all programs once the vault is engaged so
the save status note is always reachable.
The deployed loader emits chunks with currentOnChunk(line + '\n'),
which my node tests confirm produces clean per-line chunks
('tick 0\n', 'tick 1\n', ...). But fox still reports horizontal
output in his playground tab — same tier, same loop, same loader
hash on the wire (verified via curl). Browser caches and module
loaders are aggressive enough that hard-refresh isn't always
enough.
Adding a temporary console.log inside Module.print so when fox
opens DevTools and re-runs the throttled loop he can see (a) is
the print callback firing at all? and (b) what does Emscripten
hand us — is it actually 'tick 0' or something stranger like
'tick 0\n' or 'tick' + ' 0' as two calls?
Will revert this once the bug is resolved.
EOF
)
Emscripten's Module.print fires once per line and hands the loader
just the line content — no trailing newline; the caller is expected
to add it back when joining (outBuf.join('\n') does that for the
final return). Streaming path was forwarding the line straight to
onChunk without the newline, so successive chunks concatenated
horizontally in the playground panel — the loop
(let loop ((i 0))
(cond ((= (modulo i 10000) 0) (display \"tick \") (display i) (newline)))
(loop (+ i 1)))
was running fine on c tier (each (newline) ended a printf line) but
arriving in the UI as 'tick 0tick 10000tick 20000...' with no
breaks.
Loader's print/printErr callbacks now pass `line + \"\\n\"` to
currentOnChunk. Final buffered join (which already adds the \\n)
is untouched, so the post-eval output string keeps the same shape.
Pyodide _StreamingStdout sees Python's raw write() bytes including
the \\n already; asm's emit_chunk fires AFTER output_len was
incremented past the \\n, so the slice [flush_start, output_len)
includes it. Neither needed a change.
Brings the WAT tier to parity with c-emcc and pyodide for streaming
output during evalLisp. Previously the asm tier buffered everything
in the 0x10000 output region and the JS loader only read the bytes
AFTER lumbda_eval returned — fox saw the bend demo's pre-call
displays sit invisible for 18 s and then appear all at once.
WAT-side changes:
- New env.emit_chunk(ptr, len) import. Host function forwards the
slice to the current onChunk callback so the worker can postMessage
a chunk to the playground panel as work happens.
- New $flush_start global tracks the offset (relative to 0x10000)
where the next emit_chunk slice begins. Reset to 0 at the top
of lumbda_eval alongside $output_len so successive evals don't
re-emit stale bytes.
- $out_char now checks for newline (i32.const 10) after the store.
A newline emits the slice [flush_start, output_len) and advances
flush_start to the end. Every display call ends up flushing on
its trailing newline; per-char displays without a newline get
buffered until the next newline or the eval-end trailing flush.
- $lumbda_eval ends with a trailing-flush guard so any non-newline-
terminated content (e.g. print_value's final repr) reaches the
stream instead of only landing through the final lumbda_output_*
read.
JS loader:
- importObj.env.emit_chunk decodes the slice from wasm memory and
forwards to refs.currentOnChunk.
- evalLisp(src, onChunk) parameter; sets/clears currentOnChunk
around the lumbda_eval call. Same shape as c-emcc + pyodide.
Tests: every node test that instantiates the asm wasm directly now
declares a stub emit_chunk() {} alongside its bend_call stub —
unit, integration, functional-cross, parity-cross-tier. Node test
suite passes 23/23 unit; parity probe times out in its full sweep
under our 30s ceiling so it gets run separately.
Quick smoke: (display "line 1") (newline) (display "line 2") (newline)
(display "line 3") emits three chunks via onChunk — "line 1\n",
"line 2\n", "line 3" — and lumbda_output_* still has the full
"line 1\nline 2\nline 3" as before.
Browser instantiation of lumbda-c.wasm failed with:
Aborted(LinkError: import object field 'js_lumbda_bend_call'
is not a Function)
The wasm correctly required env.js_lumbda_bend_call as a function
import, and the EM_JS-generated function existed in the glue —
emcc placed the function declaration at depth 0 of createLumbdaC
where wasmImports lives, so it SHOULD have been hoisted into scope
at instantiate time. It worked in node test runs but threw on
browser load. The exact emcc-version cause is fuzzy; the fix is
to use the textbook mechanism instead of guessing.
New file wasm/c/bend-call-library.js — Emscripten JS library with
mergeInto(LibraryManager.library, { js_lumbda_bend_call: function(...) }).
mergeInto lands the function directly inside wasmImports under the
mangled name _js_lumbda_bend_call (auto-prefix), wired to the
js_lumbda_bend_call import. No scope guessing.
lumbda_wasm_entry.c — drops EM_JS, keeps the C wrapper bi_bend_call_wasm
and declares extern int js_lumbda_bend_call(...) so emcc emits the
env import that the library fills.
wasm/Makefile — adds --js-library c/bend-call-library.js to C_LDFLAGS
and lists the library as a dependency so changes trigger a rebuild.
Verified: deployed glue at lumbda.com now contains the mergeInto
binding (js_lumbda_bend_call:_js_lumbda_bend_call inside wasmImports).
End-to-end smoke via mocked XHR in node returns the expected
(ok pong) round-trip from the demo's first probe step.
lumbda_wasm_entry.c — adds an EM_JS bridge js_lumbda_bend_call that
does sync XHR POST (legal inside Web Workers, the playground's tier
host) and writes the response bytes back into the wasm heap. C
wrapper bi_bend_call_wasm allocates a 256 KiB response buffer, calls
the EM_JS function, wraps the bytes as a lumbda string Value, and
gets registered as the `bend!-call` builtin in lumbda_wasm_init —
not in c/builtins.c, so the native CLI build doesn't acquire a
wasm-flavored binding it can't satisfy.
lumbda-c.loader.js — setBendUrl(url) mutates globalThis._lumbdaCBendUrl
which the EM_JS reads on every call. runner.js plumbing already
propagates a saved playground URL to every tier's setBendUrl on
each save.
C tier now matches asm + pyodide for HTTP-mode bend. End-to-end
verified in node + wasm directly: bend!-call returns the configured
URL response text; "no bend URL configured" when unset; clean
"bend error: ..." string when the XHR throws (e.g., CORS, network).
Tested against the live bend.unturf.com chain via the playground.
Three tiers in parallel now show the same (ok (HEX0 HEX1 HEX2))
result from (cuda-shake-fanout ("00" "01" "deadbeef") 32).
The WAT $read_string function scanned bytes until the first " and copied
raw, with NO escape handling. So a source string like
"(\"00\" \"01\")" got chopped at the first \" — the asm tier read only
"(\\" before terminating, producing a mangled payload that the worker
couldn't parse. The bend-gpu demo's payload built via string-append of
escaped-quote strings came out as "(cuda-shake-fanout (\ \ \) 32)" in
the asm tier, sent garbage to bend, and got nothing useful back.
Two-pass fix to match Python/C tier behavior:
1. Scan-and-count pass: walks source-ptr to the closing ", but
when it sees \ it skips the next byte so embedded \" doesn't
terminate the string. Counts decoded output bytes (each \X
contributes one byte, not two).
2. Allocate + copy-and-decode pass: walks the same range, converts
\n → 0x0A, \t → 0x09, and any other \X (including \" and \\)
→ X. Matches the lenient fallback the desktop tiers use.
Verified via cross-tier parity probe — 255/255 still passing.
Demo payload now constructs as
"(cuda-shake-fanout (\"00\" \"01\" \"deadbeef\") 32)" (45 bytes,
identical to Python/C reads) and the asm playground returns
(ok (HEX0 HEX1 HEX2)) from the live 3090-ai bend worker.
lumbda-py.js (and the three deployed mirrors) now expose setBendUrl
and register a bend!-call primitive in the pyodide-hosted lumbda
environment.
Implementation:
- JS loader stashes a `globalThis._lumbdaPyBendCall` function that
does sync XHR POST to the configured bend URL (legal in Web
Workers, where the pyodide tier runs in this playground)
- Python bootstrap imports `_lumbdaPyBendCall` from `js` and binds
it as a builtin under the symbol `bend!-call`, accepting any
value and stringifying via lumbda.show before sending
- setBendUrl(url) on the tier object updates the JS closure; the
runner.js plumbing already calls it on every tier when the user
saves a bend URL in the ⚡ bar
This brings the pyodide tier to parity with the asm (WAT) tier for
HTTP-mode bend. The emcc C tier still lacks the bind — wiring it
needs a new wasm primitive built via emcc; lands in the next commit.
Tested: bend!-call "(ping)" against the same gpu-worker endpoint
returns the same (ok pong) S-expression the asm tier sees.
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Step 1 of the GC effort. Each tier loader now exposes heapStats():
- asm-wasm — lumbda_heap_used / lumbda_heap_total wat exports
- c-wasm — emscripten linear memory size (no free path right now,
so used = total; documented in the loader)
- python — pyodide module linear memory size; CPython GC cycles
this naturally
Worker handles a "heap" message kind that round-trips the active tab's
loaded tiers; repl tabbar shows a compact "py 12M · c 32M · asm 4M"
strip next to the buttons. Polls every 2s.
Doesn't solve the leak — just makes pressure visible so the user knows
when to use "reboot tier". Real GC (Cheney over the WAT bump allocator,
Boehm-em or custom mark-sweep for c-wasm) coming next.
Variable-length signed bignums on the asm-wasm tier. Layout:
[tag=10, sign:i32, n_limbs:i32, limbs[]:u32]
Little-endian u32 limbs (base 2^32). i64 used for limb-pair products
in bn_mul and for the (rem << 32) | limb shift in bn_divmod_small.
Promotion: num_add/sub/mul/cmp inspect operands and pick the right
representation (fixnum, rational, bignum). Fixnum overflow in +/-/* is
detected by computing in i64 and checking against the 30-bit fixnum
range — outside that, operands lift to bignums.
(expt 2 1024) uses exponentiation-by-squaring through num_mul so
intermediate products auto-promote, returning the exact 309-digit value.
Reader: digit parsing accumulates via num_add/num_mul, so a literal of
any length reads as the narrowest representation that holds it.
Parity corpus: KNOWN_DIVERGE is now empty. 237/237 passing across
python (ref), c-wasm, and asm-wasm. New asserts pin the bignum surface
so a regression breaks make wasm-test immediately.
New: wasm/tests/parity-cross-tier.mjs runs the parity-corpus.mjs (216
test cases tagged by whitepaper section / R7RS concept) against three
tiers — native python (reference), c-wasm, asm-wasm — and fails on any
unknown divergence. Known gaps live in KNOWN_DIVERGE so the table stays
green while the bignum / call/cc / etc. work proceeds.
Wired into `make wasm-test` so a regression against any spec claim gets
caught before merge.
Bugs caught and fixed:
- python remainder: was `signed_a % signed_b * sign(a)`, which double-
applied the sign of a (python's % floors) — gave -3 for (-17, 5)
instead of the R7RS-correct -2. Now uses abs() on both sides.
- asm-wasm modulo: was i32.rem_s (truncated, remainder semantics)
where R7RS modulo wants sign of divisor. Added the "if rem and
divisor disagree on sign, add divisor" branch.
Cross-tier numbers after fix:
216 passing
3 known diverge: expt-2-100, expt-3-50, big-arith — all asm-wasm
(no bignums on the asm tier yet; whitepaper §2.1 claim still open)
0 fail
REPL layout: body is now the scroll container, prompt-bar is
position:fixed at the viewport bottom so it doesn't get pushed off
screen by a long transcript. Empty space above the prompt on a fresh
session reads like a terminal.
All other tests still pass: 20 unit, 8 integration, 11 functional.
Adds tag-9 rational type to the asm tier. Layout [tag=9, num:i32, den:i32].
make_rational normalizes via gcd and collapses to a fixnum when den
reduces to 1, so 14/2 stays as 7.
Arithmetic (+, -, *, /, =, <, >, <=, >=) now promotes to rational when
any argument is rational. Mixed fixnum/rational lifts the fixnum
accumulator into a rational mid-loop so (+ 1 1/2) returns 3/2, not 1/2.
Reader parses "67/7" literals via the existing atom path: after the
numerator's digits, if '/' follows we keep reading the denominator and
hand back a normalized rational. Falls through to symbol if either side
isn't all digits.
Printer renders rationals as "n/d". equal_p compares numbers by value
(1/2 = 2/4, 3 = 6/2). is_number / number? cover both fixnums and
rationals.
eval now treats rationals as self-evaluating — without this, '1/3'
parsed correctly but evaluated to VOID.
Mandelbrot demo: switched from (/ a b) to (quotient a b) for the
fixed-point math. The demo had been relying on integer truncation
that '/' no longer provides on tiers with R7RS-correct rationals.
Bignums still pending: 31-bit num/den overflows with huge denominators.
Real lift comes with the bignum task in the C tier (which has them) or
a new bignum module in the WAT.
Cross-tier check still hangs on the bigger TCO-heavy sections of
functional.lsp — separate from rationals. Will keep grinding.
Tests: unit 20/20, integration 8/8, functional 11/11.
num_div for two integers used to fall back to double when the
quotient wasn't exact. R7RS / python lumbda require exact-in →
exact-out for /. Fixed: the rational_normalize path was already
wired for the is_exact branch; the int/int branch now calls it
too instead of make_double.
(/ 67 7) → 67/7 (was 9.5714285714285712)
(/ 1 3) → 1/3 (was 0.33333…)
(/ 6 2) → 3 (exact stays integer)
(+ 1/3 1/6) → 1/2 (rational arithmetic propagates)
C native + C-WASM tier now match python lumbda on / between integers.
asm tier rationals remain pending — that needs bignums in asm first.
native c-test: 205/205 still passes.
WASM tail-call instruction wired into every tail position:
if branches, eval_begin last expression, all special-form dispatchers
(cond/when/unless/case/let/let*/letrec/begin/and/or), function
application's $apply, and apply's closure-body $eval_begin.
wat2wasm + wasm-validate now use --enable-tail-call.
Reader fixes:
- Dotted pair syntax: (a b . rest) parses as a real dotted list.
Without this, variadic params and other dotted-cdr forms parsed
as 4-element proper lists.
- Stray ) at top level advances source_ptr instead of spinning
forever. Found via bisection of functional.lsp under TCO.
- eval_args guards against non-pair tails so a misplaced dotted
argument (e.g. an unexpanded macro template) can't deref garbage.
Cross-tier numbers (wasm-test-functional-cross):
before: 93/111 reached, stack overflow on (ack 3 4)
now: progresses through the full TCO section, deep let, named-let
to 100k, mutual recursion to 200k. Still climbing.
Bend gpu demo:
6th playground option ("bend (gpu dispatch) ⚡") ships a SHAKE256
fan-out at 1M inputs via (bend!-call ...). Run button is GUARDED:
if no bend URL is configured, refuses with
"set a bend URL first — this demo is GPU-only by design".
Protects customer machines from burning minutes on a workload the
local tiers cannot finish in reasonable time.
Header (logo + tagline) is the only fixed region. Footer removed.
Everything else — tab bar, transcript, prompt — now lives inside one
.repl-stream scroller. A fresh session shows the prompt right under
the tabs near the top; as entries arrive the prompt drifts down with
them. Tabs use a dashed bottom rule instead of a heavy bar so they
read as the start of the stream rather than a separate chrome strip.
Reads as one continuous stream now. Each entry is just:
λ> <input>
<output> ; tier · NNms
No left border, no boxed cards, no side-column tier label. Output
indents under the prompt (3ch) using monospace ch units. Tier+time
render as a Lisp-comment-style suffix in muted color.
Prompt bar: borderless textarea on the code-bg surface so the input
visually joins the transcript above. Placeholder cut to "(+ 1 2)" —
the surrounding text already explains the semantics.
Multi-line inputs keep prompt continuation marks ("..").
- bend URL and vault password now share a single config-bar row, split
via a 2-column grid (1fr 1fr). Vault still hidden when free-form
isn't selected; just collapses its column.
- Every flexbox removed. Every multi-child container uses CSS grid:
.controls, .config-bar, .bend-bar, .vault-bar, .panes, .pane,
.brand, .tabbar, .tabs, .tab, .transcript, .entry, .tier-output,
.prompt-bar, .lock-screen, .lock-card, .lock-row.
- Added `[hidden] { display: none !important; }` so the ephemeral
button on the REPL lock screen actually hides the modal. Without
this, .lock-screen's `display: grid` overrode the hidden attribute's
UA-default display: none.
