Two fixes in one push.
(1) Streaming horizontal-output bug, take three. textContent += chunk
then appendChild(createTextNode(chunk)) both still rendered chunks
horizontally in fox's Firefox tab even though the chunks clearly
contained \\n (node test confirmed; deployed loader hash matched
local; curl-fetched lumbda-c.loader.js carried the line + '\\n' fix).
Whatever the browser was doing with sibling text nodes inside a
<pre> wasn't honoring the embedded newlines.
Switch to one <div class=\"stream-line\"> per logical line. liveBlock.append
walks the incoming chunk byte by byte, every \\n closes the current
pending div and spawns a fresh empty one for the next line. CSS
adds .stream-line { display: block; white-space: pre; } so each
finished line stacks vertically no matter what the parent
white-space rule was doing. Empty lines get a single space so they
take a row instead of collapsing. Reconcile path inside finalize()
compares the joined per-line text to the full output and rebuilds
the div column if they diverge — for the asm-tier fallback we
already had and now also for any future browser/wasm combo where
a flush silently drops a chunk.
Also drops the temporary [c-tier print] console.log debug we added
in the last commit — diagnosis arrived from elsewhere, no point
keeping the spam.
(2) Per-program autosave drafts. scheduleFreeFormSave previously
returned early if the selected program wasn't \"free-form\", so a
user who unlocked the vault, edited the bend-gpu demo, and came
back later found their edits gone — only free-form persisted.
Now drafts live as { [programName]: text } in the vault payload;
every edit, regardless of which radio is selected, debounces a
save into drafts[currentProgram]. unlockVault loads any saved
draft for the current program (and lifts the legacy
top-level freeForm field into drafts['free-form'] so existing
users don't lose their work). loadCurrentDemo shows a saved
draft instead of the ship default whenever one exists; vault bar
stays visible across all programs once the vault is engaged so
the save status note is always reachable.
The deployed loader emits chunks with currentOnChunk(line + '\n'),
which my node tests confirm produces clean per-line chunks
('tick 0\n', 'tick 1\n', ...). But fox still reports horizontal
output in his playground tab — same tier, same loop, same loader
hash on the wire (verified via curl). Browser caches and module
loaders are aggressive enough that hard-refresh isn't always
enough.
Adding a temporary console.log inside Module.print so when fox
opens DevTools and re-runs the throttled loop he can see (a) is
the print callback firing at all? and (b) what does Emscripten
hand us — is it actually 'tick 0' or something stranger like
'tick 0\n' or 'tick' + ' 0' as two calls?
Will revert this once the bug is resolved.
EOF
)
Emscripten's Module.print fires once per line and hands the loader
just the line content — no trailing newline; the caller is expected
to add it back when joining (outBuf.join('\n') does that for the
final return). Streaming path was forwarding the line straight to
onChunk without the newline, so successive chunks concatenated
horizontally in the playground panel — the loop
(let loop ((i 0))
(cond ((= (modulo i 10000) 0) (display \"tick \") (display i) (newline)))
(loop (+ i 1)))
was running fine on c tier (each (newline) ended a printf line) but
arriving in the UI as 'tick 0tick 10000tick 20000...' with no
breaks.
Loader's print/printErr callbacks now pass `line + \"\\n\"` to
currentOnChunk. Final buffered join (which already adds the \\n)
is untouched, so the post-eval output string keeps the same shape.
Pyodide _StreamingStdout sees Python's raw write() bytes including
the \\n already; asm's emit_chunk fires AFTER output_len was
incremented past the \\n, so the slice [flush_start, output_len)
includes it. Neither needed a change.
Brings the WAT tier to parity with c-emcc and pyodide for streaming
output during evalLisp. Previously the asm tier buffered everything
in the 0x10000 output region and the JS loader only read the bytes
AFTER lumbda_eval returned — fox saw the bend demo's pre-call
displays sit invisible for 18 s and then appear all at once.
WAT-side changes:
- New env.emit_chunk(ptr, len) import. Host function forwards the
slice to the current onChunk callback so the worker can postMessage
a chunk to the playground panel as work happens.
- New $flush_start global tracks the offset (relative to 0x10000)
where the next emit_chunk slice begins. Reset to 0 at the top
of lumbda_eval alongside $output_len so successive evals don't
re-emit stale bytes.
- $out_char now checks for newline (i32.const 10) after the store.
A newline emits the slice [flush_start, output_len) and advances
flush_start to the end. Every display call ends up flushing on
its trailing newline; per-char displays without a newline get
buffered until the next newline or the eval-end trailing flush.
- $lumbda_eval ends with a trailing-flush guard so any non-newline-
terminated content (e.g. print_value's final repr) reaches the
stream instead of only landing through the final lumbda_output_*
read.
JS loader:
- importObj.env.emit_chunk decodes the slice from wasm memory and
forwards to refs.currentOnChunk.
- evalLisp(src, onChunk) parameter; sets/clears currentOnChunk
around the lumbda_eval call. Same shape as c-emcc + pyodide.
Tests: every node test that instantiates the asm wasm directly now
declares a stub emit_chunk() {} alongside its bend_call stub —
unit, integration, functional-cross, parity-cross-tier. Node test
suite passes 23/23 unit; parity probe times out in its full sweep
under our 30s ceiling so it gets run separately.
Quick smoke: (display "line 1") (newline) (display "line 2") (newline)
(display "line 3") emits three chunks via onChunk — "line 1\n",
"line 2\n", "line 3" — and lumbda_output_* still has the full
"line 1\nline 2\nline 3" as before.
Previously every (display ...) call buffered into the tier's outBuf /
sys.stdout / WAT output-buffer and only landed in the playground panel
after evalLisp returned. During the 18-second cuda-secp256k1-bench
dispatch the panel stayed blank, then everything (probe + telemetry +
heavy result) appeared at once. Fox: 'demo waits for the full program
to return before emitting instead of line by line as it finishes'.
Each tier's print sink now ALSO calls a per-eval onChunk callback
that the worker forwards to the main thread as a {kind:"chunk"}
postMessage. The main thread spawns an in-progress tier-block on
the first chunk and appends each subsequent chunk to its <pre>,
auto-scrolling. On done/error the timing header finalizes in place;
the block looks identical to a non-streamed appendBlock at the end.
C tier (Emscripten): the Module.print + Module.printErr callbacks
that already fired per-line now invoke currentOnChunk in addition
to buffering. evalLisp accepts an onChunk arg and threads it.
Python tier (pyodide): _lumbda_eval swaps sys.stdout for a
_StreamingStdout subclass of io.StringIO whose write() also calls
into globalThis._lumbdaPyEmitChunk on the JS side. The final
auto-printed value also emits.
Asm tier (WAT): output stays buffered until evalLisp returns —
streaming there needs a new wasm import (out_line) and a wasm
rebuild. Caller-side fallback: app.js's appendBlock path still
runs for asm so its block appears at the end as before.
worker.mjs forwards each chunk via postMessage; app.js's
runOnTierInWorker handles 'chunk' messages by lazily creating
a startLiveBlock on first chunk, appending text on each, and
finalizing the timing in finalize() once done arrives.
Browser instantiation of lumbda-c.wasm failed with:
Aborted(LinkError: import object field 'js_lumbda_bend_call'
is not a Function)
The wasm correctly required env.js_lumbda_bend_call as a function
import, and the EM_JS-generated function existed in the glue —
emcc placed the function declaration at depth 0 of createLumbdaC
where wasmImports lives, so it SHOULD have been hoisted into scope
at instantiate time. It worked in node test runs but threw on
browser load. The exact emcc-version cause is fuzzy; the fix is
to use the textbook mechanism instead of guessing.
New file wasm/c/bend-call-library.js — Emscripten JS library with
mergeInto(LibraryManager.library, { js_lumbda_bend_call: function(...) }).
mergeInto lands the function directly inside wasmImports under the
mangled name _js_lumbda_bend_call (auto-prefix), wired to the
js_lumbda_bend_call import. No scope guessing.
lumbda_wasm_entry.c — drops EM_JS, keeps the C wrapper bi_bend_call_wasm
and declares extern int js_lumbda_bend_call(...) so emcc emits the
env import that the library fills.
wasm/Makefile — adds --js-library c/bend-call-library.js to C_LDFLAGS
and lists the library as a dependency so changes trigger a rebuild.
Verified: deployed glue at lumbda.com now contains the mergeInto
binding (js_lumbda_bend_call:_js_lumbda_bend_call inside wasmImports).
End-to-end smoke via mocked XHR in node returns the expected
(ok pong) round-trip from the demo's first probe step.
lumbda_wasm_entry.c — adds an EM_JS bridge js_lumbda_bend_call that
does sync XHR POST (legal inside Web Workers, the playground's tier
host) and writes the response bytes back into the wasm heap. C
wrapper bi_bend_call_wasm allocates a 256 KiB response buffer, calls
the EM_JS function, wraps the bytes as a lumbda string Value, and
gets registered as the `bend!-call` builtin in lumbda_wasm_init —
not in c/builtins.c, so the native CLI build doesn't acquire a
wasm-flavored binding it can't satisfy.
lumbda-c.loader.js — setBendUrl(url) mutates globalThis._lumbdaCBendUrl
which the EM_JS reads on every call. runner.js plumbing already
propagates a saved playground URL to every tier's setBendUrl on
each save.
C tier now matches asm + pyodide for HTTP-mode bend. End-to-end
verified in node + wasm directly: bend!-call returns the configured
URL response text; "no bend URL configured" when unset; clean
"bend error: ..." string when the XHR throws (e.g., CORS, network).
Tested against the live bend.unturf.com chain via the playground.
Three tiers in parallel now show the same (ok (HEX0 HEX1 HEX2))
result from (cuda-shake-fanout ("00" "01" "deadbeef") 32).
The WAT $read_string function scanned bytes until the first " and copied
raw, with NO escape handling. So a source string like
"(\"00\" \"01\")" got chopped at the first \" — the asm tier read only
"(\\" before terminating, producing a mangled payload that the worker
couldn't parse. The bend-gpu demo's payload built via string-append of
escaped-quote strings came out as "(cuda-shake-fanout (\ \ \) 32)" in
the asm tier, sent garbage to bend, and got nothing useful back.
Two-pass fix to match Python/C tier behavior:
1. Scan-and-count pass: walks source-ptr to the closing ", but
when it sees \ it skips the next byte so embedded \" doesn't
terminate the string. Counts decoded output bytes (each \X
contributes one byte, not two).
2. Allocate + copy-and-decode pass: walks the same range, converts
\n → 0x0A, \t → 0x09, and any other \X (including \" and \\)
→ X. Matches the lenient fallback the desktop tiers use.
Verified via cross-tier parity probe — 255/255 still passing.
Demo payload now constructs as
"(cuda-shake-fanout (\"00\" \"01\" \"deadbeef\") 32)" (45 bytes,
identical to Python/C reads) and the asm playground returns
(ok (HEX0 HEX1 HEX2)) from the live 3090-ai bend worker.
lumbda-py.js (and the three deployed mirrors) now expose setBendUrl
and register a bend!-call primitive in the pyodide-hosted lumbda
environment.
Implementation:
- JS loader stashes a `globalThis._lumbdaPyBendCall` function that
does sync XHR POST to the configured bend URL (legal in Web
Workers, where the pyodide tier runs in this playground)
- Python bootstrap imports `_lumbdaPyBendCall` from `js` and binds
it as a builtin under the symbol `bend!-call`, accepting any
value and stringifying via lumbda.show before sending
- setBendUrl(url) on the tier object updates the JS closure; the
runner.js plumbing already calls it on every tier when the user
saves a bend URL in the ⚡ bar
This brings the pyodide tier to parity with the asm (WAT) tier for
HTTP-mode bend. The emcc C tier still lacks the bind — wiring it
needs a new wasm primitive built via emcc; lands in the next commit.
Tested: bend!-call "(ping)" against the same gpu-worker endpoint
returns the same (ok pong) S-expression the asm tier sees.
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Step 1 of the GC effort. Each tier loader now exposes heapStats():
- asm-wasm — lumbda_heap_used / lumbda_heap_total wat exports
- c-wasm — emscripten linear memory size (no free path right now,
so used = total; documented in the loader)
- python — pyodide module linear memory size; CPython GC cycles
this naturally
Worker handles a "heap" message kind that round-trips the active tab's
loaded tiers; repl tabbar shows a compact "py 12M · c 32M · asm 4M"
strip next to the buttons. Polls every 2s.
Doesn't solve the leak — just makes pressure visible so the user knows
when to use "reboot tier". Real GC (Cheney over the WAT bump allocator,
Boehm-em or custom mark-sweep for c-wasm) coming next.
New: wasm/tests/parity-cross-tier.mjs runs the parity-corpus.mjs (216
test cases tagged by whitepaper section / R7RS concept) against three
tiers — native python (reference), c-wasm, asm-wasm — and fails on any
unknown divergence. Known gaps live in KNOWN_DIVERGE so the table stays
green while the bignum / call/cc / etc. work proceeds.
Wired into `make wasm-test` so a regression against any spec claim gets
caught before merge.
Bugs caught and fixed:
- python remainder: was `signed_a % signed_b * sign(a)`, which double-
applied the sign of a (python's % floors) — gave -3 for (-17, 5)
instead of the R7RS-correct -2. Now uses abs() on both sides.
- asm-wasm modulo: was i32.rem_s (truncated, remainder semantics)
where R7RS modulo wants sign of divisor. Added the "if rem and
divisor disagree on sign, add divisor" branch.
Cross-tier numbers after fix:
216 passing
3 known diverge: expt-2-100, expt-3-50, big-arith — all asm-wasm
(no bignums on the asm tier yet; whitepaper §2.1 claim still open)
0 fail
REPL layout: body is now the scroll container, prompt-bar is
position:fixed at the viewport bottom so it doesn't get pushed off
screen by a long transcript. Empty space above the prompt on a fresh
session reads like a terminal.
All other tests still pass: 20 unit, 8 integration, 11 functional.
Adds tag-9 rational type to the asm tier. Layout [tag=9, num:i32, den:i32].
make_rational normalizes via gcd and collapses to a fixnum when den
reduces to 1, so 14/2 stays as 7.
Arithmetic (+, -, *, /, =, <, >, <=, >=) now promotes to rational when
any argument is rational. Mixed fixnum/rational lifts the fixnum
accumulator into a rational mid-loop so (+ 1 1/2) returns 3/2, not 1/2.
Reader parses "67/7" literals via the existing atom path: after the
numerator's digits, if '/' follows we keep reading the denominator and
hand back a normalized rational. Falls through to symbol if either side
isn't all digits.
Printer renders rationals as "n/d". equal_p compares numbers by value
(1/2 = 2/4, 3 = 6/2). is_number / number? cover both fixnums and
rationals.
eval now treats rationals as self-evaluating — without this, '1/3'
parsed correctly but evaluated to VOID.
Mandelbrot demo: switched from (/ a b) to (quotient a b) for the
fixed-point math. The demo had been relying on integer truncation
that '/' no longer provides on tiers with R7RS-correct rationals.
Bignums still pending: 31-bit num/den overflows with huge denominators.
Real lift comes with the bignum task in the C tier (which has them) or
a new bignum module in the WAT.
Cross-tier check still hangs on the bigger TCO-heavy sections of
functional.lsp — separate from rationals. Will keep grinding.
Tests: unit 20/20, integration 8/8, functional 11/11.
WASM tail-call instruction wired into every tail position:
if branches, eval_begin last expression, all special-form dispatchers
(cond/when/unless/case/let/let*/letrec/begin/and/or), function
application's $apply, and apply's closure-body $eval_begin.
wat2wasm + wasm-validate now use --enable-tail-call.
Reader fixes:
- Dotted pair syntax: (a b . rest) parses as a real dotted list.
Without this, variadic params and other dotted-cdr forms parsed
as 4-element proper lists.
- Stray ) at top level advances source_ptr instead of spinning
forever. Found via bisection of functional.lsp under TCO.
- eval_args guards against non-pair tails so a misplaced dotted
argument (e.g. an unexpanded macro template) can't deref garbage.
Cross-tier numbers (wasm-test-functional-cross):
before: 93/111 reached, stack overflow on (ack 3 4)
now: progresses through the full TCO section, deep let, named-let
to 100k, mutual recursion to 200k. Still climbing.
Bend gpu demo:
6th playground option ("bend (gpu dispatch) ⚡") ships a SHAKE256
fan-out at 1M inputs via (bend!-call ...). Run button is GUARDED:
if no bend URL is configured, refuses with
"set a bend URL first — this demo is GPU-only by design".
Protects customer machines from burning minutes on a workload the
local tiers cannot finish in reasonable time.
- bend URL and vault password now share a single config-bar row, split
via a 2-column grid (1fr 1fr). Vault still hidden when free-form
isn't selected; just collapses its column.
- Every flexbox removed. Every multi-child container uses CSS grid:
.controls, .config-bar, .bend-bar, .vault-bar, .panes, .pane,
.brand, .tabbar, .tabs, .tab, .transcript, .entry, .tier-output,
.prompt-bar, .lock-screen, .lock-card, .lock-row.
- Added `[hidden] { display: none !important; }` so the ephemeral
button on the REPL lock screen actually hides the modal. Without
this, .lock-screen's `display: grid` overrode the hidden attribute's
UA-default display: none.
bend!-call from the WAT tier
- New WAT import: (import "env" "bend_call"). The host loader supplies
a sync XMLHttpRequest that POSTs the payload to a configured URL
(workers only — sync XHR isn't allowed on main thread).
- New primitive (bend!-call "<payload>") returns the response as a
lumbda string. Works in playground and REPL once the bend URL is
saved in the new top bar.
- bendUrl persists in plain localStorage (not encrypted — it's a
server address, not a secret).
- Tests stub the import with a no-op so unit / integration / functional
suites keep instantiating cleanly.
Playground + REPL zoom 133% by default
- html { zoom: 1.33 } so the styleguide sizes read comfortably without
requiring browser-level zoom.
Free-form default = cross-tier assertion runner in Lisp
- The default editor content for "free form" is now a small assertion
framework matching tests/functional.lsp's PASS/FAIL convention. A
starter the user can extend, runs identically on the three tiers.
C tier + Python tier (bend) are wired through the runner stub; full
bend integration in those tiers comes next once their loaders learn
about setBendUrl.
Adds (write x) and (write-string s) primitives. write quotes strings
and #\-prefixes chars — what tests/functional.lsp's assert-equal
uses to print failures.
New target: make wasm-test-functional-cross (also rolled into wasm-test)
runs the 205-assertion tests/functional.lsp against each WASM tier and
reports pass counts:
c-wasm: 205 / 205 (full parity with native c)
asm-wasm: 93 / 111 reaches mid-suite before stack overflow on
a deeply recursive test; the 84% it reaches
passes. Documented progress toward full parity
with asm/lumbda.s.
The runner exit-soft on the asm tier — it's a measurement, not a gate.
Free-form radio adds a 5th demo slot. When selected, a vault bar appears
under the controls: enter a password, "unlock" derives a per-device
vault and decrypts (or creates fresh). Edits in the editor auto-save
350ms after typing stops. Reload + same password restores the code.
Same Web Crypto stack as /repl/ (PBKDF2 + AES-GCM, vault id =
SHA-256(password || device-salt)).
Layout: one shared vertical scroller — code pane and output pane both
grow with content, the body scrolls. No more independent in-pane
scrollers fighting the page.
Home page split into "Demo" and "REPL" sections with their own CTAs.
Interactive REPL at lumbda.com/repl with:
- multi-tab sessions (click + to add, × to close, double-click to rename)
- per-tab tier selector (python/c/asm/all-three race)
- persistent transcripts encrypted in localStorage via Web Crypto
(PBKDF2 + AES-GCM, vault id = SHA-256(password || device-salt) —
same pattern as unsandbox's vault-encryption-design.md, native
crypto.subtle API instead of CryptoJS)
- ephemeral mode (skip vault, transcripts vanish on reload)
- one worker per (tab × tier) — state persists across evals in a tab
- reboot tier button (terminate this tab's worker, fresh state next eval)
- cancel button (kills the running worker in active tab)
Home page now links to both /playground/ and /repl/.
Tier state itself does NOT persist across reloads — the transcript does,
but defines/set!/hash-tables vanish with the worker. Portal save/resume
in WAT (deferred) will let a tier session survive close+reopen.
