Adds a portal-bar to the REPL between tabbar and transcript: a save
button + chip strip showing all saved checkpoints for the active tab.
Click a chip to restore, click × to delete.
Per-tier strategy:
* c, python — call the tier's (portal-snapshot! NAME), then read the
JSON blob out of MEMFS (Emscripten/Pyodide FS) and stash it in the
encrypted vault entry. Restore reverses: hydrate MEMFS, then
(portal-load! NAME) merges the bindings into the live env.
* asm — no portal serializer in the WAT tier yet (would need a
Cheney-aware walk). Falls back to transcript replay: save snapshots
every successful prior input, restore reboots the tier and re-evals
them in order.
Plumbing:
* Worker bridge: new portal-save / portal-load message kinds wire
MEMFS reads/writes to the main thread.
* runner.js exposes portalSave / portalLoad — null when a tier
hasn't implemented portals (asm stays grey).
* C tier: replace EM_JS with extern + --js-library for js_lumbda_bend_call
(EM_JS-generated declaration was unreachable from wasmImports at
instantiate time, browsers threw "import object field ... not a
Function"). FS added to EXPORTED_RUNTIME_METHODS so JS can reach
pyodide.FS / Module.FS for MEMFS I/O.
Smoke-tested all three tiers headlessly: save → chip render → restore
round-trips clean on c / python / asm, zero console errors.
evalInTier picks up onChunkText + onChunkEol callback parameters
and routes the new typed-event chunks from worker.mjs (already
posting chunk-text + chunk-eol per 25d2765 / 89d4877) to per-tier
streaming sinks. The REPL's own worker.mjs was already the same
build as the playground's (md5-matched) so no worker changes.
sendInput pre-populates entry.results with one placeholder per tier
(output:"", streaming:true), runs the initial renderAll, then walks
the freshly-rendered DOM rows and attaches streaming refs (one per
tier) directly to each tier-result span. Chunks land in per-line
<div display:block> children inside that span — same shape the
playground uses. metaSpan flips to "<tier> · running…" until done.
On each tier's promise resolve, the placeholder gets mutated in
place (NOT push'd a second time) and the corresponding liveRow's
finalize() reconciles the streamed divs against the full output
(rebuilds from canonical text only if they diverge or there was
an error) and stamps the elapsed-ms meta. renderAll is NOT called
between tier completions any more — that was clobbering sibling
tiers still streaming in 'all three' mode.
Net effect: paste (let loop ((i 0)) (display i) (newline) (loop
(+ i 1))) in the REPL, hit run on 'all three', and you see each
tier's count tick up in its own row in real time — not a frozen
panel followed by a wall of output at the end.
A throttled forever-counter on the asm tier ran for ~5.7 s and
then trapped 'index out of bounds' — the WAT writes output to a
fixed 64 KB region at 0x10000–0x1FFFF, and a fast (display X)
(newline) loop accumulates faster than the buffer can drain. After
about 6,400 ticks at ~10 chars each, overflowed the
region into the source buffer at 0x20000 and the next i32.store8
fell off linear memory.
Fix is a contract change on emit_chunk: its signature picks up an
i32 return — 1 tells the WAT to recycle (zero output_len AND
flush_start), 0 keeps the original 'just advance flush_start'
semantics so callers that read lumbda_output_ptr/len after eval
still see the full buffer.
The asm loader returns 1 from emit_chunk, accumulating every
flushed slice into refs.accumulated. evalLisp's return value is now
refs.accumulated + the trailing (still-unflushed) buffer slice
rather than just the lumbda_output_ptr/len slice — caller still
gets the complete output, the WAT-side buffer just keeps recycling.
Test stubs already declared emit_chunk() {} which returns
undefined — JS->wasm i32 coercion turns that into 0, preserving
the no-recycle behavior they expected. unit/integration suites
(31 tests total) still pass.
Trailing flush in lumbda_eval also honors the return value: if the
host consumed, zero both offsets so the final lumbda_output_ptr/len
read returns 0 bytes (loader already accumulated the trailing
slice — no need to re-deliver). Earlier draft of this patch
double-emitted the trailing slice because we read it through both
the emit_chunk path and the final-buffer path.
Going nuclear on the c-tier-horizontal-output saga. Six attempts at
preserving the \n byte from Module.print through Web Worker
postMessage to the DOM all reported clean chunks in node tests but
still rendered horizontal in fox's Firefox tab.
Removing \n bytes from the chunk-transmission path entirely. Worker
walks each onChunk slice byte-by-byte and posts two separate kinds
of message:
{ kind: 'chunk-text', tier, text: 'tick 0' } <- visible chars only
{ kind: 'chunk-eol', tier } <- bare line break
Newlines no longer travel as bytes — they're typed events. Whatever
was eating them between Emscripten's TTY out() and the main thread
in fox's browser drops out of the path entirely.
liveBlock.append split into appendText (extend the in-progress
line) and appendNewline (close pending div, spawn fresh sibling).
Python tier paths through the same worker code so it gets the same
typed-event treatment — should still render vertical because that's
what it was doing already.
After (4) rounds of newline-debugging where node tests said chunks
contained \n but fox's Firefox tab still rendered horizontal text
for the c tier (python+asm rendered vertical with the same code
path), giving up on trying to guess where the \n was being eaten
and instead getting rid of \n bytes on the wire entirely.
Worker now splits each onChunk slice on \n at the source and posts
one message per line — { kind:'chunk', tier, chunk:'tick 0', eol:true }
— so the line break is carried as a boolean flag rather than a byte.
Main thread re-attaches the '\n' before handing to liveBlock.append,
which still walks bytes for charCodeAt 10 and stacks per line.
Effectively: the loader's currentOnChunk(line + '\n') feeds the
worker which immediately splits back on the \n, both halves still
arrive on the main thread, the main thread reconstitutes them with
a fresh \n that we now know our DOM split honors (proved by the
python tier which uses the same liveBlock.append). C-tier-specific
\n loss between Module.print and postMessage drops out of the
picture.
Reapplies the per-line-div approach lost in the autosave revert
(c2d9eca) and pins display:block + white-space:pre as INLINE styles
on each div instead of relying on the .stream-line CSS rule. We
spent a long stretch chasing what looked like a deeper rendering
bug — turned out the previous .stream-line class rule in style.css
was loaded but the cached stylesheet bundle in fox's tab didn't
include it, so each div kept its default inline display and the
chunks still rendered horizontally even though the DOM had a clean
sibling-div column.
Each chunk now: walk byte-by-byte, accumulate text in a pending
div, every \n closes the pending div with display:block already
set and spawns a fresh sibling. Empty lines get a single space so
they still take a row. finalize() reconciles against the full eval
output (asm tier fallback) using the joined per-line text rather
than comparing pre.textContent.
Fox: 'only free form should save the other demos should go back to
default.' Restoring the early-return guard in scheduleFreeFormSave
(skip unless the selected program is free-form), the original
unlockVault that loads only the legacy top-level freeForm field,
the original lockVault that resets free-form to its placeholder
without touching the other demos, and the original loadCurrentDemo
that hides the vault bar unless free-form is the selected program.
The per-program drafts map landed in ffada02 against expectation;
this fully undoes that piece. Streaming chunk-to-div changes from
the same commit stay — they're orthogonal.
Two fixes in one push.
(1) Streaming horizontal-output bug, take three. textContent += chunk
then appendChild(createTextNode(chunk)) both still rendered chunks
horizontally in fox's Firefox tab even though the chunks clearly
contained \\n (node test confirmed; deployed loader hash matched
local; curl-fetched lumbda-c.loader.js carried the line + '\\n' fix).
Whatever the browser was doing with sibling text nodes inside a
<pre> wasn't honoring the embedded newlines.
Switch to one <div class=\"stream-line\"> per logical line. liveBlock.append
walks the incoming chunk byte by byte, every \\n closes the current
pending div and spawns a fresh empty one for the next line. CSS
adds .stream-line { display: block; white-space: pre; } so each
finished line stacks vertically no matter what the parent
white-space rule was doing. Empty lines get a single space so they
take a row instead of collapsing. Reconcile path inside finalize()
compares the joined per-line text to the full output and rebuilds
the div column if they diverge — for the asm-tier fallback we
already had and now also for any future browser/wasm combo where
a flush silently drops a chunk.
Also drops the temporary [c-tier print] console.log debug we added
in the last commit — diagnosis arrived from elsewhere, no point
keeping the spam.
(2) Per-program autosave drafts. scheduleFreeFormSave previously
returned early if the selected program wasn't \"free-form\", so a
user who unlocked the vault, edited the bend-gpu demo, and came
back later found their edits gone — only free-form persisted.
Now drafts live as { [programName]: text } in the vault payload;
every edit, regardless of which radio is selected, debounces a
save into drafts[currentProgram]. unlockVault loads any saved
draft for the current program (and lifts the legacy
top-level freeForm field into drafts['free-form'] so existing
users don't lose their work). loadCurrentDemo shows a saved
draft instead of the ship default whenever one exists; vault bar
stays visible across all programs once the vault is engaged so
the save status note is always reachable.
The deployed loader emits chunks with currentOnChunk(line + '\n'),
which my node tests confirm produces clean per-line chunks
('tick 0\n', 'tick 1\n', ...). But fox still reports horizontal
output in his playground tab — same tier, same loop, same loader
hash on the wire (verified via curl). Browser caches and module
loaders are aggressive enough that hard-refresh isn't always
enough.
Adding a temporary console.log inside Module.print so when fox
opens DevTools and re-runs the throttled loop he can see (a) is
the print callback firing at all? and (b) what does Emscripten
hand us — is it actually 'tick 0' or something stranger like
'tick 0\n' or 'tick' + ' 0' as two calls?
Will revert this once the bug is resolved.
EOF
)
liveBlock.append was doing `pre.textContent += chunk` on every
streamed chunk. textContent's setter wipes all existing child
nodes and creates a single new text node from the concatenated
read+new value — perfectly correct semantically but under a fast
C-tier emit loop (one chunk per (newline) printf) the per-update
restringify was visibly dropping line breaks, rendering
tick 0tick 10000tick 20000tick 30000…
horizontally instead of stacking vertically.
`pre.appendChild(document.createTextNode(chunk))` appends a new
text node alongside existing ones — no read-back, no restringify.
Browser renders the sequence of text nodes as concatenated text
inside the <pre> which already has `white-space: pre`, so every
embedded \n produces a real line break.
Pairs with the c-tier loader's `currentOnChunk(line + "\n")` fix:
loader adds the \n that Emscripten's Module.print convention
stripped, append preserves the \n end-to-end into the DOM.
Emscripten's Module.print fires once per line and hands the loader
just the line content — no trailing newline; the caller is expected
to add it back when joining (outBuf.join('\n') does that for the
final return). Streaming path was forwarding the line straight to
onChunk without the newline, so successive chunks concatenated
horizontally in the playground panel — the loop
(let loop ((i 0))
(cond ((= (modulo i 10000) 0) (display \"tick \") (display i) (newline)))
(loop (+ i 1)))
was running fine on c tier (each (newline) ended a printf line) but
arriving in the UI as 'tick 0tick 10000tick 20000...' with no
breaks.
Loader's print/printErr callbacks now pass `line + \"\\n\"` to
currentOnChunk. Final buffered join (which already adds the \\n)
is untouched, so the post-eval output string keeps the same shape.
Pyodide _StreamingStdout sees Python's raw write() bytes including
the \\n already; asm's emit_chunk fires AFTER output_len was
incremented past the \\n, so the slice [flush_start, output_len)
includes it. Neither needed a change.
Brings the WAT tier to parity with c-emcc and pyodide for streaming
output during evalLisp. Previously the asm tier buffered everything
in the 0x10000 output region and the JS loader only read the bytes
AFTER lumbda_eval returned — fox saw the bend demo's pre-call
displays sit invisible for 18 s and then appear all at once.
WAT-side changes:
- New env.emit_chunk(ptr, len) import. Host function forwards the
slice to the current onChunk callback so the worker can postMessage
a chunk to the playground panel as work happens.
- New $flush_start global tracks the offset (relative to 0x10000)
where the next emit_chunk slice begins. Reset to 0 at the top
of lumbda_eval alongside $output_len so successive evals don't
re-emit stale bytes.
- $out_char now checks for newline (i32.const 10) after the store.
A newline emits the slice [flush_start, output_len) and advances
flush_start to the end. Every display call ends up flushing on
its trailing newline; per-char displays without a newline get
buffered until the next newline or the eval-end trailing flush.
- $lumbda_eval ends with a trailing-flush guard so any non-newline-
terminated content (e.g. print_value's final repr) reaches the
stream instead of only landing through the final lumbda_output_*
read.
JS loader:
- importObj.env.emit_chunk decodes the slice from wasm memory and
forwards to refs.currentOnChunk.
- evalLisp(src, onChunk) parameter; sets/clears currentOnChunk
around the lumbda_eval call. Same shape as c-emcc + pyodide.
Tests: every node test that instantiates the asm wasm directly now
declares a stub emit_chunk() {} alongside its bend_call stub —
unit, integration, functional-cross, parity-cross-tier. Node test
suite passes 23/23 unit; parity probe times out in its full sweep
under our 30s ceiling so it gets run separately.
Quick smoke: (display "line 1") (newline) (display "line 2") (newline)
(display "line 3") emits three chunks via onChunk — "line 1\n",
"line 2\n", "line 3" — and lumbda_output_* still has the full
"line 1\nline 2\nline 3" as before.
Previously every (display ...) call buffered into the tier's outBuf /
sys.stdout / WAT output-buffer and only landed in the playground panel
after evalLisp returned. During the 18-second cuda-secp256k1-bench
dispatch the panel stayed blank, then everything (probe + telemetry +
heavy result) appeared at once. Fox: 'demo waits for the full program
to return before emitting instead of line by line as it finishes'.
Each tier's print sink now ALSO calls a per-eval onChunk callback
that the worker forwards to the main thread as a {kind:"chunk"}
postMessage. The main thread spawns an in-progress tier-block on
the first chunk and appends each subsequent chunk to its <pre>,
auto-scrolling. On done/error the timing header finalizes in place;
the block looks identical to a non-streamed appendBlock at the end.
C tier (Emscripten): the Module.print + Module.printErr callbacks
that already fired per-line now invoke currentOnChunk in addition
to buffering. evalLisp accepts an onChunk arg and threads it.
Python tier (pyodide): _lumbda_eval swaps sys.stdout for a
_StreamingStdout subclass of io.StringIO whose write() also calls
into globalThis._lumbdaPyEmitChunk on the JS side. The final
auto-printed value also emits.
Asm tier (WAT): output stays buffered until evalLisp returns —
streaming there needs a new wasm import (out_line) and a wasm
rebuild. Caller-side fallback: app.js's appendBlock path still
runs for asm so its block appears at the end as before.
worker.mjs forwards each chunk via postMessage; app.js's
runOnTierInWorker handles 'chunk' messages by lazily creating
a startLiveBlock on first chunk, appending text on each, and
finalizing the timing in finalize() once done arrives.
The previous attempt to ship a pretty summary via an (summary "...")
field inside the structured response broke on the asm (WAT) tier
because read-from-string isn't a primitive there — the demo
extraction fell through to the raw fallback and dumped the entire
escaped S-expression.
Two-part fix that works on all three tiers:
http-listener (gpu-worker.lsp:1103-1116) — POST response builder
now checks if the handle-request return value is a string and
ships it as the raw HTTP body in that case. S-expression returns
still go through write-to-string. One-line guard, no impact on
ping/health/cuda-shake-fanout/cuda-sim-ops-bin which all keep
returning structured S-exps.
handle-cuda-secp256k1-bench — returns the formatted summary
string directly instead of an (ok ... (summary ...)) wrapping.
Drops the now-redundant structured fields; every number lives
inside the human-readable text already. asm / c / python demo
just calls (display (bend!-call "(cuda-secp256k1-bench N)")) and
the formatted output panel renders identically on all tiers.
Demo simplified accordingly: three (display (bend!-call …)) calls
with newlines between, no read-from-string / assoc / pair? dance.
Fox said 'doesn't dazzle me' on the previous run. The math is real
(160x GPU win, ~33 minutes of CPU compute in 13 seconds of GPU
kernel) but the output panel showed a wall of S-expression atoms
that buries the headline.
Worker now builds a multi-line summary string inside the response
under a new (summary "...") field:
✦ GPU just batched 100,000,000 secp256k1 public-key computations.
scalar gen : 5130 ms (random.urandom)
GPU kernel : 13028 ms (RTX 3090, --window-w 4)
GPU throughput : 7.67 million keys / second
CPU baseline ref: 0.050 Mkeys/s (libsecp256k1 single core)
CPU would need : 33.3 minutes (2000 sec)
GPU finished in : 13.0 sec
speedup : ~153× faster on GPU
Two new portable helpers — with-commas (recursive thousand
separator) and format-float (truncate decimals + drop trailing
dot when n=0) — work on all three tiers because they only use
string-append, substring, char=?, and number->string.
Demo now reads the response with read-from-string, assoc-extracts
the summary string, and displays it raw. Same source compiles
identically under asm / c / python. Existing structured fields
(n, gen-ms, gpu-ms, etc.) stay untouched for callers who want
the numbers as data.
Replaces the small cuda-shake-fanout step and the modest kickmix
4x demo with a single dispatch you can feel: 100,000,000 secp256k1
scalar*G batched multiplications.
Numbers measured end-to-end through https://bend.unturf.com/ on
3090-ai:
HTTP request size ~32 bytes (just '(cuda-secp256k1-bench 100000000)')
worker scalar gen 5.2 s (/dev/urandom into 3.2 GB BSCP)
GPU kernel 12.5 s (7.99 Mkeys/s on 3090)
total wall-clock 18.3 s
CPU equivalent ~2000 s = ~33 minutes (libsecp256k1 ref)
speedup-est ~160x
The 3.2 GB of random scalars never crosses the wire — the worker
synthesizes them from os.urandom and dispatches into the existing
cuda-secp256k1-batched-mul daemon. Response is a tiny S-expression
with the timing summary; no 6.4 GB result blob comes back.
Ping + health stay at the top so a visitor sees the chain warm up
before committing to the 18 s heavy run.
The sample-x convenience was reading the BSCR output file via
read-char in a loop. C tier's read-char does buffered UTF-8 decode
which hangs on the high-bit bytes that fill a real point's X
coordinate — the child handler stalls right after the daemon
reports back, log shows the kernel timing then nothing, HTTP
client times out.
The timing fields (gpu-ms, gpu-mkeys-per-sec, cpu-est-sec,
speedup-est) are the whole story for this demo. Killing the
sample-x output drops the read-binary-file-prefix /
read-string-bytes / sample-x-hex / hex-digit helpers and unblocks
the response. Worker now returns inside ~1s of the daemon finishing.
New op-head (cuda-secp256k1-bench N) lets HTTP callers trigger a
massive secp256k1 batched scalar*G workload without uploading the
32*N-byte BSCP payload. Worker generates the random scalars itself
via generate-bscp.py (/dev/urandom in 1 MB chunks), dispatches to
the existing cuda-secp256k1-batched-mul daemon (same daemon the
BSCP wire mode hits), times the GPU kernel, and returns a small
S-expression summarizing the run:
(ok (n N)
(gen-ms G)
(gpu-ms D)
(gpu-mkeys-per-sec R)
(cpu-rate-mkeys-per-sec 0.05) ; libsecp256k1 single-thread ref
(cpu-est-sec E)
(speedup-est S)
(sample-x HEX))
cpu-rate is the textbook libsecp256k1 single-thread number (~50K
scalar*G/sec). cpu-est-sec extrapolates from that without actually
running the CPU baseline — honest because the rate is well-known
and the daemon's GPU rate (~13.83 Mkeys/s on 3090 per Day-3 bench)
is what we measure end-to-end.
Reference numbers expected at 10M scalars on 3090-ai:
gen-ms ~3000 (urandom + write 320 MB)
gpu-ms ~720
speedup-est ~277x (gpu 13.83 Mkeys/s / cpu 0.05 Mkeys/s)
cpu-est-sec ~200 (~3 minutes of CPU work)
Three helpers added: read-binary-file-prefix (peek at the BSCR
header), sample-x-hex (format point.x as 64-char hex), and
generate-bscp-file (spawn the python helper, fail-open on missing
binary). No daemon changes — secp256k1-batch-mul stays unmodified.
Step 4 of bend-gpu.lsp now dispatches (cuda-sim-ops-bin path 141)
against /tmp/ecdsa-queue/out.bin on 3090-ai — the foxhop kickmix
circuit simulator at 9024 shots over 141 batches. The HTTP payload
is ~70 bytes; the worker does ~1.5 seconds of real Toffoli-heavy
QECC simulation, the GPU kernel comes back in ~360 ms, byte-identical
to the CPU reference. Speedup ~4x on this workload, scales further
with bigger circuits.
Why this works over HTTP where 1M-input SHAKE wouldn't: the request
references a bin file that already lives on the worker disk, so we
never have to stream the bytes through the wire. The response is a
structured (cuda-sim-result …) S-expression with timing-ms,
gates-sum, status, byte-identity, and the speedup-kernel field —
everything you need to see the GPU win directly in the playground
output panel.
Pairs with the existing small-payload (cuda-shake-fanout (...) 32)
step so the demo shows both the round-trip pattern and the real
"slow on CPU, fast on GPU" headline that the original demo header
promised but couldn't deliver until cuda-sim-ops-bin was wired up.
Browser instantiation of lumbda-c.wasm failed with:
Aborted(LinkError: import object field 'js_lumbda_bend_call'
is not a Function)
The wasm correctly required env.js_lumbda_bend_call as a function
import, and the EM_JS-generated function existed in the glue —
emcc placed the function declaration at depth 0 of createLumbdaC
where wasmImports lives, so it SHOULD have been hoisted into scope
at instantiate time. It worked in node test runs but threw on
browser load. The exact emcc-version cause is fuzzy; the fix is
to use the textbook mechanism instead of guessing.
New file wasm/c/bend-call-library.js — Emscripten JS library with
mergeInto(LibraryManager.library, { js_lumbda_bend_call: function(...) }).
mergeInto lands the function directly inside wasmImports under the
mangled name _js_lumbda_bend_call (auto-prefix), wired to the
js_lumbda_bend_call import. No scope guessing.
lumbda_wasm_entry.c — drops EM_JS, keeps the C wrapper bi_bend_call_wasm
and declares extern int js_lumbda_bend_call(...) so emcc emits the
env import that the library fills.
wasm/Makefile — adds --js-library c/bend-call-library.js to C_LDFLAGS
and lists the library as a dependency so changes trigger a rebuild.
Verified: deployed glue at lumbda.com now contains the mergeInto
binding (js_lumbda_bend_call:_js_lumbda_bend_call inside wasmImports).
End-to-end smoke via mocked XHR in node returns the expected
(ok pong) round-trip from the demo's first probe step.
lumbda_wasm_entry.c — adds an EM_JS bridge js_lumbda_bend_call that
does sync XHR POST (legal inside Web Workers, the playground's tier
host) and writes the response bytes back into the wasm heap. C
wrapper bi_bend_call_wasm allocates a 256 KiB response buffer, calls
the EM_JS function, wraps the bytes as a lumbda string Value, and
gets registered as the `bend!-call` builtin in lumbda_wasm_init —
not in c/builtins.c, so the native CLI build doesn't acquire a
wasm-flavored binding it can't satisfy.
lumbda-c.loader.js — setBendUrl(url) mutates globalThis._lumbdaCBendUrl
which the EM_JS reads on every call. runner.js plumbing already
propagates a saved playground URL to every tier's setBendUrl on
each save.
C tier now matches asm + pyodide for HTTP-mode bend. End-to-end
verified in node + wasm directly: bend!-call returns the configured
URL response text; "no bend URL configured" when unset; clean
"bend error: ..." string when the XHR throws (e.g., CORS, network).
Tested against the live bend.unturf.com chain via the playground.
Three tiers in parallel now show the same (ok (HEX0 HEX1 HEX2))
result from (cuda-shake-fanout ("00" "01" "deadbeef") 32).
The WAT $read_string function scanned bytes until the first " and copied
raw, with NO escape handling. So a source string like
"(\"00\" \"01\")" got chopped at the first \" — the asm tier read only
"(\\" before terminating, producing a mangled payload that the worker
couldn't parse. The bend-gpu demo's payload built via string-append of
escaped-quote strings came out as "(cuda-shake-fanout (\ \ \) 32)" in
the asm tier, sent garbage to bend, and got nothing useful back.
Two-pass fix to match Python/C tier behavior:
1. Scan-and-count pass: walks source-ptr to the closing ", but
when it sees \ it skips the next byte so embedded \" doesn't
terminate the string. Counts decoded output bytes (each \X
contributes one byte, not two).
2. Allocate + copy-and-decode pass: walks the same range, converts
\n → 0x0A, \t → 0x09, and any other \X (including \" and \\)
→ X. Matches the lenient fallback the desktop tiers use.
Verified via cross-tier parity probe — 255/255 still passing.
Demo payload now constructs as
"(cuda-shake-fanout (\"00\" \"01\" \"deadbeef\") 32)" (45 bytes,
identical to Python/C reads) and the asm playground returns
(ok (HEX0 HEX1 HEX2)) from the live 3090-ai bend worker.
When inner (let loop ((i ...))) shadows an outer (let loop ((a ...) (b ...))),
the inner named-let block was overwriting j->loop_slots[] without saving
the outer's slot positions. After the inner block restored loop_sym /
loop_nparams / loop_params, the outer's recursive call (loop new-a new-b)
would write the new args into the inner's stale slot positions instead
of the outer's slots, causing the outer body to see stale binding values
or trigger 'set! undefined' on tail-call args.
loop_slots is a member ARRAY (not pointer) — memcpy'd at line 876 when
setting up loop context — so the existing pointer save/restore for
loop_params didn't cover it.
Verified:
- jit_named_let_factorial still PASS (3628800)
- new nested-shadowing test: outer 3-param + 4 inner 1-param now PASS
- exact replica of squaring.lsp round84-fold structure now PASS
Bug surfaced when investigating test-round84-keep-quotient-product
failure in www.foxhop.net/ecdsa tests. Test still has a deeper substrate
bug beyond this JIT fix (width 0 ancilla in non-fast mode), but this
fix is independently correct + closes the loop_slots scope leak.
lumbda-py.js (and the three deployed mirrors) now expose setBendUrl
and register a bend!-call primitive in the pyodide-hosted lumbda
environment.
Implementation:
- JS loader stashes a `globalThis._lumbdaPyBendCall` function that
does sync XHR POST to the configured bend URL (legal in Web
Workers, where the pyodide tier runs in this playground)
- Python bootstrap imports `_lumbdaPyBendCall` from `js` and binds
it as a builtin under the symbol `bend!-call`, accepting any
value and stringifying via lumbda.show before sending
- setBendUrl(url) on the tier object updates the JS closure; the
runner.js plumbing already calls it on every tier when the user
saves a bend URL in the ⚡ bar
This brings the pyodide tier to parity with the asm (WAT) tier for
HTTP-mode bend. The emcc C tier still lacks the bind — wiring it
needs a new wasm primitive built via emcc; lands in the next commit.
Tested: bend!-call "(ping)" against the same gpu-worker endpoint
returns the same (ok pong) S-expression the asm tier sees.
gpu-worker.lsp — handle-cuda-shake-fanout, handle-cuda-sim-ops-bin,
and the four binary handlers (BSHK/BCGB/BSCP/BSRT/BSB3) now check
for the registered daemon (or, for sim-ops-bin, the bend-cuda
binary) before dispatching. Missing daemon returns (error
(daemon-not-registered <op>)) over S-exp wire, or "BERRdaemon-not-
registered: <op>" over binary wire. Before this commit any caller
whose worker host lacked a CUDA binary saw the child process crash
on (cdr #f) and got HTTP 502 / empty response with no useful
diagnostic.
Factored two helpers: daemon-or-error (S-exp result) and
with-required-daemon (binary-mode wrapper). Both keep the original
handler bodies untouched on the happy path; the guard adds one
assoc lookup per request.
www/playground/demos/bend-gpu.lsp + wasm/app/demos/bend-gpu.lsp —
demo was sending (cuda-shake256-fanout COUNT 32), an op the
dispatcher doesn't know AND a signature handle-cuda-shake-fanout
doesn't accept (it takes (inputs out-bytes)). Replaced with a
three-step probe: (ping) → (health) → small (cuda-shake-fanout
("00" "01" "deadbeef") 32). Each step prints its result so the user
gets feedback at every stage of the round-trip. Note added in the
header that browser-side bend!-call is asm-tier-only today; pyodide
and emcc tier wiring is the next commit.
WAT — leading (define ...) forms in a lambda body now bind LOCALLY
(letrec*-equivalent) instead of polluting the global env. Implementation:
apply for closures pre-processes the body in three passes:
1. hoist_internal_defines walks leading defines, env_define each name
to VOID in the new env, returns the extended env.
2. strip_leading_defines returns the body with the defines removed.
3. fill_internal_defines evaluates each define's value-expression in
the new env (so mutual references work) and env_set the real value.
(define x 1)
(define (f) (define x 99) x)
(f) ; → 99 (was 99, still 99)
x ; → 1 (was 99 wrongly — fixed)
(define (h) (define helper (lambda (x) (* x 2))) (helper 5))
helper ; → unbound (was a leaked global procedure — fixed)
C-WASM — added a thorough doc-block in lumbda_wasm_entry.c covering
the gc.c fallback malloc situation and three plausible real fixes
(Boehm-em build, custom mark-sweep over NaN-boxed heap, generational
reset). Repl tabbar already surfaces the pressure to the user.
Parity corpus locks the new scoping behavior:
internal-define-local — global x stays 1
internal-define-returns — f returns 99
internal-define-mutual — mutually-recursive internal defines
Tests: 20 unit, 8 integration, 11 functional, 249 parity all green.
WAT GC — Cheney-style two-space copying collector. Runs at end of
lumbda_eval when heap_used > 50% of memory.size — the only safe
collection point since the eval call stack has unwound and roots are
fully visible via the globals.
Implementation:
object_size(ptr) returns the byte size of any tagged heap object.
gc_forward(v) copies the object to to-space, leaves a 0xCAFEBABE
forwarding tombstone with the new address at offset 4.
gc_scan_object(ptr) walks pointer fields of pair/closure/vector/
hashtable and replaces each with its forwarded address.
gc_collect orchestrates: forward roots (global_env, intern_list,
every special-form sym), scan to-space, memmove back to 0x30000,
re-shift all pointer fields by the delta. Two passes (forward+
shift) cost the same memory bandwidth as plain Cheney does in one.
Exports: lumbda_gc (manual trigger), lumbda_heap_used, lumbda_heap_total.
Parity probe gains 3 new GC stress tests:
gc-throwaway — allocate-and-drop loop, post-eval value matches
gc-retained-length — verify the GC doesn't dropp live cons-chain
gc-survives-eval — eval after a heavy alloc still works correctly
C-WASM tier — set g_auto_compile = true in lumbda_wasm_init so every
define compiles to bytecode. Without this, the tree-walker recurses
through host C stack frames for (let loop ...) patterns and blows the
WASM linear-memory stack around N=500. With auto-compile on, the VM
uses its own explicit frame stack and TCO kicks in.
Net: 246/246 parity, 20 unit, 8 integration, 11 functional all green.
Heap diagnostics surfaced in the repl tabbar; "reboot tier" stays as
the user-side reclaim path for the c-wasm tier (which still leaks
because the Boehm-em port isn't wired yet — that's task #38).
Step 1 of the GC effort. Each tier loader now exposes heapStats():
- asm-wasm — lumbda_heap_used / lumbda_heap_total wat exports
- c-wasm — emscripten linear memory size (no free path right now,
so used = total; documented in the loader)
- python — pyodide module linear memory size; CPython GC cycles
this naturally
Worker handles a "heap" message kind that round-trips the active tab's
loaded tiers; repl tabbar shows a compact "py 12M · c 32M · asm 4M"
strip next to the buttons. Polls every 2s.
Doesn't solve the leak — just makes pressure visible so the user knows
when to use "reboot tier". Real GC (Cheney over the WAT bump allocator,
Boehm-em or custom mark-sweep for c-wasm) coming next.
Each gpu-worker.lsp now listens on both wire-TCP (existing :8320) and
HTTP/1.1+CORS (new :8321), sharing one handle-request dispatcher. Lets
a tab on https://lumbda.com/playground/ POST to its own machine via
http://localhost:8321/ — browsers permit localhost from HTTPS origins
without TLS, so no proxy, no cert, no fox-owned infra required for the
decentralized run-your-own-bend story.
main() forks at startup: child runs http-run-loop on :8321, parent
keeps existing run-loop on :8320. Adding a new op-head to handle-request
exposes it over both transports automatically. Binary modes
(BSHK/BCGB/BSCP/BSRT/BSB3) stay wire-only — they exist for native
callers who already cache the binary locally; browser callers send
S-expression recipes the worker dispatches the same way.
Two latent defects fixed to make CPU-only and Python-tier hosts work:
- vram-used-mib now file-exists? guards /usr/bin/nvidia-smi. Python
tier's spawn-process-stdio raises FileNotFoundError on missing
binary, not returning #f as the prior code expected, which crashed
every worker on a CPU-only laptop.
- fork-self return discriminated via (number? pid) not (eq? pid 0).
Python tier's (eq? 0 #f) returns #t because == conflates int 0
with bool False; pre-existing run-loop has the same risk but
C/asm tier (identity eq?) masks it for the production case.
Phase 2 (server-side factory ops: compile uploaded .lsp recipes into
.bin before bending — the foxhop champion-circuit workflow) deferred
until authentication lands; today a worker on the public internet
would let any caller occupy our GPU.
Operational Caddy + DNS proposals in plans/bend-http-deploy.md cover
the personal-remote-access endpoint chain (proxy.unturf.com edge →
ai.foxhop.net Caddy → 3090-ai:8321) gated by trusted-IP allowlist —
applied separately.
Also codifies the playground "CSS Grid only, never flexbox" rule in
CLAUDE.md: all www/ and wasm/ stylesheets are already grid-only;
documenting the invariant so future edits don't drift.
Tests: smoke-bend-http.sh — (ping)→(ok pong), unknown-op fallback,
OPTIONS CORS preflight — all PASS. Wire path unchanged, verified
round-trip via 8-digit-prefix framing.
Variable-length signed bignums on the asm-wasm tier. Layout:
[tag=10, sign:i32, n_limbs:i32, limbs[]:u32]
Little-endian u32 limbs (base 2^32). i64 used for limb-pair products
in bn_mul and for the (rem << 32) | limb shift in bn_divmod_small.
Promotion: num_add/sub/mul/cmp inspect operands and pick the right
representation (fixnum, rational, bignum). Fixnum overflow in +/-/* is
detected by computing in i64 and checking against the 30-bit fixnum
range — outside that, operands lift to bignums.
(expt 2 1024) uses exponentiation-by-squaring through num_mul so
intermediate products auto-promote, returning the exact 309-digit value.
Reader: digit parsing accumulates via num_add/num_mul, so a literal of
any length reads as the narrowest representation that holds it.
Parity corpus: KNOWN_DIVERGE is now empty. 237/237 passing across
python (ref), c-wasm, and asm-wasm. New asserts pin the bignum surface
so a regression breaks make wasm-test immediately.
New: wasm/tests/parity-cross-tier.mjs runs the parity-corpus.mjs (216
test cases tagged by whitepaper section / R7RS concept) against three
tiers — native python (reference), c-wasm, asm-wasm — and fails on any
unknown divergence. Known gaps live in KNOWN_DIVERGE so the table stays
green while the bignum / call/cc / etc. work proceeds.
Wired into `make wasm-test` so a regression against any spec claim gets
caught before merge.
Bugs caught and fixed:
- python remainder: was `signed_a % signed_b * sign(a)`, which double-
applied the sign of a (python's % floors) — gave -3 for (-17, 5)
instead of the R7RS-correct -2. Now uses abs() on both sides.
- asm-wasm modulo: was i32.rem_s (truncated, remainder semantics)
where R7RS modulo wants sign of divisor. Added the "if rem and
divisor disagree on sign, add divisor" branch.
Cross-tier numbers after fix:
216 passing
3 known diverge: expt-2-100, expt-3-50, big-arith — all asm-wasm
(no bignums on the asm tier yet; whitepaper §2.1 claim still open)
0 fail
REPL layout: body is now the scroll container, prompt-bar is
position:fixed at the viewport bottom so it doesn't get pushed off
screen by a long transcript. Empty space above the prompt on a fresh
session reads like a terminal.
All other tests still pass: 20 unit, 8 integration, 11 functional.
Adds tag-9 rational type to the asm tier. Layout [tag=9, num:i32, den:i32].
make_rational normalizes via gcd and collapses to a fixnum when den
reduces to 1, so 14/2 stays as 7.
Arithmetic (+, -, *, /, =, <, >, <=, >=) now promotes to rational when
any argument is rational. Mixed fixnum/rational lifts the fixnum
accumulator into a rational mid-loop so (+ 1 1/2) returns 3/2, not 1/2.
Reader parses "67/7" literals via the existing atom path: after the
numerator's digits, if '/' follows we keep reading the denominator and
hand back a normalized rational. Falls through to symbol if either side
isn't all digits.
Printer renders rationals as "n/d". equal_p compares numbers by value
(1/2 = 2/4, 3 = 6/2). is_number / number? cover both fixnums and
rationals.
eval now treats rationals as self-evaluating — without this, '1/3'
parsed correctly but evaluated to VOID.
Mandelbrot demo: switched from (/ a b) to (quotient a b) for the
fixed-point math. The demo had been relying on integer truncation
that '/' no longer provides on tiers with R7RS-correct rationals.
Bignums still pending: 31-bit num/den overflows with huge denominators.
Real lift comes with the bignum task in the C tier (which has them) or
a new bignum module in the WAT.
Cross-tier check still hangs on the bigger TCO-heavy sections of
functional.lsp — separate from rationals. Will keep grinding.
Tests: unit 20/20, integration 8/8, functional 11/11.
Both bytecode VMs had a latent O(n^2) defect on self-recursive tail
calls invoked from inside let/let*/letrec/letrec*/do bodies. The
self-tail-call op assumed reusing "current env" was safe, but current
env was the innermost let* frame, not the lambda body env. Each iter
pushed a fresh let* frame on top (PUSH_ENV at compile site), the
self-tail-call rebound params into that frame & jumped to ip=0 without
unwinding. Env chain grew linearly with iters; every var lookup walked
O(n) chain; effective O(n^2) behaviour.
Symptom observed 2026-06-14: 156k circ-ops walk hung > 5min instead of
1.4s. K=5 doctrine reducers ran 30+ runaway lumbda procs at 99% CPU
across multiple `make sweep-doctrine` invocations before we tracked
it back to language layer (initially misdiagnosed as K=5 substrate).
Fix: track scope depth at compile time on CodeObj (scope_depth bumped
on PUSH_ENV emit, decremented on POP_ENV emit). Record self_base at
lambda body entry (0 unless internal defines pushed a frame). At
self-tail-call emit, encode pops_needed = scope_depth - self_base in
the op arg. Runtime handler unwinds that many env frames before
rebinding params + jumping to ip=0.
Tree-walker (c/lumbda without --fast) already worked - it walks the
ast & lets recursion clean up frames naturally. Asm tier also fine -
no self-tail-call op, uses different lambda-call convention.
Verification:
python tier: 571 tests PASS, our 100k let* repro 1.04s wall (was infinite)
c tier: 205 tests PASS, same repro 0.05s wall (was infinite)
asm tier: 158 tests PASS (no fix needed, never had the bug)
Portal-resume backwards-compat: pre-fix portals stored OP_SELF_TAIL_CALL
arg as 2-tuple. Deserializer fills pops=0 when 'pops' key is absent,
so an old portal resumes at correct behaviour at the cost of slow walk
on its very next self-tail-call body (no worse than pre-fix).
Memory note saved at reference_lumbda_let_star_in_tail_loop in our
foxhop blackops memory for future agents.
num_div for two integers used to fall back to double when the
quotient wasn't exact. R7RS / python lumbda require exact-in →
exact-out for /. Fixed: the rational_normalize path was already
wired for the is_exact branch; the int/int branch now calls it
too instead of make_double.
(/ 67 7) → 67/7 (was 9.5714285714285712)
(/ 1 3) → 1/3 (was 0.33333…)
(/ 6 2) → 3 (exact stays integer)
(+ 1/3 1/6) → 1/2 (rational arithmetic propagates)
C native + C-WASM tier now match python lumbda on / between integers.
asm tier rationals remain pending — that needs bignums in asm first.
native c-test: 205/205 still passes.
WASM tail-call instruction wired into every tail position:
if branches, eval_begin last expression, all special-form dispatchers
(cond/when/unless/case/let/let*/letrec/begin/and/or), function
application's $apply, and apply's closure-body $eval_begin.
wat2wasm + wasm-validate now use --enable-tail-call.
Reader fixes:
- Dotted pair syntax: (a b . rest) parses as a real dotted list.
Without this, variadic params and other dotted-cdr forms parsed
as 4-element proper lists.
- Stray ) at top level advances source_ptr instead of spinning
forever. Found via bisection of functional.lsp under TCO.
- eval_args guards against non-pair tails so a misplaced dotted
argument (e.g. an unexpanded macro template) can't deref garbage.
Cross-tier numbers (wasm-test-functional-cross):
before: 93/111 reached, stack overflow on (ack 3 4)
now: progresses through the full TCO section, deep let, named-let
to 100k, mutual recursion to 200k. Still climbing.
Bend gpu demo:
6th playground option ("bend (gpu dispatch) ⚡") ships a SHAKE256
fan-out at 1M inputs via (bend!-call ...). Run button is GUARDED:
if no bend URL is configured, refuses with
"set a bend URL first — this demo is GPU-only by design".
Protects customer machines from burning minutes on a workload the
local tiers cannot finish in reasonable time.
Two follow-up defects from 2026-06-14 factory triage:
(1) DLQ runner classifier did not recognize "vram-oversized" reason text
introduced by foxhop dispatcher pre-flight (commit 8d45e0d on the foxhop
side). 65 of 87 rDLQ cells got escalated as class=unknown instead of a
properly named bucket. Adds pattern + escalate-class entry + reducer
test case mapped to (vram-oversized sim no — bin is dead weight on this
card, salvage skipped).
(2) sweep-doctrine reducers had no per-test timeout. K=5 doctrine tests
(test-k5-apply-forward-ipmul + 4 siblings) ran lumbda at 99% CPU for
2h43m on a remote node without ever emitting their DOCTRINE verdict
line — accumulating 30+ runaway lumbda procs under two stuck `make
sweep-doctrine` invocations. run.sh + run-parallel.sh now wrap our
lumbda invocation in `timeout ${SWEEP_DOCTRINE_TEST_TIMEOUT_S:-300}`;
hit exits 124, our existing "no DOCTRINE line" branch logs HARNESS-FAIL.
K=5 substrate has a documented non-terminating compute defect AND a
load-time buffer overflow (commit 6d18c59 on foxhop). Bisect deferred
per ticket 0007 in foxhop tree; needs qemu apparatus we currently lack.
Header (logo + tagline) is the only fixed region. Footer removed.
Everything else — tab bar, transcript, prompt — now lives inside one
.repl-stream scroller. A fresh session shows the prompt right under
the tabs near the top; as entries arrive the prompt drifts down with
them. Tabs use a dashed bottom rule instead of a heavy bar so they
read as the start of the stream rather than a separate chrome strip.
Reads as one continuous stream now. Each entry is just:
λ> <input>
<output> ; tier · NNms
No left border, no boxed cards, no side-column tier label. Output
indents under the prompt (3ch) using monospace ch units. Tier+time
render as a Lisp-comment-style suffix in muted color.
Prompt bar: borderless textarea on the code-bg surface so the input
visually joins the transcript above. Placeholder cut to "(+ 1 2)" —
the surrounding text already explains the semantics.
Multi-line inputs keep prompt continuation marks ("..").
