russell@unturf.com
373c8ea0d3
Added a write-to-string-shim.lsp for asm tier (which lacks the native builtin); asm launch script pre-defines *argv* + loads the shim so wire.lsp's wire-send finds a write-to-string definition. Python/C tiers keep the native builtin — the shim is opt-in. Bench extended with two huge workloads (100k × 16 B, 1M × 16 B). Real numbers, 3090-ai, daemon warm, both ends localhost: workload Python C tier C win small (3 × 16 B) 1.16 ms 0.14 ms 8.3× small (100 × 16 B) 3.39 ms 0.42 ms 8.1× medium (1000 × 16 B) 23.26 ms 2.67 ms 8.7× huge (100k × 16 B) 2,220 ms STALL n/a huge (1M × 16 B) 24,338 ms STALL n/a THE FINDING: at huge sizes, the bottleneck is the S-expression text wire format, not the CUDA kernel. shake256-fanout finishes 1M × 16B in ~47 ms; the Python worker takes 24 SECONDS end-to-end because wire.lsp's recv-exact accumulates chunks via string-append in a loop — O(n²) at multi-MB payload sizes. C tier fails outright. The right fix is binary wire framing between client + worker, parallel to the binary portal format the daemon + leaf already use. That's a separate piece of work; today's Web page edit calls it out honestly so visitors know when bend is the right tool. asm tier worker hosting still has process-management quirks (doesn't survive nohup detachment in this environment); bench ships with --skip-asm by default in this run.
143 lines
8.9 KiB
HTML
143 lines
8.9 KiB
HTML
<!doctype html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||
<title>Lumbda — feedback as a primitive</title>
|
||
<meta name="description" content="Lumbda — a Lisp/Scheme-derived language with four execution backends (Python, Python bytecode VM, C + x86_64 JIT, pure x86_64 GNU asm), full first-class continuations, portal-based state migration, and a formally verified universality proof.">
|
||
<link rel="stylesheet" href="style.css">
|
||
<script src="https://uncloseai.com/uncloseai.js" type="module"></script>
|
||
</head>
|
||
<body>
|
||
|
||
<header>
|
||
<h1 aria-label="lumbda.">lumbda<span class="period" aria-hidden="true">.</span></h1>
|
||
<img class="lambda-mark" src="lumbda-logo-green.png?v=2" alt="" aria-hidden="true">
|
||
<p class="tagline">feedback as a primitive</p>
|
||
</header>
|
||
|
||
<main>
|
||
|
||
<section id="what">
|
||
<p class="lead">
|
||
Lumbda names a Lisp/Scheme-derived language carrying four independently implemented execution backends — one surface syntax, one test suite. A Python tree-walker with an optional bytecode VM, a C implementation that adds an x86_64 JIT, and a pure x86_64 GNU asm interpreter (~6,600 lines, ~23 KB stripped, zero external dependencies). Every backend runs a shared <code>.lsp</code> source byte-identically, with full first-class continuations, exact rationals, records, and hygienic macros.
|
||
</p>
|
||
</section>
|
||
|
||
<section id="quick">
|
||
<h2>Get it</h2>
|
||
<pre><code>git clone https://git.unturf.com/engineering/unturf/lumbda.git
|
||
cd lumbda
|
||
make test-all</code></pre>
|
||
<p>Run a program in any tier:</p>
|
||
<pre><code>python3 lumbda.py --fast examples/fibonacci.lsp
|
||
./c/lumbda examples/fibonacci.lsp
|
||
./asm/lumbda < examples/fibonacci.lsp</code></pre>
|
||
</section>
|
||
|
||
<section id="tiers">
|
||
<h2>Four tiers, one language</h2>
|
||
<table>
|
||
<thead><tr><th>Tier</th><th>Lines</th><th>Binary</th><th>What a tier buys</th></tr></thead>
|
||
<tbody>
|
||
<tr><td>Python interpreter + bytecode VM</td><td>3,743</td><td>—</td><td>REPL hackability, debugging, reference</td></tr>
|
||
<tr><td>C tree-walker + bytecode VM</td><td>9,164</td><td>~215 KB</td><td>deep recursion, production workloads</td></tr>
|
||
<tr><td>C + x86_64 JIT</td><td>+patches</td><td>~215 KB</td><td>7–10× faster than CPython on recursive workloads</td></tr>
|
||
<tr><td>Pure x86_64 GNU asm</td><td>6,645</td><td>~23 KB</td><td>zero-dependency boot, auditability, embedded</td></tr>
|
||
<tr><td>GNU asm + naive mark-sweep GC + meta-GC arena</td><td>(same source, <code>GC_NAIVE=1</code>)</td><td>~27 KB</td><td>bounded memory without manual arena discipline</td></tr>
|
||
</tbody>
|
||
</table>
|
||
</section>
|
||
|
||
<section id="portal">
|
||
<h2>Portal: feedback across time</h2>
|
||
<p>A continuation carries feedback within a process. A portal carries feedback across processes. Same primitive, different scope: capture machine state, serialize, reload elsewhere, resume. Lumbda ships three portal formats with distinct trade-offs:</p>
|
||
<ul>
|
||
<li><strong>S-expression portal</strong> — Scheme source as a wire protocol. 16 of 16 producer×consumer cells green across Python, C, asm no-GC, and asm GC.</li>
|
||
<li><strong>JSON portal</strong> — graph-aware, preserves closures and live continuations (Python, C).</li>
|
||
<li><strong>Binary heap dump</strong> — asm only. 1.5 ms save+resume between two processes.</li>
|
||
</ul>
|
||
</section>
|
||
|
||
<section id="bend">
|
||
<h2>bend: dispatch to a GPU without rewriting your code</h2>
|
||
<p>Wrap any registered GPU-able form in <code>(bend …)</code> and lumbda decides per call whether to run it locally or ship it to a CUDA worker over our wire protocol. The decision uses a cost estimator on the argument shape, not the operation name. Tiny inputs stay local; heavy inputs bend to a worker that holds a warm CUDA context across requests.</p>
|
||
|
||
<h3>Start a GPU worker</h3>
|
||
<pre><code># On any host with nvcc + a CUDA-capable GPU:
|
||
make gpu-worker
|
||
# → builds examples/cuda-fanout/shake256-fanout
|
||
# → builds the C tier (~10× faster wire orchestration than Python)
|
||
# → launches gpu-worker.lsp on port 9091
|
||
|
||
# Override tier or port:
|
||
make gpu-worker LUMBDA=python PORT=9001 # easier debugging
|
||
make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
|
||
|
||
<h3>Call it from any tier</h3>
|
||
<pre><code>;; bend works on every tier — Python, C, asm — through the same
|
||
;; tcp-* + portal primitives lumbda already ships.
|
||
(load "examples/cuda-fanout/wire.lsp")
|
||
(load "examples/cuda-fanout/bend.lsp")
|
||
(load "examples/cuda-fanout/bend-macros.lsp") ; Python/C only — asm uses bend-call
|
||
|
||
;; Tiny — cost below threshold, evaluates locally
|
||
(bend (cuda-shake-fanout '("00" "01" "deadbeef") 32))
|
||
|
||
;; Heavy — cost above threshold, ships to the GPU worker
|
||
(bend (cuda-shake-fanout one-million-inputs 32))</code></pre>
|
||
|
||
<p>On a single RTX 3090 with a warm daemon, fan-out matched <code>hashlib.shake_256</code> byte-for-byte and won by 1.5–10× across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.</p>
|
||
<h3>Tier choice for the worker host</h3>
|
||
<p>The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of N calls per workload, daemon warm, both client & worker on localhost):</p>
|
||
<table>
|
||
<thead><tr><th>workload</th><th>Python tier</th><th>C tier</th><th>C win</th></tr></thead>
|
||
<tbody>
|
||
<tr><td>small (3 × 16 B)</td><td>1.16 ms</td><td>0.14 ms</td><td>8.3×</td></tr>
|
||
<tr><td>small (100 × 16 B)</td><td>3.39 ms</td><td>0.42 ms</td><td>8.1×</td></tr>
|
||
<tr><td>medium (1000 × 16 B)</td><td>23.26 ms</td><td>2.67 ms</td><td>8.7×</td></tr>
|
||
<tr><td>huge (100k × 16 B)</td><td>2,220 ms</td><td>(wire stalls)</td><td>n/a</td></tr>
|
||
<tr><td>huge (1M × 16 B)</td><td>24,338 ms</td><td>(wire stalls)</td><td>n/a</td></tr>
|
||
</tbody>
|
||
</table>
|
||
<p>At small & medium sizes, C tier wins by ~9× — that's the ratio between Python's S-expression parser and the C tier's reader. <strong>At huge sizes, both tiers stall on the wire, not the kernel.</strong> The shake256-fanout CUDA kernel on this 3090 finishes 1M × 16 B in ~47 ms — three orders of magnitude faster than the Python tier's 24 s end-to-end. The S-expression hex framing in <code>wire.lsp</code> is O(n²) at huge payload sizes because <code>recv-exact</code> accumulates chunks through <code>string-append</code> in a loop. C tier fails outright at this size; Python tier just suffers.</p>
|
||
<p>What this means: at small inputs, tier orchestration matters & C wins. At huge inputs, wire framing dominates regardless of tier — the right fix is a binary wire mode between client & worker, parallel to the binary portal mode the daemon already supports between worker & leaf. Until that lands, keep payloads under ~10 MB or call <code>cuda-shake-fanout</code> against pre-staged files instead of inline lists.</p>
|
||
<p>The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written <code>pipe2 + fork + execve</code> syscalls — no libc anywhere on the chain.</p>
|
||
<p>See <a href="https://git.unturf.com/engineering/unturf/lumbda/-/blob/master/examples/cuda-fanout/README.md">examples/cuda-fanout/</a> for the wire contract, daemon protocol, bench data, and per-tier integration sketch.</p>
|
||
</section>
|
||
|
||
<section id="proof">
|
||
<h2>EML universality proof</h2>
|
||
<p>A single operator <code>eml(x, y) = exp(x) − ln(y)</code> with a constant 1 generates all elementary functions: <code>exp</code>, <code>ln</code>, arithmetic, negation, complex-plane access, trigonometry. Verified numerically in Python, verified in Lumbda's own bytecode, proven formally in Lean 4 with zero <code>sorry</code>. Lumbda's native symbolic-rewrite checker runs five theorems in 46 ms cold or 7 ms cached — roughly 16× faster than Lean's cold rebuild on identical hardware.</p>
|
||
</section>
|
||
|
||
<section id="doc">
|
||
<h2>Whitepaper</h2>
|
||
<p>Full language reference, tier-by-tier architecture, benchmarks, meta-GC design, universality proof.</p>
|
||
<p>
|
||
<a class="cta" href="lumbda-whitepaper.html">Read in browser (HTML)</a>
|
||
<a class="cta" href="lumbda-whitepaper.pdf">Download PDF (~2.7 MB)</a>
|
||
</p>
|
||
</section>
|
||
|
||
<section id="license">
|
||
<h2>License</h2>
|
||
<p>AGPL-3.0-only. Public domain for whitepapers, proofs, and disclosures through <a href="https://undefect.com">undefect.com</a>. Companion to <a href="https://unturf.com">unturf.com</a>’s permacomputer project.</p>
|
||
</section>
|
||
|
||
</main>
|
||
|
||
<footer>
|
||
<p>
|
||
<a href="https://git.unturf.com/engineering/unturf/lumbda">source</a>
|
||
·
|
||
<a href="lumbda-whitepaper.html">whitepaper (HTML)</a>
|
||
·
|
||
<a href="lumbda-whitepaper.pdf">whitepaper (PDF)</a>
|
||
·
|
||
<a href="https://unturf.com">unturf.com</a>
|
||
</p>
|
||
</footer>
|
||
|
||
</body>
|
||
</html>
|