russell@unturf.com
f24afcc5d9
Wrote examples/cuda-fanout/bench_tiers.py — spawns a worker per tier, fires N TCP round-trips at three workload sizes through the warm daemon, reports median + p99. Measured on 3090-ai, daemon warm: workload Python C tier C win small (3 × 16 B) 1.27 ms 0.14 ms 9.1× small (100 × 16 B) 3.46 ms 0.41 ms 8.4× medium (1000 × 16 B) 23.51 ms 2.67 ms 8.8× Ratio stays at ~9× across the grid — the per-byte cost of Python's S-expression reader/printer compared to the C tier's reader. Justifies the LUMBDA=c default landed in the previous commit. asm tier worker starts up & listens (after the launch script predefines *argv* '()), but bench script saw malformed responses on this run — likely a write-to-string format difference between asm & Python/C reader. Leaving for follow-up; published numbers cover the tiers that completed end-to-end. www/index.html bend section gains the measured table under a new 'Tier choice for the worker host' subsection. Replaces the earlier hand-wavy ~10× claim with the actual measured numbers.
140 lines
8 KiB
HTML
140 lines
8 KiB
HTML
<!doctype html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||
<title>Lumbda — feedback as a primitive</title>
|
||
<meta name="description" content="Lumbda — a Lisp/Scheme-derived language with four execution backends (Python, Python bytecode VM, C + x86_64 JIT, pure x86_64 GNU asm), full first-class continuations, portal-based state migration, and a formally verified universality proof.">
|
||
<link rel="stylesheet" href="style.css">
|
||
<script src="https://uncloseai.com/uncloseai.js" type="module"></script>
|
||
</head>
|
||
<body>
|
||
|
||
<header>
|
||
<h1 aria-label="lumbda.">lumbda<span class="period" aria-hidden="true">.</span></h1>
|
||
<img class="lambda-mark" src="lumbda-logo-green.png?v=2" alt="" aria-hidden="true">
|
||
<p class="tagline">feedback as a primitive</p>
|
||
</header>
|
||
|
||
<main>
|
||
|
||
<section id="what">
|
||
<p class="lead">
|
||
Lumbda names a Lisp/Scheme-derived language carrying four independently implemented execution backends — one surface syntax, one test suite. A Python tree-walker with an optional bytecode VM, a C implementation that adds an x86_64 JIT, and a pure x86_64 GNU asm interpreter (~6,600 lines, ~23 KB stripped, zero external dependencies). Every backend runs a shared <code>.lsp</code> source byte-identically, with full first-class continuations, exact rationals, records, and hygienic macros.
|
||
</p>
|
||
</section>
|
||
|
||
<section id="quick">
|
||
<h2>Get it</h2>
|
||
<pre><code>git clone https://git.unturf.com/engineering/unturf/lumbda.git
|
||
cd lumbda
|
||
make test-all</code></pre>
|
||
<p>Run a program in any tier:</p>
|
||
<pre><code>python3 lumbda.py --fast examples/fibonacci.lsp
|
||
./c/lumbda examples/fibonacci.lsp
|
||
./asm/lumbda < examples/fibonacci.lsp</code></pre>
|
||
</section>
|
||
|
||
<section id="tiers">
|
||
<h2>Four tiers, one language</h2>
|
||
<table>
|
||
<thead><tr><th>Tier</th><th>Lines</th><th>Binary</th><th>What a tier buys</th></tr></thead>
|
||
<tbody>
|
||
<tr><td>Python interpreter + bytecode VM</td><td>3,743</td><td>—</td><td>REPL hackability, debugging, reference</td></tr>
|
||
<tr><td>C tree-walker + bytecode VM</td><td>9,164</td><td>~215 KB</td><td>deep recursion, production workloads</td></tr>
|
||
<tr><td>C + x86_64 JIT</td><td>+patches</td><td>~215 KB</td><td>7–10× faster than CPython on recursive workloads</td></tr>
|
||
<tr><td>Pure x86_64 GNU asm</td><td>6,645</td><td>~23 KB</td><td>zero-dependency boot, auditability, embedded</td></tr>
|
||
<tr><td>GNU asm + naive mark-sweep GC + meta-GC arena</td><td>(same source, <code>GC_NAIVE=1</code>)</td><td>~27 KB</td><td>bounded memory without manual arena discipline</td></tr>
|
||
</tbody>
|
||
</table>
|
||
</section>
|
||
|
||
<section id="portal">
|
||
<h2>Portal: feedback across time</h2>
|
||
<p>A continuation carries feedback within a process. A portal carries feedback across processes. Same primitive, different scope: capture machine state, serialize, reload elsewhere, resume. Lumbda ships three portal formats with distinct trade-offs:</p>
|
||
<ul>
|
||
<li><strong>S-expression portal</strong> — Scheme source as a wire protocol. 16 of 16 producer×consumer cells green across Python, C, asm no-GC, and asm GC.</li>
|
||
<li><strong>JSON portal</strong> — graph-aware, preserves closures and live continuations (Python, C).</li>
|
||
<li><strong>Binary heap dump</strong> — asm only. 1.5 ms save+resume between two processes.</li>
|
||
</ul>
|
||
</section>
|
||
|
||
<section id="bend">
|
||
<h2>bend: dispatch to a GPU without rewriting your code</h2>
|
||
<p>Wrap any registered GPU-able form in <code>(bend …)</code> and lumbda decides per call whether to run it locally or ship it to a CUDA worker over our wire protocol. The decision uses a cost estimator on the argument shape, not the operation name. Tiny inputs stay local; heavy inputs bend to a worker that holds a warm CUDA context across requests.</p>
|
||
|
||
<h3>Start a GPU worker</h3>
|
||
<pre><code># On any host with nvcc + a CUDA-capable GPU:
|
||
make gpu-worker
|
||
# → builds examples/cuda-fanout/shake256-fanout
|
||
# → builds the C tier (~10× faster wire orchestration than Python)
|
||
# → launches gpu-worker.lsp on port 9091
|
||
|
||
# Override tier or port:
|
||
make gpu-worker LUMBDA=python PORT=9001 # easier debugging
|
||
make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
|
||
|
||
<h3>Call it from any tier</h3>
|
||
<pre><code>;; bend works on every tier — Python, C, asm — through the same
|
||
;; tcp-* + portal primitives lumbda already ships.
|
||
(load "examples/cuda-fanout/wire.lsp")
|
||
(load "examples/cuda-fanout/bend.lsp")
|
||
(load "examples/cuda-fanout/bend-macros.lsp") ; Python/C only — asm uses bend-call
|
||
|
||
;; Tiny — cost below threshold, evaluates locally
|
||
(bend (cuda-shake-fanout '("00" "01" "deadbeef") 32))
|
||
|
||
;; Heavy — cost above threshold, ships to the GPU worker
|
||
(bend (cuda-shake-fanout one-million-inputs 32))</code></pre>
|
||
|
||
<p>On a single RTX 3090 with a warm daemon, fan-out matched <code>hashlib.shake_256</code> byte-for-byte and won by 1.5–10× across the workloads we measured. Below the break-even (~100 MB of bulk hash work) host CPU stays faster — the cost estimator picks correctly.</p>
|
||
<h3>Tier choice for the worker host</h3>
|
||
<p>The CUDA kernel runs inside the leaf binary, so the tier we pick for the worker host only affects wire orchestration (S-expression parse, portal write, pipe to daemon, response format). Measured per-call round-trip on the 3090 (median of 20 calls per workload, daemon warm):</p>
|
||
<table>
|
||
<thead><tr><th>workload</th><th>Python tier</th><th>C tier</th><th>C win</th></tr></thead>
|
||
<tbody>
|
||
<tr><td>small (3 × 16 B)</td><td>1.27 ms</td><td>0.14 ms</td><td>9.1×</td></tr>
|
||
<tr><td>small (100 × 16 B)</td><td>3.46 ms</td><td>0.41 ms</td><td>8.4×</td></tr>
|
||
<tr><td>medium (1000 × 16 B)</td><td>23.51 ms</td><td>2.67 ms</td><td>8.8×</td></tr>
|
||
</tbody>
|
||
</table>
|
||
<p>C tier wins by ~9× across the grid — consistent with the ratio between Python's S-expression parser and the C tier's reader. At very heavy workloads (where the kernel itself takes seconds) the tier choice becomes noise; at light workloads (where bend stays local anyway) the tier choice doesn't matter either. The middle ground is where C tier earns its default.</p>
|
||
<p>The CUDA toolchain stays isolated to the leaf binary the worker spawns. No tier links libcudart; no tier requires nvcc at build time. Asm tier hosts workers through hand-written <code>pipe2 + fork + execve</code> syscalls — no libc anywhere on the chain.</p>
|
||
<p>See <a href="https://git.unturf.com/engineering/unturf/lumbda/-/blob/master/examples/cuda-fanout/README.md">examples/cuda-fanout/</a> for the wire contract, daemon protocol, bench data, and per-tier integration sketch.</p>
|
||
</section>
|
||
|
||
<section id="proof">
|
||
<h2>EML universality proof</h2>
|
||
<p>A single operator <code>eml(x, y) = exp(x) − ln(y)</code> with a constant 1 generates all elementary functions: <code>exp</code>, <code>ln</code>, arithmetic, negation, complex-plane access, trigonometry. Verified numerically in Python, verified in Lumbda's own bytecode, proven formally in Lean 4 with zero <code>sorry</code>. Lumbda's native symbolic-rewrite checker runs five theorems in 46 ms cold or 7 ms cached — roughly 16× faster than Lean's cold rebuild on identical hardware.</p>
|
||
</section>
|
||
|
||
<section id="doc">
|
||
<h2>Whitepaper</h2>
|
||
<p>Full language reference, tier-by-tier architecture, benchmarks, meta-GC design, universality proof.</p>
|
||
<p>
|
||
<a class="cta" href="lumbda-whitepaper.html">Read in browser (HTML)</a>
|
||
<a class="cta" href="lumbda-whitepaper.pdf">Download PDF (~2.7 MB)</a>
|
||
</p>
|
||
</section>
|
||
|
||
<section id="license">
|
||
<h2>License</h2>
|
||
<p>AGPL-3.0-only. Public domain for whitepapers, proofs, and disclosures through <a href="https://undefect.com">undefect.com</a>. Companion to <a href="https://unturf.com">unturf.com</a>’s permacomputer project.</p>
|
||
</section>
|
||
|
||
</main>
|
||
|
||
<footer>
|
||
<p>
|
||
<a href="https://git.unturf.com/engineering/unturf/lumbda">source</a>
|
||
·
|
||
<a href="lumbda-whitepaper.html">whitepaper (HTML)</a>
|
||
·
|
||
<a href="lumbda-whitepaper.pdf">whitepaper (PDF)</a>
|
||
·
|
||
<a href="https://unturf.com">unturf.com</a>
|
||
</p>
|
||
</footer>
|
||
|
||
</body>
|
||
</html>
|