Commit graph

25 commits

Author SHA1 Message Date
a95277cef4
c: fix buffer overflow in load_file on non-seekable inputs
Bash process substitution <(...) passes /proc/self/fd/N — a pipe, not
a regular file. load_file used fseek(SEEK_END)+ftell to size a single-
read buffer; on a pipe ftell returns -1, which casts to SIZE_MAX as
fread's nbyte argument and blows the heap. Glibc fortify caught it
as '*** buffer overflow detected ***'.

Detect non-seekable input via the fseek return code and fall back to
a doubling growable buffer instead. Seekable path unchanged.

Repro: ~/git/lumbda/c/lumbda <(echo '(display 1)(newline)')
2026-06-04 12:44:43 -04:00
03a3534f0e
c: retire TODO-named-let-bytecode.md
Resolved by 45a90b8 (vm restores cur_code across CALL/RETURN).
The reproducer at lines 5-11 of the TODO file runs cleanly now
under both `lumbda --fast` and `lumbda -j --fast`, and the
proof/eml_proof_in_lumbda.lsp `normalize` function no longer
needs the `(define (iter ...))` workaround that the TODO
documented. Future regressions are caught by
tests/regression-named-let-leak.{lsp,sh}.
2026-06-04 01:11:26 -04:00
45a90b84e9
c: vm restores cur_code across CALL/RETURN, fixing JIT named-let hang
Five-line fix that ends the F1 hang in foxhop.net
ecdsa/tests/unit/probe-c-confirm.lsp.

Symptom — under --fast, a defined function whose body is a
tail-recursive named-let that calls another user-defined function
per iteration loops forever at 100 percent CPU. Trace pins the
bytecode dispatch:

  walk1.body: PUSHE MKCLO DUP BIND LOOKUP TCALL ->loop
  loop:       LOOKUP NULL? JIF LOOKUP CALL ->always-true
  always-true: CONST RET (returns #t)
  always-true (!): JIF LOOKUP CDR STAIL -> ip=0 of always-true (!)
  loop forever

cur_code was the call-frame-local register holding the currently
executing CodeObj. OP_CALL updated it on entry but neither OP_RETURN
nor the builtin-fallback restore path in OP_TAIL_CALL put it back
on return. Subsequent OP_SELF_TAIL_CALL read cur_code->self_params
from the still-stale callee proc (NULL for always-true since it has
no named-let), guard skipped the env rebind, then set ip=0 — without
ever updating the loop variable. Loop variable stayed pinned at the
initial list and our walk never reached its base case.

Fix — VMFrame gains a cur_code field. Three frame-push sites save
it on entry (OP_CALL, OP_TAIL_CALL builtin fallback frame-restore,
OP_CALL_CC compiled-proc entry); two frame-pop sites restore it on
return (OP_RETURN, OP_TAIL_CALL builtin fallback).

Verified inside foxhop.net's ecdsa QEMU guest:

- foxhop.net/ecdsa/tests/unit/probe-c-confirm.lsp F1..F4 — all pass
- foxhop.net/ecdsa/tests/unit/test-sim.lsp under --fast — 21/21 pass
- foxhop.net/ecdsa/lumbda/main.lsp under --fast — 6 shots, score 18,
  byte-identical with our Python tier
- make functional-test — 205/205 on Python and C tiers
- make regression-named-let-leak — 4/4 across Python, tree-walker,
  and --fast JIT

c/TODO-named-let-bytecode.md can stop applying its `(define (iter ...))`
workaround once this lands.
2026-06-04 01:11:18 -04:00
88c4b05032
c: enable Boehm GC by default, GC_INIT, file ops, regression test
Three coupled changes that unblock the ecdsa quantum-circuit
simulator's run on the C tier from neoblanka.

1. c/Makefile autodetects libgc-dev — if /usr/include/gc.h is
   present, the build links Boehm and defines USE_BOEHM_GC. Without
   GC, ul_free is a no-op (lumbda.h:35) and every allocation leaks;
   small REPL snippets work but workloads with thousands of envs
   OOM the process. Override with USE_GC=0 to force the malloc-only
   path for diagnostics.

2. c/main.c calls GC_INIT before init_symbols, then GC_disable.
   GC_INIT registers the stack base for conservative scan — without
   it some Linux configs miss roots. GC_disable is a deliberate
   stopgap: lumbda Values are NaN-boxed pointers that conservative
   Boehm cannot recognize as pointers, so live targets get reclaimed
   (env binding symbol payloads, SymbolEntry strings) and lookups
   fail with "undefined: <sym>". Reproducing this without GC_disable
   on the GC build: any sim.lsp call chain triggers the corruption
   after ~100 named-let iterations. Until tracing is precise,
   growing the heap is safer than wrong results. Long-running
   workloads run under ulimit -v.

3. c/builtins.c gains rename-file and delete-file matching the
   Python tier (lumbda.py:3468). sim.lsp's write-portal! pattern
   (write to .tmp, rename) needs rename-file to land cross-tier
   identical results.

4. tests/regression-named-let-leak.lsp + .sh pin four shapes that
   blew up ecdsa: the c/TODO-named-let-bytecode.md repro, the F1
   shape from foxhop.net's lumbda-c-tier-leak-SP.md (12-line
   minimum), a 200-iter scaled variant, and a sim.lsp run-ops!
   mirror. Wired into root Makefile as regression-named-let-leak;
   added to test-all. Wrapper caps memory at 256 MB virt and 15s
   per tier so a leak regression fails the run instead of consuming
   host RAM.

Known limits:
- --fast JIT still has the named-let + inner user-fn call hang
  (separate TODO; tree-walker handles this fine).
- GC_disable means the heap grows; workloads must bound their work
  budget. ecdsa's sim runs comfortably in 5 MB.

Verified inside a 2G/2vCPU QEMU guest (foxhop.net ecdsa/vm-runner.sh):
- test-c (tree-walker) — 35/35 PASS
- bench-c (tree-walker) — score 18 matches Python tier byte-identical
- F1 probe (tree-walker) — all four steps PASS
2026-06-04 00:55:31 -04:00
192118388f cl-compat: run Zoë Trout's favorites unchanged (ticket 0004)
Zoë Trout's favorites at wedgewack.org/ursa.lisp.txt are Common Lisp:
iterative LOOP macros, setf cascades, defun with &optional, image-
based stone-lisp culture. Her first contribution to lumbda was a
question — "do we care for our programs, and how long are they alive
for?" — and the answer now extends beyond the RNG portal (§7.5) to
iteration style itself.

Four-phase delivery, all under ticket 0004:

  Phase A — idiomatic Scheme ports at examples/ursa-scheme.lsp.
    Every Zoë defun rewritten as named-let + tail recursion + list-
    backed work queue + type-predicate dispatch.

  Phase B — CL compat shim at cl-compat.lsp.
    defun (with &optional), setf (simple vars, multi-pair), flet,
    multiple-value-bind, t / nil (nil=#f so cond/if compose),
    evenp/oddp/plusp/minusp/zerop, mod/ash/logbitp/nreverse,
    cl-when/cl-unless (plain when is a void-returning lumbda special
    form), declare (no-op), cddddr (missing accessor).

  Phase C — cl-loop macro covering 14 patterns.
    while/until/repeat, for VAR from A to/below/downto B, for VAR =
    INIT [then STEP], for VAR across VEC, of-type T, do, when/unless
    return, finally (return VAL). Sequential do*-style stepping via
    gensym + cl-subst. Look-ahead termination so `repeat 4 for s = 4
    then (- (* s s) 2) finally (return s)` returns 37634 (pre-step)
    rather than 1416317954 (post-step). Every expansion ends in a
    named-let tail call — TCO holds for loops of any length.

  Phase D — load examples/ursa.lisp.txt with minimal annotation.
    Preserves Zoë's CL. Minimal edits documented in file header:
    load cl-compat.lsp, loop→cl-loop, when→cl-when, random→random-int,
    &key→&optional. rho/digits omitted (need make-array/CLOS — see
    ticket 0004 for scope boundary).

Defect uncovered along the way (c/types.c env_lookup): a "global
shortcut" checked global env immediately after missing the local
frame, SKIPPING intermediate parent scopes. Broke lexical scoping
whenever a parent scope shadowed a global. Reproduced with
  (define s 4)
  (let ((s 100)) (let ((m 0)) s))  ; returned 4, should return 100
Any nested let whose body referenced a shadowed name silently read
the global. Fix: remove the shortcut, walk the parent chain end-to-
end. 1255 assertions across five suites pass unchanged after fix —
surfaced only because cl-loop iterator names routinely collide with
globals accumulated in a stone-lisp image.

Whitepaper §9.2 documents the CL-in-Scheme design and the guarantees
that survive (TCO, portal determinism, cross-impl reproducibility).
Zoë added to authors + acknowledgments; reacknowledgment reframes
her first contribution as the deeper program-lifetime question, with
RNG portal as a derivative (§7.5) and cl-loop as the follow-up.

Tests: tests/cl-compat.lsp (44 assertions) and tests/ursa.lsp (28
assertions) exercise both paths under Python + C via tests/zoe-
favorites-test.sh, wired into make test-all.

MOAD notes: unmoad flags memq/assq in cl-compat.lsp over cl-loop-
keywords (~30 elements, constant) and var->new (≤4 state vars per
loop). Both are macro-expansion-time, bounded-small-N — not runtime
hot paths. Pre-existing c/types.c findings (strcmp-in-loop for
record-type lookup) are not from this change.
2026-04-24 07:02:21 -04:00
4960381c67 portal-rng: add (random-seed-from-os!) across all three tiers
Ticket 0002 — reads 8 bytes from /dev/urandom (little-endian u64) and
seeds xoshiro256**. Opt-in kernel entropy for stochastic runs; the
default stays deterministic (k=0 at startup), so ticket 0001's
portal-reproducibility contract is unchanged.

Real-world flow now one call away:
  Machine A: (random-seed-from-os!) + run simulation + portal-save
  Machine B: portal-resume — same stream, bit-for-bit

All three impls fail loud on /dev/urandom trouble (LispErr in Python
and C, stderr + exit(1) in asm) — no silent fallback to a weak seed.

Tests:
- tests/functional.lsp: 2 new shared asserts (entropic + replay)
- asm/test.sh: 2 new asm-local checks (149 total, was 147)
- make test-all green across Python (205), C (205), asm (149)

Whitepaper §7.5 gains one sentence noting the OS-seed path.
unmoad: zero new findings in added code.
2026-04-20 15:50:26 -04:00
54c4c651bb portal-rng: asm xoshiro256** + cross-impl tests + default seed=0
Completes ticket 0001 started in 27f468c. All three impls now carry
bit-identical xoshiro256**; portal state round-trips across process
boundaries in every producer x consumer cell (Python <-> C <-> asm).

asm impl:
- 4 new builtins: random-seed!, random-int, random-state, random-state!
- g_rng_state in BSS (4 x u64); rng_splitmix64_step, rng_seed, rng_next
- Binary portal header bumped LUMBDAB1/48 -> LUMBDAB2/80; carries
  32 bytes of rng state at offsets 40..64, reserved moved to 72
- No float support in asm, so (random) intentionally omitted there
- _start seeds with 0 so the stream is deterministic from startup

Python + C (supplements 27f468c):
- rng_seed(0) auto-invoked at module load / register_portal_builtins
  so (random) without explicit (random-seed!) returns a real value
  instead of the all-zero xoshiro fixed point

Tests:
- tests/functional.lsp: 7 new shared assertions (Python + C)
- asm/test.sh: 5 new asm-local assertions (142 -> 147)
- tests/portal-rng-save.lsp / portal-rng-load.lsp: portable S-expression
  portal that captures both state AND next-5 baseline so loader self-
  verifies without a separate harness
- tests/portal-cross-test.sh: 9 new producer x consumer RNG cells; all
  18 cells pass end-to-end

Verified: seed=42, (random-int 1000000) draws 1..10 =
558742 543102 559009 124193 317476 750584 200754 814407 344958 929085
identical in Python, C, and asm.

unmoad scan: zero new findings in added code.
2026-04-20 11:11:47 -04:00
27f468c5b7 portal-rng: Python + C impls of xoshiro256** + portal state capture
Zoe's contribution question: does our portal preserve RNG state so a
simulation can continue in another process with the same random stream?
Answer today: no — no RNG existed. Answer now (Python + C): yes, bit-identical.

- New builtins: random-seed!, random, random-int, random-state, random-state!
- xoshiro256** (Blackman & Vigna 2018) — deterministic, portable, no libc rand
- State = 4 x u64; portal-v1 JSON gains 'rng' field with 8 x u32 halves
- Python and C produce bit-identical streams (verified: seed=42, 10 draws)
- Asm impl + cross-impl tests + whitepaper note: next commits

Ticket: docs/tickets/0001-portal-rng.md
2026-04-20 10:57:39 -04:00
6e9d3ea52f isqrt: add integer square root builtin to all three impls
Semantics: (isqrt n) → floor(sqrt(n)). Negative argument errors.
Matches Python 3.8+ math.isqrt and R7RS exact-integer-sqrt contract.

- Python: wraps math.isqrt via lambda registration
- C: hand-rolled bit-by-bit algorithm in bi_isqrt (O(log n), no FPU)
- asm: new BI_ISQRT=109, bit-by-bit algorithm in integer registers
       (%r8/%r9/%r10). Negative input → stderr + exit(1) like other
       errors. GC builtin constants bumped to 110-114.

Tests:
- tests/functional.lsp: 7 shared tests (0, 1, perfect squares, floor
  cases, large values). Python + C now 196 each (was 189).
- asm/test.sh: 5 asm-local tests. asm suite now 142 (was 137).

MOAD: all three implementations O(log n), no O(N²) hazards.
2026-04-20 08:50:24 -04:00
f7352b51b0 rename: uncommonlisp -> lumbda throughout the repo
Historical internal name "uncommonlisp" retired in favor of the
public name "lumbda" ahead of lumbda.com going live. Scope of
this commit:

Source files renamed:
  uncommonlisp.py                     -> lumbda.py
  asm/uncommonlisp.s                  -> asm/lumbda.s
  c/uncommonlisp.h                    -> c/lumbda.h
  whitepaper/uncommonlisp-whitepaper  -> whitepaper/lumbda-whitepaper (.rst + .pdf)

Binaries renamed (tracked ones; c/ was always gitignored):
  asm/uncommonlisp, asm/uncommonlisp-gc, asm/uncommonlisp.o,
  asm/uncommonlisp-gc.o                -> asm/lumbda(-gc)(.o)
  c/.gitignore                          -> ignores lumbda

Internal string updates (sed pass ordered longest-first):
  asm/uncommonlisp -> asm/lumbda
  c/uncommonlisp   -> c/lumbda
  uncommonlisp.py  -> lumbda.py
  UNCOMMONLISP_BIN -> LUMBDA_BIN (asm/test.sh env var)
  "uncommonlisp> " -> "lumbda> " (asm REPL prompt baked into binary)
  UNCOMMONLISP     -> LUMBDA (macros, comments)
  uncommonlisp     -> lumbda (prose)

Binary portal magic updated:
  "ULPORTAL" -> "LUMBDAB1"   # "Lumbda Binary v1"
Old portal files are not backward-compatible — this is a deliberate
break since it's the rename moment. S-expression portals already
carry their own ";; lumbda-portal v1" header and remain cleanly
versioned.

WHITEPAPER.pdf / WHITEPAPER.rst symlinks repointed to the renamed
files. Makefile's whitepaper target targets lumbda-whitepaper.pdf.

Not changed (intentional, separate phases):
  - Filesystem directory /home/fox/git/uncommonlisp itself
    (fox renames locally and the gitlab repo URL in a follow-up)
  - tests.py hardcoded cwd=/home/fox/git/uncommonlisp
    (matches the current on-disk location; will flip when the
    directory rename ships)
  - Git history (immutable; old commits still say uncommonlisp,
    which is correct — that's what they were)

Verified:
  137 asm no-GC + 137 asm GC + 571 Python + 83 C + 189 shared
  functional tests all pass under the new names.
  bench-gc-http (2000 req): all 4 cells behave as expected
  (cells 1/2 flat, 3 leaks, 4 bounded at 1 chunk).
  Python REPL, C REPL, asm REPL all start cleanly.
2026-04-19 10:20:11 -04:00
3463fadd3f C --fast named-let bug: minimal repro + workaround, all 4 tiers pass now
Hunted the C --fast compiler bug that was hanging on the EML proof.
Narrowed to a specific pattern:

  (let loop ((t start))
    (let ((next (fn t)))
      (if next (loop next) t)))

A named-let whose body is (let ((x (...))) (if x (recurse x) base)).
The recursive call inside the inner let+if branch never reaches the
loop closure — hangs or segfaults.

Reproducible with a 4-line test case; filed as
c/TODO-named-let-bytecode.md with minimal repro, suspected cause
(env-chain mismatch between PUSH_ENV and TAIL_CALL), and a known-
good workaround.

Workaround landed in proof/eml_proof_in_lumbda.lsp's `normalize`:
replaced the named-let with an internal recursive `define`, which
compiles correctly under --fast. Same logic, different surface
syntax. All four Lumbda tiers now verify the proof.

Benchmark refreshed (make bench-proof):

                              cold     cached
  Lumbda asm                   46 ms    7 ms
  Lumbda C --fast              65 ms    9 ms
  Lumbda C (tree-walker)       87 ms   12 ms
  Lumbda Python --fast        651 ms  232 ms
  Lean 4                      722 ms    5 ms

All four tiers now green. Asm still fastest (46 ms cold vs Lean's
722 ms — ~16× faster). Cached Lumbda asm 7 ms vs Lean 5 ms (within
1.5×). The C --fast tier went from "hangs" to 65 ms cold — competitive
with asm once the compiler bug is dodged.

Whitepaper §8.6 table updated; prior "(hangs)" row is gone;
footnote on the named-let workaround links the TODO file.
2026-04-17 20:56:00 -04:00
027017f01d C: --fast documented for deep recursion; whitepaper numbers corrected
ack(3,8) was reported as "segfault" for the C impl in the previous
whitepaper revision. That was a stale observation — C has --fast
(bytecode VM with explicit frame stack) that handles deep recursion
cleanly. The benchmark table compared the wrong modes.

Corrected apples-to-apples:
- Python --fast (bytecode VM) — 17,004 ms on ack(3,8)
- C --fast      (bytecode VM) — 1,433 ms  **fastest of the three**
- asm native    (tree-walker) — 2,322 ms

C's --fast wins every workload. asm still beats Python --fast by
~7x despite being a tree-walker, because it skips Python's per-op
overhead entirely.

c/main.c: --help text updated to clarify that --fast is required
(or `ulimit -s unlimited`) for deep recursion in the default
tree-walker mode. Attempted flipping --fast to default; reverted
because that surfaced a cumulative-state buffer overflow in the
bytecode compiler that only triggers after the full 189-test
functional suite but not on isolated scripts. Left as a TODO in
the code comment. 189 C tests + full test-all still pass.

Whitepaper §6.4 table now shows all three impls in their
high-performance configuration. Also noted that a pthread-with-
larger-stack wrapper would let the C tree-walker handle deep
recursion without --fast — tracked as low-priority future work
since --fast is strictly faster regardless.
2026-04-17 18:54:06 -04:00
06b93c588a portal over HTTP: 9/9 cross-runtime, plus eval-to-global-env fix
Closes the last loop promised in the whitepaper's Future Work: a
node serves its state as an S-expression portal over HTTP, another
node pulls it down with tcp-connect + tcp-recv and materializes the
bindings locally via (eval (read-from-string line)).

examples/portal-http-server.lsp (90 lines):
- Holds some state (counter, my-int, my-list, my-fib, my-str)
- GET /portal → S-expression body: a sequence of (define ...) forms
- GET / → HTML index
- Uses heap-snapshot / heap-restore for O(1) memory on asm

examples/portal-http-client.lsp (90 lines):
- tcp-connect, send HTTP/1.0 GET, receive full response
- Strip headers (walk to first \r\n\r\n)
- Split body by \n, eval each non-empty, non-comment line
- The remote bindings are now live locally

3×3 server/client matrix: all 9 combinations green. Every runtime
hosts, every runtime consumes. The wire format is Scheme source;
no schema, no JSON, no Protobuf.

Prerequisite fix: `eval` semantics aligned across all three impls.

Python and C's `eval` special form previously evaluated its result
in the CALLER's env, so a nested (eval (read-from-string
"(define x 42)")) would install x in the local function scope —
invisible to later top-level code. asm's bi_eval always used the
global env (r14). With this commit, all three impls evaluate the
eval'd result in the global env, matching asm's existing behavior.

Python: uncommonlisp.py leval eval-handler now does `env = env.g`
before continuing the trampoline.
C: c/eval.c SYM_EVAL branch now does `env = env->global`.
asm: no change (already correct).

One pre-existing Python defect surfaced by the client:
`count` is a SRFI-1-style builtin (`d(S('count'), ...)`), so a
local let-loop variable named `count` collides with it in the
inline-cache lookup path and OP_LOOK_ADD1 fires on the builtin
instead of the local. Worked around by renaming the loop
accumulator to `cnt`. Underlying Env.lookup shortcut-to-global
issue is out of scope for this commit.

Regression: 975 tests still green.
2026-04-17 13:41:48 -04:00
574ddc50d7 S-expressions over sockets — RPC + remote REPL in portable Scheme
Fuses portal (Scheme-source-as-interchange) with sockets (bytes over
the network). Wire protocol: one S-expression per connection. Same
server + client .lsp runs byte-identically in Python, C, and asm.

New primitives in all three impls:
- read-from-string — parse one sexp from a string

Asm gets two more:
- symbol->string — standard R7RS, was missing
- eval — evaluate a Scheme value in the global env (Python + C had
  it as a special form; asm exposes it as a builtin)

examples/rpc-server.lsp (port 9080):
- Whitelisted dispatch: ping / add / mul / fib / echo
- Never calls eval on client input; safe by construction
- Uses heap-snapshot/restore for O(1) memory on asm
- ~90 lines, portable

examples/rpc-client.lsp:
- Sends one request, reads one response, displays both
- 45 lines, portable

examples/repl-server.lsp (port 9081):
- DANGER: full remote eval. Any Scheme form accepted and evaluated
  in the server's global env. Persistent across connections.
- Deliberately does NOT use heap-snapshot — remote (define x ...)
  lives in the global env above any snapshot point; rewinding would
  invalidate the new binding. The ulimit -v 512 MB safety net
  (documented in CLAUDE.md) ensures an escaped process can't crash
  the machine.
- ~70 lines, portable. Demonstrates what "the language IS the
  interchange format" gets you at the limit: a single socket and
  a single primitive (eval) carry a full-powered REPL.

Verified 3×3 server×client matrix: all 9 combinations green.
All 132 asm + 571 py + 189 shared + 83 c tests still pass.

One quirk discovered and worked around: in asm, a closure captures
its env chain by pointer at define time. Forward-referenced names
in mutually-recursive toplevel defines can fail under specific
heap-restore patterns — see the leaf-first ordering note in
rpc-server.lsp.
2026-04-17 09:00:04 -04:00
b8d6afdeb3 heap-snapshot + native HTTP client + MOAD-0001 cleanup
Three wins in one commit.

1) heap-snapshot / heap-restore (asm arena primitive)
   asm has no GC. Long-running servers leaked ~64 MB per heap growth.
   Two new builtins let a programmer capture r15 and later rewind to
   it, recycling intermediate allocations in O(1) memory.
   Python + C get no-op versions so portable .lsp code can call them
   unconditionally.

   examples/http-server.lsp now takes a snapshot at top level and
   rewinds after every request. Measured asm RSS: 88 KB initial,
   100 KB after 100 requests, 100 KB after 1100 requests — flat.
   Prior behavior was +64 MB per few thousand requests.

2) examples/http-client-bench.lsp — native HTTP load generator
   Uses only the six tcp-* primitives + current-time-ms. Runs
   identically in all three impls. Eliminates curl's ~2 ms/req
   fork+exec overhead, so real server throughput shows up:

     Python server ← Python client   2403 rps
     C      server ← C      client   2439 rps
     asm    server ← asm    client   2994 rps
     asm    server ← C      client   2500 rps

   The earlier curl-based bench was clamped near 400 rps by the
   client; the actual servers handle 6–7× that.

3) MOAD-0001 cleanup
   - c/builtins.c bi_string_replace: strncmp-at-every-position
     (hand-rolled, sedimentary) → strstr (libc-tuned, typically
     Boyer-Moore-Horspool). O(N*k) → O(N + matches*k).
   - uncommonlisp.py _tokenize_lines: per-token src.count('\n', 0, pos)
     → precompute line_starts once, bisect_right per token.
     O(N*M) → O(M + N log M).

Also adds current-time-ms to all three impls so benchmarks can
time themselves without relying on the Python/C float `current-time`
(asm has no floats). Seconds-since-epoch tagged as a 61-bit int.

Test counts unchanged: 571 py + 132 asm + 189 shared + 83 c = 975.
All green via make test-all.
2026-04-16 20:49:32 -04:00
bfd4ec7ec8 sockets + portable HTTP server — 6 primitives, same server runs in all 3
Added tcp-listen/accept/connect/recv/send/close to Python, C, and asm.
One examples/http-server.lsp runs identically in all three impls and
serves HTTP/1.0 with routing, content-type, and content-length headers.

asm additions:
- SYS_SOCKET/BIND/LISTEN/ACCEPT/CONNECT/SETSOCKOPT syscalls
- 6 tcp-* builtins using the existing port encoding (SPECIAL ≥ 1000)
- bi_tcp_connect: dotted-quad IPv4 parser, no DNS dependency

Defects fixed along the way (surfaced by the HTTP server):
- string-append: was 2-arg only; now variadic (walks arg list twice)
- number->string: was stubbed to VAL_VOID; now correctly writes digits
  into a heap-allocated string (incl. negative handling)
- String-literal reader: \r and \0 escape sequences now handled (was
  silently dropping backslash, treating them as literal 'r' / '0')
- tcp_accept: sockaddr buffer was 8 bytes, now 16 (was corrupting
  caller's stack when accept wrote full struct sockaddr_in)

Pinocchio benchmark (tests/web-benchmark.sh):
At concurrency=20, 1000 requests, serving a 1KB body:

  uncommonlisp Python   373 req/s
  uncommonlisp C        370 req/s
  uncommonlisp asm      370 req/s
  python3 http.server   381 req/s  (stdlib reference)
  busybox httpd         382 req/s  (production reference)

All five converge within 3% — the client (curl fork/exec) is the
bottleneck, not the server. Our single-threaded blocking servers
are indistinguishable from battle-tested ones at this load.

Binary sizes:
  uncommonlisp asm    45 KB   (HTTP + everything else)
  busybox httpd       2.1 MB  (multi-call binary)
  python3             8 MB    (interpreter)

The asm HTTP server is 46× smaller than busybox and 176× smaller
than Python, serves from 7 Linux syscalls, and the entire protocol
handler is 70 lines of portable Scheme.

Test counts: 132 asm (up 1), rest unchanged. All green.
2026-04-16 18:58:27 -04:00
57f3c9fab1 asm/c/py: add (load), ports, write-file/file->string — full cross-impl parity
Asm gains the file I/O surface Python and C already had, unlocking
9/9 cells of the portal producer×consumer matrix (previously 6/9).

asm:
- (load "path") — mmaps file, swaps input source, loops scheme_read+eval,
  restores on exit. Nestable. Uses SYS_LSEEK + SYS_MUNMAP.
- Output ports: (open-output-file), (close-port), (port?). Encoded as
  SPECIAL values ≥ 1000 (fd = (val>>3) − PORT_SPECIAL_BASE), no tag-bit
  expansion needed.
- (display), (write), (newline) accept optional port arg; printer
  writes via output_fd global, swapped by port-aware builtins.
- (write-file path content) / (file->string path) — bytes in/out.

c, py: (write-file) / (file->string) added for parity.

tests: 131 asm (up 23), 189 functional (up 8, shared py+c),
tests/portal-cross-test.sh exercises 3×3 save×load matrix.
2026-04-16 16:37:40 -04:00
610c93e8c8 C: add full continuations, portal save/resume, 7 new tests
Full continuations: FullCont captures frames/stack/env with deep copy.
VM trampoline via setjmp/longjmp. Multi-shot safe via deep_copy_env.

Portal (new c/portal.c): serialize env + continuation to JSON,
resume on another machine. portal-checkpoint! triggers mid-VM save.

83/83 C unit tests + 181/181 functional tests pass.
2026-04-15 20:07:09 -04:00
30d7279be2 C: add deep_copy_env, VM frame stack for continuations
asm: fix builtin dispatch, improve apply_proc_raw

C changes: deep_copy_env() for multi-shot continuations,
explicit frame stack in VM for compiled code call/cc support.

asm changes: improved builtin implementations, fixed dispatch paths.

All tests pass: asm 75, C 76+114 functional.
2026-04-15 19:57:17 -04:00
22571fa470 Fix MOAD-0001 defects across all implementations
asm/uncommonlisp.s — intern_symbol: replaced O(N) linear scan with
djb2 hash table (1024 buckets, chaining). 2.9x faster symbol interning
on programs with many symbols. 75 tests pass.

uncommonlisp.py — _define_record_type: replaced list.index() O(N)
with dict lookup O(1) for field→index mapping. 571 tests pass.

MOAD-0002 documented: _portal_checkpoint, _call_stack, _auto_compile
are intentional globals (hot loop performance). cc_escape_val/cc_active_jmp
are required by setjmp/longjmp call/cc approach. Comments added.

All 836 assertions pass across Python + C + Assembly + functional.
2026-04-15 14:29:58 -04:00
d373f80aaf JIT: add named-let loops, let/let*, and/or, car/cdr/cons, 18 new tests
JIT now covers: if, cond, and, or, let, let*, named-let (native loops),
car, cdr, cons, null?, pair?, arithmetic, comparisons, recursion, TCO.
1309 lines of x86_64 codegen. 76 C tests + 114 functional tests pass.

Named-let loops compile to native jmp (zero call overhead):
  sum-to(50k): 0.33ms JIT vs 7.8ms CPython (24x faster than Python)
  ack(3,4):    0.20ms JIT vs 2.0ms CPython (10x faster)
  fib-rec(20): 0.42ms JIT vs 2.7ms CPython (6x faster)

EML benchmark added: integer-domain exp/ln composition under JIT.
2026-04-14 20:50:24 -04:00
c80eabac47 x86_64 JIT: 12-21x faster than CPython, 230x faster than interpreter
Real native machine code via mmap(PROT_EXEC). No exec(). No strings.
Raw x86_64 bytes: mov, add, sub, imul, cmp, je, jne, call, ret, jmp.

ack(3,4):    0.12ms JIT vs 1.5ms CPython vs 28ms interpreter
fib-rec(20): 0.16ms JIT vs 3.4ms CPython vs 40ms interpreter

Added cond support to JIT (cascaded comparisons → conditional jumps).
Fixed JIT cache: sentinel value prevents retry on unjittable functions.
System V AMD64 ABI: args in rdi/rsi/rdx, callee-saved r12-r15.
Tail calls use jmp (true TCO at machine code level).

691 lines of jit.c. 114 functional tests pass. All C tests pass.
2026-04-14 19:58:26 -04:00
46812e1885 Add GPU architecture notes and JIT header
docs/gpu-architecture.md — roadmap for GPU lambda execution:
  Phase 1: map/reduce (CUDA thread per element)
  Phase 2: trampolining (recursive lambdas without stack)
  Phase 3: interaction combinators (Bend/HVM approach, 74K MIPS)

c/jit.h — x86_64 JIT header: JitBlock, JitFunc typedef,
  jit_compile/jit_free API. Uses mmap for executable memory.
  System V AMD64 ABI calling convention.

jit.c implementation in progress (x86 instruction encoding).
2026-04-14 19:43:16 -04:00
db2cd77c62 Add shared functional test suite: 114 tests, both implementations pass
tests/functional.lsp — single .lsp file, runs identically in Python and C.
Covers: arithmetic, comparison, booleans, pairs, lists, strings, characters,
vectors, hash tables, control flow, let/lambda/closures, do loops, define,
recursion, TCO (100k depth), quasiquote, macros, type predicates, call/cc,
error handling, mergesort, higher-order programs.

Fixed C call/cc: proper escape continuations via setjmp/longjmp.

make test-all runs: Python unit (571) + C unit (58) + shared functional (114).
2026-04-14 15:21:17 -04:00
fc9eb5350c Add C implementation: 7,429 lines, 58 tests, identical output
Complete C port of the Scheme interpreter. Same .lsp files run in
both Python and C with identical output.

Architecture:
- NaN-boxed 64-bit values (zero-alloc numbers)
- Hash-map environments with parent chain + global shortcut
- Interned symbols
- TCO via explicit loop (eval) and TAIL_CALL/SELF_TAIL_CALL (VM)
- Bytecode compiler with all opcodes including superinstructions
- 58 unit + integration tests

Makefile targets:
  make test-all    run Python (571) + C (58) tests
  make examples    run examples in both, compare output
  make friction    benchmark same .lsp in Python vs C
  make c-build     build C interpreter
  make c-test      run C tests
  make c-repl      C REPL
2026-04-14 14:55:17 -04:00