Commit graph

23 commits

Author SHA1 Message Date
f7352b51b0 rename: uncommonlisp -> lumbda throughout the repo
Historical internal name "uncommonlisp" retired in favor of the
public name "lumbda" ahead of lumbda.com going live. Scope of
this commit:

Source files renamed:
  uncommonlisp.py                     -> lumbda.py
  asm/uncommonlisp.s                  -> asm/lumbda.s
  c/uncommonlisp.h                    -> c/lumbda.h
  whitepaper/uncommonlisp-whitepaper  -> whitepaper/lumbda-whitepaper (.rst + .pdf)

Binaries renamed (tracked ones; c/ was always gitignored):
  asm/uncommonlisp, asm/uncommonlisp-gc, asm/uncommonlisp.o,
  asm/uncommonlisp-gc.o                -> asm/lumbda(-gc)(.o)
  c/.gitignore                          -> ignores lumbda

Internal string updates (sed pass ordered longest-first):
  asm/uncommonlisp -> asm/lumbda
  c/uncommonlisp   -> c/lumbda
  uncommonlisp.py  -> lumbda.py
  UNCOMMONLISP_BIN -> LUMBDA_BIN (asm/test.sh env var)
  "uncommonlisp> " -> "lumbda> " (asm REPL prompt baked into binary)
  UNCOMMONLISP     -> LUMBDA (macros, comments)
  uncommonlisp     -> lumbda (prose)

Binary portal magic updated:
  "ULPORTAL" -> "LUMBDAB1"   # "Lumbda Binary v1"
Old portal files are not backward-compatible — this is a deliberate
break since it's the rename moment. S-expression portals already
carry their own ";; lumbda-portal v1" header and remain cleanly
versioned.

WHITEPAPER.pdf / WHITEPAPER.rst symlinks repointed to the renamed
files. Makefile's whitepaper target targets lumbda-whitepaper.pdf.

Not changed (intentional, separate phases):
  - Filesystem directory /home/fox/git/uncommonlisp itself
    (fox renames locally and the gitlab repo URL in a follow-up)
  - tests.py hardcoded cwd=/home/fox/git/uncommonlisp
    (matches the current on-disk location; will flip when the
    directory rename ships)
  - Git history (immutable; old commits still say uncommonlisp,
    which is correct — that's what they were)

Verified:
  137 asm no-GC + 137 asm GC + 571 Python + 83 C + 189 shared
  functional tests all pass under the new names.
  bench-gc-http (2000 req): all 4 cells behave as expected
  (cells 1/2 flat, 3 leaks, 4 bounded at 1 chunk).
  Python REPL, C REPL, asm REPL all start cleanly.
2026-04-19 10:20:11 -04:00
9e54c7aaf1 asm-gc portal v1: explicit version header + resume-side version check
GC-build portal files now start with ";; lumbda-portal v1\n". The
line is a Scheme comment the reader already skips, so loading a
v1 file via bi_load works unchanged. What's new is that
portal-resume actively validates the header before delegating to
bi_load:

  file starts with ";; lumbda-portal v1\n"   -> load normally
  file starts with ";;" but different text   -> return #f (rejected)
  file does not start with ";;" at all       -> load as legacy (back-compat)

The check reads the first 32 bytes of the file, compares the
first two bytes against ";;", and on match compares the full
20-byte v1 prefix. Closes the versioning-friction concern a SEW
reviewer raised after reading §7.4.1: we can now add a v2 format
with new syntax (complex numbers, records, whatever) without
older consumers silently parsing new files into garbage — they
will cleanly return #f.

Verified:
  v1 portal -> resume loads all bindings, returns #<void>
  v2 portal -> resume returns #f without evaluating any forms
  legacy portal (no ;; header) -> resume loads normally
  missing file -> resume returns #f

The deeper framing worth writing down: this is a migration format
for handoff across process / tier / machine, not an archive format.
If archival becomes a real use case it earns its own format with
proper schema evolution and a builtin-rename table. The v1 tag is
the minimum hook that lets v2 happen cleanly when someone needs it.

137 asm no-GC + 137 asm GC still pass. §6.6.4 HTTP cells still
green at 10K: GC + no-snapshot at 543 req/s, peak 1.1 MB, 972 KB
growth (one chunk, steady state).
2026-04-19 09:43:15 -04:00
a606b6087e asm-gc: movb-not-orq type patch (kills residual "unbound variable")
Root-cause fix for the residual crashes I had documented as known
issues in §6.6.4. Every heap_alloc call site was setting its type
byte with `orq $(HT_X << 8), -8(%rax)` — but OR merges with the
stale type byte from a free-list-reused block. A pair previously
used as a vector (type 5 = 0b101) re-allocated as pair (type 1 =
0b001) ends up with merged type 0b101 = still vector. Walker then
treats the pair as a vector, reads the pair's car as a "length",
and walks off the block end — hence the hash-set bench's "unbound
variable: t", memory bench's "unbound variable: lst", arena bench's
"unbound variable: k".

Fix: overwrite the byte instead of OR-ing. 18 sites converted from
`orq $(HT_X << 8), -8(%rax)` to `movb $HT_X, -7(%rax)`. Every
previously-residual crash gone on first rerun.

Refreshed benchmark numbers throughout §6.6:

§6.6 Memory table: 122× less memory at 26% slowdown (was 124×,
30%). Range shifted because the fix also accelerated the common
paths; ratio stable.

§6.6.4 HTTP soak at 50,000 requests × 16 concurrent × 4 cells:
  no-GC + snapshot       630 req/s   peak 100 KB     growth 4 KB
  GC    + snapshot       633 req/s   peak 120 KB     growth 4 KB
  no-GC + no snapshot    625 req/s   peak 458 MB     OOM at cap
  GC    + no snapshot    610 req/s   peak 1,092 KB   growth 852 KB

Cell 4 now sustains 50K requests with steady-state 1-chunk memory.
Previous residual edge at 50K (cell 4 failing to start) was a
manifestation of the same type-byte bug, now gone.

§6.6.3 Adaptive numbers collapsed to within ~1% across all three
workloads (was 6% / 7% / 17% deltas). Paper updated to honestly
report adaptive as a null experiment on these shapes — neutral
cost, same stats surface, default on.

§6.6 diagram: bench-gc.png refreshed to match new numbers.

137 asm no-GC + 137 asm GC + 189 shared functional all pass.
Hash-set / memory / arena / adaptive / HTTP benches all clean.
2026-04-18 20:28:31 -04:00
58025a37bc asm-gc Fix 2: S-expression portal for the GC build
Binary heap dump can't work under GC because the heap is a linked
chunk list with typed block headers and a free list. Raw-byte
serialization would lose structure. Rather than invent portal v2
with chunk tables and pointer relocation, the GC build uses the
S-expression format that already works across all other tiers:

  # bi_portal_save in GC build:
  walk %r14 (env chain); for each non-builtin, non-closure binding,
  emit `(define <sym> (quote <val>))` to the opened file via
  scheme_print with output_fd redirected to that fd.

  # bi_portal_resume in GC build:
  jmp bi_load — read every form from the file, eval each in %r14.

The quote wrapper makes data values round-trip cleanly: lists,
vectors, strings, symbols, numbers, pairs all re-read as literals.
Closures and builtins are explicitly skipped — closures can't
faithfully re-read from their printed form; builtins reconstruct
from the target's prelude. Same treatment the JSON portal gives.

The no-GC build keeps the binary portal format unchanged (wrapped
in .ifndef GC_NAIVE). Users get the fast format on the fast build,
the portable format on the safe build. Same API, different wire
format by build.

Cross-tier verified: GC-asm producer -> Python consumer passes
with `x=42`, `nums=(1 2 3 4 5)`. The §7.2 cross-impl matrix
expands from 9 to 16 cells, all green.

Whitepaper §7.4 now notes it's the no-GC format; new §7.4.1
documents the GC build's S-expression portal with the trade-off
(slower than binary dump, stricter about what round-trips, but no
architecture constraint and no "same binary" requirement).

137 asm no-GC + 137 asm GC + 189 shared functional tests pass.
All four HTTP cells from §6.6.4 still bounded under sustained
load (GC + no-snapshot at ~630 req/s peak, 1.1 MB steady state).
2026-04-18 19:41:52 -04:00
5ec9eff5fe asm-gc Fix 1: precise block typing kills conservative-scan class of bugs
Replaces header format from [size:63 | mark:1] with
[size:48 | type:8 | flags:8 (mark in bit 0)]. Every heap_alloc
call site in the GC build now sets its type byte via one extra
`orq $(HT_X << 8), -8(%rax)` after return. Ten types defined:
HT_PAIR, HT_CLOSURE, HT_STRING, HT_SYMBOL, HT_VECTOR,
HT_HASHTABLE, HT_HASHSET, HT_ENVNODE, HT_CHAINNODE, HT_PADDING.

The mark / sweep / arena-escape walkers now dispatch on the
type byte instead of heuristically guessing from block size.
Deletes the special-case "negative sentinel at offset 0" branch
in gc_mark_drain (hash-table vs hash-set vs vector discrimination
was encoded there), the "size == 24 and TAG_SYM at offset 0"
check in gc_mark_env, and the "length fits block" sanity check
in the vector walker. All that logic collapses into a single
compare on the type byte.

Also routed the remaining direct-%r15-bump allocators
(bi_strref, bi_vector, bi_makevec, bi_listtovec, bi_substr)
through heap_alloc so they get proper headers + type bytes.
These had been silently broken under the GC build because they
bypassed the header-emitting path entirely; any direct-bump'd
data appeared to the sweep walker as garbage headers.

§6.6.4 cell 4 (asm GC + no snapshot) was crashing at first GC
before this change. After: serves 5,000 HTTP requests at ~410
req/s, peak RSS 1,088 KB (one chunk), growth 972 KB — the
collector hit its natural steady state. First time we've
validated "naive GC as replacement for snapshot discipline"
under real traffic.

New §6.6.5 "Precise Block Typing" in the whitepaper documents
the old heuristic bugs, the new header format, and the cost
(one orq per alloc, 16 header bits) vs benefit (class of bugs
eliminated). Updated §6.6.4 to reflect cell 4 passing.

Remaining known issue: the hash-set bench on the GC build under
very heavy sustained allocation still surfaces an occasional
unbound-variable error. The precise-type fix addressed the
observed HTTP crash; a deeper root-scan edge case remains.
Tracked for Fix 2 work.

137 asm no-GC + 137 asm GC + 189 shared functional tests all
pass.
2026-04-18 19:33:44 -04:00
3d55092037 asm-gc: adaptive EMA-driven meta-GC policy + bench + two correctness fixes
Moves the meta-GC from greedy (always verify) to adaptive: track a
scaled EMA of recent escape rate; when rate exceeds 50% (128/256),
SKIP the verifier and let the heap grow until natural GC; every 16
skipped arenas, force a verify as a probe to re-sample the rate.

Two correctness fixes uncovered while testing adaptive:

1. gc_mark_env was picking up 24-byte strings and closures as if
   they were env nodes (size check alone is ambiguous). Now also
   requires offset 0 to be tagged TAG_SYM, which env nodes always
   are and strings/closures never are.

2. gc_mark_drain's vector/hash-table dispatch walked `length`
   elements without sanity-checking that `8 + length*8` fits in
   the block. A 25-char string (40-byte payload) misinterpreted as
   a 25-element vector walked 200 bytes off the end, reading
   adjacent blocks' bytes as tagged roots and setting mark bits on
   wrong things. Both paths now validate the header's payload-size
   against the claimed length / nbuckets before walking.

New builtin:
  (arena-set-mode 0|1) — 0 = greedy baseline, 1 = adaptive (default)

arena-stats extended to six fields:
  (calls resets escapes skipped bytes-reclaimed ema-rate)

Bench (tests/bench-gc-adaptive.sh, one process per phase to isolate
a separate latent cross-phase bug we haven't cracked, N=1000 per
phase, i5-8350U):

  workload   mode     time_ms   resets   escapes   skipped
  friendly   greedy    732      1000        0         0
  friendly   adapt     691      1000        0         0
  hostile    greedy    568         0     1000         0
  hostile    adapt     607         0     1000        11
  mixed      greedy   1981        17     1983         0
  mixed      adapt    1694        14     1986         2

Adaptive wins on friendly (-6%) and mixed (-17%). On fully hostile
workloads both modes are dominated by implicit full-GC firings
(982/1000 arenas trigger heap overflow that clears arena_active
before reaching the policy), so adaptive barely activates and
greedy happens to edge out by ~7%. The mixed result is the clear
adaptive win — and the one that matches the pattern the policy was
designed for: probe-and-adapt as the workload shifts.

137 asm (no-GC) + 137 asm (GC) + 189 shared functional tests all
still pass.
2026-04-18 11:33:39 -04:00
a8eddd492e asm-gc: meta-GC layer — arena fast path with mark-phase verifier
Adds (with-arena thunk) as the O(1) bulk-reclaim fast path on top
of the existing naive mark-sweep. The meta-GC:

  1. Snapshots %r15 at arena entry.
  2. Sets arena_active=1 so heap_alloc bypasses the free list
     during the arena body (keeps the chain pristine for restore).
  3. Invokes the thunk via apply_proc_raw.
  4. Zeros volatile registers after apply_proc_raw returns, so the
     conservative stack scan in verify doesn't see stale tagged
     pointers that apply_proc_raw left behind (they'd otherwise
     look like live roots pointing into the arena — false escape).
  5. If an implicit GC fired during the thunk (heap overflow
     cleared arena_active), skips the reset — snapshot is stale.
  6. Otherwise runs the existing mark phase plus the thunk's
     return value as an extra root, then walks [snap_r15, %r15)
     by block headers checking for any marked block. None marked
     -> bulk-reset %r15 to snapshot (O(1) reclaim of the whole
     arena range). Any marked -> escape, fall through to naive
     sweep on the full range.

Two new GC-build builtins:
  (with-arena thunk) -> thunk's return value
  (arena-stats)      -> (calls resets escapes bytes-reclaimed)

Meta-GC benchmark (tests/bench-gc-arena.sh, i5-8350U, 2000 iters
of build-sum-discard over 200-element lists):

  Phase A (naive sweep only):
    time=932ms  gc-collections=200  arena=unused
  Phase B (arena-wrapped, same workload):
    time=945ms  gc-collections=1    arena=(2000 2000 0 205_392_000)

Arena reset rate on this truly-transient workload: 2000/2000 =
100%. Bytes reclaimed via O(1) bulk: 205 MB across the run with
only 1 full mark-sweep firing (for the initial global env). Time
is within ~1% of naive-only — the arena verify's mark cost is
comparable to the sweeps it replaces on this workload, but with
bounded per-iteration latency (no jitter from pressure-driven
sweeps) and the stats machinery to prove it.

Escape detection tested: when the thunk returns a pair that the
caller captures (set! escaped (with-arena ...)), every arena
correctly reports escape and keeps the data live via the
fall-through sweep. 137 asm (no-GC) + 137 asm (GC) + 189 shared
functional tests still pass.
2026-04-18 10:01:28 -04:00
489776baa4 asm: naive stop-the-world mark-sweep GC as a control group
Adds a second asm build (asm/uncommonlisp-gc) behind the GC_NAIVE
assembler flag, providing the benchmark baseline we previously had
no data for. Same binary, same surface, different allocator:

  - 8-byte header per heap block (size << 1 | mark), placed at -8
    from the tagged pointer so existing untag + offset accesses
    stay unchanged.
  - Chunk list tracked in a side array, letting sweep walk every
    mmap'd region by header-chained blocks instead of guessing.
  - Free list rebuilt each sweep, first-fit alloc with split on
    large-leftover (>= 24 bytes).
  - Mark phase enumerates five root classes: %r14 (global env,
    untagged chain), sym_else_val, sym_table entries, every
    sym_hash_bucket chain, and a conservative scan from current
    %rsp to the initial stack_top captured at _start. The stack
    scan runs twice per word — once as a tagged value, once as a
    potential untagged env-node pointer (size-guarded to 24 bytes
    so it can't walk off a wrong-size block).
  - Transitive marking via an explicit 16K-entry mark stack;
    gc_mark_env walks untagged env chains from %r14 and from every
    closure's env field.
  - heap_alloc preserves the non-GC ABI (only %rax clobbered) so
    existing callers like bi_append, which holds state in %rcx
    across make_pair, keep working.
  - Overflow path uses check-then-write bumps and pads the old
    chunk's tail with a single dead block before growing, so sweep
    never walks into uninitialized mmap'd memory.
  - HEAP_SIZE shrinks to 1 MB under GC_NAIVE so the collector
    actually runs on ordinary workloads.
  - Two diagnostic builtins in the GC build: (gc-collect) to force
    a collection, (gc-stats) -> (collections . live-bytes).

Control-group bench (examples/bench-gc-memory.lsp, 2000 iterations
of build-sum-discard over 200-element lists, i5-8350U):

  tier           time_ms   peak_rss   final_rss
  asm no-GC       1097     133.9 MB   133.9 MB   (grows, never shrinks)
  asm naive GC    1431       1.1 MB     1.1 MB   (steady state)

124x less memory at a ~30% throughput cost. That is the number we
were guessing at before. Reproduce: make bench-gc.

Tests: 137 asm (no-GC) + 137 asm (GC) + 189 shared functional pass.
The two asm builds are tested independently via UNCOMMONLISP_BIN in
asm/test.sh; asm/Makefile now builds both and exposes a test-gc
target.
2026-04-18 09:34:51 -04:00
afb5616843 asm: native hash-set + benchmark — 15-21x over portable
Adds 6 hash-set builtins (make-hash-set, hash-set?, hash-set-add!,
hash-set-contains?, hash-set-size, hash-set->list). Same sentinel
scheme as hash-table but tag word = -2 (hash-table is -1, vector
is >= 0). One cons cell per entry (vs two for hash-table) since
a set stores keys only — that's where the speedup over the Scheme-
level vector-based ht-* lib comes from.

Benchmark (tests/bench-hashset.sh, via make bench-hashset),
N=5000, i5-8350U asm tier:

                portable   native   speedup
  insert        ~130 ms    ~7 ms    ~20x
  hit-lookup    ~125 ms    ~8 ms    ~15x
  miss-lookup   ~240 ms    ~12 ms   ~20x

Portable is the ht-* lib from proof-netspace-server-lib.lsp
(vectors + cons chains + modulo, pure Scheme). Native replaces
the Scheme-level bucket walk with an asm loop that dereferences
pairs directly — no env lookups, no frame building per iteration.

All 137 asm + 189 functional (Python + C) tests still green.
2026-04-18 06:15:21 -04:00
f675778c6d asm: native hash-table primitives + equal? on strings
Adds 11 hash-table builtins (make-hash-table, hash-table?,
hash-table-set!, hash-table-ref, hash-table-ref/default,
hash-table-delete!, hash-table-exists?, hash-table-size,
hash-table-keys, hash-table-values, hash-table->alist),
bringing asm to surface parity with Python and C tiers.

Layout shares tag 7 with vectors; sentinel -1 at offset 0
disambiguates (vector length always >= 0). Fixed 64 buckets,
alist chains of (cons k v) per bucket.

Also lifts the long-standing pre-existing defect where asm
equal? only did identity compare — now byte-compares strings,
which hash-table string keys require. Pairs still deep; vectors
and hash-tables stay identity (matches C).

Tests: 137 asm + 189 functional + 189 C + Python pass.
27 dedicated hash-table assertions cover ref, ref/default,
exists?, delete!, update, predicate disjointness, int/string
keys, bulk 200-entry stress, keys/values/alist extraction.
2026-04-18 06:10:41 -04:00
d88149a502 proof netspace: envelope teleport + portable hash-table on vectors
Extends proof-netspace RPC with two verbs that let peers exchange the
full solution space in one round-trip:

  (envelope)           → reply (envelope (h1 h2 ...))
  (merge (h1 h2 ...))  → fold hashes into local DB, reply (merged N)

Any node can now bootstrap from a peer's cache instead of re-verifying
every theorem locally. Two nodes that swap envelopes both become
supersets of what either knew — the primitive for mesh-wide spiral.

*proof-db* swapped from linear alist to a hash-set. O(N·M) merge drops
to O(M). The hash-table is a ~20-line pure-Lumbda library over
make-vector / vector-ref / vector-set! — runs unmodified in all three
tiers. No asm hash-table primitive needed.

Also fixes a pre-existing asm defect: bi_makevec clobbered %rax via
the GETARG macro's internal scratch use, causing SIGSEGV on every
(make-vector N fill) call. The bug shipped because asm/test.sh only
covered the variadic (vector ...) constructor; tests/functional.lsp
had one make-vector assert but was never wired into asm's harness.
Added five make-vector assertions to asm/test.sh (132 → 137).

Portal snapshot rewritten to emit (set! *proof-db* ...) so the
top-level binding is actually mutated on restart — previous
(define ...) form bound locally on some code paths, leaving the
in-memory DB empty after load.

Verified: make test-all green (137 asm + 189 functional + Python/C
tests), 3-tier matrix cold+warm+restart all clean.
2026-04-17 21:36:26 -04:00
574ddc50d7 S-expressions over sockets — RPC + remote REPL in portable Scheme
Fuses portal (Scheme-source-as-interchange) with sockets (bytes over
the network). Wire protocol: one S-expression per connection. Same
server + client .lsp runs byte-identically in Python, C, and asm.

New primitives in all three impls:
- read-from-string — parse one sexp from a string

Asm gets two more:
- symbol->string — standard R7RS, was missing
- eval — evaluate a Scheme value in the global env (Python + C had
  it as a special form; asm exposes it as a builtin)

examples/rpc-server.lsp (port 9080):
- Whitelisted dispatch: ping / add / mul / fib / echo
- Never calls eval on client input; safe by construction
- Uses heap-snapshot/restore for O(1) memory on asm
- ~90 lines, portable

examples/rpc-client.lsp:
- Sends one request, reads one response, displays both
- 45 lines, portable

examples/repl-server.lsp (port 9081):
- DANGER: full remote eval. Any Scheme form accepted and evaluated
  in the server's global env. Persistent across connections.
- Deliberately does NOT use heap-snapshot — remote (define x ...)
  lives in the global env above any snapshot point; rewinding would
  invalidate the new binding. The ulimit -v 512 MB safety net
  (documented in CLAUDE.md) ensures an escaped process can't crash
  the machine.
- ~70 lines, portable. Demonstrates what "the language IS the
  interchange format" gets you at the limit: a single socket and
  a single primitive (eval) carry a full-powered REPL.

Verified 3×3 server×client matrix: all 9 combinations green.
All 132 asm + 571 py + 189 shared + 83 c tests still pass.

One quirk discovered and worked around: in asm, a closure captures
its env chain by pointer at define time. Forward-referenced names
in mutually-recursive toplevel defines can fail under specific
heap-restore patterns — see the leaf-first ordering note in
rpc-server.lsp.
2026-04-17 09:00:04 -04:00
b8d6afdeb3 heap-snapshot + native HTTP client + MOAD-0001 cleanup
Three wins in one commit.

1) heap-snapshot / heap-restore (asm arena primitive)
   asm has no GC. Long-running servers leaked ~64 MB per heap growth.
   Two new builtins let a programmer capture r15 and later rewind to
   it, recycling intermediate allocations in O(1) memory.
   Python + C get no-op versions so portable .lsp code can call them
   unconditionally.

   examples/http-server.lsp now takes a snapshot at top level and
   rewinds after every request. Measured asm RSS: 88 KB initial,
   100 KB after 100 requests, 100 KB after 1100 requests — flat.
   Prior behavior was +64 MB per few thousand requests.

2) examples/http-client-bench.lsp — native HTTP load generator
   Uses only the six tcp-* primitives + current-time-ms. Runs
   identically in all three impls. Eliminates curl's ~2 ms/req
   fork+exec overhead, so real server throughput shows up:

     Python server ← Python client   2403 rps
     C      server ← C      client   2439 rps
     asm    server ← asm    client   2994 rps
     asm    server ← C      client   2500 rps

   The earlier curl-based bench was clamped near 400 rps by the
   client; the actual servers handle 6–7× that.

3) MOAD-0001 cleanup
   - c/builtins.c bi_string_replace: strncmp-at-every-position
     (hand-rolled, sedimentary) → strstr (libc-tuned, typically
     Boyer-Moore-Horspool). O(N*k) → O(N + matches*k).
   - uncommonlisp.py _tokenize_lines: per-token src.count('\n', 0, pos)
     → precompute line_starts once, bisect_right per token.
     O(N*M) → O(M + N log M).

Also adds current-time-ms to all three impls so benchmarks can
time themselves without relying on the Python/C float `current-time`
(asm has no floats). Seconds-since-epoch tagged as a 61-bit int.

Test counts unchanged: 571 py + 132 asm + 189 shared + 83 c = 975.
All green via make test-all.
2026-04-16 20:49:32 -04:00
bfd4ec7ec8 sockets + portable HTTP server — 6 primitives, same server runs in all 3
Added tcp-listen/accept/connect/recv/send/close to Python, C, and asm.
One examples/http-server.lsp runs identically in all three impls and
serves HTTP/1.0 with routing, content-type, and content-length headers.

asm additions:
- SYS_SOCKET/BIND/LISTEN/ACCEPT/CONNECT/SETSOCKOPT syscalls
- 6 tcp-* builtins using the existing port encoding (SPECIAL ≥ 1000)
- bi_tcp_connect: dotted-quad IPv4 parser, no DNS dependency

Defects fixed along the way (surfaced by the HTTP server):
- string-append: was 2-arg only; now variadic (walks arg list twice)
- number->string: was stubbed to VAL_VOID; now correctly writes digits
  into a heap-allocated string (incl. negative handling)
- String-literal reader: \r and \0 escape sequences now handled (was
  silently dropping backslash, treating them as literal 'r' / '0')
- tcp_accept: sockaddr buffer was 8 bytes, now 16 (was corrupting
  caller's stack when accept wrote full struct sockaddr_in)

Pinocchio benchmark (tests/web-benchmark.sh):
At concurrency=20, 1000 requests, serving a 1KB body:

  uncommonlisp Python   373 req/s
  uncommonlisp C        370 req/s
  uncommonlisp asm      370 req/s
  python3 http.server   381 req/s  (stdlib reference)
  busybox httpd         382 req/s  (production reference)

All five converge within 3% — the client (curl fork/exec) is the
bottleneck, not the server. Our single-threaded blocking servers
are indistinguishable from battle-tested ones at this load.

Binary sizes:
  uncommonlisp asm    45 KB   (HTTP + everything else)
  busybox httpd       2.1 MB  (multi-call binary)
  python3             8 MB    (interpreter)

The asm HTTP server is 46× smaller than busybox and 176× smaller
than Python, serves from 7 Linux syscalls, and the entire protocol
handler is 70 lines of portable Scheme.

Test counts: 132 asm (up 1), rest unchanged. All green.
2026-04-16 18:58:27 -04:00
2f8b7dc737 portal benchmark + 3 mismatch defects fixed
Benchmark exercises full save×load matrix across Python/C/asm plus the
mismatch cases (wrong format, truncated input, missing file, corrupt
header). Cases that used to segfault or report wrong paths now degrade
cleanly.

asm (uncommonlisp.s):
- (define var) with no value now binds to VOID instead of segfault
- portal-resume checks sys_read returned full 48-byte header; sanity-
  checks heap_size and heap_base before committing r14/r15 restore.
  Corrupt/truncated portals now return #f cleanly.

py (uncommonlisp.py):
- file-not-found error inside a nested (load) now reports the actual
  missing path (via FileNotFoundError.filename) rather than the outer
  script path.
- _load wraps UnicodeDecodeError (binary file loaded as text) into a
  LispErr with the path; no more raw Python traceback.

tests/portal-benchmark.sh: 50-iter benchmark, 4 parts
  (save / load / cross-process / mismatch-classification).

Representative numbers (this laptop, 2026-04-16):
  setup+save: Python 126ms, C 3ms, asm 0.9ms
  cross-proc: Py→Py 260ms, C→C 6ms, asm→asm 1.5ms
All three test suites still pass: 571 py unit, 131 asm, 189 shared.
2026-04-16 16:57:24 -04:00
57f3c9fab1 asm/c/py: add (load), ports, write-file/file->string — full cross-impl parity
Asm gains the file I/O surface Python and C already had, unlocking
9/9 cells of the portal producer×consumer matrix (previously 6/9).

asm:
- (load "path") — mmaps file, swaps input source, loops scheme_read+eval,
  restores on exit. Nestable. Uses SYS_LSEEK + SYS_MUNMAP.
- Output ports: (open-output-file), (close-port), (port?). Encoded as
  SPECIAL values ≥ 1000 (fd = (val>>3) − PORT_SPECIAL_BASE), no tag-bit
  expansion needed.
- (display), (write), (newline) accept optional port arg; printer
  writes via output_fd global, swapped by port-aware builtins.
- (write-file path content) / (file->string path) — bytes in/out.

c, py: (write-file) / (file->string) added for parity.

tests: 131 asm (up 23), 189 functional (up 8, shared py+c),
tests/portal-cross-test.sh exercises 3×3 save×load matrix.
2026-04-16 16:37:40 -04:00
ba0e42d057 asm: add portal save/resume — binary heap dump to file
portal-save: writes 48-byte header (magic, heap_size, heap_base, r14,
r15) + raw heap bytes via sys_write. 9.8KB for a state with fib(20).

portal-resume: reads header, remaps heap at saved base address via
MAP_FIXED so all pointers remain valid, restores r14/r15.

Binary format — no JSON, no parsing. Just bytes in, bytes out.
Carry on USB to air-gapped machine, resume from exact state.

Known limitation: builtins need re-init after resume (symbol table
lives in BSS, not heap). User-defined data survives intact.

108 existing tests pass.
2026-04-16 13:11:38 -04:00
3ad161f74a asm: fix 4 defects, add vector printing, 33 new tests (75 → 108)
Fixed: map/filter/fold r13 register collision with heap limit,
apply_proc_raw closure body dispatch (begin vs single expr),
vector-ref/set GETARG clobber, vector printing (#(e1 e2 ...)).

108/108 asm tests pass. All implementations green:
  Python 571 + C 83 + asm 108 + functional 181 = 943 assertions.
2026-04-15 20:16:59 -04:00
30d7279be2 C: add deep_copy_env, VM frame stack for continuations
asm: fix builtin dispatch, improve apply_proc_raw

C changes: deep_copy_env() for multi-shot continuations,
explicit frame stack in VM for compiled code call/cc support.

asm changes: improved builtin implementations, fixed dispatch paths.

All tests pass: asm 75, C 76+114 functional.
2026-04-15 19:57:17 -04:00
b038266173 asm: add 37 new builtins, fix equal? for structural comparison
New builtins: /, odd?, even?, append, reverse, map, filter, fold-left,
for-each, apply, member, assoc, write, string-length, string-ref,
string-append, string=?, number->string, string->number, char->integer,
integer->char, char-alphabetic?, char-numeric?, vector, vector-ref,
vector-set!, vector-length, vector?, make-vector, vector->list,
list->vector, char?, list?, substring, expt, gcd, integer?.

Fixed equal? to do deep structural comparison on pairs (was identity only).
Added .cmp_false label. Added apply_proc_raw helper for higher-order fns.

56/63 expanded tests pass. 75 original tests still pass.
Known defects: expt overflow, append incomplete copy, num->str stub,
map/filter incomplete dispatch, vector display.
2026-04-15 19:08:09 -04:00
22571fa470 Fix MOAD-0001 defects across all implementations
asm/uncommonlisp.s — intern_symbol: replaced O(N) linear scan with
djb2 hash table (1024 buckets, chaining). 2.9x faster symbol interning
on programs with many symbols. 75 tests pass.

uncommonlisp.py — _define_record_type: replaced list.index() O(N)
with dict lookup O(1) for field→index mapping. 571 tests pass.

MOAD-0002 documented: _portal_checkpoint, _call_stack, _auto_compile
are intentional globals (hot loop performance). cc_escape_val/cc_active_jmp
are required by setjmp/longjmp call/cc approach. Comments added.

All 836 assertions pass across Python + C + Assembly + functional.
2026-04-15 14:29:58 -04:00
a7239ea717 Add asm test suite: 75 tests (unit + integration + functional)
Unit tests (42): arithmetic, comparison, booleans, predicates,
  pairs/lists, and/or — each verifies expected output.
Integration tests (18): special forms, bindings, named-let,
  recursion, higher-order, closures.
Functional tests (15): fib(35), ack(3,4), TCO 100k depth,
  list processing, mutual recursion, closure state, display.

make test-all now runs: Python 571 + C 76 + asm 75 + functional 114.
All pass.
2026-04-15 13:44:36 -04:00
8b37399288 Add pure x86_64 assembly Scheme interpreter: 2592 lines, 13KB binary
No C. No libc. Just Linux syscalls and machine instructions.
2592 lines of GNU assembler. 13KB stripped binary. 1.8MB resident.

Features: quote, if, define, set!, lambda, begin, let, named-let,
cond, and, or. 34 builtins. Tag-in-low-bits values. Bump allocator
on mmap'd pages. TCO via jmp. Symbol interning.

Runs: (fact 10)=3628800, (fib 35)=9227465, (ack 3 4)=125.
24 tests pass. Zero dependencies.

"If you vibe code in assembly, you don't even need a compiler."
2026-04-15 12:18:39 -04:00