Commit graph

11 commits

Author SHA1 Message Date
b8d6afdeb3 heap-snapshot + native HTTP client + MOAD-0001 cleanup
Three wins in one commit.

1) heap-snapshot / heap-restore (asm arena primitive)
   asm has no GC. Long-running servers leaked ~64 MB per heap growth.
   Two new builtins let a programmer capture r15 and later rewind to
   it, recycling intermediate allocations in O(1) memory.
   Python + C get no-op versions so portable .lsp code can call them
   unconditionally.

   examples/http-server.lsp now takes a snapshot at top level and
   rewinds after every request. Measured asm RSS: 88 KB initial,
   100 KB after 100 requests, 100 KB after 1100 requests — flat.
   Prior behavior was +64 MB per few thousand requests.

2) examples/http-client-bench.lsp — native HTTP load generator
   Uses only the six tcp-* primitives + current-time-ms. Runs
   identically in all three impls. Eliminates curl's ~2 ms/req
   fork+exec overhead, so real server throughput shows up:

     Python server ← Python client   2403 rps
     C      server ← C      client   2439 rps
     asm    server ← asm    client   2994 rps
     asm    server ← C      client   2500 rps

   The earlier curl-based bench was clamped near 400 rps by the
   client; the actual servers handle 6–7× that.

3) MOAD-0001 cleanup
   - c/builtins.c bi_string_replace: strncmp-at-every-position
     (hand-rolled, sedimentary) → strstr (libc-tuned, typically
     Boyer-Moore-Horspool). O(N*k) → O(N + matches*k).
   - uncommonlisp.py _tokenize_lines: per-token src.count('\n', 0, pos)
     → precompute line_starts once, bisect_right per token.
     O(N*M) → O(M + N log M).

Also adds current-time-ms to all three impls so benchmarks can
time themselves without relying on the Python/C float `current-time`
(asm has no floats). Seconds-since-epoch tagged as a 61-bit int.

Test counts unchanged: 571 py + 132 asm + 189 shared + 83 c = 975.
All green via make test-all.
2026-04-16 20:49:32 -04:00
bfd4ec7ec8 sockets + portable HTTP server — 6 primitives, same server runs in all 3
Added tcp-listen/accept/connect/recv/send/close to Python, C, and asm.
One examples/http-server.lsp runs identically in all three impls and
serves HTTP/1.0 with routing, content-type, and content-length headers.

asm additions:
- SYS_SOCKET/BIND/LISTEN/ACCEPT/CONNECT/SETSOCKOPT syscalls
- 6 tcp-* builtins using the existing port encoding (SPECIAL ≥ 1000)
- bi_tcp_connect: dotted-quad IPv4 parser, no DNS dependency

Defects fixed along the way (surfaced by the HTTP server):
- string-append: was 2-arg only; now variadic (walks arg list twice)
- number->string: was stubbed to VAL_VOID; now correctly writes digits
  into a heap-allocated string (incl. negative handling)
- String-literal reader: \r and \0 escape sequences now handled (was
  silently dropping backslash, treating them as literal 'r' / '0')
- tcp_accept: sockaddr buffer was 8 bytes, now 16 (was corrupting
  caller's stack when accept wrote full struct sockaddr_in)

Pinocchio benchmark (tests/web-benchmark.sh):
At concurrency=20, 1000 requests, serving a 1KB body:

  uncommonlisp Python   373 req/s
  uncommonlisp C        370 req/s
  uncommonlisp asm      370 req/s
  python3 http.server   381 req/s  (stdlib reference)
  busybox httpd         382 req/s  (production reference)

All five converge within 3% — the client (curl fork/exec) is the
bottleneck, not the server. Our single-threaded blocking servers
are indistinguishable from battle-tested ones at this load.

Binary sizes:
  uncommonlisp asm    45 KB   (HTTP + everything else)
  busybox httpd       2.1 MB  (multi-call binary)
  python3             8 MB    (interpreter)

The asm HTTP server is 46× smaller than busybox and 176× smaller
than Python, serves from 7 Linux syscalls, and the entire protocol
handler is 70 lines of portable Scheme.

Test counts: 132 asm (up 1), rest unchanged. All green.
2026-04-16 18:58:27 -04:00
2f8b7dc737 portal benchmark + 3 mismatch defects fixed
Benchmark exercises full save×load matrix across Python/C/asm plus the
mismatch cases (wrong format, truncated input, missing file, corrupt
header). Cases that used to segfault or report wrong paths now degrade
cleanly.

asm (uncommonlisp.s):
- (define var) with no value now binds to VOID instead of segfault
- portal-resume checks sys_read returned full 48-byte header; sanity-
  checks heap_size and heap_base before committing r14/r15 restore.
  Corrupt/truncated portals now return #f cleanly.

py (uncommonlisp.py):
- file-not-found error inside a nested (load) now reports the actual
  missing path (via FileNotFoundError.filename) rather than the outer
  script path.
- _load wraps UnicodeDecodeError (binary file loaded as text) into a
  LispErr with the path; no more raw Python traceback.

tests/portal-benchmark.sh: 50-iter benchmark, 4 parts
  (save / load / cross-process / mismatch-classification).

Representative numbers (this laptop, 2026-04-16):
  setup+save: Python 126ms, C 3ms, asm 0.9ms
  cross-proc: Py→Py 260ms, C→C 6ms, asm→asm 1.5ms
All three test suites still pass: 571 py unit, 131 asm, 189 shared.
2026-04-16 16:57:24 -04:00
57f3c9fab1 asm/c/py: add (load), ports, write-file/file->string — full cross-impl parity
Asm gains the file I/O surface Python and C already had, unlocking
9/9 cells of the portal producer×consumer matrix (previously 6/9).

asm:
- (load "path") — mmaps file, swaps input source, loops scheme_read+eval,
  restores on exit. Nestable. Uses SYS_LSEEK + SYS_MUNMAP.
- Output ports: (open-output-file), (close-port), (port?). Encoded as
  SPECIAL values ≥ 1000 (fd = (val>>3) − PORT_SPECIAL_BASE), no tag-bit
  expansion needed.
- (display), (write), (newline) accept optional port arg; printer
  writes via output_fd global, swapped by port-aware builtins.
- (write-file path content) / (file->string path) — bytes in/out.

c, py: (write-file) / (file->string) added for parity.

tests: 131 asm (up 23), 189 functional (up 8, shared py+c),
tests/portal-cross-test.sh exercises 3×3 save×load matrix.
2026-04-16 16:37:40 -04:00
ba0e42d057 asm: add portal save/resume — binary heap dump to file
portal-save: writes 48-byte header (magic, heap_size, heap_base, r14,
r15) + raw heap bytes via sys_write. 9.8KB for a state with fib(20).

portal-resume: reads header, remaps heap at saved base address via
MAP_FIXED so all pointers remain valid, restores r14/r15.

Binary format — no JSON, no parsing. Just bytes in, bytes out.
Carry on USB to air-gapped machine, resume from exact state.

Known limitation: builtins need re-init after resume (symbol table
lives in BSS, not heap). User-defined data survives intact.

108 existing tests pass.
2026-04-16 13:11:38 -04:00
3ad161f74a asm: fix 4 defects, add vector printing, 33 new tests (75 → 108)
Fixed: map/filter/fold r13 register collision with heap limit,
apply_proc_raw closure body dispatch (begin vs single expr),
vector-ref/set GETARG clobber, vector printing (#(e1 e2 ...)).

108/108 asm tests pass. All implementations green:
  Python 571 + C 83 + asm 108 + functional 181 = 943 assertions.
2026-04-15 20:16:59 -04:00
30d7279be2 C: add deep_copy_env, VM frame stack for continuations
asm: fix builtin dispatch, improve apply_proc_raw

C changes: deep_copy_env() for multi-shot continuations,
explicit frame stack in VM for compiled code call/cc support.

asm changes: improved builtin implementations, fixed dispatch paths.

All tests pass: asm 75, C 76+114 functional.
2026-04-15 19:57:17 -04:00
b038266173 asm: add 37 new builtins, fix equal? for structural comparison
New builtins: /, odd?, even?, append, reverse, map, filter, fold-left,
for-each, apply, member, assoc, write, string-length, string-ref,
string-append, string=?, number->string, string->number, char->integer,
integer->char, char-alphabetic?, char-numeric?, vector, vector-ref,
vector-set!, vector-length, vector?, make-vector, vector->list,
list->vector, char?, list?, substring, expt, gcd, integer?.

Fixed equal? to do deep structural comparison on pairs (was identity only).
Added .cmp_false label. Added apply_proc_raw helper for higher-order fns.

56/63 expanded tests pass. 75 original tests still pass.
Known defects: expt overflow, append incomplete copy, num->str stub,
map/filter incomplete dispatch, vector display.
2026-04-15 19:08:09 -04:00
22571fa470 Fix MOAD-0001 defects across all implementations
asm/uncommonlisp.s — intern_symbol: replaced O(N) linear scan with
djb2 hash table (1024 buckets, chaining). 2.9x faster symbol interning
on programs with many symbols. 75 tests pass.

uncommonlisp.py — _define_record_type: replaced list.index() O(N)
with dict lookup O(1) for field→index mapping. 571 tests pass.

MOAD-0002 documented: _portal_checkpoint, _call_stack, _auto_compile
are intentional globals (hot loop performance). cc_escape_val/cc_active_jmp
are required by setjmp/longjmp call/cc approach. Comments added.

All 836 assertions pass across Python + C + Assembly + functional.
2026-04-15 14:29:58 -04:00
a7239ea717 Add asm test suite: 75 tests (unit + integration + functional)
Unit tests (42): arithmetic, comparison, booleans, predicates,
  pairs/lists, and/or — each verifies expected output.
Integration tests (18): special forms, bindings, named-let,
  recursion, higher-order, closures.
Functional tests (15): fib(35), ack(3,4), TCO 100k depth,
  list processing, mutual recursion, closure state, display.

make test-all now runs: Python 571 + C 76 + asm 75 + functional 114.
All pass.
2026-04-15 13:44:36 -04:00
8b37399288 Add pure x86_64 assembly Scheme interpreter: 2592 lines, 13KB binary
No C. No libc. Just Linux syscalls and machine instructions.
2592 lines of GNU assembler. 13KB stripped binary. 1.8MB resident.

Features: quote, if, define, set!, lambda, begin, let, named-let,
cond, and, or. 34 builtins. Tag-in-low-bits values. Bump allocator
on mmap'd pages. TCO via jmp. Symbol interning.

Runs: (fact 10)=3628800, (fib 35)=9227465, (ack 3 4)=125.
24 tests pass. Zero dependencies.

"If you vibe code in assembly, you don't even need a compiler."
2026-04-15 12:18:39 -04:00