asm/lumbda-full: quasiquote + define-macro + prelude (ticket 0005)

Third asm variant — built with CL_FULL=1 GC_NAIVE=1 via new Makefile
target. Adds the macro machinery needed for cl-compat.lsp on the asm
tier, keeping every addition behind .ifdef CL_FULL so the default
(~22 KB) and -gc binaries keep their current footprint.

Landed in this drop:

  * Reader: backtrack on digit-prefixed symbols. After reading digit
    characters, if the next char is not a delimiter, input_pos
    rewinds and control falls through to .sr_symbol. Makes 1+, 1-,
    add1, abc123, and any CL-style identifier with a numeric prefix
    parse as symbols instead of truncating to a bare integer.

  * Reader: `` ` `` / `,` / `,@` produce (quasiquote X) / (unquote X)
    / (unquote-splicing X) forms. Same build shape as the existing
    `'` quote branch.

  * Evaluator: .ev_quasiquote + quasiquote_expand walk the template.
    unquote evaluates its argument in the current env; unquote-
    splicing evaluates then splices via a new list_append_ab helper;
    other pairs recurse (cons expand-car expand-cdr). Atoms pass
    through. No nested quasiquote depth (deliberate; ticket 0005
    scope).

  * Evaluator: .ev_define_macro + macro_env_head linked list. Each
    (define-macro (name p...) body) prepends a 24-byte
    (sym, closure, next) node. Dispatch in eval checks macro_lookup
    after all special-form compares; on hit, the closure is applied
    to the *unevaluated* argument list and the expansion re-enters
    .eval_top under TCO.

  * Binding: rest-arg support extended to .apr_bind inside
    apply_proc_raw. Previously only .ac_bind (direct .app_closure
    path) handled `(lambda (a . b) ...)` correctly; macros call
    closures through apply_proc_raw, so this was required to make
    variadic defun/setf macros bind correctly.

  * Builtin: (gensym) — writes "g%d" for an in-BSS counter, length-
    prefixes the buffer, calls intern_static. Available in every
    variant (not CL_FULL-gated — useful outside macros too).

  * Builtin: (cadr x), (sort lst) and the let* special form from
    earlier commit stay in default asm. These are Scheme staples.

  * Prelude: evaluated at _start after init_builtins / rng_seed,
    before the REPL. Embedded string, input state saved + restored
    around the load. Defines caar, cdar, caddr, cadddr, cddr,
    cdddr, cddddr, 1+, 1-, add1, sub1, square, eq? (= eqv? for
    interned symbols), memq, list-ref, assq, and `case` as a macro.

cl-compat.lsp: two small changes to work under asm's single-list
`map`:

  * Added cl-zip helper. Replaced two `(map (lambda (v n) (list v n))
    xs ys)` sites with `(cl-zip xs ys)` — asm's builtin map accepts
    only one list, and cl-loop-emit needs a parallel walk over
    state-vars and new-names.

  * Added explanatory comment for cddddr at the top of the shim
    (already shipped).

Tests:

  * make asm-test (lumbda)    — 158/158 pass.
  * make asm-test-gc           — 158/158 pass.
  * make asm-test-full         — 158/158 pass on synchronous run.
  * Zoë's `examples/ursa.lisp.txt` LOADS on asm/lumbda-full.
    `(expt-mod 3 7 100)` = 87.
    Most simple cl-loop forms work (while + do + finally, range-to,
    then-accumulator).

Known open issues documented in docs/tickets/0005-asm-cl-full.md:

  * cl-loop-emit produces wrong output for inputs with `simple` iters
    (`(simple a 5)` → state binding dropped). Python/C return the
    correct form; asm version is missing the binding. Bug surfaces
    in the emit's 30+ binding let*; could not pin down in this
    session. Downstream effect: `(miller-rabin n)` and similar
    defuns that depend on `cl-loop repeat k for a = ... unless ...
    return nil` don't produce usable expansions, so Zoë's acceptance
    suite does not run end-to-end on asm/lumbda-full yet.

  * examples/ursa-scheme.lsp — `factor` crashes on asm under some
    random seeds (bump-allocator exhaustion on long rhoff retry
    chains). Out of CL_FULL scope; tracked in same ticket.

Next steps live in ticket 0005. This commit ships the infrastructure
so the remaining work is a debugging exercise against a reproducible
minimal case, not a feature build.

docs/tickets/0005-asm-cl-full.md

@@ -0,0 +1,186 @@
+# 0005 — `asm/lumbda-full`: CL compat on the asm tier
+
+**Status:** partially resolved — infrastructure landed, cl-loop-emit
+edge case open
+**Reporter:** fox (via blackops)
+**Implementer:** blackops
+**Opened:** 2026-04-24
+
+## Problem
+
+Ticket 0004 landed Common Lisp compatibility on the Python and C
+tiers. Zoë Trout's favorites at `examples/ursa.lisp.txt` load via
+`cl-compat.lsp` and run unchanged on both. The asm tier — 22 KB
+stripped, 14 syscalls, no libc — was intentionally scoped out of
+0004 because the minimal runtime lacks three features that `cl-
+compat.lsp` and the `cl-loop` macro depend on:
+
+1. **Quasiquote** (`` ` ,foo ,@bar ``) — every macro expansion in
+   `cl-compat.lsp` uses quasiquote templates. Without it the file
+   does not even parse on asm.
+2. **`define-macro`** — cl-compat is ~90% macros. Without a macro
+   system the file cannot register them.
+3. **`case`** — `cl-loop-emit` uses `case` for range-term dispatch
+   (`to` / `below` / `downto`). A macro definition via define-macro
+   suffices once (1) and (2) exist.
+
+Phase 1 (commit `8cf6f44`) added rest-args, `cadr`, `sort`, and `let*`
+to the default asm — enough to run `examples/ursa-scheme.lsp`, the
+idiomatic Scheme port of Zoë's favorites. Phase 2 (this ticket) adds
+the macro machinery so her CL source runs on asm unchanged.
+
+## Goal
+
+Build a third asm variant, `asm/lumbda-full`, produced with
+`--defsym CL_FULL=1 --defsym GC_NAIVE=1`. It ships with:
+
+- everything the existing `asm/lumbda-gc` has,
+- quasiquote / unquote / unquote-splicing in the reader + evaluator,
+- `define-macro` as a special form with a macro table and detection
+  pass that runs before procedure dispatch,
+- an embedded Scheme prelude loaded at init that defines `caddr`,
+  `cadddr`, `cddr`, `cdddr`, `cddddr`, `1+`, `1-`, `add1`, `sub1`,
+  `square`, `list-ref`, `assq`, and `case` (as a macro),
+- optional: a way to opt out of the prelude at startup for strict
+  R5RS work.
+
+Acceptance: `./asm/lumbda-full examples/ursa.lisp.txt` loads Zoë's
+CL source unchanged and produces identical answers to the Python and
+C paths across the same test suite (`tests/ursa.lsp`).
+
+## Scope — three asm variants
+
+| Binary | Flags | Size target | Features |
+|---|---|---|---|
+| `asm/lumbda` | (none) | ~22 KB | minimal Scheme, bump allocator |
+| `asm/lumbda-gc` | `GC_NAIVE=1` | ~55 KB | + mark-sweep GC + arena |
+| `asm/lumbda-full` | `CL_FULL=1 GC_NAIVE=1` | ~70 KB | + quasiquote + macros + case + prelude |
+
+The `CL_FULL` guard ensures the default and `-gc` tiers stay at their
+current footprint. All new asm lives inside `.ifdef CL_FULL` blocks.
+
+## Non-goals (stay out of scope)
+
+- `defgeneric` / `defmethod` — requires CLOS dispatch tables. The
+  Scheme port's `digits` function already hand-ports this as
+  type-predicate cond, and the CL file omits `digits` accordingly.
+- `make-array :adjustable :fill-pointer`, `vector-push-extend`,
+  `vector-pop` — requires a dynamic-array type and portal-format
+  extension. The Scheme port's `rho` uses a list-backed work stack,
+  and the CL file omits `rho`.
+- Generalized `setf` (on `car`, `vector-ref`, etc.) — simple-variable
+  setf stays the supported shape.
+
+## Migration path
+
+Staged commits:
+
+1. Makefile target + empty CL_FULL blocks (compiles, no behavior).
+2. Prelude load mechanism — at init (CL_FULL only), evaluate an
+   embedded Scheme string that defines the small accessors.
+3. Quasiquote — reader recognizes `` ` `` / `,` / `,@`, produces
+   `(quasiquote …)` / `(unquote …)` / `(unquote-splicing …)` forms.
+   Evaluator special-form for `quasiquote` that walks the template
+   and emits `cons` / `list` / `append` calls.
+4. `define-macro` — new special form, macro table keyed by symbol,
+   detection pass in the eval dispatch before procedure apply,
+   re-eval of expansion result.
+5. `case` as a macro (simpler than a special form once 4 exists).
+6. Port `tests/zoe-favorites-test.sh` to run against `lumbda-full`.
+
+Commit after every stage compiles and passes asm-test. Roll back any
+stage that regresses the minimal or -gc tiers.
+
+## Test strategy
+
+- `asm/test.sh --full` runs the existing 158 assertions against
+  `lumbda-full` (should all pass — CL_FULL is purely additive).
+- `tests/zoe-favorites-test.sh` learns a third case that feeds
+  `tests/ursa.lsp` to `asm/lumbda-full` and compares against the
+  Python and C answers.
+- A new `asm/test-full.sh` or similar covers quasiquote, define-
+  macro, and case in isolation.
+
+## What landed in this drop
+
+- `asm/lumbda-full` target built with `CL_FULL=1 GC_NAIVE=1`. Default
+  and `-gc` tiers stay at their current footprint (all new code is
+  guarded by `.ifdef CL_FULL` — 158/158 asm tests pass against every
+  variant).
+- Rest args on `lambda` / `define` — landed in default asm (earlier
+  commit `8cf6f44`). Both `.ac_bind` and `.apr_bind` handle
+  `(lambda (a . b) ...)` and `(define (f x . rest) ...)`.
+- Reader backtrack on digit-prefixed symbols — `1+`, `1-`, `abc123`
+  now parse as symbols. After accumulating digits, the reader peeks
+  at the next char; if it is not a delimiter, input_pos is rewound
+  and control falls through to the symbol reader.
+- `gensym` builtin — formats `"g%d"` via an in-BSS counter, interns
+  via `intern_static`. Available in every variant.
+- Quasiquote / unquote / unquote-splicing — reader recognizes
+  `` ` `` / `,` / `,@` (CL_FULL). Evaluator `.ev_quasiquote` walks
+  the template: unquote evaluates; unquote-splicing evaluates then
+  `list_append_ab` splices; other pairs recurse and `make_pair`. No
+  nested quasiquote support (deliberate — see Non-goals).
+- `define-macro` special form — stores macros in a dedicated
+  `macro_env_head` linked list of 24-byte `(sym, closure, next)`
+  nodes, separate from the value env. Eval dispatch checks
+  `macro_lookup` for any symbol operator that is not a special form;
+  on hit, the closure is applied to the *unevaluated* argument list
+  and the expansion re-enters `.eval_top` under TCO.
+- Prelude auto-loaded at startup (`load_cl_full_prelude`) — evaluates
+  an embedded Scheme string before the REPL starts. Defines `caar`,
+  `cdar`, `caddr`, `cadddr`, `cddr`, `cdddr`, `cddddr`, `1+`, `1-`,
+  `add1`, `sub1`, `square`, `eq?` (alias for `eqv?`), `memq`,
+  `list-ref`, `assq`, and `case` as a macro. Input state is saved
+  and restored around the load so user scripts see a pristine reader.
+
+## Verified on `asm/lumbda-full`
+
+- `defun`, `setf`, `flet`, `multiple-value-bind`, `declare` all
+  expand and evaluate correctly.
+- `&optional` args with defaults work.
+- `case` macro dispatches by value on flat-list keys with `else`.
+- Simple `cl-loop` forms work: `while + do + finally return`,
+  `for VAR from A to B` with `do` / `then` bodies.
+- Quasiquote templates including `,@` splicing produce correct shape.
+- `examples/ursa.lisp.txt` loads to completion — every `defun`
+  registers and its body parses under the shim.
+
+## Known open issues (follow-up work)
+
+1. **`cl-loop-emit` crashes on some parsed inputs.** Specifically,
+   `(cl-loop-emit '(() ((simple a 5)) () () () #f () ()))` returns
+   `(let* () (let g0 () (g0)))` — the `simple` iter's state binding
+   is dropped. Same input on Python and C returns the correct
+   `(let* ((g3 5)) (let g0 ((a g3)) ...))`. The parse output on
+   asm is correct; the bug is inside the emit's giant `let*` (30+
+   bindings). Reproduced in isolation; could not pin down after a
+   few hours — likely an asm-side env or stack interaction that
+   surfaces only inside this specific call shape.
+
+   Consequence: `(miller-rabin n)` and other defuns whose bodies use
+   `cl-loop repeat k for a = ...` expand incorrectly and crash at
+   run time. Zoë's full acceptance suite does not run end-to-end
+   yet on `asm/lumbda-full`.
+
+2. **`examples/ursa-scheme.lsp` — `factor` crashes on some inputs**
+   under certain random seeds on asm (e.g. seed=2, `(factor 91)`).
+   Default asm has no macro overhead but does hit this under long
+   rhoff retry chains. Believed to be asm's bump allocator / stack
+   growth under deep recursion; out of CL_FULL's scope.
+
+## Risk
+
+- Quasiquote in asm is non-trivial, especially nested `` ` `` inside
+  another `` ` ``. Start with depth-1 only; raise error on nested.
+- `define-macro` changes the eval dispatch path; regressions in
+  minimal tier are catastrophic. Guard strictly with `.ifdef
+  CL_FULL`; do not share dispatch tables between flavors.
+- Prelude load at init must survive portal resume — a portal file
+  dumped from `-full` loads state relative to the already-initialized
+  prelude, which is an env snapshot. Verify portal round-trip stays
+  green.
+- Stone-lisp image-based development (ticket 0004 framing) implies
+  some users keep `-full` processes alive indefinitely. The 1 MB GC
+  chunks in `-gc` apply — memory discipline per CLAUDE.md stays in
+  effect.