Add peephole optimizer, --help, examples, README update

- Peephole optimizer: eliminates VOID+POP pairs, JUMP-to-next-instruction. Adjusts jump targets after dead code removal. - --help/-h and --version/-v flags. - 4 example programs: fibonacci, generator, mergesort, objects. - README rewritten: documents bytecode compiler, --fast flag, all features. - 529 tests green.
2026-04-13 16:56:21 -04:00 · 2026-04-13 16:56:21 -04:00 · 1326e8a106
commit 1326e8a106
parent f89d3bce6f
6 changed files with 314 additions and 96 deletions
--- a/README.md
+++ b/README.md
@ -1,6 +1,6 @@
 # uncommonlisp

-A Scheme-like Lisp interpreter in one Python file.
+A Scheme interpreter in one Python file, with a bytecode compiler.

 ```
 λ> (define (fib n)
@ -13,28 +13,59 @@ A Scheme-like Lisp interpreter in one Python file.
 ## Usage

 ```bash
-python3 uncommonlisp.py              # interactive REPL
-python3 uncommonlisp.py script.lsp   # run a file
-python3 uncommonlisp.py -e '(+ 1 2)' # eval an expression
+python3 uncommonlisp.py                # interactive REPL
+python3 uncommonlisp.py script.lsp     # run a file
+python3 uncommonlisp.py -e '(+ 1 2)'  # eval an expression
+python3 uncommonlisp.py --fast script.lsp  # auto-compile (7-19x faster)
 ```

+## Bytecode compiler
+
+uncommonlisp includes a stack-based bytecode compiler and VM. Enable it with `--fast` or `(auto-compile! #t)`:
+
+```bash
+python3 uncommonlisp.py --fast examples/fibonacci.lsp
+```
+
+```scheme
+(auto-compile! #t)
+(define (ack m n)
+  (cond ((= m 0) (+ n 1))
+        ((= n 0) (ack (- m 1) 1))
+        (else    (ack (- m 1) (ack m (- n 1))))))
+(compiled? ack)  ; => #t
+(ack 3 4)        ; => 125
+```
+
+The compiler handles: `if`, `begin`, `and`, `or`, `when`, `unless`, `cond`, `define`, `set!`, `lambda`, `let`, named-let, `let*`, `letrec`, `do`, `call/cc`, function calls with tail-call optimization. Macros are expanded at compile time. 20 specialized opcodes for hot builtins (`+`, `-`, `*`, `=`, `<`, `car`, `cdr`, `cons`, `null?`, etc.) avoid function call overhead.
+
+Features:
+- **Explicit frame stack** — compiled-to-compiled calls don't grow the Python stack
+- **Full continuations** — `call/cc` supports upward continuations; generators work
+- **Constant folding** — `(+ 1 2)` folds to `3` at compile time
+- **Peephole optimizer** — eliminates dead code (VOID+POP, JUMP-to-next)
+- **`(disassemble proc)`** — inspect generated bytecode
+
 ## What's implemented

 **Core language**
 - Full lexical scoping and closures
- Tail-call optimization (TCO) via explicit loop — deep recursion never blows the stack
+- Tail-call optimization (TCO) — deep recursion never blows the stack
 - Hygienic macros via `syntax-rules` with ellipsis (`...`) support
- `define-macro` / `defmacro` for procedural macro transformers
- `call/cc` (escape continuations)
+- `define-macro` / `defmacro` for procedural macros
+- `call/cc` — full continuations (escape + upward) in compiled code
 - `values` / `call-with-values`
 - `dynamic-wind`, `guard`, `with-exception-handler`
 - `quasiquote` / `unquote` / `unquote-splicing` with proper nesting
- R7RS internal defines → letrec* body semantics (mutual recursion in bodies)
- R7RS error objects — `error-object?` `error-object-message` `error-object-irritants`
- Exact rational arithmetic via `Fraction` — `(/ 1 3)` → `1/3`, `(+ 1/4 3/4)` → `1`
- String ports — `open-input-string` `open-output-string` `get-output-string` `read` on ports
- Module system — `module` / `import` with explicit export lists
+- R7RS internal defines with letrec* body semantics
+- R7RS error objects
+- Exact rational arithmetic — `(/ 1 3)` → `1/3`, `(+ 1/4 3/4)` → `1`
+- String ports — `open-input-string` `open-output-string` `read` on ports
+- Mutable strings — `string-set!` `string-fill!` `string-copy!`
+- Module system — `module` / `import` with export lists
 - `define-record-type` with `(inherit parent)` for single-inheritance
+- Pretty-print — `pp` / `pretty-print`
+- Tracing — `(trace fn)` / `(untrace fn)`

 **Special forms**
 `define` `set!` `lambda` `λ` `if` `cond` `case` `and` `or` `when` `unless`
@ -45,107 +76,70 @@ python3 uncommonlisp.py -e '(+ 1 2)' # eval an expression
 `module` `import` `define-record-type`

 **Built-ins**
- Arithmetic: `+` `-` `*` `/` `quotient` `remainder` `modulo` `expt` `sqrt` `abs` `floor` `ceiling` `round` `truncate` `min` `max` `gcd` `lcm` `log` `exp` `sin` `cos` `tan` `atan` `numerator` `denominator` and more
- Rationals: `(/ 1 3)` → `1/3`, literal `1/3` syntax, exact arithmetic, `exact` / `inexact` conversion
+- Arithmetic: `+` `-` `*` `/` `quotient` `remainder` `modulo` `expt` `sqrt` `abs` `floor` `ceiling` `round` `truncate` `min` `max` `gcd` `lcm` `log` `exp` trig functions, `numerator` `denominator`
+- Rationals: `(/ 1 3)` → `1/3`, literal `1/3` syntax, `exact` / `inexact` conversion
 - Comparison: `=` `<` `>` `<=` `>=` `zero?` `positive?` `negative?` `odd?` `even?`
- Booleans: `not` `boolean?` `boolean=?`
- Equality: `eq?` `eqv?` `equal?`
- Pairs & lists: `cons` `car` `cdr` `set-car!` `set-cdr!` `list` `list*` `length` `append` `reverse` `list-ref` `list-tail` `memq` `memv` `member` `assq` `assv` `assoc` `iota` `map` `for-each` `filter` `fold-left` `fold-right` `reduce` `any` `every` `count` `flat-map` `sort` `sort-by` `partition` `find` `take` `drop` `take-while` `drop-while` `take-right` `drop-right` `zip` `flatten` `concatenate` `list-tabulate` `unfold` and more
- SRFI-1: `last` `first`–`fifth` `delete` `lset-union` `lset-intersection` `lset-difference` `proper-list?` `dotted-list?`
- Strings: `string-length` `string-ref` `substring` `string-append` `string-upcase` `string-downcase` `string->list` `list->string` `string->symbol` `symbol->string` `string->number` (with `#x`/`0x`/`#b`/`#o` prefix detection) `number->string` `string-contains` `string-split` `string-join` `string-trim` `string-replace` `format` and more
- Characters: `char->integer` `integer->char` `char-alphabetic?` `char-numeric?` `char-upcase` `char-downcase`
- Vectors: `make-vector` `vector` `vector-ref` `vector-set!` `vector->list` `list->vector`
- Hash tables: `make-hash-table` `hash-table-set!` `hash-table-ref` `hash-table-ref/default` `hash-table-delete!` `hash-table-exists?` `hash-table-keys` `hash-table-values` `hash-table->alist` `hash-table-walk` and more
- Type predicates: `number?` `integer?` `real?` `rational?` `string?` `symbol?` `pair?` `null?` `list?` `char?` `vector?` `boolean?` `procedure?` `exact?` `inexact?`
- I/O: `display` `write` `newline` `read` `read-char` `peek-char` `read-line` `load` `with-output-to-string` `open-input-string` `open-output-string` `get-output-string`
- Error objects: `error-object?` `error-object-message` `error-object-irritants`
- Pretty-print: `pp` / `pretty-print` — indented output respecting line width
+- Pairs & lists: `cons` `car` `cdr` `list` `length` `append` `reverse` `map` `for-each` `filter` `fold-left` `fold-right` `reduce` `any` `every` `sort` `partition` `find` `take` `drop` `zip` `flatten` and more
+- SRFI-1: `last` `first`–`fifth` `delete` `lset-union` `lset-intersection` `lset-difference` `unfold` `list-tabulate`
+- Strings: `string-length` `string-ref` `string-set!` `substring` `string-append` `string-copy` `string-copy!` `string-fill!` `string->list` `string->number` `format` and more
+- Characters: `char->integer` `integer->char` `char-alphabetic?` `char-upcase` `char-downcase`
+- Vectors: `make-vector` `vector` `vector-ref` `vector-set!` `vector-copy` `vector-copy!`
+- Hash tables: `make-hash-table` `hash-table-set!` `hash-table-ref` `hash-table-keys` `hash-table-values` `hash-table-walk` and more
+- I/O: `display` `write` `newline` `read` `read-char` `read-line` `open-input-string` `open-output-string` `with-output-to-string`
+- File system: `file-exists?` `delete-file` `rename-file` `directory-files` `current-directory`
+- System: `command-line` `get-environment-variable` `current-time` `exit`
 - Python interop: `py-eval` `py-exec` `py-import` `py-call` `py-attr`
+- Compiler: `compile` `compiled?` `disassemble` `auto-compile!`

-**Prelude** (loaded automatically)
-`when` `unless` `case` `while` `for` `define-record-type` (with inheritance) `1+` `1-` `add1` `sub1` `square` `cube` `compose` `atom?` `range` `flatten` `string-map` `string-for-each` `call-with-string-output-port`
-
-**Standard library** (`stdlib.lsp`, load explicitly)
-Syntax-rules versions of `let`/`and`/`or`/`cond`/`case`/`do`, `fluid-let`, `receive` (SRFI-8), `begin0`, `while`/`until`, `dotimes`/`dolist`, `push!`/`pop!`, `and-let*` (SRFI-2), string utilities, list utilities (`sum` `product` `maximum` `minimum` `average` `enumerate` `transpose` `chunks` `interleave`), numeric utilities (`factorial` `fib` `prime?` `primes-up-to` `clamp`), alist/hash utilities, tree utilities, simple object system, coroutines via `call/cc`
+**Standard library** (`stdlib.lsp`)
+Additional macros, string/list/numeric/tree utilities, alist/hash helpers, simple object system, SRFI-2/8/64 test framework.

 ## Examples

+```bash
+python3 uncommonlisp.py --fast examples/fibonacci.lsp
+python3 uncommonlisp.py --fast examples/generator.lsp
+python3 uncommonlisp.py --fast examples/mergesort.lsp
+python3 uncommonlisp.py examples/objects.lsp
+```
+
 ```scheme
-; Closures
-(define (make-counter)
-  (let ((n 0))
-    (lambda () (set! n (+ n 1)) n)))
+;; Generator using full continuations
+(auto-compile! #t)
+(define (make-gen thunk)
+  (let ((k #f) (done #f))
+    (lambda ()
+      (if done 'done
+          (call/cc (lambda (return)
+            (if k (k return)
+                (begin (thunk (lambda (val)
+                         (call/cc (lambda (next)
+                           (set! k next) (return val)))))
+                       (set! done #t) (return 'done)))))))))

-(define c (make-counter))
-(c) ; => 1
-(c) ; => 2
-
-; Hygienic macro (syntax-rules)
-(define-syntax my-or
-  (syntax-rules ()
-    ((my-or) #f)
-    ((my-or e) e)
-    ((my-or e1 e2 ...)
-     (let ((t e1))
-       (if t t (my-or e2 ...))))))
-
-; define-record-type
-(define-record-type point
-  (make-point x y)
-  point?
-  (x point-x)
-  (y point-y set-point-y!))
-
-(define p (make-point 3 4))
-(point-x p)  ; => 3
-
-; Named let (looping)
-(let loop ((i 0) (acc '()))
-  (if (= i 5)
-      (reverse acc)
-      (loop (+ i 1) (cons (* i i) acc))))
-; => (0 1 4 9 16)
-
-; Hash tables
-(define freq
-  (let ((h (make-hash-table)))
-    (for-each (lambda (x)
-      (hash-table-set! h x (+ 1 (hash-table-ref/default h x 0))))
-      '(a b a c b a))
-    h))
-(hash-table-ref freq 'a)  ; => 3
-
-; Tail calls — no stack overflow even at depth 1,000,000
-(define (count-down n)
-  (if (= n 0) 'done (count-down (- n 1))))
-(count-down 1000000)  ; => done
-
-; Python interop
-(define re (py-import "re"))
-(py-call (py-attr re 'findall) "[0-9]+" "abc123def456")
-; => ["123", "456"]
+(define counter (make-gen (lambda (yield)
+  (let loop ((i 0)) (yield i) (loop (+ i 1))))))
+(counter) ; => 0
+(counter) ; => 1
+(counter) ; => 2
 ```

-## Running tests
+## Running tests & benchmarks

 ```bash
-make test          # run 455 tests
+make test          # run 529 tests
 make test-verbose  # verbose output
-```
-
-## Running benchmarks
-
-```bash
-python3 bench.py     # compare against CPython baseline
-python3 bench.py -v  # show result values too
+make bench         # compare interpreter vs bytecode vs CPython
+make lint          # syntax check all Python files
 ```

 ## File layout

 ```
-uncommonlisp.py   interpreter (self-contained, one file)
-stdlib.lsp        extended standard library (load manually)
-tests.py          test suite (396 tests)
+uncommonlisp.py   interpreter + bytecode compiler (one file, ~2600 lines)
+stdlib.lsp        extended standard library
+tests.py          test suite (529 tests)
 bench.py          benchmarks vs CPython
-Makefile          make test / make repl
+examples/         example programs
+Makefile          make test / make bench / make repl
 ```
--- a/examples/fibonacci.lsp
+++ b/examples/fibonacci.lsp
@ -0,0 +1,21 @@
+;;; fibonacci.lsp — iterative and recursive Fibonacci
+;;; Run: python3 uncommonlisp.py --fast examples/fibonacci.lsp
+
+(define (fib-iter n)
+  (let loop ((a 0) (b 1) (i 0))
+    (if (= i n) a (loop b (+ a b) (+ i 1)))))
+
+(define (fib-rec n)
+  (if (<= n 1) n (+ (fib-rec (- n 1)) (fib-rec (- n 2)))))
+
+(display "fib(30) iterative: ")
+(display (fib-iter 30))
+(newline)
+
+(display "fib(20) recursive: ")
+(display (fib-rec 20))
+(newline)
+
+(display "First 15 Fibonacci numbers: ")
+(display (map fib-iter (iota 15)))
+(newline)
--- a/examples/generator.lsp
+++ b/examples/generator.lsp
@ -0,0 +1,50 @@
+;;; generator.lsp — coroutine-style generators using full continuations
+;;; Run: python3 uncommonlisp.py --fast examples/generator.lsp
+
+(define (make-generator thunk)
+  (let ((k #f) (done #f))
+    (lambda ()
+      (if done 'done
+          (call/cc
+            (lambda (return)
+              (if k
+                  (k return)
+                  (begin
+                    (thunk (lambda (val)
+                             (call/cc (lambda (next)
+                                        (set! k next)
+                                        (return val)))))
+                    (set! done #t)
+                    (return 'done)))))))))
+
+;; Range generator
+(define (range-gen start end)
+  (make-generator
+    (lambda (yield)
+      (let loop ((i start))
+        (when (< i end)
+          (yield i)
+          (loop (+ i 1)))))))
+
+(define gen (range-gen 0 5))
+(display "Generator: ")
+(let loop ()
+  (let ((v (gen)))
+    (unless (eq? v 'done)
+      (display v) (display " ")
+      (loop))))
+(newline)
+
+;; Fibonacci generator (infinite)
+(define (fib-gen)
+  (make-generator
+    (lambda (yield)
+      (let loop ((a 0) (b 1))
+        (yield a)
+        (loop b (+ a b))))))
+
+(define fibs (fib-gen))
+(display "First 10 fibs: ")
+(do ((i 0 (+ i 1))) ((= i 10))
+  (display (fibs)) (display " "))
+(newline)
--- a/examples/mergesort.lsp
+++ b/examples/mergesort.lsp
@ -0,0 +1,27 @@
+;;; mergesort.lsp — functional merge sort
+;;; Run: python3 uncommonlisp.py --fast examples/mergesort.lsp
+
+(define (merge a b)
+  (cond ((null? a) b)
+        ((null? b) a)
+        ((< (car a) (car b))
+         (cons (car a) (merge (cdr a) b)))
+        (else
+         (cons (car b) (merge a (cdr b))))))
+
+(define (split lst)
+  (let loop ((l lst) (a '()) (b '()))
+    (if (null? l)
+        (list a b)
+        (loop (cdr l) b (cons (car l) a)))))
+
+(define (msort lst)
+  (if (or (null? lst) (null? (cdr lst)))
+      lst
+      (let ((halves (split lst)))
+        (merge (msort (car halves))
+               (msort (cadr halves))))))
+
+(define data (reverse (iota 20)))
+(display "Input:  ") (display data) (newline)
+(display "Sorted: ") (display (msort data)) (newline)
--- a/examples/objects.lsp
+++ b/examples/objects.lsp
@ -0,0 +1,49 @@
+;;; objects.lsp — record types, closures as objects, hash tables
+;;; Run: python3 uncommonlisp.py examples/objects.lsp
+
+;; Record types with inheritance
+(define-record-type <point>
+  (make-point x y)
+  point?
+  (x point-x)
+  (y point-y))
+
+(define-record-type <point3d>
+  (inherit <point>)
+  (make-point3d x y z)
+  point3d?
+  (x point3d-x)
+  (y point3d-y)
+  (z point3d-z))
+
+(define p (make-point 3 4))
+(define p3 (make-point3d 1 2 3))
+
+(display "point: ") (display (point-x p)) (display ",") (display (point-y p)) (newline)
+(display "point3d: ") (display (point3d-x p3)) (display ",")
+  (display (point3d-y p3)) (display ",") (display (point3d-z p3)) (newline)
+(display "point3d is point? ") (display (point? p3)) (newline)
+
+;; Closure-based counter
+(define (make-counter)
+  (let ((n 0))
+    (lambda (msg . args)
+      (cond ((eq? msg 'inc) (set! n (+ n 1)) n)
+            ((eq? msg 'dec) (set! n (- n 1)) n)
+            ((eq? msg 'get) n)
+            ((eq? msg 'reset) (set! n 0) n)
+            (else (error "unknown message" msg))))))
+
+(define c (make-counter))
+(c 'inc) (c 'inc) (c 'inc)
+(display "Counter after 3 increments: ") (display (c 'get)) (newline)
+(c 'dec)
+(display "Counter after decrement: ") (display (c 'get)) (newline)
+
+;; Hash table
+(define ht (make-hash-table))
+(hash-table-set! ht 'name "uncommonlisp")
+(hash-table-set! ht 'version "1.0.0")
+(hash-table-set! ht 'features '(bytecode continuations macros))
+(display "Name: ") (display (hash-table-ref ht 'name)) (newline)
+(display "Features: ") (display (hash-table-ref ht 'features)) (newline)
--- a/uncommonlisp.py
+++ b/uncommonlisp.py
@ -1457,9 +1457,64 @@ def _bc_lambda(body, params, rest, env, name=None):
        for nm in def_names: inner.emit(OP_VOID); inner.emit(OP_BIND, nm)
    _bc_body(body_list, inner, env, tail=True)
    inner.emit(OP_RETURN)
+    _peephole(inner)
    return inner


+_JUMP_OPS = frozenset([OP_JUMP, OP_JUMP_IF_FALSE, OP_JUMP_IF_FALSE_KEEP,
+                        OP_JUMP_IF_TRUE_KEEP])
+
+def _peephole(code):
+    """Peephole optimization: eliminate dead code and redundant ops."""
+    instrs = code.instrs
+    n = len(instrs)
+    if n < 2: return
+    # Mark instructions to remove
+    remove = set()
+    for i in range(n - 1):
+        op, arg = instrs[i]
+        nop, _ = instrs[i + 1]
+        # VOID POP → remove both
+        if op == OP_VOID and nop == OP_POP:
+            remove.add(i); remove.add(i + 1)
+        # Dead code after RETURN (unless it's a jump target)
+        if op == OP_RETURN and nop not in (OP_RETURN,) and i + 1 not in _jump_targets(instrs):
+            # Only remove if next instruction is not a jump target
+            if nop not in (OP_PUSH_ENV, OP_POP_ENV):  # be conservative
+                pass  # skip for safety — jump target analysis is complex
+    # JUMP to next instruction → remove
+    for i in range(n):
+        op, arg = instrs[i]
+        if op == OP_JUMP and arg == i + 1:
+            remove.add(i)
+    if not remove: return
+    # Build index mapping: old → new
+    mapping = {}; new_idx = 0
+    for i in range(n):
+        mapping[i] = new_idx
+        if i not in remove: new_idx += 1
+    mapping[n] = new_idx  # for jumps pointing past the end
+    # Rebuild with adjusted jumps
+    new_instrs = []
+    for i in range(n):
+        if i in remove: continue
+        op, arg = instrs[i]
+        if op in _JUMP_OPS and isinstance(arg, int):
+            new_instrs.append((op, mapping.get(arg, arg)))
+        else:
+            new_instrs.append((op, arg))
+    code.instrs = new_instrs
+
+
+def _jump_targets(instrs):
+    """Return set of instruction indices that are jump targets."""
+    targets = set()
+    for op, arg in instrs:
+        if op in _JUMP_OPS and isinstance(arg, int):
+            targets.add(arg)
+    return targets
+
+
 class _ContInvoked(Exception):
    """Raised when a full continuation is invoked."""
    __slots__ = ('cont', 'val')
@ -2609,6 +2664,28 @@ def main():

    args = sys.argv[1:]

+    # --help / -h
+    if '--help' in args or '-h' in args:
+        print('''uncommonlisp — a Scheme in one Python file
+
+Usage: uncommonlisp [options] [script.lsp] [args...]
+       uncommonlisp -e '(+ 1 2)'
+       uncommonlisp                    (interactive REPL)
+
+Options:
+  -e EXPR       evaluate expression and print result
+  -f, --fast    auto-compile all defines (bytecode VM, 7-19x faster)
+  -h, --help    show this help
+  -v, --version show version
+
+Features: R7RS core, bytecode compiler, full continuations, macros,
+          syntax-rules, modules, rationals, string ports, SRFI-1/2/8/64.''')
+        return
+
+    # --version / -v
+    if '--version' in args or '-v' in args:
+        print(f'uncommonlisp 1.0.0'); return
+
    # --fast / -f: enable auto-compile
    if '--fast' in args or '-f' in args:
        _auto_compile[0] = True