russell@unturf.com
a8eddd492e
Adds (with-arena thunk) as the O(1) bulk-reclaim fast path on top
of the existing naive mark-sweep. The meta-GC:
1. Snapshots %r15 at arena entry.
2. Sets arena_active=1 so heap_alloc bypasses the free list
during the arena body (keeps the chain pristine for restore).
3. Invokes the thunk via apply_proc_raw.
4. Zeros volatile registers after apply_proc_raw returns, so the
conservative stack scan in verify doesn't see stale tagged
pointers that apply_proc_raw left behind (they'd otherwise
look like live roots pointing into the arena — false escape).
5. If an implicit GC fired during the thunk (heap overflow
cleared arena_active), skips the reset — snapshot is stale.
6. Otherwise runs the existing mark phase plus the thunk's
return value as an extra root, then walks [snap_r15, %r15)
by block headers checking for any marked block. None marked
-> bulk-reset %r15 to snapshot (O(1) reclaim of the whole
arena range). Any marked -> escape, fall through to naive
sweep on the full range.
Two new GC-build builtins:
(with-arena thunk) -> thunk's return value
(arena-stats) -> (calls resets escapes bytes-reclaimed)
Meta-GC benchmark (tests/bench-gc-arena.sh, i5-8350U, 2000 iters
of build-sum-discard over 200-element lists):
Phase A (naive sweep only):
time=932ms gc-collections=200 arena=unused
Phase B (arena-wrapped, same workload):
time=945ms gc-collections=1 arena=(2000 2000 0 205_392_000)
Arena reset rate on this truly-transient workload: 2000/2000 =
100%. Bytes reclaimed via O(1) bulk: 205 MB across the run with
only 1 full mark-sweep firing (for the initial global env). Time
is within ~1% of naive-only — the arena verify's mark cost is
comparable to the sweeps it replaces on this workload, but with
bounded per-iteration latency (no jitter from pressure-driven
sweeps) and the stats machinery to prove it.
Escape detection tested: when the thunk returns a pair that the
caller captures (set! escaped (with-arena ...)), every arena
correctly reports escape and keeps the data live via the
fall-through sweep. 137 asm (no-GC) + 137 asm (GC) + 189 shared
functional tests still pass.
211 lines
7.7 KiB
Makefile
211 lines
7.7 KiB
Makefile
# ═══════════════════════════════════════════════════════════════════
|
||
# uncommonlisp — Python + C + Assembly implementations
|
||
# ═══════════════════════════════════════════════════════════════════
|
||
#
|
||
# Implementations:
|
||
# Python uncommonlisp.py bytecode VM, full continuations, portal
|
||
# C c/uncommonlisp tree-walker + bytecode VM + x86_64 JIT
|
||
# Assembly asm/uncommonlisp pure x86_64, no libc, 13KB binary
|
||
#
|
||
# Test suites:
|
||
# make test Python unit/integration (571 tests)
|
||
# make c-test C unit/integration + JIT (76 tests) + functional (114)
|
||
# make asm-test Assembly unit/integration/functional (75 tests)
|
||
# make functional-test Shared .lsp suite in Python + C (114 each)
|
||
# make test-all Everything (836 total)
|
||
#
|
||
# Benchmarks (each generates reproducible numbers referenced in the
|
||
# whitepaper; hardware-independent commands, safety envelope built in):
|
||
# make bench Python bench.py (tree-walker vs bytecode VM)
|
||
# make c-bench C unit-level microbenchmarks
|
||
# make bench-3way §6.4: Python vs C vs asm on sum-to/ack
|
||
# make bench-portal §7.5: S-exp/JSON/binary portal save+load timings
|
||
# make bench-portal-cross §7.2: 3x3 cross-impl portal save×load matrix
|
||
# make bench-web §11.3: HTTP benchmark vs busybox + python http.server
|
||
# make bench-rpc-chain §11.4: Py → C relay → asm backend chain timing
|
||
# make bench-proof §8.6: EML proof — Lean 4 vs Lumbda tiers
|
||
# make bench-all run every bench above back to back
|
||
# make friction head-to-head timing: Python vs C vs CPython
|
||
#
|
||
# Other:
|
||
# make examples Run examples in Python + C, compare output
|
||
# make docs Generate architecture diagrams
|
||
# make whitepaper Build PDF whitepaper
|
||
# make clean-all Clean everything
|
||
|
||
all: test
|
||
|
||
# ─── Python implementation ────────────────────────────────────────
|
||
|
||
test:
|
||
python3 tests.py
|
||
|
||
test-verbose:
|
||
python3 tests.py -v
|
||
|
||
bench:
|
||
python3 bench.py
|
||
|
||
bench-verbose:
|
||
python3 bench.py -v
|
||
|
||
repl:
|
||
python3 uncommonlisp.py
|
||
|
||
lint:
|
||
python3 -m py_compile uncommonlisp.py
|
||
python3 -m py_compile tests.py
|
||
python3 -m py_compile bench.py
|
||
|
||
# ─── C implementation ─────────────────────────────────────────────
|
||
|
||
c-build:
|
||
$(MAKE) -C c all
|
||
|
||
c-test: c-build
|
||
$(MAKE) -C c test
|
||
@echo "── C functional tests (shared .lsp suite) ──"
|
||
./c/uncommonlisp tests/functional.lsp
|
||
|
||
c-bench: c-build
|
||
$(MAKE) -C c bench
|
||
|
||
c-repl: c-build
|
||
./c/uncommonlisp
|
||
|
||
c-clean:
|
||
$(MAKE) -C c clean
|
||
|
||
# ─── Assembly implementation ──────────────────────────────────────
|
||
|
||
asm-build:
|
||
$(MAKE) -C asm all
|
||
|
||
asm-test: asm-build
|
||
$(MAKE) -C asm test
|
||
|
||
asm-repl: asm-build
|
||
./asm/uncommonlisp
|
||
|
||
asm-clean:
|
||
$(MAKE) -C asm clean
|
||
|
||
# ─── All implementations ─────────────────────────────────────────
|
||
|
||
functional-test: c-build
|
||
@echo "═══ Shared functional tests (114 tests) ═══"
|
||
@echo "── Python ──"
|
||
@python3 uncommonlisp.py --fast tests/functional.lsp | tail -3
|
||
@echo "── C ──"
|
||
@./c/uncommonlisp tests/functional.lsp | tail -3
|
||
|
||
test-all: test c-test asm-test functional-test
|
||
@echo "════════════════════════════════════════════════════"
|
||
@echo "All tests passed (Python + C + Assembly + functional)"
|
||
|
||
# ─── Benchmarks (reproducible; referenced in whitepaper §6–§11) ───
|
||
|
||
bench-3way: c-build asm-build
|
||
@bash tests/bench-3way.sh
|
||
|
||
bench-portal: c-build asm-build
|
||
@bash tests/portal-benchmark.sh
|
||
|
||
bench-portal-cross: c-build asm-build
|
||
@bash tests/portal-cross-test.sh
|
||
|
||
bench-web: c-build asm-build
|
||
@bash tests/web-benchmark.sh
|
||
|
||
bench-rpc-chain: c-build asm-build
|
||
@bash tests/rpc-chain-bench.sh
|
||
|
||
bench-proof: c-build asm-build
|
||
@bash tests/bench-proof.sh
|
||
|
||
bench-hashset: asm-build
|
||
@bash tests/bench-hashset.sh
|
||
|
||
bench-gc: asm-build
|
||
@bash tests/bench-gc-memory.sh
|
||
|
||
bench-gc-arena: asm-build
|
||
@bash tests/bench-gc-arena.sh
|
||
|
||
bench-all: bench c-bench bench-3way bench-portal bench-portal-cross bench-web bench-rpc-chain bench-proof
|
||
@echo "═══════════════════════════════════════════════════════════"
|
||
@echo "All benchmarks complete. Numbers in the whitepaper §6.4,"
|
||
@echo "§7.2, §7.5, §11.3, §11.4 are reproducible from these targets."
|
||
|
||
# ─── Examples ─────────────────────────────────────────────────────
|
||
|
||
examples: c-build
|
||
@echo "═══ fibonacci.lsp ═══"
|
||
@echo "--- Python ---" && python3 uncommonlisp.py --fast examples/fibonacci.lsp
|
||
@echo "--- C ---" && ./c/uncommonlisp examples/fibonacci.lsp
|
||
@echo
|
||
@echo "═══ mergesort.lsp ═══"
|
||
@echo "--- Python ---" && python3 uncommonlisp.py --fast examples/mergesort.lsp
|
||
@echo "--- C ---" && ./c/uncommonlisp examples/mergesort.lsp
|
||
@echo
|
||
@echo "═══ objects.lsp ═══"
|
||
@echo "--- Python ---" && python3 uncommonlisp.py examples/objects.lsp
|
||
@echo "--- C ---" && ./c/uncommonlisp examples/objects.lsp
|
||
|
||
# ─── Friction benchmark ──────────────────────────────────────────
|
||
|
||
friction: c-build asm-build
|
||
bash friction.sh
|
||
|
||
# ─── Documentation ────────────────────────────────────────────────
|
||
|
||
DOT_FILES := $(wildcard docs/*.dot)
|
||
DOT_PNGS := $(DOT_FILES:.dot=.png)
|
||
|
||
docs: $(DOT_PNGS)
|
||
|
||
docs/%.png: docs/%.dot
|
||
dot -Tpng $< -o $@
|
||
|
||
# ─── Whitepaper ───────────────────────────────────────────────────
|
||
|
||
VENV := whitepaper/.venv
|
||
RST := whitepaper/uncommonlisp-whitepaper.rst
|
||
PDF := whitepaper/uncommonlisp-whitepaper.pdf
|
||
STYLE := whitepaper/whitepaper.style
|
||
|
||
$(VENV)/bin/rst2pdf:
|
||
python3 -m venv $(VENV)
|
||
$(VENV)/bin/pip install --upgrade pip
|
||
$(VENV)/bin/pip install rst2pdf
|
||
|
||
whitepaper: $(PDF)
|
||
|
||
$(PDF): $(RST) $(STYLE) $(VENV)/bin/rst2pdf
|
||
cd whitepaper && ../$(<D)/.venv/bin/rst2pdf \
|
||
-s whitepaper.style \
|
||
--fit-background-mode=scale \
|
||
uncommonlisp-whitepaper.rst \
|
||
-o uncommonlisp-whitepaper.pdf
|
||
@ln -sf uncommonlisp-whitepaper.pdf whitepaper/WHITEPAPER.pdf
|
||
@echo "Built: $(PDF)"
|
||
|
||
# ─── Clean ────────────────────────────────────────────────────────
|
||
|
||
clean:
|
||
rm -rf __pycache__ *.pyc
|
||
|
||
clean-whitepaper:
|
||
rm -rf $(VENV) $(PDF) whitepaper/WHITEPAPER.pdf
|
||
|
||
clean-docs:
|
||
rm -f docs/*.png
|
||
|
||
clean-all: clean clean-whitepaper clean-docs c-clean asm-clean
|
||
|
||
.PHONY: all test test-verbose bench bench-verbose repl lint \
|
||
c-build c-test c-bench c-repl c-clean \
|
||
asm-build asm-test asm-repl asm-clean \
|
||
test-all bench-all examples friction functional-test \
|
||
bench-3way bench-portal bench-portal-cross bench-web bench-rpc-chain bench-proof \
|
||
docs whitepaper clean clean-whitepaper clean-docs clean-all
|