lumbda/tests/bench-proof.sh
russell@unturf.com f7352b51b0 rename: uncommonlisp -> lumbda throughout the repo
Historical internal name "uncommonlisp" retired in favor of the
public name "lumbda" ahead of lumbda.com going live. Scope of
this commit:

Source files renamed:
  uncommonlisp.py                     -> lumbda.py
  asm/uncommonlisp.s                  -> asm/lumbda.s
  c/uncommonlisp.h                    -> c/lumbda.h
  whitepaper/uncommonlisp-whitepaper  -> whitepaper/lumbda-whitepaper (.rst + .pdf)

Binaries renamed (tracked ones; c/ was always gitignored):
  asm/uncommonlisp, asm/uncommonlisp-gc, asm/uncommonlisp.o,
  asm/uncommonlisp-gc.o                -> asm/lumbda(-gc)(.o)
  c/.gitignore                          -> ignores lumbda

Internal string updates (sed pass ordered longest-first):
  asm/uncommonlisp -> asm/lumbda
  c/uncommonlisp   -> c/lumbda
  uncommonlisp.py  -> lumbda.py
  UNCOMMONLISP_BIN -> LUMBDA_BIN (asm/test.sh env var)
  "uncommonlisp> " -> "lumbda> " (asm REPL prompt baked into binary)
  UNCOMMONLISP     -> LUMBDA (macros, comments)
  uncommonlisp     -> lumbda (prose)

Binary portal magic updated:
  "ULPORTAL" -> "LUMBDAB1"   # "Lumbda Binary v1"
Old portal files are not backward-compatible — this is a deliberate
break since it's the rename moment. S-expression portals already
carry their own ";; lumbda-portal v1" header and remain cleanly
versioned.

WHITEPAPER.pdf / WHITEPAPER.rst symlinks repointed to the renamed
files. Makefile's whitepaper target targets lumbda-whitepaper.pdf.

Not changed (intentional, separate phases):
  - Filesystem directory /home/fox/git/uncommonlisp itself
    (fox renames locally and the gitlab repo URL in a follow-up)
  - tests.py hardcoded cwd=/home/fox/git/uncommonlisp
    (matches the current on-disk location; will flip when the
    directory rename ships)
  - Git history (immutable; old commits still say uncommonlisp,
    which is correct — that's what they were)

Verified:
  137 asm no-GC + 137 asm GC + 571 Python + 83 C + 189 shared
  functional tests all pass under the new names.
  bench-gc-http (2000 req): all 4 cells behave as expected
  (cells 1/2 flat, 3 leaks, 4 bounded at 1 chunk).
  Python REPL, C REPL, asm REPL all start cleanly.
2026-04-19 10:20:11 -04:00

92 lines
3.7 KiB
Bash
Executable file

#!/bin/bash
# bench-proof.sh — verify the EML proof across every tier we have.
#
# Same five theorems, same symbolic-rewrite strategy, different hosts:
# Lean 4 (cached & cold) vs Lumbda (Python, C tree-walker, C --fast, asm).
# Prints best-of-3 in milliseconds.
set -u
cd "$(dirname "$0")/.."
ulimit -v 1048576 -s unlimited
CACHE=/tmp/lumbda-eml.cache
bestof_cold() {
# Delete cache before each run so every measurement re-verifies.
local cmd="$1" best=999999
for _ in 1 2 3; do
rm -f "$CACHE"
local t0 t1 ms
t0=$(date +%s%N)
eval "$cmd" >/dev/null 2>&1 || true
t1=$(date +%s%N)
ms=$(( (t1 - t0) / 1000000 ))
[ "$ms" -lt "$best" ] && best="$ms"
done
echo "$best"
}
bestof_cached() {
# Ensure cache exists once, then measure cache-hit paths.
local cmd="$1" best=999999
rm -f "$CACHE"
eval "$cmd" >/dev/null 2>&1 || true # prime
for _ in 1 2 3; do
local t0 t1 ms
t0=$(date +%s%N)
eval "$cmd" >/dev/null 2>&1 || true
t1=$(date +%s%N)
ms=$(( (t1 - t0) / 1000000 ))
[ "$ms" -lt "$best" ] && best="$ms"
done
echo "$best"
}
echo "══════════════════════════════════════════════════════"
echo "EML proof verification — best of 3 runs (ms)"
echo " i5-8350U, same five theorems, same symbolic strategy"
echo " cold = cache cleared first ('full re-verification')"
echo " cached = prior run's cache artifact is trusted"
echo "══════════════════════════════════════════════════════"
printf " %-38s %s\n" "Lumbda tier" "cold cached"
printf " %-38s %s\n" "─────────────────────────────────────" "───────────────"
printf " %-38s %4s ms %4s ms\n" "Python --fast" \
"$(bestof_cold 'python3 lumbda.py --fast proof/eml_proof_in_lumbda.lsp')" \
"$(bestof_cached 'python3 lumbda.py --fast proof/eml_proof_in_lumbda.lsp')"
printf " %-38s %4s ms %4s ms\n" "C (tree-walker)" \
"$(bestof_cold 'c/lumbda proof/eml_proof_in_lumbda.lsp')" \
"$(bestof_cached 'c/lumbda proof/eml_proof_in_lumbda.lsp')"
printf " %-38s %4s ms %4s ms\n" "C --fast (bytecode VM)" \
"$(bestof_cold 'timeout 15 c/lumbda --fast proof/eml_proof_in_lumbda.lsp')" \
"$(bestof_cached 'timeout 15 c/lumbda --fast proof/eml_proof_in_lumbda.lsp')"
printf " %-38s %4s ms %4s ms\n" "asm" \
"$(bestof_cold 'asm/lumbda < proof/eml_proof_in_lumbda.lsp')" \
"$(bestof_cached 'asm/lumbda < proof/eml_proof_in_lumbda.lsp')"
echo ""
echo " Lean 4 comparison:"
if command -v lake >/dev/null 2>&1; then
printf " %-38s %4s ms\n" "Lean 4 (cached replay)" "$(bestof_cached 'cd proof/lean && lake build')"
best=999999
for _ in 1 2 3; do
(cd proof/lean && lake clean >/dev/null 2>&1)
t0=$(date +%s%N)
(cd proof/lean && lake build >/dev/null 2>&1)
t1=$(date +%s%N)
ms=$(( (t1 - t0) / 1000000 ))
[ "$ms" -lt "$best" ] && best="$ms"
done
printf " %-38s %4s ms\n" "Lean 4 (cold rebuild)" "$best"
else
echo " (Lean 4 not installed — skipping Lean rows)"
fi
rm -f "$CACHE"
echo "══════════════════════════════════════════════════════"
echo " Notes:"
echo " - Cached replay re-reads an already-checked artifact;"
echo " cold rebuild/run is the fair end-to-end compare."
echo " - Clearing the cache: rm -f $CACHE"
echo " - C --fast has a known cumulative-state compiler bug."