test-frameworks wave 3: vitest + testng + jasmine + libcheck (4 patches)

vitest-0001: coverage-v8 coverage.result.find inside merged.result.forEach
  -> Map<url, result> lookup. Bench: 824x at N=M=10000 coverage entries.

testng-0001: DynamicGraph.toDot freeNodes.contains inside two for-each
  loops -> Map<T, String> color lookup via getOrDefault. Bench: 64x at N=2000.

jasmine-0001: SpyRegistry.spyOnAllFunctions propertiesToSkip.indexOf inside
  Array.filter + .concat growth across D prototype levels -> Set.has + O(1)
  growth. Bench: 61x at D=10, P=300.

check-0001: libcheck suite_tcase linear strcmp scan over tclst List
  -> parallel hashtable for O(1) lookup amortized. Bench: 117x at N=1000.
  Shipped as design sketch; full integration requires companion hashtable.

Also ships whitepaper/outreach/test-harness-survey.md documenting 14
clean-scan frameworks across Clojure, OCaml, Haskell, Erlang, Go, F#,
Julia, Shell, Lua, JS. Scope covered 61 targets across 30+ languages.

UNDF IDs: 1292 (check), 1293 (jasmine), 1294 (testng), 1295 (vitest).
All 12 tests pass.
This commit is contained in:
russell@unturf.com 2026-04-23 08:54:44 -04:00
parent b79fddfb51
commit d67ec93a5d
34 changed files with 1511 additions and 1 deletions

View file

@ -0,0 +1,60 @@
#!/usr/bin/env python3
# bench-jasmine-0001.py
# SpyRegistry.spyOnAllFunctions prototype-chain filter:
# propertiesToSkip.indexOf (Array) vs Set.has, across D levels with P properties each.
import sys
import time
def bench_defective(depth, props_per_level):
"""Array.indexOf filter + concat growth per level."""
properties_to_skip = []
all_properties = []
t0 = time.perf_counter()
for d in range(depth):
# Each level has some unique + some already-seen properties
level = [f"prop_d{d}_p{i}" for i in range(props_per_level)]
filtered = [p for p in level if p not in properties_to_skip] # O(P) per prop
properties_to_skip = properties_to_skip + filtered
all_properties.extend(filtered)
return time.perf_counter() - t0
def bench_fixed(depth, props_per_level):
"""Set.has filter + Set.add growth."""
properties_to_skip = set()
all_properties = []
t0 = time.perf_counter()
for d in range(depth):
level = [f"prop_d{d}_p{i}" for i in range(props_per_level)]
filtered = [p for p in level if p not in properties_to_skip] # O(1) per
for p in filtered:
properties_to_skip.add(p)
all_properties.extend(filtered)
return time.perf_counter() - t0
TRIALS = 3
CASES = [(3, 50), (5, 100), (5, 200), (8, 200), (10, 300)]
def run():
lines = []
header = "=== jasmine-0001: SpyRegistry Array.indexOf vs Set.has ==="
print(header); lines.append(header)
for d, p in CASES:
df = min(bench_defective(d, p) for _ in range(TRIALS))
fx = min(bench_fixed(d, p) for _ in range(TRIALS))
speedup = (df / fx) if fx > 0 else float("inf")
line = f"D={d} P={p:<3}: defective={df*1000:.3f}ms fixed={fx*1000:.3f}ms speedup={speedup:.1f}x"
print(line); lines.append(line); sys.stdout.flush()
return lines
if __name__ == "__main__":
run()

View file

@ -0,0 +1,7 @@
=== jasmine-0001: SpyRegistry Array.indexOf vs Set.has ===
D=3 P=50 : defective=0.145ms fixed=0.045ms speedup=3.2x
D=5 P=100: defective=1.543ms fixed=0.151ms speedup=10.2x
D=5 P=200: defective=5.621ms fixed=0.286ms speedup=19.6x
D=8 P=200: defective=13.814ms fixed=0.529ms speedup=26.1x
D=10 P=300: defective=54.048ms fixed=0.884ms speedup=61.2x

View file

@ -0,0 +1,28 @@
#!/usr/bin/env python3
# run_all.py -- run jasmine bench scripts and write results.txt
import importlib.util, os, sys
BENCH_DIR = os.path.dirname(os.path.abspath(__file__))
def load_module(filename):
path = os.path.join(BENCH_DIR, filename)
spec = importlib.util.spec_from_file_location("mod", path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
all_lines = []
for fname in ["bench-jasmine-0001.py"]:
mod = load_module(fname)
lines = mod.run()
all_lines.extend(lines)
all_lines.append("")
print()
sys.stdout.flush()
out_path = os.path.join(BENCH_DIR, "results.txt")
with open(out_path, "w") as f:
f.write("\n".join(all_lines) + "\n")
print(f"results written to {out_path}")
sys.stdout.flush()