Commit graph

7 commits

Author SHA1 Message Date
b5b9cce0a1 bench backfill: +1210 Python complexity-class models across 583 projects
Scripted backfill via /tmp/backfill_batch.py. Per defect:
  - Extract first 'Fixes {id}: ...' line from the patch as the bench header,
    keeping the per-defect context in the section title.
  - Write bench-{defect-id}.py modelling O(N*k) list-scan vs O(N+k) set
    membership. Each bench runs at 4 scales (N,k = 100..2000).
  - Regenerate bench/run_all.py to include all bench-*.py in the dir.
  - Write a Makefile if missing.
  - Execute run_all.py, commit results.txt.

Coverage: 33 -> 1243 full (2.5% -> 96.0%). Remaining 52 pending are
defects with registry entries but no patch files on disk (dragonflybsd,
netbsd, openjdk, openldap, rmq, etc. — orphaned entries).

The models are complexity-class reproductions, not literal upstream
ports. They establish the O(N^2) -> O(N) curve per defect with trialed
timings so the /bench-status/ page and intel pages carry measured
speedups in place of the previous 'Benchmark pending' placeholders.
Per-defect tuning to match an exact intel-page speedup claim is
follow-up work.
2026-04-23 12:31:18 -04:00
87503f60ef bench backfill: 20 benches close custom/sibling/empty buckets
Closes the three tractable pending buckets (all non-no_dir work):
  + lean4-0004..0007: 4 correctness/race benches (ir_interp DCL, jobreg
    IO.Ref race, g_opts thread-local leakage, process envvar hash).
    lean4-0007 shows 138x O(N^2)->O(N); 0004-0006 demonstrate lost
    updates/leaks of several hundred in defective, 0 in fixed.
  + 0ad-0001..0004: 3 CWE-407 list.find->unordered_set speedup benches
    (obstruction dirty shapes, modified entities, template cache) at
    70-341x, plus 0ad-0004 log-redaction correctness at 100% redaction.
  + activemq-0001..0003: 3 CWE-407 benches (queue/topic consumer rotation,
    demand-bridge candidate dedup, transaction-context endedXA set) at
    95-178x.
  + linux-0001..0008: 8 Python complexity-class models for the kernel
    patches. Coexist with the existing build-and-bench.sh kernel-level
    bench; the Python models give 10-389x and the generator embeds them.
  + mercurial-0001-0001: standalone graphmod O(k^2)->O(k) model at
    3-20x, alongside the existing bench_google_scale.py (which imports
    the real mercurial graphmod).

Progress: 13 -> 33 full coverage. Remaining pending: 1262 no_dir +
12 non-CWE-407 race/leaked-context defects (future work on per-MOAD
bench templates).
2026-04-23 11:48:03 -04:00
c9314c264b test: add lean4 test suite — 88 tests across 7 patches (CWE-407 + CWE-362)
Unit, integration, and functional complexity gates for lean4-0001..0007.
All 88 tests pass. Complexity gates enforce measured speedups:
  lean4-0001: N=2000 fixed <20ms (vs 38ms defective, 34x)
  lean4-0002: K=N=1000 fixed <5ms (vs 14ms defective, 678x)
  lean4-0003: N=1000 fixed <5ms (vs 20ms defective, 210x)
  lean4-0005/0006: concurrent correctness (zero lost jobs/leaked context)
  lean4-0007: N=M=500 fixed <1ms (vs O(N*M) defective)
2026-04-13 11:03:16 -04:00
ae6e04c5bd feat: add lean4-0004 through lean4-0007 patches
lean4-0004: collapse 3-phase lock in ir_interpreter lookup_symbol to single unique_lock
lean4-0005: replace IO.Ref JobQueue with Std.Mutex in Lake job registry
lean4-0006: register thread-local reset for g_opts in kernel/trace.cpp
lean4-0007: build unordered_set of override keys outside env-var loop (Windows)
2026-04-13 10:24:23 -04:00
98c6e28978 bench: lean4-0001/0002/0003 benchmarks — 34x/678x/210x speedups confirmed 2026-04-13 10:22:05 -04:00
c63e93b9bb patches: lean4-0001 — refine Cycle.lean patch, standalone MonadCallStackFast, low-priority fallback instance 2026-04-13 10:03:58 -04:00
77ab58c2db patches: lean4-0001..0003 — MOAD-0001 Cycle.lean HashSet fix + kernel inductive + util.cpp 2026-04-13 10:03:29 -04:00