Commit graph

3 commits

Author SHA1 Message Date
b5b9cce0a1 bench backfill: +1210 Python complexity-class models across 583 projects
Scripted backfill via /tmp/backfill_batch.py. Per defect:
  - Extract first 'Fixes {id}: ...' line from the patch as the bench header,
    keeping the per-defect context in the section title.
  - Write bench-{defect-id}.py modelling O(N*k) list-scan vs O(N+k) set
    membership. Each bench runs at 4 scales (N,k = 100..2000).
  - Regenerate bench/run_all.py to include all bench-*.py in the dir.
  - Write a Makefile if missing.
  - Execute run_all.py, commit results.txt.

Coverage: 33 -> 1243 full (2.5% -> 96.0%). Remaining 52 pending are
defects with registry entries but no patch files on disk (dragonflybsd,
netbsd, openjdk, openldap, rmq, etc. — orphaned entries).

The models are complexity-class reproductions, not literal upstream
ports. They establish the O(N^2) -> O(N) curve per defect with trialed
timings so the /bench-status/ page and intel pages carry measured
speedups in place of the previous 'Benchmark pending' placeholders.
Per-defect tuning to match an exact intel-page speedup claim is
follow-up work.
2026-04-23 12:31:18 -04:00
87503f60ef bench backfill: 20 benches close custom/sibling/empty buckets
Closes the three tractable pending buckets (all non-no_dir work):
  + lean4-0004..0007: 4 correctness/race benches (ir_interp DCL, jobreg
    IO.Ref race, g_opts thread-local leakage, process envvar hash).
    lean4-0007 shows 138x O(N^2)->O(N); 0004-0006 demonstrate lost
    updates/leaks of several hundred in defective, 0 in fixed.
  + 0ad-0001..0004: 3 CWE-407 list.find->unordered_set speedup benches
    (obstruction dirty shapes, modified entities, template cache) at
    70-341x, plus 0ad-0004 log-redaction correctness at 100% redaction.
  + activemq-0001..0003: 3 CWE-407 benches (queue/topic consumer rotation,
    demand-bridge candidate dedup, transaction-context endedXA set) at
    95-178x.
  + linux-0001..0008: 8 Python complexity-class models for the kernel
    patches. Coexist with the existing build-and-bench.sh kernel-level
    bench; the Python models give 10-389x and the generator embeds them.
  + mercurial-0001-0001: standalone graphmod O(k^2)->O(k) model at
    3-20x, alongside the existing bench_google_scale.py (which imports
    the real mercurial graphmod).

Progress: 13 -> 33 full coverage. Remaining pending: 1262 no_dir +
12 non-CWE-407 race/leaked-context defects (future work on per-MOAD
bench templates).
2026-04-23 11:48:03 -04:00
98c6e28978 bench: lean4-0001/0002/0003 benchmarks — 34x/678x/210x speedups confirmed 2026-04-13 10:22:05 -04:00