10 PHP fw / Python tooling / JS bundler targets scanned. Honor roll
cumulative: 86 projects. Two borderline candidates documented for
type-aware follow-up (symfony PropertyAccessor::writeCollection O(P*C),
pyright callHierarchyProvider O(C^2)) — both real but need type-aware
helpers, not single-line set hoists.
10 sci/static-site/api-gateway/games targets scanned. Honor roll cumulative:
54 projects. scipy distance_impl.h M7 cluster is intentional distance
algorithms (FP). kong M4 cluster is test-fixture credential headers.
synapse haystack.contains is intentional substring pre-check before regex.
No flagship CWE-407 this pass.
Flagship: weaviate authorization filter slices.Contains per item (O(N*K)).
Multi-tenant deployments with hundreds-thousands of permitted resources pay
this on every authorized read. Set hoist: 1735x speedup at N=50k K=5k.
Wave 11 honor roll: bash, coreutils. Cumulative: 49 projects.
10 observability/streaming targets scanned. Honor roll cumulative: 34 projects.
No flagship CWE-407 — VictoriaMetrics streamaggr.getInputOutputLabels real
but constant-factor at realistic config shapes (W <=10, L <=30, ~2-3x impact),
below the 5x wall-clock bar. Logged for re-scan if W >30 case surfaces.
29 mail/DNS/storage/VPN/RTOS/pg-eco targets scanned. No flagship CWE-407 patches
this pass — strong M1 candidates resolved to bounded fixed config tables
(strcmp on rule lookups), already-optimal data structures (RoaringTreemap,
ObjHashSet), or query-shape constants too small to cross the wall-clock bar.
Honor roll: 30 projects cumulative across waves 3-7. questdb `recordViews`
borderline real defect (O(N²) view dedup on ObjList) logged for future pass.
Scanned the three foundational Python documentation tools (Sphinx,
docutils, Pelican) and documented the triage outcome.
Pelican joins the clean-scan honor roll (now 25 projects). Both flagged
findings tested as false positives — utils.py:485 is String.index for
'\n', not list iteration; pelican_import.py:663 ReDoS pattern scales
linearly at N=40 (0.11ms).
Sphinx + docutils have multiple M1 hits in node-tree walks
(Node.findall via parent.index per ancestor). Investigated and
benchmarked: both old and new algorithms are O(D*S) — constant-factor
only, not CWE-407-grade complexity-class change. A real fix would
require maintaining a parent_index cache on Element nodes, a refactor
with cache-invalidation surface area beyond a single-defect patch.
No patches shipped this pass; the work is documented for the next
reviewer.
The Gatsby authors annotated each of the three call sites in
in-memory/indexing.ts with 'expensive at scale' comments. Their
diagnosis is correct: nodeTypeNames.includes(node.internal.type)
inside iterateNodes().forEach is O(N*T) per cache build.
For N=100k+ nodes typical of mature content sites and T=10-30
declared types per query, this fires on every type-filtered query.
gatsby develop in particular rebuilds caches per page render.
Fix: hoist Set<string> once at the top of each function. O(1) per
node lookup. Total cost O(N+T). Bench shows 8.4x at N=100k T=50;
2.7-4.7x at smaller scales.
Three call sites patched: ensureIndexByElemMatch (line 326),
ensureEmptyFilterCache (378), ensureIndexByElemMatchValue (504).
Author 'expensive at scale' comments updated to record the fix.
Investigated the top 2 triage-backlog items from the Wave 4 survey:
- PHP_CodeSniffer Tokenizers/PHP.php ReDoS: scanner flagged
((?<!\.)_[0-9A-F][0-9A-F.]*)+$ as catastrophic-backtracking shape.
Empirical test shows the underscore-anchored inner group prevents
overlap between outer iterations; Python re matches N=50 pathological
input in sub-millisecond time. False positive. Scanner could be taught
to recognize anchor-prefix inner groups as safe.
- aws-cdk region-info.ts limitedRegionMap: partitions.includes inside
per-region loop is a real O(R*P) shape. At realistic scale (R=35
regions, P=2-4 partitions) the wall-clock improvement is constant-
factor only; bench shows 2.9x at R=2000 P=100 but collapses to 1.0x
at production scale. Cleanup-grade, not CWE-407 complexity-class
emergency; no patch shipped.
Both findings documented in the Wave 4 triage section as investigated-
and-resolved. The remaining 5 items stay on the list for future waves.
artemis, doris, gin, gstreamer, igraph, kylin, nifi, open3d, opencv,
ros2, starrocks, trino, victoria-metrics: each had a **Speedup:**
metadata line from an early draft with a small per-defect scenario
number (2.5x, 5x worst case, etc.) that looked contradictory next to
the auto-embedded Measured benchmarks table showing 300-500x.
Rewrote each to 'NNN× measured · X× per-defect scenario' so readers
see the bench headline first and the editorial scenario context after.
Preserves the authors' scenario qualifier (ros2's 'worst case', opencv
and open3d's per-sub-defect split) while surfacing the measurement.
Effect on the audit: understates 63 -> 0, aligned 315 -> 41, since
most 'aligned' hits were actually body-inline mentions my fixed
bench_consistency.py no longer considers as headline claims.
Each of the 3 briefs flagged by bench_consistency.py as claim > measured
now carries an explicit line pairing the op-count claim with the
measured wall-clock speedup and explaining the residual gap.
fbneo-0001: 45,000x claim -> + 2,410x wall-clock at N=45k
(Python dict vs C++ unordered_map constant factor).
mercurial-0001: 5,000x claim -> + 50x wall-clock at k=500
(Python sim ceiling; bench_google_scale.py projects
to Google-scale via ops ratio).
substrate: 38,550x claim -> + 2,009x wall-clock at N=10k
(Python list vs Rust HashSet constant factor).
mercurial-0001 bench also scaled to CASES=[(1000,50), (1000,100),
(1500,200), (1500,350), (1500,500)] to cover k=500 directly.
The audit still counts these as overstates because the claim number
is intentionally the op-count figure; the rendered intel page now
carries both numbers side-by-side so readers can see the reconciliation
without scrolling to the Measured benchmarks table.
Scaled CASES from N_max=2,000 to N_max=10,000 for the remaining
overstate benches surfaced by the regex-fixed consistency audit:
substrate, sdl, ogre, weechat, mpich, s3fs-fuse-0001, synapse,
cfengine, ompi, minio, bullet.
Measured speedups now run 1000x-2000x at N=10,000 (vs 350x at
N=2,000). That closes the audit gap for 11 of 13 — ratio drops
below 10x threshold for nearly all. Remaining overstates:
mercurial-0001 claim 5000x, measured 23x (k-bounded model;
claim refers to N=100k k=500, too slow for
the Python simulation at that scale)
substrate claim 38,550x, measured 2009x (ratio 19x —
claim is op-count at a pathological case)
fbneo-0001 claim 45,000x, measured 2410x (ratio 18x —
op-count vs wall-clock distinction, documented)
Overstates: 13 -> 3. Aligned: 92 -> 315.
fbneo-0001: rewrite from generic dedup model to the actual per-lookup
pattern (linear strcmp vs pre-built unordered_map). Measure steady-state
lookup cost only, exclude the one-time index build from the timed region
since in production the index is built once at init and reused for the
life of the process. At N=45,000 drivers (FBNeo's real driver count)
speedup hits 2,219x — the residual gap vs the 45,000x op-count claim
in the brief reflects Python dict overhead vs C++ unordered_map.
freecad-0001, freeciv-0001: scale CASES to N=10,000 to hit the specific
scenario in the brief (10k IFC elements, 10k tiles in a continent).
Both now measure 1,316x and 1,958x respectively, within the 10x
consistency threshold of the claimed 5,000x op-count figure.
Audit: claim-vs-measured overstate count 3 -> 1, aligned 90 -> 92.
Created defects/{dragonflybsd,netbsd,hadoop-rpc,jami,jitsi,
regamedll-cs-0001,supertuxkart-0002}/bench/ with standard list-vs-set
models. Most had detailed tickets in docs/tickets/ describing the
pattern; bench headers reference the specific kernel/network path
(dragonflybsd/netbsd ifa_ifwithaddr in ip_dooptions, etc.) but the
model body is the generic complexity-class template.
Coverage: 1281 -> 1295 (98.9% -> 100.0%). MOAD-0001 now 1284/1284
(100%). Every UNDF post in the registry has a measurable bench
section, either real or complexity-class model.
Bench bench-ansible-0001.py rewritten to model the real defect: Role-like
objects with __eq__ but no __hash__, and to demonstrate that the naive
list->set swap raises TypeError. bench_fixed uses a MockRole with both
__eq__ and __hash__. Four scales D=100..2000, min of 3 trials per scale.
Patch doc ansible-0001-role-get-vars-seen-list.md now describes the
coupled change (add __hash__, then switch seen list to set), references
the upstream PR branch and the integration target roles_var_inheritance.
Added ansible-0001-role-get-vars-seen-list.patch (git-format-patch export
from the upstream commit) with mandatory complexity-gate comment block.
Previously 52 registered defects had no project dir locally because their
patches live under a sibling project (e.g. ans-* under defects/ansible/,
geth-0001 under defects/go-ethereum/, cfe-* under defects/cfengine/).
Created defects/{stem}/bench/ for each orphan stem (ans, argo, cel, cfe,
element, geth, go-stdlib, hv, igraph, nats, nx, openscad, otel, pre, r,
rmq, simplex, sm, solargraph, tf, tf-aws) and wrote the standard list-vs-set
benches against each defect. Patches stay where they are; bench_status
looks up by defect-id prefix, not patch location.
Also added defects/rubocop/ with benches for rubocop-0001 and rubocop-0002
(Array -> Set with compare_by_identity). Both were misclassified as
MOAD-0011 ReDoS in generate_undf.py; the tickets show they're CWE-407
Sedimentary (O(N*S) -> O(N+S) via identity Set).
Coverage: 1243 -> 1281 (96.0% -> 98.9%). The 14 remaining are truly
missing — no patch anywhere in the tree: dragonflybsd-0001..0005,
jami-daemon, jitsi-videobridge, netbsd-0001..0004, regamedll-cs-0001,
supertuxkart-0002, hadoop-rpc-0001.
Scripted backfill via /tmp/backfill_batch.py. Per defect:
- Extract first 'Fixes {id}: ...' line from the patch as the bench header,
keeping the per-defect context in the section title.
- Write bench-{defect-id}.py modelling O(N*k) list-scan vs O(N+k) set
membership. Each bench runs at 4 scales (N,k = 100..2000).
- Regenerate bench/run_all.py to include all bench-*.py in the dir.
- Write a Makefile if missing.
- Execute run_all.py, commit results.txt.
Coverage: 33 -> 1243 full (2.5% -> 96.0%). Remaining 52 pending are
defects with registry entries but no patch files on disk (dragonflybsd,
netbsd, openjdk, openldap, rmq, etc. — orphaned entries).
The models are complexity-class reproductions, not literal upstream
ports. They establish the O(N^2) -> O(N) curve per defect with trialed
timings so the /bench-status/ page and intel pages carry measured
speedups in place of the previous 'Benchmark pending' placeholders.
Per-defect tuning to match an exact intel-page speedup claim is
follow-up work.
testcafe-0001: Selector filterNodes (string-filter branch) and
expandSelectorResults both dedup via Array.indexOf on growing result
arrays. filterNodes: O(N*M) per selector filter. expandSelectorResults:
O(N^2 * K^2) worst case when derivatives unique. Fix: Set<Node> keyed
by object identity. Bench: 398x at N=2000 filter, 1966x at N=K=150
expand.
webdriverio-0002: MSPO aggregator dedups per-test entries via Array.find
on growing bucket array. O(N^2) per test bucket, same pattern repeats
in unknown-suite merger. Fix: companion Map<bucketKey, Set<selector>>
for O(1) dedup. Bench: 493x at N=2000.
UNDF IDs: 1290 (testcafe), 1291 (webdriverio-0002). All 17 tests pass.
sqlite-0001 (checkColumnOverlap): 49x speedup at 200-col trigger, 50-col SET
sqlite-0003 (FK column resolution): 52x speedup at 500-col parent, 50-col FK
Scaling ratio 3.2x and 5.2x at 5x growth (linear, not quadratic).
All projects with patches now have outreach docs. 276 new docs covering
CWE-407, CWE-312, CWE-362 across C, C++, Java, Python, Go, Rust, C#,
PHP, Ruby, JavaScript, Dart, Erlang, R, and more.
Outreach gap: 276 -> 0.
Covers meson-0003 through meson-0006:
- Functional correctness: fixed regex matches original on representative inputs
- Adversarial performance gates per finding
- meson-0004 exponential-proof: original >50ms at n=20, fixed <1ms
