B&W print-friendly diagrams + tinkerpop-0001 + wave-3 proof sections. Squash of 94 local commits onto remote master.
1285 lines
69 KiB
Markdown
1285 lines
69 KiB
Markdown
# CWE-407 Fix — First, Second, Third Order Effects & Blast Radius
|
||
|
||
**Internal only. No external distribution until all patches, tests, and white paper are complete.**
|
||
|
||
---
|
||
|
||
## The unlock
|
||
|
||
Every defect in this map is the same structural error: a list used where a set/map belongs,
|
||
inside a graph traversal loop. The fix is local, mechanical, and provably correct. But because
|
||
these defects live in compilers, build tools, package managers, and language runtimes — tools
|
||
that sit at the base of the software stack — the effects propagate upward through every layer
|
||
that was ever built with them.
|
||
|
||
The complexity change is O(V²) → O(V+E) for the hot path. At V=1000 nodes (a large Java
|
||
inference graph, a TypeScript checker cycle, a Tarjan SCC over package deps), this is a
|
||
**1000× reduction in membership-check operations** for that path. In practice most graphs are
|
||
small and the speedup is 2–10×, but the effect is real, measurable, and cumulative across
|
||
every build.
|
||
|
||
---
|
||
|
||
## First order effects — the patched tools themselves
|
||
|
||
These are direct. Each patched tool gets faster and its users see it immediately.
|
||
|
||
| Tool | Defect(s) | What gets faster |
|
||
|------|-----------|-----------------|
|
||
| **javac** | javac-0001..0005 | Type inference, dependency analysis, every Java compilation |
|
||
| **TypeScript tsc** | ts-0001..0003 | Cycle detection in module resolution and symbol merging |
|
||
| **GHC** | ghc-0001..0004 | SCC decode, codegen edge queries, register allocation, type-class checking |
|
||
| **Kotlin compiler** | kotlin-0001 | Non-expansive inheritance restriction checking |
|
||
| **Scala 3** | scala3-0001 | Constraint solving in type inference (currently O(n³)) |
|
||
| **CPython peg_generator** | cpython-0001 | Grammar SCC detection (affects CPython devs building Python itself) |
|
||
| **pip / distlib** | distlib-0001 | Dependency cycle detection during `pip install` |
|
||
| **GCC** | gcc-0001 | Johnson's algorithm in gcov coverage analysis |
|
||
| **LLVM / Clang** | llvm-0001 | Link-time optimization call graph traversal |
|
||
| **rustc** | rustc-0001..0002 | Match exhaustiveness checking, specialization graph build |
|
||
| **Maven** | maven-0001..0003 | Project dependency graph edge removal and cycle reporting |
|
||
| **CMake** | cmake-0001 | Link dependency group traversal |
|
||
| **npm arborist** | npm-0002 | Peer dep placement (`npm-0001` was NOT-A-DEFECT — `_depsSeen` already a `Set`) |
|
||
| **Cargo** | cargo-0001 | `cargo tree` display (display-only, bounded) |
|
||
| **Erlang stdlib** | erlang-0001 | `digraph:get_path`, `get_cycle`, `get_short_path` (`erlang-0002` FIXABLE-UPSTREAM — requires sltab in digraph.erl) |
|
||
| **Linux headerdep** | linux-0001 | Header dependency cycle detection (build tooling) |
|
||
|
||
### Blast radius at first order
|
||
- **Low.** All patches are local, behavioral equivalence is provable, and we have unit tests
|
||
with exact operation counts that guard against regression.
|
||
- **Risk:** A patch that changes iteration order in SCC output could break a downstream
|
||
consumer that assumed a specific ordering. Mitigation: test SCC output order explicitly.
|
||
|
||
---
|
||
|
||
## Second order effects — ecosystems built on the patched tools
|
||
|
||
### Java / JVM ecosystem (javac patches)
|
||
|
||
Everything compiled by javac benefits from faster type inference. This includes:
|
||
|
||
- **Spring Framework / Spring Boot** — millions of annotations processed per build
|
||
- **Apache Kafka, Hadoop, Cassandra, HBase** — large codebases with heavy generics
|
||
- **Android SDK toolchain** — every Android app build
|
||
- **Gradle / Maven builds** — CI/CD time drops globally
|
||
- **Bazel Java rules** — incremental builds get faster at the inference layer
|
||
|
||
#### JVM-based blockchain infrastructure
|
||
|
||
The javac patches propagate into every blockchain project built on the JVM. These are not
|
||
marginal systems — several are production financial infrastructure handling billions of dollars
|
||
in value daily.
|
||
|
||
| Project | Language | Role | Why javac matters |
|
||
|---------|----------|------|-------------------|
|
||
| **Hyperledger Besu** | Java | Full Ethereum execution client (EVM, P2P, state) | Entire codebase compiled with javac; own source is a HIGH scan target |
|
||
| **Hedera Hashgraph** | Java | Hashgraph consensus network (HBAR) | Full Java stack; consensus algorithm has graph traversal |
|
||
| **Corda / R3** | Kotlin | Enterprise permissioned ledger (financial institutions) | Kotlin compiles via javac; also benefits from kotlin-0001 patch |
|
||
| **Tron** | Java | Smart contract platform (TVM, DPoS) | Full Java validator stack |
|
||
| **Waves** | Scala | Smart contract platform | Scala compiles via javac; also benefits from scala3-0001 |
|
||
| **NEM / Symbol** | Java/TypeScript | Enterprise blockchain | Java core benefits from javac patches |
|
||
| **Hyperledger Fabric Java SDK** | Java | Permissioned ledger used by IBM, banks | SDK compiled with javac |
|
||
| **IOTA** | Rust | DAG-based ledger | Clean (Rust ecosystem; already confirmed) |
|
||
|
||
**Besu scan priority:** Hyperledger Besu is the only full Java Ethereum execution client
|
||
(alternatives geth/Nethermind/Erigon are Go/C#/Go — see CLEAN list). Besu compiles Solidity
|
||
to EVM bytecode, maintains a P2P peer graph, manages a Merkle-Patricia trie for state, and
|
||
runs EVM execution for every transaction. Graph traversal is endemic. Besu is flagged as a
|
||
HIGH scan target; scan deferred pending current wave completion.
|
||
|
||
**Second-order blast radius:** High surface area, low risk per change. The javac patches are
|
||
already tested against the installed JDK 21. The risk is that enterprise teams using older JDK
|
||
versions get the fix at different times, creating a fragmented rollout window. For financial
|
||
infrastructure (Corda, Besu, Hedera), the rollout coordination matters — these teams have
|
||
their own release cycles and may not pick up a JDK patch quickly.
|
||
|
||
### Python ecosystem (cpython, pip/distlib)
|
||
|
||
- **pip install** — every Python developer, every Docker build, every CI/CD pipeline
|
||
- **virtualenv, pipenv, poetry** — all vendor distlib or depend on pip
|
||
- **PyPI infrastructure** — resolver runs on the server side too
|
||
- **Conda** — uses its own solver but pip-compatible layer is affected
|
||
- **Docker Python base images** — `pip install -r requirements.txt` in Dockerfile layers
|
||
is a global hotspot; faster dep resolution → faster Docker builds → faster CI
|
||
|
||
**pip specifically:** The distlib Tarjan SCC runs during `pip install` when detecting circular
|
||
dependencies in the candidate resolution set. For projects with large transitive dep graphs
|
||
(e.g., `pip install tensorflow`, `pip install scipy`), this is a non-trivial path.
|
||
|
||
### TypeScript / JavaScript ecosystem (tsc patches)
|
||
|
||
- **React, Angular, Vue, Next.js** — type-checked with tsc on every save and CI run
|
||
- **VS Code** — ships its own tsc fork; language server does cycle detection constantly
|
||
during editing
|
||
- **Deno** — uses TypeScript compiler internals
|
||
- **Vite, esbuild, webpack** — type checking layer
|
||
- **npm, pnpm, yarn** — arborist patch (npm-0001/0002) affects every `npm install`
|
||
|
||
**Note on VS Code:** The language server runs tsc continuously. ts-0001/0002/0003 affect
|
||
interactive editing performance (symbol resolution latency, auto-complete lag in large
|
||
codebases). Fixing these is a user-visible UX improvement, not just a build-time win.
|
||
|
||
### Erlang ecosystem (erlang-0001 patched, erlang-0002 FIXABLE-UPSTREAM)
|
||
|
||
`digraph` and `digraph_utils` are OTP stdlib — the graph library for the entire Erlang
|
||
and Elixir ecosystem. The erlang-0002 fix (`loop_vertices/1`, `is_simple/1`: O(V²) →
|
||
O(V)) propagates to every application that calls those functions on OTP upgrade.
|
||
|
||
**Systems that get faster:**
|
||
- **RabbitMQ** — uses `digraph` for exchange routing graph validation (topology cycles,
|
||
simplicity checks on the exchange graph during reconfiguration)
|
||
- **ejabberd** — XMPP routing graph; topology validation at cluster join
|
||
- **Rebar3** — Erlang build tool; dependency graph analysis
|
||
- **Mix / Hex** — Elixir build tool; same OTP digraph calls
|
||
- **Any Erlang/Elixir application** that calls `loop_vertices/1` or `is_simple/1`
|
||
|
||
**The speed increase is real and correct. But it carries secondary risk for
|
||
timing-sensitive systems.**
|
||
|
||
#### The throttle risk — queue-based and financial systems
|
||
|
||
Erlang is the runtime of choice for telecom infrastructure, financial messaging, and
|
||
high-throughput queue systems. Some of these systems have been capacity-planned and
|
||
operationally tuned around *observed* performance characteristics of the current OTP
|
||
runtime, including graph operations that run during topology changes.
|
||
|
||
**Risk pattern:** If `loop_vertices` or `is_simple` was running slowly enough to act
|
||
as an implicit throttle during exchange graph reconfiguration (e.g., RabbitMQ
|
||
vhost topology change, ejabberd MUC room graph validation), downstream consumers
|
||
may have been sized assuming that rate. A sudden 100×–1000× speedup in that path
|
||
changes the rate at which topology changes are processed, potentially triggering
|
||
thundering-herd behavior in systems that were never expected to handle topology
|
||
changes at that speed.
|
||
|
||
**Specific systems to audit before deploying the OTP patch:**
|
||
|
||
| System | Risk | Why |
|
||
|--------|------|-----|
|
||
| **RabbitMQ** | Medium | Exchange topology validation rate increases; downstream consumers of topology-change events need capacity review |
|
||
| **Financial messaging (LMAX Disruptor-style Erlang systems)** | Medium-High | Queue scheduling and backpressure logic may be calibrated to current digraph latency |
|
||
| **Stock exchange order routing** | High if affected | Any Erlang-based order router where exchange graph validation is in the latency-critical path must be re-benchmarked |
|
||
| **ejabberd MUC** | Low | Room graph ops are infrequent and not in the message routing hot path |
|
||
| **Rebar3 / Mix** | None | Build tooling only; faster is unambiguously good |
|
||
|
||
**The general principle:** This is the "faster is dangerous when slow was load-limiting"
|
||
problem. It applies to any OTP-based system where:
|
||
1. `loop_vertices/1` or `is_simple/1` runs during a state-change event
|
||
2. That event feeds a downstream system with a fixed processing budget
|
||
3. That downstream system was sized against the current (slow) call latency
|
||
|
||
**Mitigation:** Before deploying the OTP patch in any financial or queue-based
|
||
production system:
|
||
1. Identify all call sites of `digraph_utils:loop_vertices/1` and `is_simple/1`
|
||
in the application and all dependencies
|
||
2. Measure the current call latency under production-representative load
|
||
3. Model the downstream effect of 100× speedup at those sites
|
||
4. Adjust backpressure, rate limiting, or consumer capacity as needed
|
||
5. Stage the rollout: canary → 10% → 100% with monitoring on downstream queue depth
|
||
|
||
### Haskell ecosystem (GHC patches)
|
||
|
||
- **Pandoc** — compiled with GHC, used everywhere for document conversion
|
||
- **Cardano** — blockchain written in Haskell; smart contract compilation is affected
|
||
- **XMonad, Yi** — Haskell tooling
|
||
- **Stack, Cabal** — build tools that invoke GHC; faster GHC = faster Haskell builds
|
||
- **ghc-0002 codegen** — every function that generates LLVM IR via GHC gets the benefit
|
||
|
||
### Rust ecosystem (rustc patches)
|
||
|
||
- **Firefox** — compiled with rustc; match exhaustiveness checker (rustc-0001) runs
|
||
on every enum
|
||
- **ripgrep, fd, bat, exa** — popular CLI tools; faster specialization builds
|
||
- **Servo** — rendering engine in Rust
|
||
- **Cargo itself** — cargo-0001 is display-only, bounded, but still a principle violation
|
||
|
||
### Browser ecosystem
|
||
|
||
Browsers are among the largest and most performance-critical C++/Rust codebases on the
|
||
planet. All three major engines are affected by patches already in this map.
|
||
|
||
**Firefox**
|
||
- Compiled with Clang; LLVM LTO enabled in release builds → llvm-0001 applies directly
|
||
- Large Rust codebase: WebRender (GPU compositor), Stylo (CSS engine), Servo components
|
||
→ rustc-0001 (match exhaustiveness) and rustc-0002 (specialization graph) both apply
|
||
- SpiderMonkey IonMonkey JS engine: **CLEAN** (confirmed — uses hash containers throughout)
|
||
- TypeScript used in Firefox DevTools and web-ext tooling → ts-0001..0003 apply
|
||
|
||
**Chrome / Chromium**
|
||
- Compiled with Clang; LLVM LTO in all release builds → llvm-0001 applies
|
||
- V8 TurboFan JIT compiler: **CLEAN** (confirmed — no CWE-407 candidates)
|
||
- TypeScript used in DevTools, Chrome Extensions API, web platform test tooling
|
||
- Large npm dependency graph for web platform tooling → npm arborist patch applies
|
||
|
||
**Safari / WebKit**
|
||
- Compiled with Clang; LLVM LTO applies → llvm-0001 applies
|
||
- JavaScriptCore (JSC): not yet scanned; lower probability than SpiderMonkey/V8 given
|
||
Apple's engineering culture, but a candidate
|
||
- WebKit build system uses CMake → cmake-0001 applies
|
||
|
||
**The LTO point:** All three browsers ship release builds with link-time optimization
|
||
enabled. LLVM's GlobalsModRef call-graph traversal (llvm-0001) runs across the entire
|
||
binary during LTO. Firefox is ~10M LOC, Chromium is ~35M LOC. For Chromium specifically,
|
||
LTO build time is a known bottleneck — the GlobalsModRef fix is material.
|
||
|
||
**Browser JS engine scan summary:**
|
||
|
||
| Engine | Browser | Result |
|
||
|--------|---------|--------|
|
||
| V8 TurboFan | Chrome | **CLEAN** |
|
||
| SpiderMonkey IonMonkey | Firefox | **CLEAN** |
|
||
| JavaScriptCore | Safari | Not yet scanned |
|
||
|
||
### C/C++ ecosystem (GCC, LLVM, CMake patches)
|
||
|
||
This is the broadest surface area. GCC and LLVM compile essentially everything:
|
||
|
||
- **PostgreSQL** — compiled with GCC/Clang; build time improves
|
||
- **SQLite** — compiled with GCC/Clang
|
||
- **MySQL / MariaDB** — compiled with CMake + GCC/Clang; CMake patch (cmake-0001)
|
||
directly speeds up the MySQL build's link-dependency resolution
|
||
- **Apache httpd (apache2)** — autotools + GCC; build time improves
|
||
- **nginx** — GCC; build time improves
|
||
- **Caddy** — written in Go (Go compiler already clean ✓); no effect here
|
||
- **OpenSSL, libssl** — GCC/Clang compilation benefits
|
||
- **Linux kernel** — GCC + headerdep.pl (linux-0001) patched
|
||
|
||
**LLVM LTO specifically (llvm-0001):** Link-time optimization is used by default in release
|
||
builds of Firefox, Chrome, Rust's standard library, LLVM itself, and PostgreSQL with `--enable-lto`.
|
||
The GlobalsModRef call-graph traversal runs during LTO. For large LTO builds (Firefox is
|
||
~10M LOC), this is a meaningful path.
|
||
|
||
### Second-order blast radius summary
|
||
|
||
| Ecosystem | Risk level | Primary concern |
|
||
|-----------|-----------|-----------------|
|
||
| JVM / Android | Medium | JDK rollout fragmentation across versions |
|
||
| Python / pip | Low-Medium | pip is heavily tested; distlib change is isolated |
|
||
| TypeScript / npm | Medium | VS Code ships its own tsc; needs separate coordination |
|
||
| Haskell | Low | GHC releases are infrequent, community is small |
|
||
| Rust | Low | rustc team has strong test infra |
|
||
| C/C++ / GCC / LLVM | Medium-High | Widest surface; GCC/LLVM release cycles are long |
|
||
|
||
---
|
||
|
||
## Third order effects — infrastructure and runtime systems
|
||
|
||
### Databases
|
||
|
||
**PostgreSQL**
|
||
- Build-time: benefits from GCC/CMake fix (faster to compile from source)
|
||
- Runtime: PostgreSQL's query planner has **5 confirmed CWE-407 defects** (postgresql-0001
|
||
through -0005) in `tlist.c`, `preptlist.c`, `equivclass.c`, `analyzejoins.c`, and `list.c`.
|
||
All are **DEFERRED** — the standard fix (replace list with hash set) is blocked by the
|
||
absence of a generic expression hash function in PostgreSQL core. See the executive summary
|
||
doc for full analysis. These sites require upstream PostgreSQL core collaboration before
|
||
a patch is possible.
|
||
- Extension ecosystem: PL/Python, PL/Perl, PostGIS — all pull in the patched language runtimes
|
||
|
||
**SQLite**
|
||
- Build-time: GCC fix applies
|
||
- Runtime: SQLite's query optimizer is simpler (no join reordering); lower risk of CWE-407
|
||
in its own source. However, SQLite is used as an embedded DB in Python (`sqlite3` module),
|
||
Ruby, PHP, Node.js — all of which are getting faster runtimes from our patches.
|
||
|
||
**MySQL / MariaDB**
|
||
- Build-time: CMake patch (cmake-0001) directly applies — MySQL build uses CMake heavily
|
||
- Runtime: MySQL's optimizer handles join graphs; candidate for its own CWE-407 scan
|
||
|
||
**MongoDB**
|
||
- Compiled with SCons + GCC/Clang; build improves
|
||
- Aggregation pipeline planner: candidate for scan
|
||
|
||
### Web servers
|
||
|
||
**apache2 (httpd)**
|
||
- Compiled with GCC; build-time improvement
|
||
- `mod_proxy`, `mod_rewrite` rule graphs: low complexity, bounded inputs
|
||
- Apache Traffic Server (ATS) — more complex routing graph; candidate for scan
|
||
|
||
**nginx**
|
||
- Compiled with GCC; build-time improvement
|
||
- nginx config parsing is linear, low-complexity graph work; low risk of CWE-407 in nginx itself
|
||
|
||
**Caddy**
|
||
- Written in Go — Go compiler already clean (✓ in our clean list)
|
||
- Caddy's own routing graph uses Go maps throughout; low risk
|
||
|
||
### GeoIP and geographic routing — special considerations
|
||
|
||
This is the most subtle third-order effect.
|
||
|
||
**The issue:** GeoIP accuracy is imperfect. GeoIP databases (MaxMind GeoLite2, IP2Location,
|
||
etc.) have known error rates — typically 95-99% accurate at country level, 60-80% at city
|
||
level. These errors cause:
|
||
- Misrouted CDN requests (user in Frankfurt hits London PoP)
|
||
- Payment fraud false positives (billing address country ≠ detected country)
|
||
- Content geo-restrictions misfiring
|
||
|
||
**The connection to our fix:** Geographic load balancers and CDN routing systems are built on
|
||
top of the language runtimes we're patching. Faster compilation and package resolution means:
|
||
1. Routing rule updates deploy faster → errors propagate faster too
|
||
2. If a GeoIP correction patch ships faster (because pip install / npm install / maven build is
|
||
faster), it reaches production faster — which is good when the correction is right, and
|
||
propagates faster when the correction itself contains an error
|
||
|
||
**Specific risk:** MaxMind's geoip2 Python library runs on CPython. If CPython's own build
|
||
tooling improves (cpython-0001 is in peg_generator, used when regenerating the parser), and
|
||
pip's dep resolution improves (distlib-0001), then GeoIP library updates reach production
|
||
faster. At scale (millions of IPs routed per second), even a brief incorrect GeoIP DB update
|
||
is amplified.
|
||
|
||
**Mitigation:** GeoIP database deployments should be blue/green with traffic validation at 1%
|
||
before full rollout — this is good practice regardless of our patches but becomes more
|
||
important as deployment velocity increases.
|
||
|
||
**The broader geo paradox:** Our fix makes the whole stack faster. Faster stacks reduce
|
||
latency. Reduced latency can shift requests between geographic regions (requests that were
|
||
timing out now succeed, from further away). This very slightly shifts the apparent distribution
|
||
of traffic origins — which feeds back into GeoIP accuracy metrics. This is a genuine second-
|
||
order effect that geo-aware systems (ad targeting, fraud detection, CDN) should be aware of.
|
||
|
||
### CI/CD and cloud infrastructure
|
||
|
||
**Docker image builds**
|
||
- Python base images: `pip install` in Dockerfile layers is the single biggest time sink
|
||
in most CI pipelines. distlib-0001 + cpython-0001 together reduce this.
|
||
- Maven/Gradle builds: Java CI pipelines benefit from javac patches
|
||
- npm install: arborist patches reduce dep tree construction time
|
||
|
||
**At scale:** GitHub Actions processes ~50M workflow runs/month. If each Java/Python/TS
|
||
workflow saves 5-15 seconds of build time, the aggregate is millions of compute-hours/month.
|
||
This is real money and real carbon.
|
||
|
||
**Serverless cold starts**
|
||
- AWS Lambda Python runtime: pip-installed dependencies ship in the Lambda layer;
|
||
faster resolution = smaller/simpler layers = faster cold starts
|
||
- Lambda Java runtime: javac inference improvements are baked into compiled JARs — no direct
|
||
cold-start benefit, but the JIT (C2 compiler, which uses LLVM-like graph analysis) may benefit
|
||
indirectly
|
||
|
||
---
|
||
|
||
## Blast radius mitigation plan
|
||
|
||
### Tier 1 — Before any patch is submitted upstream
|
||
|
||
1. **Every patch has a behavioral equivalence proof** — not just "tests pass" but a written
|
||
argument that output is identical for all inputs (SCC membership, ordering, cycle reporting)
|
||
2. **Operation-count unit tests** — if a test does not assert O(1) membership, it does not count
|
||
3. **Fuzz testing on graph structure** — random DAGs, random dense graphs, self-loops,
|
||
disconnected components, very large graphs (V=10,000+)
|
||
4. **No patch touches error messages or exception types** — changing "contains" to a set
|
||
must not change what gets thrown or printed when a cycle is detected
|
||
|
||
### Tier 2 — Before coordinated disclosure
|
||
|
||
5. **Upstream maintainer contact before public patch** — privately share the patch and proof
|
||
with the maintainer; give them 90 days to merge and release
|
||
6. **Sequence disclosure by blast radius** — patch low-surface tools first (peg_generator,
|
||
distlib, erlang stdlib) before high-surface tools (javac, tsc, GHC)
|
||
7. **Version compatibility testing** — test each patch against the last 3 major releases of
|
||
the affected tool, not just HEAD
|
||
|
||
### Tier 3 — Infrastructure-specific
|
||
|
||
8. **Database query planners** — scan PostgreSQL, MySQL, MongoDB source for CWE-407 before
|
||
disclosing compiler fixes. If db planners have the same defect, coordinate disclosure
|
||
together. Revealing "compilers are fixed" while "your DB planner has the same bug" creates
|
||
an exploit window.
|
||
9. **GeoIP deployment velocity** — note in the white paper that faster deployment pipelines
|
||
increase the importance of staged rollouts for data updates (not just code)
|
||
10. **CDN / routing system operators** — brief major CDN operators (Cloudflare, Fastly, Akamai)
|
||
as part of coordinated disclosure. Their build pipelines are affected; their traffic routing
|
||
systems may independently contain the same defect.
|
||
|
||
### Tier 4 — Post-disclosure monitoring
|
||
|
||
11. **Regression watch** — monitor upstream repos for 6 months post-disclosure for any
|
||
performance regression reports that could be attributed to ordering changes in SCC output
|
||
12. **CVE coordination** — CWE-407 in a build tool is typically a DoS via crafted input
|
||
(an adversary can construct a source file that maximizes the quadratic behavior). File CVEs
|
||
for tools that accept untrusted input (tsc, javac, GCC/Clang — all accept user source).
|
||
Do NOT file CVEs for internal-only tools where input is trusted.
|
||
|
||
---
|
||
|
||
## Fourth frontier: scientific computing and numerical analysis
|
||
|
||
This is the domain where the topology defect may be causing the most *invisible* damage.
|
||
Scientific computing works on genuinely large graphs — protein interaction networks (V=20,000+),
|
||
genomics dependency graphs, finite element meshes, neural computation graphs, Monte Carlo
|
||
dependency chains. At these scales, O(V²) is not "a bit slow" — it is **computationally
|
||
unobservable**. Researchers simply never run the algorithm on the full dataset; they subsample,
|
||
they approximate, they accept that "large graphs are slow."
|
||
|
||
### NetworkX (Python)
|
||
|
||
NetworkX is the dominant pure-Python graph library. Used by: bioinformatics, social network
|
||
analysis, quantum circuit simulation, ML pipeline graphs, physics simulations.
|
||
|
||
NetworkX implements Tarjan SCC, Kosaraju SCC, DFS, topological sort, cycle detection,
|
||
dominator trees, and dozens of other graph algorithms entirely in Python. The entire library
|
||
predates the widespread adoption of O(1)-first idioms in Python graph code.
|
||
|
||
**High-probability CWE-407 targets in NetworkX:**
|
||
- `networkx/algorithms/components/strongly_connected.py` — Tarjan/Kosaraju SCC
|
||
- `networkx/algorithms/cycles.py` — `simple_cycles()`, `find_cycle()`
|
||
- `networkx/algorithms/dag.py` — topological sort, cycle detection
|
||
- `networkx/algorithms/dominance.py` — dominator tree construction
|
||
- `networkx/algorithms/traversal/depth_first_search.py` — DFS with visited tracking
|
||
|
||
If any of these use `list` for the visited/stack/path set, every scientific computing
|
||
workflow that calls them on large graphs has been running at O(V²) instead of O(V+E).
|
||
At V=10,000, that is a 10,000× error in expected runtime. Researchers would observe this
|
||
as "NetworkX doesn't scale" and switch to igraph or switch languages — never knowing the
|
||
fix was one data structure change.
|
||
|
||
**Scan priority: CRITICAL.** NetworkX is installed in virtually every scientific Python
|
||
environment. Clone and scan immediately.
|
||
|
||
### SciPy `csgraph`
|
||
|
||
`scipy.sparse.csgraph` implements Dijkstra, Bellman-Ford, Floyd-Warshall, minimum spanning
|
||
tree, connected components, and shortest paths. The core algorithms are written in Cython
|
||
and compiled to C — the hot paths are likely clean. But:
|
||
|
||
- The Python dispatch layer wraps these C routines and may do list-based bookkeeping
|
||
- `scipy.sparse.csgraph.depth_first_order` — DFS with predecessor/successor arrays;
|
||
the Python-level visited set tracking is a candidate
|
||
- `scipy.sparse.csgraph.minimum_spanning_tree` — Kruskal uses union-find (clean) but
|
||
Prim variants may not
|
||
|
||
SciPy is used in: finite element analysis, fluid dynamics simulation, computational
|
||
chemistry, signal processing pipelines. Wrong graph complexity at this layer means
|
||
numerical simulations are taking longer than the physics requires.
|
||
|
||
### NumPy
|
||
|
||
NumPy itself does not implement graph algorithms. But NumPy arrays are frequently used as
|
||
adjacency matrices fed into graph libraries — and the conversion layer (numpy array →
|
||
graph structure) may introduce O(n) membership patterns. Lower priority than NetworkX,
|
||
but the `numpy.lib.arraysetops` and `numpy.unique` functions used in graph preprocessing
|
||
pipelines are worth scanning for misuse.
|
||
|
||
### igraph (C core + Python/R bindings)
|
||
|
||
igraph is a C library widely used in network science and bioinformatics. The C core is
|
||
likely clean (C programmers tend to use arrays with boolean flags). But the Python and R
|
||
binding layers, and the igraph R package's pure-R wrappers, are candidates.
|
||
|
||
### Graph-ML frameworks
|
||
|
||
- **PyTorch Geometric (PyG)** — graph neural networks. Uses Python-level graph traversal
|
||
for neighborhood sampling, subgraph extraction, message passing setup.
|
||
- **DGL (Deep Graph Library)** — similar. Graph partitioning and traversal in Python layer.
|
||
- **TensorFlow graph executor** — C++. Execution graph SCC and topological sort are
|
||
internal; likely clean (Google engineers), but worth scanning.
|
||
- **JAX** — computation graph tracing in Python. `jax.core` builds and traverses Jaxpr
|
||
graphs during tracing. Candidate for CWE-407 in the trace/compilation path.
|
||
|
||
**The ML training implication:** If graph traversal in a GNN framework's data loading or
|
||
batching code is O(V²), large-graph training runs that appear to stall at the data
|
||
preparation stage may actually be fixable with a one-line patch. This would directly
|
||
reduce training costs at scale.
|
||
|
||
### Numerical analysis: the "wrong answer" risk
|
||
|
||
This is the most serious concern beyond performance. Some numerical algorithms use graph
|
||
traversal to determine computation order (e.g., sparse matrix factorization, automatic
|
||
differentiation, constraint propagation). If the traversal produces a **different ordering**
|
||
due to a latent defect — not a performance defect but an **ordering defect** — the
|
||
numerical results themselves could be subtly wrong.
|
||
|
||
Example: sparse Cholesky factorization uses a fill-reduction ordering step (AMD, METIS)
|
||
that involves graph traversal. If a visited-set defect causes a node to be processed twice
|
||
or skipped, the fill pattern changes. The factorization still "works" but has higher fill
|
||
than optimal, consuming more memory and producing different round-off error patterns.
|
||
|
||
This is speculative but must be ruled out. The white paper needs a section specifically
|
||
addressing whether any of the defects in our map could produce incorrect output (not just
|
||
slow output) under any input. Current assessment: **no** — all confirmed defects degrade
|
||
to O(n²) but produce correct output. But numerical computing chains using these libraries
|
||
must be individually verified.
|
||
|
||
---
|
||
|
||
## Fifth frontier: network routing protocols and infrastructure
|
||
|
||
This is where the topology defect ceases to be a software quality issue and becomes a
|
||
**live infrastructure reliability issue.**
|
||
|
||
Network routing protocols are graph algorithms running continuously on production hardware,
|
||
reacting to topology changes in real time. If their graph traversal has quadratic membership
|
||
checks, the convergence behavior of the internet itself is degraded relative to theoretical
|
||
bounds.
|
||
|
||
### BGP (Border Gateway Protocol)
|
||
|
||
BGP is the routing protocol of the internet — it maintains reachability between all
|
||
autonomous systems (ASes). BGP routers maintain route tables with 900,000+ IPv4 prefixes
|
||
and process updates continuously.
|
||
|
||
**AS-path loop detection:** BGP prevents routing loops by checking if the local AS number
|
||
appears in the AS-path of an incoming route. In a naive implementation, this is a linear
|
||
scan of the path list. For typical paths (4-8 ASes) this is negligible. But:
|
||
|
||
- During BGP route storms (mass withdrawal + re-advertisement), a router may process
|
||
millions of updates/second
|
||
- If the loop detection iterates a list rather than checking a set/bitmap, the cost per
|
||
update multiplies with path length
|
||
- Route reflectors in large ISP networks see paths of 20-50 ASes for international routes
|
||
|
||
**FRRouting (FRR)** — the most widely deployed open-source BGP/OSPF/IS-IS implementation.
|
||
Used by major cloud providers (Meta, Microsoft, LinkedIn use FRR derivatives). Written in C.
|
||
`bgpd/bgp_aspath.c` — AS-path manipulation and loop detection. **High-priority scan target.**
|
||
|
||
**BIRD** — widely used in IXP (Internet Exchange Point) route servers. Written in C.
|
||
`proto/bgp/` — BGP implementation. Candidate.
|
||
|
||
**ExaBGP** — Python BGP implementation used for route injection and traffic engineering.
|
||
Pure Python. If its path-traversal code uses list membership, it inherits the defect
|
||
directly. **Very high probability of CWE-407 given the language.**
|
||
|
||
**GoBGP** — Go implementation. Go uses maps natively. Likely clean. Worth verifying.
|
||
|
||
**OpenBGPD** — OpenBSD BGP daemon. C. BSD codebase tends to be careful but written in
|
||
pre-modern-idiom era. Candidate.
|
||
|
||
### OSPF (Open Shortest Path First)
|
||
|
||
OSPF runs Dijkstra's Shortest Path First (SPF) algorithm on the link-state database. SPF
|
||
is triggered every time the topology changes (link up/down, metric change). On large
|
||
networks (enterprise core, ISP backbone), SPF runs on graphs of hundreds to thousands
|
||
of nodes.
|
||
|
||
**If Dijkstra's visited set is a list:** O(V²) per SPF run. OSPF specs require SPF to
|
||
complete in milliseconds. A quadratic implementation on a 1000-node network doing 1M
|
||
operations instead of 1000 would fail to meet convergence timers, causing route
|
||
oscillation and potentially forming routing black holes during convergence.
|
||
|
||
**FRRouting `ospfd`** — `ospfd/ospf_spf.c`. SPF implementation in C. **Critical scan target.**
|
||
If this has CWE-407, it is a live network reliability defect in production ISP infrastructure.
|
||
|
||
**The convergence timer implication:** OSPF defines `SPF_DELAY` (default 200ms) and
|
||
`SPF_HOLDTIME` (default 1000ms). If SPF takes longer than expected due to quadratic
|
||
behavior, the hold-time backs off and convergence slows — making the network appear
|
||
to be "under load" when it is actually hitting a complexity defect.
|
||
|
||
### IS-IS (Intermediate System to Intermediate System)
|
||
|
||
IS-IS is the other major link-state IGP, preferred by many large ISPs (Google, Comcast)
|
||
and most carrier backbone networks. Also uses SPF. Same risk as OSPF.
|
||
|
||
**FRRouting `isisd`** — `isisd/isis_spf.c`. Candidate.
|
||
**CLNS IS-IS in IOS/IOS-XR** — Cisco's implementation, closed-source, cannot scan directly.
|
||
But if FRR has the defect, Cisco almost certainly inherited similar code from the same
|
||
1990s-era algorithm literature.
|
||
|
||
### MPLS and traffic engineering
|
||
|
||
MPLS label-switched paths are computed using RSVP-TE or SR-TE path computation. Path
|
||
computation involves constrained shortest-path first (CSPF) — Dijkstra with constraints.
|
||
CSPF runs on a graph of the entire network for each LSP setup. If visited-set is a list:
|
||
O(V²) per tunnel setup. In a network with thousands of MPLS tunnels being re-signaled
|
||
after a failure, this would cause a tunnel re-establishment storm.
|
||
|
||
**OpenDaylight (ODL)** — Java SDN controller. Implements PCE (Path Computation Element)
|
||
for MPLS-TE. Java + graph algorithms = **very high probability of CWE-407**. Used by
|
||
major telcos for network automation. Scan target.
|
||
|
||
**ONOS (Open Network Operating System)** — Java SDN controller used by AT&T, NTT, Comcast.
|
||
`core/api/src/main/java/org/onosproject/net/topology/` — topology service. Java list-based
|
||
graph traversal almost certain. **High-priority scan target.**
|
||
|
||
### Network middleware and service meshes
|
||
|
||
**HAProxy** — C, load balancer. Route selection is simple weighted round-robin or least-conn,
|
||
not graph-based. Low risk of CWE-407 in its own code. But HAProxy's configuration validator
|
||
may parse ACL dependency graphs — candidate for the config-parse path.
|
||
|
||
**Envoy Proxy** — C++, service mesh. Envoy's cluster graph, endpoint discovery, and routing
|
||
rule evaluation are all graph-structured. The xDS API builds a runtime graph of clusters,
|
||
endpoints, and listeners. `source/common/upstream/` — cluster dependency resolution.
|
||
**Medium-priority scan target.**
|
||
|
||
**Istio (control plane)** — Go. Pilot builds an Envoy configuration graph and pushes it.
|
||
Go-based, likely uses maps. But `pilot/pkg/networking/core/` — virtual service graph
|
||
resolution. Worth verifying.
|
||
|
||
**Consul** — Go service mesh. Likely clean.
|
||
|
||
**Linkerd** — Rust. Very likely clean.
|
||
|
||
**Cilium** — Go + eBPF. Policy graph in Go. Likely clean.
|
||
|
||
### The internet reliability implication
|
||
|
||
If FRR's OSPF SPF or BGP path-selection has a quadratic membership check:
|
||
|
||
1. **Every network failure event** triggers slower-than-specified convergence
|
||
2. **BGP route storms** (which happen regularly at major IXPs) cause CPU spikes that are
|
||
currently attributed to "BGP flapping load" but may actually be algorithmic overhead
|
||
3. **The internet's recovery time from fiber cuts, hardware failures, and DDoS attacks
|
||
is longer than it needs to be** — not by a little, but potentially by orders of
|
||
magnitude on large networks
|
||
|
||
This is not hypothetical. There are documented cases of BGP convergence taking minutes
|
||
instead of seconds on large networks. The standard explanation is "BGP is slow by design."
|
||
The actual explanation may include quadratic graph traversal.
|
||
|
||
### Routing scan backlog — immediate priority
|
||
|
||
| System | Language | Key file to scan | Why critical |
|
||
|--------|----------|-----------------|--------------|
|
||
| FRRouting bgpd | C | `bgpd/bgp_aspath.c`, `bgpd/bgp_route.c` | AS-path loop detection |
|
||
| FRRouting ospfd | C | `ospfd/ospf_spf.c` | Dijkstra SPF on every topology change |
|
||
| FRRouting isisd | C | `isisd/isis_spf.c` | IS-IS SPF, carrier backbone |
|
||
| FRRouting ldpd | C | `ldpd/lde_lib.c` | MPLS label distribution |
|
||
| BIRD bgp | C | `proto/bgp/bgp.c` | IXP route servers globally |
|
||
| ExaBGP | Python | `exabgp/bgp/message/update/attribute/aspath.py` | Pure Python, high probability |
|
||
| OpenDaylight PCE | Java | `pcep/` topology service | MPLS-TE path computation |
|
||
| ONOS topology | Java | `core/net/src/main/java/org/onosproject/net/topology/` | Major telco production |
|
||
| NetworkX | Python | `algorithms/components/`, `algorithms/cycles.py` | Scientific computing globally |
|
||
| ExaBGP | Python | all graph traversal paths | Very high CWE-407 probability |
|
||
|
||
---
|
||
|
||
## Sixth frontier: MATLAB, CAD, and engineering simulation software
|
||
|
||
### MATLAB
|
||
|
||
MATLAB is the primary computational tool for control systems, signal processing, circuit
|
||
simulation, and numerical methods in engineering. MathWorks ships MATLAB with a graph
|
||
and network algorithms toolbox and the core language runtime includes graph traversal
|
||
in multiple places.
|
||
|
||
**MATLAB's own graph algorithms (`graph` / `digraph` objects, introduced R2015b):**
|
||
- `conncomp()` — connected components (SCC for directed graphs)
|
||
- `toposort()` — topological sort with cycle detection
|
||
- `shortestpath()`, `shortestpathtree()` — Dijkstra/Bellman-Ford
|
||
- `isdag()` — cycle detection
|
||
|
||
These are implemented in MATLAB's compiled C/C++ runtime (MathWorks closed source).
|
||
Cannot scan directly. But the behavioral signatures are observable: benchmark
|
||
`conncomp(G)` on random digraphs as V grows. If runtime is O(V²) instead of O(V+E),
|
||
the defect is present.
|
||
|
||
**MATLAB's build/dependency tooling:**
|
||
- Simulink uses a signal-flow graph to determine block execution order. Block sorting is
|
||
topological sort. If the visited set in that sort uses MATLAB cell array membership
|
||
(the MATLAB equivalent of list membership — O(n) via `ismember()`), every Simulink
|
||
model compilation has this defect.
|
||
- MATLAB's `depfun()` and the newer `matlab.codetools.requiredFilesAndProducts()` build
|
||
dependency graphs. If linear membership is used, large codebases are slower than necessary.
|
||
|
||
**MATLAB m-file graph code:** The MATLAB community writes enormous amounts of graph
|
||
algorithm code in `.m` files. `ismember(x, list)` in MATLAB is O(n) by default (it sorts
|
||
and binary-searches, so O(n log n) for the sort + O(log n) query — better than naive O(n)
|
||
but still not O(1)). Code that uses `ismember()` in a DFS loop is O(V·n·log n). The
|
||
idiomatic fix is `containers.Map` (hash map) or logical indexing arrays.
|
||
|
||
The MATLAB file exchange (100,000+ submissions) and most academic graph theory `.m` files
|
||
predate idiomatic O(1) membership in MATLAB. The defect is endemic in the research codebase.
|
||
|
||
**Octave** (open-source MATLAB-compatible): same patterns, scannable. GNU Octave was scanned
|
||
(2026-03-23) and is **CLEAN** — all graph algorithms use vectorized ops and compiled C routines,
|
||
not list-backed visited sets. The MATLAB `ismember` risk applies to user-authored `.m` files,
|
||
not Octave's own implementations.
|
||
|
||
### Simulink and Model-Based Design
|
||
|
||
Simulink is used to design control systems for aircraft, automobiles, medical devices, and
|
||
industrial machinery. The compiled output (via Embedded Coder) runs in safety-critical
|
||
hardware. The design-time graph traversal (execution order, algebraic loop detection,
|
||
rate transition analysis) uses the MATLAB runtime.
|
||
|
||
**Algebraic loop detection** is Tarjan SCC on the block diagram graph. If this runs at
|
||
O(V²), large Simulink models (aerospace, automotive — common at V=10,000 blocks) are
|
||
taking far longer to compile than necessary. Engineers accept slow model compilation as a
|
||
fact of life. It may not be.
|
||
|
||
**DO-178C / ISO 26262 implication:** If the model compiler has a performance defect, it
|
||
affects the model development cycle time but not (directly) the correctness of generated
|
||
code. However, if Simulink's cycle detection produces incorrect results due to an ordering
|
||
defect (not just a performance defect), that is a safety-critical issue. This must be
|
||
ruled out explicitly.
|
||
|
||
### CAD / EDA (Electronic Design Automation)
|
||
|
||
EDA tools are the compilers of hardware. They process netlists — graphs of logic gates,
|
||
wires, and timing constraints — and produce manufacturable chip designs. The graph
|
||
algorithms in EDA are among the most performance-critical in all of engineering.
|
||
|
||
**Key graph algorithms in EDA:**
|
||
- **Technology mapping** — covering a DAG of logic operations with library cells (graph
|
||
covering, DFS-based)
|
||
- **Static timing analysis (STA)** — longest path in a DAG (topological sort + DP)
|
||
- **Place and route** — graph partitioning, Steiner tree, maze routing
|
||
- **Equivalence checking** — SCC-based circuit comparison
|
||
- **Power analysis** — reachability in switching activity graph
|
||
|
||
**Open-source EDA tools (scannable):**
|
||
|
||
| Tool | Language | Graph algorithm | Scan target |
|
||
|------|----------|----------------|-------------|
|
||
| **Yosys** | C++ | synthesis, technology mapping, SCC | `passes/opt/`, `kernel/rtlil.cc` |
|
||
| **OpenROAD** | C++ | placement, routing, timing | `src/odb/`, `src/sta/` |
|
||
| **OpenSTA** | C++ | static timing analysis, DAG traversal | `graph/`, `search/` |
|
||
| **ABC** (Berkeley) | C | logic synthesis, DAG rewriting | `src/base/abci/` |
|
||
| **VPR** (Verilog-to-Routing) | C++ | FPGA place and route | `vpr/src/route/` |
|
||
| **Icarus Verilog** | C++ | netlist elaboration, dependency graph | `tgt-vvp/` |
|
||
| **Verilator** | C++ | RTL simulation, SCC for clock domains | `src/V3Graph.cpp` |
|
||
|
||
**Verilator specifically:** `V3Graph.cpp` and `V3GraphAlg.cpp` are Verilator's internal graph
|
||
library. Used for SCC computation on hardware description language graphs. Verilator is used
|
||
by Google, lowRISC, and chip startups to verify RISC-V designs. If its SCC has a quadratic
|
||
membership check, large chip verification runs are slower than necessary.
|
||
|
||
**Commercial EDA (Cadence, Synopsys, Mentor):** Closed source, cannot scan. But the same
|
||
algorithm literature was used. Performance benchmarks of commercial tools on large netlists
|
||
may reveal the signature of quadratic behavior.
|
||
|
||
**The chip design implication:** EDA tool runtime directly determines chip design cycle time.
|
||
Longer compile times → fewer design iterations → worse final chip quality. If O(V²) graph
|
||
traversal is embedded in EDA tools, the chips being designed today are suboptimal relative
|
||
to what the tools could produce with correct complexity.
|
||
|
||
### Other CAD / simulation systems
|
||
|
||
**FreeCAD / OpenCASCADE** — C++. Parametric dependency graph for feature ordering. If
|
||
feature rebuild order uses list-based visited tracking, complex assemblies rebuild slowly.
|
||
|
||
**KiCad** — C++. PCB netlist graph for DRC (design rule check) and copper pour. DRC
|
||
involves connectivity analysis. `pcbnew/connectivity/` — scan target.
|
||
|
||
**Blender** — C/Python. Node graph compositor and geometry nodes use topological sort for
|
||
execution order. `source/blender/blenkernel/intern/node.cc` — dependency graph. Candidate.
|
||
|
||
**FEniCS / OpenFOAM** — finite element / computational fluid dynamics. Build mesh adjacency
|
||
graphs. Python and C++ layers. Mesh partitioning algorithms involve graph traversal.
|
||
|
||
---
|
||
|
||
## Backlog — systems not yet scanned, priority order
|
||
|
||
**Confirmed CLEAN (scanned 2026-03-23, no action needed):**
|
||
ONOS, OpenDaylight, MySQL optimizer, V8 TurboFan, SpiderMonkey IonMonkey, Bazel, sbt,
|
||
GNU Octave, KiCad, Yosys, Verilator — all use O(1) hash containers for graph traversal state.
|
||
|
||
**PostgreSQL (5 sites DEFERRED):** Scanned and confirmed defective; patch blocked by missing
|
||
`nodeHash()` infrastructure. Requires upstream collaboration. See executive summary.
|
||
|
||
**Remaining unscanned — priority order:**
|
||
|
||
| System | Language | Why critical | Scan approach |
|
||
|--------|----------|-------------|---------------|
|
||
| **FRRouting ospfd SPF** | C | `ospf_spf.c` Dijkstra — live router convergence (TI-LFA already patched) | scan (C) |
|
||
| **FRRouting bgpd** | C | `bgp_aspath.c` AS-path loop detection | scan (C) |
|
||
| **FRRouting isisd** | C | `isis_spf.c` IS-IS SPF, carrier backbone | scan (C) |
|
||
| **NetworkX** | Python | Scientific graph library; academic/research global baseline | clone + scan (Python) |
|
||
| **ExaBGP** | Python | Pure Python BGP; very high probability | clone + scan (Python) |
|
||
| **OpenSTA** | C++ | Static timing analysis | clone + scan (C++) |
|
||
| **Blender node graph** | C/Python | Geometry nodes, compositor | clone + scan |
|
||
| **BIRD bgp** | C | IXP route servers | clone + scan (C) |
|
||
| **OpenBGPD** | C | BSD BGP | clone + scan (C) |
|
||
| **Buck2** | Rust | Build target graph | clone + scan (Rust) |
|
||
| **Pants** | Python | Build target graph | clone + scan (Python) |
|
||
| **NuGet** | C# | .NET dep resolution | clone + scan (C#) |
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
The paradigm unlock is real and the blast radius is broad, but it is manageable because:
|
||
|
||
1. The fix is **local** — one data structure change per site, no algorithm redesign
|
||
2. The fix is **provably correct** — set/map membership is semantically identical to list membership for these use cases; only the complexity changes
|
||
3. The fix is **testable** — operation counts are measurable and we can assert them in tests
|
||
4. The blast radius grows upward through layers that are **already heavily tested** — we are not inserting new behavior into PostgreSQL or nginx; we are making their build tools faster
|
||
|
||
**The routing/scientific computing concern is qualitatively different from the compiler concern.**
|
||
For compilers and build tools, the defect causes slow builds. For network routing protocols,
|
||
the defect may be causing live convergence failures. For scientific computing, the defect may
|
||
be causing researchers to accept wrong performance baselines and design experiments around
|
||
them. For EDA tools, it may be extending chip design cycles. These are not "software quality"
|
||
issues — they are infrastructure reliability and scientific integrity issues.
|
||
|
||
The geo/CDN/routing concern is real but indirect: our fix accelerates deployment pipelines,
|
||
which amplifies both good updates and bad ones. The mitigation is staged rollout discipline
|
||
in those systems — which is good practice regardless of our work.
|
||
|
||
**Scope of work is larger than initially mapped.** The 35-site defect map covers compilers
|
||
and build tools. Routing protocols, scientific computing, EDA, and numerical simulation are
|
||
a separate wave — same defect pattern, different domain, potentially higher real-world impact.
|
||
|
||
---
|
||
|
||
## Seventh frontier: web infrastructure stack
|
||
|
||
This layer sits between the internet and application code. Every HTTP request passes through
|
||
one or more of these systems. Graph algorithms appear in: module dependency resolution,
|
||
request routing rule evaluation, VCL/config compilation, PHP opcode compilation, and
|
||
upstream cluster topology management.
|
||
|
||
### PHP — Zend Engine (HIGH PROBABILITY)
|
||
|
||
PHP's Zend Engine compiles PHP source to opcodes at runtime. The compilation pipeline is a
|
||
classic compiler pipeline with full graph algorithm infrastructure:
|
||
|
||
- **`Zend/zend_cfg.c`** — Control Flow Graph (CFG) construction. Basic block discovery,
|
||
predecessor/successor lists. DFS-based.
|
||
- **`Zend/zend_dfg.c`** — Data Flow Graph. Liveness analysis, reaching definitions.
|
||
Iterative dataflow over CFG — the fixed-point loop visits nodes and may check
|
||
visited/changed state with list membership.
|
||
- **`Zend/zend_ssa.c`** — SSA (Static Single Assignment) form construction. Requires
|
||
dominator tree, dominance frontiers — classic graph algorithms.
|
||
- **`Zend/zend_optimizer.c`** — Optimizer over SSA form. DCE, SCCP, type inference.
|
||
|
||
PHP is executed on every web request (unless opcode cached). The opcode cache
|
||
(OPcache) means CFG/SSA is built once per file, not per-request. But any CWE-407 in
|
||
CFG/SSA construction affects every PHP deploy's warm-up time and memory usage.
|
||
|
||
**`ext/opcache/Optimizer/`** — OPcache optimizer passes. Multiple graph traversal passes.
|
||
`zend_ssa.c`, `zend_call_graph.c`, `zend_func_info.c` are all graph algorithm files.
|
||
**Highest-probability CWE-407 target in this tier.**
|
||
|
||
**PHP Composer** — the PHP package manager. Pure PHP dependency resolver.
|
||
`src/Composer/DependencyResolver/` uses a pool-based SAT solver with graph operations.
|
||
If any visited/cycle set is a PHP array (O(n) `in_array()`), it inherits distlib-0001's
|
||
pattern in the PHP ecosystem.
|
||
|
||
### Redis
|
||
|
||
Redis uses graph structures in:
|
||
- **Cluster topology** (`src/cluster.c`) — cluster nodes maintain predecessor/follower
|
||
state, path-finding for slot migration, and reachability for failover detection.
|
||
Redis Cluster's `clusterGetSlotByQuery`, `clusterNodeGetSlotBit`, node reachability —
|
||
if visited tracking uses a C array/list rather than a bitfield or hash, CWE-407 applies.
|
||
- **Lua scripting** — Redis embeds LuaJIT; Lua scripts can trigger Redis commands in
|
||
dependency chains. Not directly graph-traversal, but Lua's own compiler (LuaJIT's
|
||
`lj_ir.c`, `lj_opt_fold.c`) may have graph algorithm issues.
|
||
- **Module dependency** (`src/module.c`) — Redis modules declare dependencies. If
|
||
topological sort of module load order uses list membership, that's CWE-407.
|
||
|
||
Redis is C; patterns to look for: linear array scan in cluster path computation.
|
||
|
||
### Memcached
|
||
|
||
Simpler architecture — primarily hash tables for key storage, slab allocator for memory.
|
||
Graph algorithms are minimal. **Low probability.** The `assoc.c` (hash table) and
|
||
`items.c` (LRU chains) don't do graph traversal. Skip for now unless scan reveals hits.
|
||
|
||
### Varnish Cache
|
||
|
||
Varnish uses VCL (Varnish Configuration Language) which is compiled to C and then loaded
|
||
as a shared library. The VCL compiler (`lib/libvcc/`) has:
|
||
- **AST construction and traversal** — VCL is parsed into an AST, then compiled.
|
||
If the compiler uses list-based visited sets in tree/graph traversal, CWE-407 applies.
|
||
- **`vcc_compile.c`, `vcc_backend.c`** — backend (upstream) dependency tracking.
|
||
If backends form a dependency graph (director chains), cycle detection may use lists.
|
||
|
||
Varnish's VCL compiler runs at config load time, not per-request — lower urgency.
|
||
But large Varnish deployments with complex VCL (hundreds of backends, subroutine chains)
|
||
could see slow reload times.
|
||
|
||
### nginx
|
||
|
||
nginx's architecture is event-driven with minimal graph structure. However:
|
||
- **`src/core/ngx_resolver.c`** — DNS resolver. Resolves chains of CNAMEs.
|
||
CNAME chains are effectively a linked list, but cycle detection (detecting CNAME loops)
|
||
uses a linear scan of the chain. For most cases this is bounded (max 8 CNAME hops),
|
||
but the pattern is CWE-407 if implemented naively.
|
||
- **`src/http/ngx_http_upstream.c`** — Upstream group management. If upstream
|
||
health-check state uses list-based membership, it's O(n) per check.
|
||
- **`src/http/ngx_http_rewrite_module.c`** — Rewrite rule chains. `break`/`last` flags
|
||
terminate chains, so bounded. Low probability.
|
||
|
||
nginx is largely clean architecturally — it doesn't do complex graph computation at
|
||
runtime. Build-time scan still warranted.
|
||
|
||
### Apache2 (httpd)
|
||
|
||
- **`server/config.c`** — Module configuration merging. If module dependency graph
|
||
uses `ap_array_make` (Apache's C array) for visited tracking, CWE-407 applies.
|
||
- **`modules/proxy/mod_proxy_balancer.c`** — Load balancer worker state. If worker
|
||
health state uses linear scan, O(n) per check on every request.
|
||
- **`modules/mappers/mod_rewrite.c`** — Rewrite rule evaluation. Chain of rules with
|
||
conditions — if rule graph uses list membership for cycle detection, CWE-407.
|
||
- **`server/request.c`** — Request handler chain. `ap_run_*` hooks traverse handler
|
||
lists — linear by design but bounded.
|
||
|
||
### PHP-FPM / mod_php / CGI
|
||
|
||
These are PHP execution environments, not independent graph algorithm implementations.
|
||
They execute PHP code (which uses Zend Engine) and manage process pools. The process
|
||
pool management (`fpm/fpm_children.c`, `fpm/fpm_scoreboard.c`) uses simple arrays —
|
||
not graph algorithms. The interesting graph code is in Zend Engine itself (above).
|
||
|
||
### Web infrastructure scan backlog
|
||
|
||
Add to `tools/Makefile` and `tools/scans/`:
|
||
|
||
| Target | Language | Key files | Priority |
|
||
|--------|----------|-----------|----------|
|
||
| `php-zend` | C | `Zend/zend_cfg.c`, `zend_dfg.c`, `zend_ssa.c`, `ext/opcache/Optimizer/` | HIGH |
|
||
| `php-composer` | PHP | `src/Composer/DependencyResolver/` | MEDIUM |
|
||
| `redis` | C | `src/cluster.c`, `src/module.c` | MEDIUM |
|
||
| `varnish` | C | `lib/libvcc/vcc_compile.c`, `vcc_backend.c` | MEDIUM |
|
||
| `nginx` | C | `src/core/ngx_resolver.c`, `src/http/` | LOW |
|
||
| `apache2` | C | `server/config.c`, `modules/proxy/mod_proxy_balancer.c` | LOW |
|
||
|
||
---
|
||
|
||
## P2P and Anonymity Network Infrastructure
|
||
|
||
Scanned 2026-03-24. Six systems checked; one confirmed defect.
|
||
|
||
### Confirmed defect — Tor
|
||
|
||
**tor-0001** — `src/feature/nodelist/routerlist.c:2179`
|
||
|
||
`router_load_routers_from_string()` validates received descriptors against a `smartlist_t`
|
||
of requested fingerprints using `smartlist_contains_string()` — a `for`-loop strcmp scan —
|
||
inside a `SMARTLIST_FOREACH_BEGIN` over all received descriptors. Cost: O(R²), where R is
|
||
the number of router descriptors in the batch.
|
||
|
||
- R at a directory authority: ~7,000-8,000 (full relay consensus at startup)
|
||
- `smartlist_contains_string` confirmed O(n) linear scan at smartlist.c:97
|
||
- Same pattern in extrainfo path at line 2263-2295
|
||
- Fix: replace `smartlist_t *requested_fingerprints` with `digestmap_t *` — Tor's
|
||
existing O(1) map, already used correctly in the same file at lines 2689, 2717
|
||
|
||
**Status:** UNPATCHED — ticket `tor-0001.md`
|
||
|
||
### CLEAN systems
|
||
|
||
| System | Language | Key finding |
|
||
|--------|----------|-------------|
|
||
| I2P Java router | Java | `tunnel/pool/TunnelPeerSelector.java` uses `Set<Hash>` (HashSet) throughout |
|
||
| libtorrent | C++ | `std::find` calls are assert-only or protocol-bounded (≤10 items per fast-set) |
|
||
| Transmission | C++ | Minimal `std::find`; no graph traversal hot paths |
|
||
| Kubo (go-ipfs / IPFS) | Go | Map-first idiom; no slice-backed visited sets in DAG traversal |
|
||
| Deluge | Python | List `.index()` calls are UI-only GTK operations |
|
||
|
||
### P2P scan backlog
|
||
|
||
| Target | Language | Key files | Priority |
|
||
|--------|----------|-----------|----------|
|
||
| `i2p.i2p router` | Java | `router/java/src/net/i2p/router/networkdb/` — KBucket/NetDB operations | MEDIUM |
|
||
| `libp2p-go` | Go | `routing/` — Kademlia DHT traversal | LOW |
|
||
| `zeromq` | C++ | `src/` — message routing graph | LOW |
|
||
|
||
---
|
||
|
||
## Eighth frontier: financial markets infrastructure
|
||
|
||
Financial markets are the highest-stakes environment in which this defect map operates.
|
||
The patches touch every layer of the financial stack — from the network routing that
|
||
carries market data, to the compilers that build trading systems, to the message brokers
|
||
that route orders, to the databases that hold positions. Each layer has its own blast
|
||
radius profile.
|
||
|
||
### Layer 1 — Network (OSPF in exchange co-location)
|
||
|
||
**frrouting-0002 is unpatched and directly affects financial market reliability.**
|
||
|
||
Stock exchanges, dark pools, and electronic trading venues operate in co-location
|
||
facilities where low-latency connectivity is the product. These facilities run OSPF
|
||
internally for routing between cabinets and switching layers. Every link failure —
|
||
a transceiver fault, a scheduled maintenance failover, a cable pull — triggers an
|
||
OSPF SPF recalculation.
|
||
|
||
With frrouting-0002 unpatched, SPF on a hub-and-spoke co-location topology (all racks
|
||
connected to a core switch layer — the dominant design) is O(V²) per topology change.
|
||
For a facility with 500 connected endpoints, that is ~125,000 comparisons per failover
|
||
event instead of ~500. OSPF convergence delay is directly proportional to the time
|
||
trading systems are unreachable during failover.
|
||
|
||
**The market impact:** If OSPF convergence takes longer than expected, trading systems
|
||
that rely on co-location connectivity experience unexpected latency spikes or brief
|
||
disconnection. For algorithmic trading systems with sub-millisecond latency requirements,
|
||
this is indistinguishable from a market data outage. Orders may be rejected, hedges may
|
||
fail to execute, and risk positions may be left unhedged during the convergence window.
|
||
|
||
Priority: patch frrouting-0002 and notify co-location facility operators (Equinix,
|
||
NYSE Mahwah, CME Aurora, CBOE Lenexa) as part of coordinated disclosure.
|
||
|
||
### Layer 2 — FIX protocol engines
|
||
|
||
The Financial Information eXchange (FIX) protocol is the message layer of every
|
||
electronic market. Every order, cancel, execution report, and market data update flows
|
||
through a FIX engine.
|
||
|
||
**QuickFIX/J** (Java) — the dominant open-source Java FIX engine. Used by brokers,
|
||
hedge funds, and exchanges globally. Compiled with javac → javac patches apply to every
|
||
QuickFIX/J build. Session management and routing logic involves graph traversal for
|
||
session dependency resolution.
|
||
|
||
**QuickFIX** (C++) — the C++ FIX engine. Compiled with GCC/Clang. LTO used in
|
||
production builds → llvm-0001 applies. The session graph and message routing logic are
|
||
candidates for their own CWE-407 scan; QuickFIX C++ has not been directly scanned.
|
||
|
||
**Recommendation:** Scan QuickFIX C++ `src/` for `std::find` / `std::vector::contains`
|
||
patterns in session graph and routing code. Given the age of the codebase (2000s-era)
|
||
and the language, probability of CWE-407 candidates is medium-high.
|
||
|
||
### Layer 3 — Order management and trading systems
|
||
|
||
**Java trading systems** — the majority of exchange-facing trading systems at major
|
||
financial institutions are built on Java. Order management systems (OMS), execution
|
||
management systems (EMS), and smart order routers all compile with javac. All benefit
|
||
directly from javac-0001 through javac-0005.
|
||
|
||
**Scala/Akka trading systems** — Akka is the dominant actor framework for high-throughput
|
||
Scala trading backends. Used at LMAX, Goldman Sachs (SecDB), Morgan Stanley, and
|
||
quantitative hedge funds. **scala3-0001 (O(n³) type inference) hits every Scala 3 trading
|
||
codebase directly.** A cubic constraint solver in the compiler means every build of a
|
||
type-heavy Akka or Cats Effect trading application was running at cubic cost. The patch
|
||
reduces this to linear.
|
||
|
||
**C++ HFT systems** — high-frequency trading firms (Virtu, Citadel Securities, Jane
|
||
Street, Two Sigma) build almost exclusively in C++ for sub-microsecond latency. All
|
||
benefit from llvm-0001 (LLVM LTO in release builds) and gcc-0001. The HFT build cycle
|
||
is aggressive — rebuilds happen frequently as strategies are updated. Faster LTO
|
||
directly reduces the time between strategy change and live deployment.
|
||
|
||
**Kotlin fintech backends** — kotlin-0001 (inheritance restriction checking) affects
|
||
every Kotlin financial services backend. Corda/R3 is the canonical example, but
|
||
Kotlin is now the default language at many fintech firms (Revolut, Monzo, N26,
|
||
Stripe's backend services).
|
||
|
||
### Layer 4 — Message brokers and event streaming
|
||
|
||
**Apache Kafka** — the dominant event streaming platform for financial data. Used at
|
||
every major exchange, bank, and trading venue for market data feeds, trade events, and
|
||
risk streams. Kafka is Java, compiled with javac. The Kafka broker's internal dependency
|
||
graph and topic partition assignment logic benefit from javac patches. Kafka Streams
|
||
(Scala/Java) benefits from both javac and scala3-0001.
|
||
|
||
**RabbitMQ** — Erlang-based message broker used heavily in financial messaging
|
||
infrastructure. Faster digraph ops (erlang-0001 patched, erlang-0002 FIXABLE-UPSTREAM)
|
||
improve exchange graph validation. **Throttle risk applies** — see Erlang ecosystem
|
||
section above. RabbitMQ at financial scale (stock exchanges, clearinghouses) must be
|
||
audited before the OTP patch is deployed.
|
||
|
||
**LMAX Disruptor** — Java ring buffer / event processing framework designed specifically
|
||
for financial low-latency systems. Used at LMAX Exchange and widely adopted in financial
|
||
middleware. Pure Java; compiled with javac. Benefits from inference patches in any
|
||
generic-heavy usage.
|
||
|
||
### Layer 5 — Risk and position databases
|
||
|
||
**PostgreSQL in financial analytics** — risk management systems, position databases,
|
||
P&L calculation engines, and regulatory reporting systems (MIFID II, Dodd-Frank) run
|
||
heavily on PostgreSQL. The five deferred CWE-407 defects in the PostgreSQL query planner
|
||
are directly material here:
|
||
|
||
- `postgresql-0002` (MERGE/UPDATE planning) — financial systems use MERGE heavily for
|
||
upsert patterns in position and trade tables. Wide position tables (50–200 columns)
|
||
and complex MERGE statements hit the O(W²×C²) defect directly.
|
||
- `postgresql-0003` (equivalence class matching) — analytical risk queries with many
|
||
join predicates (risk factor joins, scenario analysis) hit the O(M×E) inner loop.
|
||
- `postgresql-0004` (join elimination) — self-join patterns common in slowly-changing
|
||
dimension tables (instrument reference data, counterparty master) trigger this path.
|
||
|
||
PostgreSQL in financial infrastructure is one of the strongest arguments for prioritizing
|
||
the `nodeHash()` contribution. The query planner defects are not abstract — they affect
|
||
every complex analytical query against wide financial tables.
|
||
|
||
**TimescaleDB** — time-series extension to PostgreSQL, used for market data storage.
|
||
Inherits all five PostgreSQL planner defects. Time-series financial queries (OHLCV,
|
||
tick data, order book snapshots) against wide tables trigger the same paths.
|
||
|
||
### Layer 6 — TypeScript trading platforms
|
||
|
||
Modern trading platforms, broker portals, and market data dashboards are TypeScript-
|
||
heavy. Bloomberg's web terminal, Refinitiv Eikon Web, and the majority of broker
|
||
execution portals are TypeScript SPAs. ts-0001 through ts-0003 affect every TypeScript
|
||
trading frontend:
|
||
|
||
- **Developer experience:** VS Code language server performance on large trading
|
||
platform codebases. Symbol resolution latency, auto-complete lag in complex
|
||
type-parameterized components. Financial UI codebases are type-heavy by design
|
||
(price types, instrument types, order state machines).
|
||
- **CI/CD build time:** TypeScript type-checking in CI for every trading platform
|
||
frontend. Faster type-checking means faster deployment of trading UI changes.
|
||
|
||
**npm dependency resolution** (npm-0002, arborist patch) — every `npm install` for
|
||
trading platform frontends. Financial firms run npm install constantly in CI.
|
||
|
||
### Layer 7 — DeFi and on-chain financial systems
|
||
|
||
**solc-0001 (HIGH, unpatched)** — every Solidity contract compiled with `--via-ir`
|
||
or `--optimize` is affected. DeFi protocols — Uniswap, Aave, Compound, Curve,
|
||
MakerDAO — compile all production contracts through the Yul IR pipeline. For contracts
|
||
with deep internal function call graphs (lending protocols with complex liquidation
|
||
logic, DEX routers with many hop paths), the O(F×D²) Yul cycle detection is a real
|
||
compile-time cost. More critically: **solc is part of the security audit process**.
|
||
Every smart contract security audit involves multiple recompilations with different
|
||
optimization settings. A slow compiler increases audit costs and may compress the time
|
||
auditors spend on each compilation step.
|
||
|
||
### Cross-layer risk: deployment velocity in financial systems
|
||
|
||
Faster build pipelines mean faster deployment of fixes — and faster deployment of
|
||
mistakes. This is the dual-use concern for financial systems specifically:
|
||
|
||
**The upside:** A critical trading system bug discovered at market open can be
|
||
hotfixed and deployed faster. The window between discovery and remediation shrinks.
|
||
For financial systems where a bug can cost millions per minute, this is real value.
|
||
|
||
**The downside:** Financial systems have strict change management. Deployments go
|
||
through approval chains, pre-deployment testing, and regulatory notification for
|
||
certain changes. A faster build pipeline does not shorten the approval chain. The
|
||
risk is that development teams, experiencing faster builds, develop habits around
|
||
faster iteration that collide with the change management requirements. "It builds
|
||
faster so we can deploy faster" is not a valid rationale for bypassing change control.
|
||
|
||
**Mitigation:** Ensure that change management processes are decoupled from build
|
||
time. Faster CI should translate to more test coverage per deployment, not fewer
|
||
gates before production.
|
||
|
||
### Financial markets blast radius summary
|
||
|
||
| Layer | Systems affected | Patches | Risk profile |
|
||
|-------|-----------------|---------|--------------|
|
||
| Network routing | OSPF in co-location | frrouting-0002 (unpatched) | HIGH — live reliability |
|
||
| FIX engines | QuickFIX/J, QuickFIX C++ | javac, llvm-0001 | Medium — build pipeline |
|
||
| Trading systems | Java OMS/EMS, Scala/Akka, C++ HFT, Kotlin fintech | javac, scala3, llvm, kotlin | Low-Medium — faster builds |
|
||
| Message brokers | Kafka, RabbitMQ | javac, erlang | Medium — Erlang throttle risk |
|
||
| Risk databases | PostgreSQL, TimescaleDB | deferred ×5 | Medium — planning defects in prod |
|
||
| Trading UIs | TypeScript platforms | ts-0001..0003 | Low — build/dev experience |
|
||
| DeFi/on-chain | Solidity (Ethereum) | solc-0001 (unpatched) | Medium — audit pipeline |
|
||
| Build pipeline | All of the above | All patches | Dual-use — velocity is good and dangerous |
|
||
|
||
---
|
||
|
||
## Ninth frontier: game engine ecosystems
|
||
|
||
### Minecraft Java Edition — server-26.1
|
||
|
||
Minecraft Java Edition is the world's best-selling PC game and one of the most widely
|
||
deployed custom server ecosystems in existence. Hundreds of thousands of servers run
|
||
community-operated instances; the modded server ecosystem (Forge, Fabric, NeoForge)
|
||
adds thousands of additional mods per major version. The server jar is bytecode-only
|
||
(no source); analysis was performed via CFR decompiler on the extracted inner jar
|
||
(`META-INF/versions/26.1/server-26.1.jar`, 7,351 classes, server-26.1).
|
||
|
||
**Scan method:** CFR decompiler + Python bytecode string scan for `List/contains`
|
||
patterns across all 7,351 classes. Core graph utilities verified clean.
|
||
|
||
#### minecraft-0001 — DependencySorter.isCyclic (HIGH)
|
||
|
||
**File:** `net/minecraft/util/DependencySorter.java` (decompiled)
|
||
**Called from:** `net/minecraft/tags/TagLoader` — tag dependency resolution
|
||
**Trigger:** Every world load, every `/reload`, every `/datapack enable`
|
||
**Complexity:** O(E^D) worst case — exponential, no visited set in recursive DFS
|
||
|
||
`DependencySorter.isCyclic()` performs a recursive DFS to check whether adding a
|
||
dependency edge would create a cycle. No visited set. For a diamond dependency graph
|
||
of depth D, the number of node visits is 2^D:
|
||
|
||
```java
|
||
private static <K> boolean isCyclic(Multimap<K, K> directDependencies, K from, K to) {
|
||
Collection dependencies = directDependencies.get(to);
|
||
if (dependencies.contains(from)) {
|
||
return true;
|
||
}
|
||
return dependencies.stream().anyMatch(
|
||
dep -> DependencySorter.isCyclic(directDependencies, from, dep)
|
||
);
|
||
}
|
||
```
|
||
|
||
**Impact:** Called from `TagLoader` — Minecraft's classification system (`#minecraft:logs`,
|
||
`#forge:ores/iron`, etc.). Tags can reference other tags. The dependency sort runs on
|
||
every world load and every `/reload`.
|
||
|
||
Vanilla Minecraft has hundreds of tags — tolerable. Large modpacks (Create, Applied
|
||
Energistics 2, Mekanism, Thermal Expansion) have thousands of cross-mod tag
|
||
dependencies with endemic diamond inheritance patterns. The "tag loading lag" reported
|
||
by modpack server operators — multi-second freezes on every server start and `/reload` —
|
||
is consistent with O(E^D) revisiting on diamond-shaped tag dependency graphs.
|
||
|
||
**Fix:** Add `Set<K> visited` parameter to track explored nodes. Per-call cost drops
|
||
from O(E^D) to O(E); total tag loading from O(E^D × E) to O(E²). Better fix: single
|
||
SCC pass over the complete graph after all edges are added, reducing total cost to
|
||
O(V+E).
|
||
|
||
**Disclosure path:** bugs.mojang.com (public bug tracker, "Performance" category)
|
||
|
||
#### minecraft-0002 — PistonStructureResolver.toPush (LOW)
|
||
|
||
**File:** `net/minecraft/world/level/block/piston/PistonStructureResolver.java`
|
||
**Pattern:** `this.toPush.contains(start)` — `toPush` is `ArrayList<BlockPos>`
|
||
**Complexity:** O(P²) — bounded at P≤12 by game design
|
||
|
||
Every piston activation resolves a push chain. Duplicate detection uses
|
||
`ArrayList.contains()` — linear scan. Minecraft hardcodes a maximum of 12 pushed
|
||
blocks per piston:
|
||
|
||
```java
|
||
if (blockCount + this.toPush.size() > 12) { return false; }
|
||
```
|
||
|
||
P≤12 caps the defect at 144 comparisons per piston. However, redstone contraptions
|
||
with many pistons firing simultaneously compound this: a 16×16 piston array at 20
|
||
TPS produces 737,280 list comparisons per second. Principle violation; low priority.
|
||
|
||
**Fix:** Parallel `HashSet<BlockPos>` for O(1) duplicate detection, same as javac-0001
|
||
(Tarjan stack → parallel set pattern).
|
||
|
||
#### Confirmed CLEAN
|
||
|
||
| Class | Why clean |
|
||
|-------|-----------|
|
||
| `util/Graph.depthFirstSearch` | Uses `Set<T>` for both `discovered` and `currentlyVisiting` — O(1) contains |
|
||
| `util/FeatureSorter` | Uses `TreeSet` for visited/onStack — O(log n), deliberate for deterministic ordering |
|
||
| `util/DependencySorter.visitDependenciesAndElement` | Uses `HashSet alreadyVisited` — O(1) |
|
||
| `world/level/lighting/DynamicGraphMinFixedPoint` | No list containers in bytecode |
|
||
| `world/level/chunk/status/ChunkDependencies` | No list containers in bytecode |
|
||
|
||
**Note on broader scan:** 34 classes had the `ArrayList + contains` bytecode signature.
|
||
All other hits are bounded inputs (pack selection: tens of packs), non-hot paths
|
||
(advancement layout, crash report categories), or operate on `HolderSet` (Minecraft's
|
||
own set wrapper, likely O(1)). `DependencySorter.isCyclic` is the only site where
|
||
unbounded graph traversal occurs without visited tracking.
|
||
|
||
#### Modding ecosystem blast radius
|
||
|
||
The Minecraft modded ecosystem amplifies minecraft-0001 specifically:
|
||
|
||
| Actor | Impact |
|
||
|-------|--------|
|
||
| **Vanilla server operators** | Hundreds of tags — tolerable; load time unnoticed |
|
||
| **Small modpack servers (50–200 mods)** | Thousands of tags — measurable lag on `/reload` |
|
||
| **Large modpack servers (Create, ATM, Omnifactory)** | Thousands of cross-mod diamond deps — multi-second freeze per reload |
|
||
| **Modpack developers** | `/reload` during development is slow; iteration cycle harmed |
|
||
| **Server hosting providers** | Restart time SLAs affected for large-modpack plans |
|
||
|
||
minecraft-0001 is a live performance defect affecting every large modpack server start
|
||
worldwide. The tag loading lag is user-visible and widely reported; the root cause has
|
||
not previously been identified.
|
||
|
||
### Mod source scan — Create, AE2, Mekanism
|
||
|
||
Three major open-source mods were independently scanned for CWE-407:
|
||
|
||
**Create mod — create-0001 (MEDIUM).** `TrackGraph.findDisconnectedGraphs()` uses
|
||
`ArrayList.remove(0)` as the BFS frontier queue. `ArrayList.remove(0)` is O(n) — the
|
||
backing array shifts all remaining elements on every dequeue. O(V²) BFS instead of
|
||
O(V+E). Trigger: every track removal event. Fix: `ArrayDeque.removeFirst()`.
|
||
|
||
**Applied Energistics 2 — CLEAN.** `GridNode.java` uses `ArrayDeque` + integer
|
||
generation counter for visited tracking — O(1). `PathingService.java` uses `HashSet`
|
||
in loop — O(1).
|
||
|
||
**Mekanism — CLEAN.** `OrphanPathFinder` uses `ObjectOpenHashSet<BlockPos>` (fastutil)
|
||
+ `Deque<BlockPos>` — both O(1). Written with performance awareness.
|
||
|
||
| Mod | Defect | Severity | Trigger |
|
||
|-----|--------|----------|---------|
|
||
| Minecraft (all mods) | minecraft-0001 `DependencySorter.isCyclic` | HIGH/EXPONENTIAL | Every world load |
|
||
| Minecraft (all mods) | minecraft-0002 `PistonStructureResolver` | LOW | Piston activation |
|
||
| Create mod | create-0001 `TrackGraph.findDisconnectedGraphs` | MEDIUM | Track removal |
|
||
| AE2 | — | CLEAN | — |
|
||
| Mekanism | — | CLEAN | — |
|