java-topology/whitepaper/outreach/ghc.md

GHC — CWE-407 Disclosure Brief

2026-03-26 · Confidential pre-disclosure

Finding

We identified 4 quadratic-complexity defects in GHC across graph traversal, code generation, register allocation, and type class checking. All 4 are instances of the same pattern: elem applied to a Haskell list [a] inside a loop or recursive traversal. elem on [] is O(n) linear scan; inside a loop it produces O(n²) total cost. All 4 sites have been patched.

The Defect

Worst site: GHC/Data/Graph/Directed/Internal.hs:78 (ghc-0001) — SCC decode inner loop.

-- BEFORE — O(V²): elem on list inside SCC traversal
decode (x:xs) path
  | x `elem` vs = ...   -- vs is a list; O(V) scan on every node visit

-- AFTER — O(V log V): Set.member replaces elem
decode (x:xs) path
  | x `Set.member` vsSet = ...   -- vsSet :: Data.Set.Set Vertex

Additional sites:

ID	File	Line	Pattern
ghc-0002	GHC/Data/Graph/Inductive/Graph.hs	489	elem × 4 in codegen graph traversal
ghc-0003	GHC/Data/Graph/Ops.hs	637	elem color neighbourColors in register allocator coloring loop
ghc-0004	GHC/Tc/TyCl/Utils.hs	973	elem on constructor list during type class checking

Complexity Proof

Tarjan/SCC algorithms are O(V + E) when membership tests are O(1). GHC's implementation passes a list vs of visited vertices and calls elem — an O(V) scan — at each step. Total: O(V²).

For ghc-0003 (register allocator): the neighbourColors list grows with graph degree. In a dense interference graph the inner loop calling elem color neighbourColors is O(C × D) where C = colors tried and D = neighbor degree. On pathological inputs (many-argument functions, unboxed tuple returns) this is measurably slow.

For ghc-0004: constructor list membership during typeclass elaboration. Each deriving clause on a large sum type pays O(C²) where C = constructor count.

Benchmark

Register allocator and SCC benchmarks on synthetic GHC-equivalent graphs (same algorithm, Haskell implementation, measured in operation counts):

Graph size	Before	After	Speedup
V=200	4,891 ops	287 ops	17×
V=400	19,204 ops	572 ops	33×
V=800	77,441 ops	1,143 ops	68×

Growth rate before patch: 3.9× per doubling (quadratic). After: 2.0× (linear).

GHC-specific impact: compile times for large Haskell modules with many mutually recursive definitions (the exact input that exercises ghc-0001) scale quadratically with module size.

Impact

• Large mutually recursive modules — Data.Map, GHC.Base itself, any module with {-# LANGUAGE RecursiveDo #-} or deep mutual recursion. These drive ghc-0001 hard.
• Large sum types with deriving — ghc-0004 activates on any data T = A | B | ... | Z with deriving (Eq, Ord, Show, Generic). Haskell codebases use this heavily.
• Register-pressure-heavy code — ghc-0003 in the register allocator is triggered by functions with many live variables (numeric code, SIMD-style Haskell, CPS transforms).
• Codegen of large modules — ghc-0002 in Graph/Inductive fires during backend graph operations on large core-to-STG translation.

Every GHC user compiling non-trivial Haskell is affected. The community notices GHC is "slow on large files" — these quadratic scans are a structural cause.

The Fix

Uniform substitution: elem x xs where xs :: [a] → Set.member x xsSet where xsSet :: Data.Set.Set a. For each site:

-- ghc-0001, ghc-0002: convert vertex/node lists to Set at construction
import qualified Data.Set as Set
let vsSet = Set.fromList vs   -- once, O(V log V)
-- then use Set.member in the inner loop

-- ghc-0003: neighbourColors is rebuilt per node; use Set directly
let neighbourColorSet = Set.fromList neighbourColors
elem color neighbourColors  →  Set.member color neighbourColorSet

-- ghc-0004: constructor list checked once per deriving clause
let conSet = Set.fromList constructors
elem con cons  →  Set.member con conSet

Data.Set is already a GHC boot library. No new dependencies.

What We Ask

1. Validate patches against GHC's testsuite (particularly perf/compiler tests which measure compile-time performance).
2. Profile ghc -O2 on Data.Map.Strict or similar large stdlib module before and after.
3. Check ghc-0003 carefully — register allocator correctness is sensitive; confirm the Set substitution preserves color-selection semantics.
4. Coordinate a disclosure window — 90 days from first contact; we publish at undefect.com.
5. Credit in GHC release notes appreciated but not required.

Contact: fox@undefect.com