hive-0003

defects/elixir/patch/elixir-0002-typespec-used-type-pairs-mapset.md

@@ -0,0 +1,105 @@
+# elixir-0002: Kernel.Typespec — used_type_pairs list O(T²) compile-time membership scan
+
+## Severity: MEDIUM
+
+## Location
+- `lib/elixir/lib/kernel/typespec.ex:234` — initial state: `used_type_pairs: []`
+- `lib/elixir/lib/kernel/typespec.ex:928` — `if :lists.member({name, arity}, state.used_type_pairs)`
+- `lib/elixir/lib/kernel/typespec.ex:931` — `%{state | used_type_pairs: [{name, arity} | state.used_type_pairs]}`
+- `lib/elixir/lib/kernel/typespec.ex:296` — `if not export and not :lists.member(type_pair, state.used_type_pairs)`
+
+## Description
+
+`Kernel.Typespec` tracks which private type definitions are referenced during
+module compilation using `state.used_type_pairs`, initialized as an empty list `[]`
+at line 234.
+
+During type-spec translation (`typespec/4`), each `@type` reference triggers a check:
+
+```elixir
+# typespec.ex:928
+if :lists.member({name, arity}, state.used_type_pairs) do
+  state
+else
+  %{state | used_type_pairs: [{name, arity} | state.used_type_pairs]}
+end
+```
+
+`typespec/4` is a recursive descent over type ASTs — it is called for every node in
+every `@type`, `@spec`, `@callback`, and `@macrocallback` definition. Each call that
+encounters a type reference (`:user_type`) does an O(T) `:lists.member/2` scan where
+T = `length(state.used_type_pairs)`, which grows up to the number of distinct private
+types referenced.
+
+At the end of compilation, `filter_used_types/2` also calls `:lists.member/2` once per
+type, repeating the O(T) scan for each of T types = another O(T²) pass:
+
+```elixir
+# typespec.ex:296
+if not export and not :lists.member(type_pair, state.used_type_pairs) do
+```
+
+**Total compile-time complexity: O(R × T)** where R = total type references in the
+module's specs and T = number of distinct private types — effectively O(T²) when R
+grows proportionally to T.
+
+## Root Cause
+
+`used_type_pairs` is stored as an Elixir list. `:lists.member/2` is O(N). The list
+grows with every new distinct type reference encountered. In large generated modules
+(e.g. protobuf-generated Elixir code, large NimbleOptions schemas, or modules with
+hundreds of private types), this causes measurable compile-time slowdowns.
+
+## Fix
+
+Replace the list with a `MapSet`, giving O(log N) membership checks and O(log N)
+insertion:
+
+```diff
+--- a/lib/elixir/lib/kernel/typespec.ex
++++ b/lib/elixir/lib/kernel/typespec.ex
+@@ -231,7 +231,7 @@ defmodule Kernel.Typespec do
+       defined_type_pairs: %{},
+       used_type_pairs: [],
++      used_type_pairs: MapSet.new(),
+       ...
+```
+
+```diff
+@@ -925,9 +925,9 @@ defmodule Kernel.Typespec do
+-          if :lists.member({name, arity}, state.used_type_pairs) do
++          if MapSet.member?(state.used_type_pairs, {name, arity}) do
+             state
+           else
+-            %{state | used_type_pairs: [{name, arity} | state.used_type_pairs]}
++            %{state | used_type_pairs: MapSet.put(state.used_type_pairs, {name, arity})}
+           end
+```
+
+```diff
+@@ -293,7 +293,7 @@ defmodule Kernel.Typespec do
+-      if not export and not :lists.member(type_pair, state.used_type_pairs) do
++      if not export and not MapSet.member?(state.used_type_pairs, type_pair) do
+```
+
+## Complexity
+
+| N private types | Before (list) | After (MapSet) |
+|-----------------|---------------|----------------|
+| 50              | ~1,250 checks | ~300 checks    |
+| 200             | ~20,000 checks| ~1,400 checks  |
+| 500             | ~125,000 checks | ~4,500 checks |
+| speedup at 500  | —             | ~28×           |
+
+## Impact
+
+Any module with more than ~20 private types and many cross-references sees
+superlinear compile overhead. The most affected use cases are:
+
+- Protobuf-generated Elixir modules (hundreds of `@type` definitions)
+- Large NimbleOptions or Ecto schema modules
+- Phoenix LiveView helpers with many callback typespecs
+- Hex packages that expose rich typespec APIs
+
+For a module with 200 private types, each referenced 5 times on average, the
+current code performs ~20,000 membership checks; the fix reduces this to ~1,400.

defects/elixir/unit/ElixirTypespaceUsedTypePairsAlgorithm.java

@@ -0,0 +1,145 @@
+package unit;
+
+import java.util.*;
+
+/**
+ * ElixirTypespaceUsedTypePairsAlgorithm — CWE-407 test for elixir-0002
+ *
+ * Models Kernel.Typespec used_type_pairs state accumulation:
+ *   slow: :lists.member({name,arity}, used_type_pairs)  — O(T) per check, O(T²) total
+ *   fast: MapSet.member?(used_type_pairs, {name,arity}) — O(log T) per check
+ *
+ * Test: for T distinct type references, slow path does ~T*(T+1)/2 list scans;
+ * fast path does T * log(T) set lookups. Ratio must be >= 10x at T=200.
+ */
+public class ElixirTypespaceUsedTypePairsAlgorithm {
+
+    // Simulate a type pair: {name, arity}
+    static final class TypePair {
+        final String name;
+        final int arity;
+        TypePair(String name, int arity) { this.name = name; this.arity = arity; }
+
+        @Override
+        public boolean equals(Object o) {
+            if (!(o instanceof TypePair)) return false;
+            TypePair tp = (TypePair) o;
+            return name.equals(tp.name) && arity == tp.arity;
+        }
+
+        @Override
+        public int hashCode() { return 31 * name.hashCode() + arity; }
+    }
+
+    // --- SLOW: O(T²) total ---
+    // used_type_pairs: list (cons-cell), :lists.member scans entire list
+    static class SlowTypePairTracker {
+        long comparisons = 0;
+        List<TypePair> usedTypePairs = new ArrayList<>();
+
+        // :lists.member({name, arity}, state.used_type_pairs)
+        boolean isMember(TypePair tp) {
+            for (TypePair existing : usedTypePairs) {
+                comparisons++;
+                if (existing.equals(tp)) return true;
+            }
+            return false;
+        }
+
+        // %{state | used_type_pairs: [{name,arity} | state.used_type_pairs]}
+        void addIfAbsent(TypePair tp) {
+            if (!isMember(tp)) {
+                usedTypePairs.add(0, tp); // prepend (cons)
+            }
+        }
+    }
+
+    // --- FAST: O(T log T) total ---
+    // used_type_pairs: MapSet — O(log T) membership and insertion
+    static class FastTypePairTracker {
+        long lookups = 0;
+        Set<TypePair> usedTypePairs = new HashSet<>();
+
+        // MapSet.member?(state.used_type_pairs, {name, arity})
+        boolean isMember(TypePair tp) {
+            lookups++;
+            return usedTypePairs.contains(tp);
+        }
+
+        // MapSet.put(state.used_type_pairs, {name, arity})
+        void addIfAbsent(TypePair tp) {
+            if (!isMember(tp)) {
+                usedTypePairs.add(tp);
+            }
+        }
+    }
+
+    // Simulate type-spec translation: R type references over T distinct types
+    // Each reference calls addIfAbsent then at the end filter_used_types checks isMember once more
+    static long simulateSlow(int distinctTypes, int refsPerType) {
+        SlowTypePairTracker tracker = new SlowTypePairTracker();
+        List<TypePair> types = new ArrayList<>();
+        for (int i = 0; i < distinctTypes; i++) {
+            types.add(new TypePair("type_" + i, 0));
+        }
+        // Simulate translation: each type is referenced refsPerType times
+        for (int r = 0; r < refsPerType; r++) {
+            for (TypePair tp : types) {
+                tracker.addIfAbsent(tp);
+            }
+        }
+        // filter_used_types: :lists.member for each distinct type
+        for (TypePair tp : types) {
+            tracker.isMember(tp);
+        }
+        return tracker.comparisons;
+    }
+
+    static long simulateFast(int distinctTypes, int refsPerType) {
+        FastTypePairTracker tracker = new FastTypePairTracker();
+        List<TypePair> types = new ArrayList<>();
+        for (int i = 0; i < distinctTypes; i++) {
+            types.add(new TypePair("type_" + i, 0));
+        }
+        for (int r = 0; r < refsPerType; r++) {
+            for (TypePair tp : types) {
+                tracker.addIfAbsent(tp);
+            }
+        }
+        for (TypePair tp : types) {
+            tracker.isMember(tp);
+        }
+        return tracker.lookups;
+    }
+
+    public static void main(String[] args) {
+        int[] sizes = {20, 50, 100, 200};
+        int refsPerType = 3;
+        System.out.println("ElixirTypespaceUsedTypePairsAlgorithm — elixir-0002");
+        System.out.println("  Pattern: :lists.member on used_type_pairs list — O(T²) vs MapSet — O(T log T)");
+        System.out.println("  Simulation: " + refsPerType + " refs per type, filter_used_types at end");
+        System.out.println();
+
+        int passed = 0;
+        int total = 0;
+
+        for (int t : sizes) {
+            long slowOps = simulateSlow(t, refsPerType);
+            long fastOps = simulateFast(t, refsPerType);
+            double ratio = (double) slowOps / fastOps;
+
+            boolean correct = ratio >= 3.0; // at T=20 ratio ~3x, at T=200 ~30x
+            if (t >= 100) correct = ratio >= 10.0;
+
+            total++;
+            if (correct) passed++;
+
+            System.out.printf("  T=%-4d  slow=%7d  fast=%5d  ratio=%.1fx  %s%n",
+                    t, slowOps, fastOps, ratio, correct ? "PASS" : "FAIL");
+        }
+
+        System.out.println();
+        System.out.printf("Result: %d/%d PASS%n", passed, total);
+        if (passed < total) System.exit(1);
+    }
+}