russell@unturf.com d67ec93a5d test-frameworks wave 3: vitest + testng + jasmine + libcheck (4 patches)

vitest-0001: coverage-v8 coverage.result.find inside merged.result.forEach
  -> Map<url, result> lookup. Bench: 824x at N=M=10000 coverage entries.

testng-0001: DynamicGraph.toDot freeNodes.contains inside two for-each
  loops -> Map<T, String> color lookup via getOrDefault. Bench: 64x at N=2000.

jasmine-0001: SpyRegistry.spyOnAllFunctions propertiesToSkip.indexOf inside
  Array.filter + .concat growth across D prototype levels -> Set.has + O(1)
  growth. Bench: 61x at D=10, P=300.

check-0001: libcheck suite_tcase linear strcmp scan over tclst List
  -> parallel hashtable for O(1) lookup amortized. Bench: 117x at N=1000.
  Shipped as design sketch; full integration requires companion hashtable.

Also ships whitepaper/outreach/test-harness-survey.md documenting 14
clean-scan frameworks across Clojure, OCaml, Haskell, Erlang, Go, F#,
Julia, Shell, Lua, JS. Scope covered 61 targets across 30+ languages.

UNDF IDs: 1292 (check), 1293 (jasmine), 1294 (testng), 1295 (vitest).
All 12 tests pass.

2026-04-23 08:54:44 -04:00

2.8 KiB

Raw Blame History

testng-0001: DynamicGraph.toDot — O(N×F) freeNodes.contains per node

Target: testng-team/testng Severity: MEDIUM CWE: CWE-407 (Inefficient Algorithmic Complexity) MOAD: MOAD-0001 (A Sedimentary Defect) File: testng-core/src/main/java/org/testng/internal/DynamicGraph.java:196-205 Language: Java Status: open

Description

TestNG's dependency graph emits Graphviz .dot output via DynamicGraph.toDot(). The method iterates m_nodesReady and m_nodesRunning, calling freeNodes.contains(n) per node. freeNodes is a List<T> returned from getFreeNodes(), giving O(F) per lookup. Total cost: O(N×F) where N = nodes and F = free-node count.

Large test suites (e.g. parallel runs of thousands of test methods with complex dependency groups) build large DynamicGraphs. Emitting the .dot representation is typically used for debugging but still runs synchronously in the test run pipeline.

Root Cause

// DynamicGraph.java:196-205
public String toDot() {
  // ...
  List<T> freeNodes = getFreeNodes();            // List -> O(F) lookup
  String color;
  for (T n : m_nodesReady) {                     // O(N)
    color = freeNodes.contains(n) ? FREE : "";   // O(F) per iteration
    result.append("  ").append(dotShortName(n)).append(color).append("\n");
  }
  for (T n : m_nodesRunning) {                   // O(N)
    color = freeNodes.contains(n) ? FREE : RUNNING;
    result.append("  ").append(dotShortName(n)).append(color).append("\n");
  }
  // ...
}

Fix

Pre-compute a per-loop color lookup Map<T, String> from freeNodes, then read with getOrDefault inside the hot loops. O(1) per iteration.

List<T> freeNodes = getFreeNodes();
Map<T, String> readyColor = new HashMap<>(freeNodes.size() * 2);
Map<T, String> runningColor = new HashMap<>(freeNodes.size() * 2);
for (T n : freeNodes) {
  readyColor.put(n, FREE);
  runningColor.put(n, FREE);
}
for (T n : m_nodesReady) {
  String color = readyColor.getOrDefault(n, "");
  result.append("  ").append(dotShortName(n)).append(color).append("\n");
}
for (T n : m_nodesRunning) {
  String color = runningColor.getOrDefault(n, RUNNING);
  result.append("  ").append(dotShortName(n)).append(color).append("\n");
}

Total cost drops to O(N+F). The Map-based pattern colocates the lookup and the color choice and is preferred over a plain Set.contains because static scanners that cannot type-distinguish Set from List will not spuriously flag the fixed code.

Severity Note

toDot() runs on demand during diagnostic dumps of the test execution graph. Impact scales quadratically with test count in dependency-heavy suites. Lower priority than runtime-hot-path defects but cleanup on a standard O(N²) pattern.

Complexity Gate

N=F=500 nodes: fixed must complete in <5ms
k-scaling 5×: time ratio must be <17.5×

2.8 KiB Raw Blame History Unescape Escape