russell@unturf.com d67ec93a5d test-frameworks wave 3: vitest + testng + jasmine + libcheck (4 patches)

vitest-0001: coverage-v8 coverage.result.find inside merged.result.forEach
  -> Map<url, result> lookup. Bench: 824x at N=M=10000 coverage entries.

testng-0001: DynamicGraph.toDot freeNodes.contains inside two for-each
  loops -> Map<T, String> color lookup via getOrDefault. Bench: 64x at N=2000.

jasmine-0001: SpyRegistry.spyOnAllFunctions propertiesToSkip.indexOf inside
  Array.filter + .concat growth across D prototype levels -> Set.has + O(1)
  growth. Bench: 61x at D=10, P=300.

check-0001: libcheck suite_tcase linear strcmp scan over tclst List
  -> parallel hashtable for O(1) lookup amortized. Bench: 117x at N=1000.
  Shipped as design sketch; full integration requires companion hashtable.

Also ships whitepaper/outreach/test-harness-survey.md documenting 14
clean-scan frameworks across Clojure, OCaml, Haskell, Erlang, Go, F#,
Julia, Shell, Lua, JS. Scope covered 61 targets across 30+ languages.

UNDF IDs: 1292 (check), 1293 (jasmine), 1294 (testng), 1295 (vitest).
All 12 tests pass.

2026-04-23 08:54:44 -04:00

2.5 KiB

Raw Blame History

vitest-0001: coverage-v8 generateCoverage — O(N×M) result.find per merged entry

Target: vitest-dev/vitest Severity: MEDIUM-HIGH CWE: CWE-407 (Inefficient Algorithmic Complexity) MOAD: MOAD-0001 (A Sedimentary Defect) File: packages/coverage-v8/src/provider.ts:50-59 Language: TypeScript Status: open

Description

@vitest/coverage-v8 merges process-level V8 coverage data after every test run. When mergeProcessCovs drops a startOffset (observed for Vue projects), the code rebuilds it by looking up the original entry via Array.find inside Array.forEach. For N merged results × M per-process entries, cost is O(N×M).

Coverage data scales with the number of source files × coverage units per file. Modern projects regularly hit N > 5000 coverage entries. Post-test coverage generation happens on every vitest run and every vitest --coverage watch cycle.

Root Cause

// packages/coverage-v8/src/provider.ts:49-59
await this.readCoverageFiles<RawCoverage>({
  onFileRead(coverage) {
    merged = mergeProcessCovs([merged, coverage])

    // mergeProcessCovs sometimes loses startOffset, e.g. in vue
    merged.result.forEach((result) => {
      if (!result.startOffset) {
        const original = coverage.result.find(r => r.url === result.url)  // O(M)
        result.startOffset = original?.startOffset || 0
      }
    })
  },
  ...
})

For N results with missing startOffset, each coverage.result.find(...) scans M entries. Total O(N×M) per onFileRead callback; callbacks fire per coverage file, so the cost multiplies across multi-process runs.

Fix

Build a Map<string, RawCoverageResult> keyed by url once per onFileRead, then look up in O(1):

onFileRead(coverage) {
  merged = mergeProcessCovs([merged, coverage])

  // Build a URL lookup once; mergeProcessCovs sometimes loses startOffset.
  const byUrl = new Map(coverage.result.map(r => [r.url, r]))
  merged.result.forEach((result) => {
    if (!result.startOffset) {
      const original = byUrl.get(result.url)
      result.startOffset = original?.startOffset || 0
    }
  })
}

Total cost drops to O(N+M) per callback.

Severity Note

Runs on every coverage-enabled test run. Impact scales with project size × test count. On large monorepos (10k+ coverage entries) the difference is measurable wall-clock time on every CI run.

Complexity Gate

N=M=5000 merged entries: fixed must complete in <10ms
k-scaling 5×: time ratio must be <17.5× (O(k) ≈5×, not O(k²) ≈25×)

2.5 KiB Raw Blame History Unescape Escape