russell@unturf.com d67ec93a5d test-frameworks wave 3: vitest + testng + jasmine + libcheck (4 patches)

vitest-0001: coverage-v8 coverage.result.find inside merged.result.forEach
  -> Map<url, result> lookup. Bench: 824x at N=M=10000 coverage entries.

testng-0001: DynamicGraph.toDot freeNodes.contains inside two for-each
  loops -> Map<T, String> color lookup via getOrDefault. Bench: 64x at N=2000.

jasmine-0001: SpyRegistry.spyOnAllFunctions propertiesToSkip.indexOf inside
  Array.filter + .concat growth across D prototype levels -> Set.has + O(1)
  growth. Bench: 61x at D=10, P=300.

check-0001: libcheck suite_tcase linear strcmp scan over tclst List
  -> parallel hashtable for O(1) lookup amortized. Bench: 117x at N=1000.
  Shipped as design sketch; full integration requires companion hashtable.

Also ships whitepaper/outreach/test-harness-survey.md documenting 14
clean-scan frameworks across Clojure, OCaml, Haskell, Erlang, Go, F#,
Julia, Shell, Lua, JS. Scope covered 61 targets across 30+ languages.

UNDF IDs: 1292 (check), 1293 (jasmine), 1294 (testng), 1295 (vitest).
All 12 tests pass.

2026-04-23 08:54:44 -04:00

2.6 KiB

Raw Blame History

libcheck — CWE-407 Disclosure Brief

Project: libcheck (libcheck/check) Disclosure date: 2026-04-23 Severity: LOW-MEDIUM Speedup: 117x at N=1000 tcases (test-name lookup), confirmed by benchmark Status: patch-ready sketch, needs companion hashtable integration

Summary

libcheck (the C unit test framework behind thousands of C/C++ projects) looks up test cases by name via a linear scan of a List, calling strcmp per entry. The runner filter path invokes this per suite × per filter application. For N tcases × N filter calls, cost is O(N²).

The Defects

check-0001 (MOAD-0001 — LOW-MEDIUM): src/check.c:76-94

int suite_tcase(Suite *s, const char *tcname) {
    List *l;
    if(s == NULL) return 0;

    l = s->tclst;
    for(check_list_front(l); !check_list_at_end(l); check_list_advance(l)) {
        TCase *tc = (TCase *)check_list_val(l);
        if(strcmp(tcname, tc->name) == 0) return 1;
    }
    return 0;
}

The same linear-scan pattern repeats in src/check_run.c for the suite/tcase name filter applied during run initialization.

Fix: Maintain a parallel hashtable keyed by test-case name alongside the ordered List. Lookups drop to O(1) amortized. The List stays authoritative for ordered iteration (test-run order is deterministic-by-design in libcheck). Integration requires adding a small hashtable implementation (or wiring against glib's GHashTable where available) and updating tcase_add/suite_add_tcase to insert into both structures.

Benchmark (N tcases, N lookups)	defective	fixed	speedup
200	0.63ms	0.02ms	26.8x
500	4.13ms	0.07ms	61.8x
1000	16.80ms	0.14ms	117.1x

Impact scales with test suite size. Typical unit-test projects have fewer than 50 tcases per suite, where the effect is negligible. Larger integration/e2e harnesses with hundreds of tcases and filter patterns see measurable slowdown. LOW-MEDIUM priority; cleanup rather than hotspot.

Scanner Evidence

unmoad detects the pattern at HIGH severity via the strcmp-in-loop rule. Trigger + clean fixture pair in tests/integration/fixtures/moad_0001/.

Patches

check-0001-suite-tcase_by_name-linear-strcmp.patch (UNDF-2026-000001292)

Patch is shipped as a design sketch; full upstream integration requires the companion hashtable (libcheck does not currently ship one). Full test + bench suite at defects/check/ in the java-topology research repo.

2.6 KiB Raw Blame History Unescape Escape