java-topology/whitepaper/outreach/scanner-enhancement-session.md
russell@unturf.com 8b3a38bcab
scanner-enhancement intel: 10 unmoad fixes, ~51679 FPs cleared across 19 codebases
Documents this autonomous-loop session's pivot from breadth wave scanning
to unmoad scanner enhancements. Each fix targeted a documented FP class
(noted across waves 11/15/18/21/24/26 + others), shipped + validated
end-to-end against re-cloned source.

Architecture: per-file symbol tables on ScanState (hash_vars for M1,
tl_vars for M3) + path/header/argument-shape heuristics. Three repos in
sync, all unmoad commits remain local (no remote per project CLAUDE.md).

Notable byproduct: with M3 noise dropped wildfly 4840 -> 37 (commit
1f48798), one real ThreadLocal leak surfaces as a MOAD-0003 candidate
for fox's review — ElytronSecurityIntegration.securityContext.set(context)
with zero remove() calls anywhere in the WildFly codebase. Out of CWE-407
scope; MOAD-0003 follow-up.
2026-04-26 07:24:28 -04:00

10 KiB

unmoad scanner enhancement session — 10 fixes, ~51,679 FPs cleared

Session date: 2026-04-26 Scope: Single autonomous-loop session targeting the false-positive backlog documented across waves 7-28 (scanner FPs noted 30+ times).


Summary

Ten scanner enhancements landed in unmoad.com, all validated end-to-end against re-cloned source. ~51,679 false positives cleared across 19 codebases. Three repos in sync after each commit. All commits remain local on unmoad.com (no remote configured per the project's CLAUDE.md).

The session pattern: pick the most-documented FP from triage backlog, write a per-file symbol-table or path-shape suppression, validate against two real codebases that exhibited the FP, commit. Rinse, repeat.

Commits + validated impact

# Commit Detector Languages FPs cleared (validated)
1 73caddd M1 Java/Kotlin/Scala/Groovy wildfly 65
2 79dbac6 M1 Rust meilisearch 25
3 1f48798 M3 Java/Kotlin/Groovy wildfly 4803
4 9f62cdf M1 C# PowerShell 42
5 e94ec26 M3 Python paperless-ngx 108 + patroni 49
6 1048451 scanner All WordPress 540 + TYPO3 427
7 c152ad9 scanner All aws-sdk-go-v2 18,582 + azure-sdk-for-go 25,755
8 26958c0 M1 Lisp/Scheme/Racket sbcl 706 + racket 306
9 0fa41ae M7 All hydrogen 47 + Hyprland 37 + abseil 22 + spack 6
10 7770325 M1+M7 Rust + all helix 37 + alacritty 40 + wezterm 81

Total: ~51,679 FPs cleared across 19 codebases.

Architecture

Three reusable pieces of state added to ScanState:

char hash_vars[64][64];   /* moad_0001: vars declared as Set/Map/HashSet */
int  hash_vars_count;
char tl_vars[64][64];     /* moad_0003: vars declared as ThreadLocal/ContextVar */
int  tl_vars_count;

Eight of ten enhancements use a per-file symbol-table tracker that recognizes declarations on each line and consults the table at call sites:

Tracker Recognizes Suppresses
track_java_hash_decls Set<...>, HashSet<...>, EnumSet<...>, Map<...>, HashMap<...>, ConcurrentHashMap<...>, ImmutableSet/Map<...> var.contains(...) / var.indexOf(...)
track_rust_hash_decls let var: HashSet<...>, let var = HashSet::new(), plus FxHashSet, BTreeSet, RoaringBitmap, SmallBitmap, IndexSet, BitSet, DashSet, etc. var.contains(...) / var.position(...)
track_csharp_hash_decls HashSet<...>, Dictionary<...>, ImmutableHashSet<...>, FrozenSet<...> var.Contains(...) / var.IndexOf(...)
track_java_tl_decls ThreadLocal<...>, InheritableThreadLocal<...>, ScopedValue<...>, FastThreadLocal<...> var.set(...) (no-remove leak check)
track_python_tl_decls var = ContextVar(...), var: ContextVar[T] = ..., _local = threading.local() var.set(...) (no-reset leak check)

Two enhancements use file-level path/header heuristics:

Heuristic Suppresses
should_skip_filename *.min.js, *.min.css, *.bundle.js, *.umd.js, *.global.prod.js, *-bundle.js, *-min.js
is_generated First-16-line scan for // Code generated by ..., DO NOT EDIT, <auto-generated>, @generated, Auto-generated, automatically generated, This file was generated

Plus expanded SKIP_DIRS: 3rdParty, external, Externals, third_party, thirdparty, Contrib.

Two enhancements operate on argument-shape and definition-shape:

Helper Recognizes
csharp_indexof_is_char_literal IndexOf('X') (PowerShell single-char position scan, not collection lookup)
looks_like_definition (M7) bool ClassName::contains(...), function body ) { opener, def/fn/function keywords
looks_like_string_contains (M7) .contains('X') / .contains("X") / .contains(b'X') substring search
contains_arg_is_path_constant (M1+M7) .contains(Type::SCREAMING_SNAKE) — bitflags / enum flag check

One severity-only adjustment:

Rule Change
list-member-in-loop (Lisp/Scheme/Racket) SEV_HIGHSEV_MEDIUM. Lisp (member ...) returns the matched-tail not just bool; idiomatic for small-list membership in graph algorithms.

Test surface

Throughout the session, the test count grew from 105 → 111 integration + 29 unit + 33 functional. Eight new fixtures:

  • moad_0001/clean_java_set_decl.javaSet<String>.contains + EnumSet.contains + HashMap.containsKey
  • moad_0001/clean_rust_hash_decl.rsHashSet<u32>.contains + BTreeSet.contains + let _ = HashSet::new() constructor pattern
  • moad_0001/clean_csharp_set_decl.csHashSet<string>.Contains + Dictionary.ContainsKey + ImmutableHashSet.Contains + IndexOf(':') char position
  • moad_0001/clean_rust_bitflags.rsflags.contains(KeyModifiers::SHIFT) + mods.contains(Self::CTRL)
  • moad_0003/clean_atomic_set_no_threadlocal.javaAtomicLong.set(0L) + AtomicReference.set(null) + JavaBean setters
  • moad_0003/clean_python_django_set.py — Django M2M relation.set(items) + threading.Event.set() + custom cache.set(k,v)
  • Functional: vendored UI suffix exclusion (.min.js/.bundle.js/.umd.js)
  • Functional: codegen marker suppression (// DO NOT EDIT + // @generated)

All 173 tests pass after every commit.

Validated impact summary

By codebase (sorted by FPs cleared):

Codebase Before After Cleared Reduction
azure-sdk-for-go (codegen) 25,898 143 25,755 99.4%
aws-sdk-go-v2 (codegen) 18,582 0 18,582 100%
wildfly (Java Set + ThreadLocal) 4,933 65 4,868 98.7%
sbcl (Lisp member) 727 21 706 97.1%
WordPress (vendored UI) 1,050 510 540 51.4%
TYPO3 (vendored UI) 934 507 427 45.7%
racket (Lisp member) 383 77 306 79.9%
paperless-ngx (Python ContextVar) 109 1 108 99.1%
wezterm (Rust bitflags) 135 54 81 60.0%
patroni (Python ContextVar) 49 0 49 100%
hydrogen (Qt geometry) 64 17 47 73.4%
PowerShell (C# IndexOf char) 169 127 42 24.9%
alacritty (Rust bitflags) 51 11 40 78.4%
Hyprland (geometry) 46 9 37 80.4%
helix (Rust bitflags) 75 38 37 49.3%
meilisearch (Rust HashSet) 68 43 25 36.8%
abseil-cpp (M7 definitions) 60 38 22 36.7%
spack (M7 method calls) 72 66 6 8.3%
ripgrep (Rust HashSet) 5 4 1 20.0%
Total 53,510 1,831 51,679 96.6%

Findings worth investigating from the cleaner output

After the M3 detector noise dropped from wildfly 4840 → 37 finds (commit 1f48798), the remaining 37 are real ThreadLocal patterns. Manual triage identified one clear MOAD-0003 candidate worth follow-up:

  • connector/src/main/java/org/jboss/as/connector/security/ElytronSecurityIntegration.java:38 declares private final ThreadLocal<SecurityContext> securityContext = new ThreadLocal<>(); with setSecurityContext(...) at line 51 calling .set(context) and zero securityContext.remove() / securityContext.set(null) anywhere in the WildFly codebase (verified by grep -rn). For a JCA WorkManager that reuses thread-pool threads across Work items from different security principals, a leftover SecurityContext from a prior Work item could be visible to a subsequent one if the new item reads via getSecurityContext() before its own setSecurityContext(...) call. Real defense-in-depth candidate at minimum, possibly an exploitable identity-leak.

This is out of the CWE-407 disclosure pipeline (MOAD-0001) but goes into the MOAD-0003 (A Leaked Context) follow-up queue for fox's review.

Effect on the wave-survey backlog

The 30+ scanner-enhancement candidates documented across wave surveys (every wave from 11 onward listed at least one) are now mostly closed:

  • Java declared-type Set/EnumSet awareness — shipped (commit 1)
  • Rust HashSet/FxHashSet/Bitmap awareness — shipped (commits 2, 10)
  • JUnit ExtensionContext suppression — partially shipped via Java ThreadLocal-scoping in commit 3 (eliminates the .set() noise; the getStore().put() patterns remain unflagged today)
  • Java ThreadLocal-scoped .set()shipped (commit 3)
  • Python ContextVar-scoped .set()shipped (commit 5)
  • C# HashSet declared-type — shipped (commit 4)
  • C# IndexOf(char_literal) — shipped (commit 4)
  • Codegen artifact suppression — shipped (commit 7)
  • Vendored UI tree exclusion — shipped (commit 6)
  • Rust bitflags::bitflags! macro — shipped (commit 10)
  • Lisp (member ...) algorithm awareness — shipped (commit 8)
  • M7 definition / String.contains distinction — shipped (commit 9)
  • Qt .contains() family awareness — partially shipped via M7 definition heuristic (catches bool QRect::contains(...) definitions but call-site qrect.contains(point) still fires; needs Qt-typed receiver tracking)
  • Smalltalk language module — not shipped (deferred; pharo still unscannable)
  • Spack/conan-style intersects() algorithm awareness — partially shipped via M7 definition heuristic (catches definitions; call-site spec.intersects(other) still fires)

What's left

The remaining noise sources after this session are dominated by:

  • Test files that exercise patterns the scanner correctly flags as defects in production code. A path-prefix suppression for *Test.java, *_test.go, tests/, test/, __tests__/ directories would help but requires care not to mask real defects in test-helper code.
  • Spack/conan-style algorithm methods at call site (intersects / overlaps / contains / distance) where the method is the algorithm but the receiver type isn't a recognized geometry primitive.
  • Method-chain receivers (a.getX().contains(b) — extract_receiver rejects chained calls). This is conservative-correct but misses legitimate suppressions when the chain returns an O(1) container.

References

  • unmoad detection engine: git.unturf.com/engineering/unmoad.com
  • ten unmoad commits: 73caddd through 7770325, all on main, no remote configured (per CLAUDE.md)
  • prior session shipped 5 flagship CWE-407 patches (UNDF-1300 through 1304): weaviate, pyroscope, ghost, ghidra-0001, ghidra-0002
  • waves 7-28 of breadth scanning: 192 cumulative honor roll, 5 flagships