# UNDF: UNDF-2026-000000290
# spark-deeper — Scheduler + Catalyst + Physical Plan + Storage + Shuffle CLEAN

## Areas Scanned

### Core Scheduler
- `core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala` — BFS traversals (spark-0002), waitingStages filter (spark-0004), `submitWaitingChildStages`, `stageDependsOn`, `getMissingParentStages`, all 6 BFS methods — all already covered by existing defects
- `core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala` — all `contains`/`exists` calls use `HashMap`/`mutable.Map` → O(1)
- `core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala` — no List.contains in hot paths
- `core/src/main/scala/org/apache/spark/scheduler/TaskSetExcludeList.scala` — `excludedExecs`, `excludedNodes` are `HashSet` → O(1)
- `core/src/main/scala/org/apache/spark/scheduler/HealthTracker.scala` — clean
- `core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala` — `queues.asScala.find(_.name == queue)` on bounded small list (typically 4 queues)

### Catalyst Optimizer
- `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala` — `excludeList` is `AttributeSet` (O(1)), `projectList.exists(...)` is O(E) but called once per plan node, not in a nested loop
- `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala` — all `projectList.exists()` / `aggList.exists()` calls are O(E) single-pass pattern-match guards
- `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala` — `assignments.exists(isEqual(_, fieldPath))` is O(A×F) in `filterSchema`, but this is a schema-time operation on small data
- `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CapturesConfig.scala` — `configPrefixDenyList` has ~11 fixed entries, not a scaling concern

### Physical Plan Operators
- `sql/core/src/main/scala/org/apache/spark/sql/execution/joins/` — all `contains`/`exists` calls use `HashSet`, `OpenHashSet`, `AttributeSet`, or `ExpressionSet` → O(1)
- `sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala` — `childrenIndexes` is bounded (≤ 2 for joins)
- `sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/` — clean
- `sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/` — `modes.contains(Final)` etc. is on a small fixed-size list

### Dynamic Partition Pruning (Spark SQL)
- `sql/core/src/main/scala/org/apache/spark/sql/execution/dynamicpruning/PartitionPruning.scala` — `joinKeys.indexOf(filteringKeys.head)` is O(K) single call, not in a loop

### Storage and Shuffle
- `core/src/main/scala/org/apache/spark/storage/` — no list.contains patterns in hot paths
- `core/src/main/scala/org/apache/spark/shuffle/` — no list.contains patterns in hot paths

### RDD Operations
- `core/src/main/scala/org/apache/spark/rdd/RDD.scala` — `getNarrowAncestors` uses `mutable.HashSet` → O(1)
- `core/src/main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala` — `partitionIndices.contains` uses Scala `Range.contains` → O(1)
- `core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala` — all `contains` calls use `HashMap`-backed structures

### MLlib (spot check)
- `mllib/src/main/scala/org/apache/spark/ml/feature/RFormulaParser.scala` — `dotTerms.contains` is O(D) per filter call on plan-time formula parsing — negligible at runtime

## Verdict: No new CWE-407 defects found beyond spark-0001 through spark-0004

All identified `contains`/`exists`/`find` calls in hot paths either:
1. Operate on hash-based structures (O(1) lookup), or
2. Are called once per plan node / RDD construction (not in a doubly-nested loop), or
3. Operate on bounded-size data (≤ join side count, ≤ mode count, etc.)

The 6 BFS `ListBuffer.remove(0)` defects are already captured as spark-0002.
The `submitWaitingChildStages` O(W×P) defect is already captured as spark-0004.

## Scan date: 2026-03-27
