engineering/java-topology
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

beam+hive: CWE-407 scan — Beam CLEAN, Hive 2 defects (mergeSchema + averageColumnSizes)

Browse source 
Beam: exhaustive scan of sdks/java/core + runners — all membership tests already
use proper Set types (HashSet, LinkedHashSet, ImmutableSet, TreeSet). CLEAN.

Hive hive-0001: SharedWorkOptimizer.mergeSchema() uses List.contains() in loops
for neededColumnIDs/neededColumns/virtualCols dedup. O(D*R) per list. MEDIUM, 3-4x.

Hive hive-0002: HiveRelMdSize.averageColumnSizes() uses ImmutableList.contains(i)
in column loop during Calcite metadata queries. O(C*L). MEDIUM, 3-5x.

Both patched with HashSet wrappers. 2/2 unit tests PASS.
This commit is contained in:
russell@unturf.com 2026-03-30 13:05:40 -04:00
parent db958a2038
commit 1a1abc2154
4 changed files with 265 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

34
defects/beam/patch/CLEAN.md
View file

@ -1,27 +1,25 @@
# Apache Beam — CWE-407 Scan Result: CLEAN
**Date:** 2026-03-30
**Scanner:** agent blackops
**Scope:** sdks/java/core/src/main/java/, runners/
**Target:** Apache Beam (Java SDK + Runners)
**Scope:** `sdks/java/core/src/main/java/`, `runners/*/src/main/java/`
## Summary
Apache Beam's Java SDK and runners are clean of CWE-407 algorithmic complexity
defects. The codebase consistently uses HashSet/LinkedHashSet for membership
tests in graph traversal, pipeline fusion, and transform hierarchy operations.
No CWE-407 (algorithmic complexity via linear membership test in loop) defects found.
## Key observations
Apache Beam consistently uses proper data structures for membership testing:
- `TransformHierarchy.visit()`: `Set<PValue>` and `Set<Node>` for visitedValues/visitedNodes
- `PortablePipelineJarCreator.copyResourcesFromJar()`: `HashSet<String>` for previousEntryNames
- `PipelineTranslation`: `HashSet<String>` for viewTransforms
- `GreedyStageFuser`: `LinkedHashSet` for fusedCollections/materializedPCollections
- `Networks.reachableNodes()`: `HashSet` for visitedNodes
- `OutputDeduplicator`: `HashMultimap` for pcollectionProducers
- `SamzaTimerInternalsFactory`: `TreeSet` for eventTimeBuffer
- `DisplayData.Builder`: `IdentityHashSet` for visitedComponents
- `FieldAccessDescriptor.union()`: `LinkedHashSet` for fieldsAccessed
- `PipelineOptionsFactory`: `ImmutableSet` for IGNORED_METHODS, PIPELINE_OPTIONS_FACTORY_CLASSES
- `GreedyStageFuser`: uses `LinkedHashSet` for fusedCollections/materializedPCollections
- `GreedyPipelineFuser`: uses `LinkedHashSet`/`HashSet` throughout; has O(N²) comment
at groupSiblings but this is inherent sibling compatibility checking, not a membership defect
- `Networks`: uses `visitedNodes` Set for BFS reachability
- `TransformHierarchy`: all visited tracking uses `Set<Node>`
- `PipelineTranslation`: viewTransforms is `HashSet<String>`
- `Schema.indexOf()`: backed by `fieldIndices` HashMap
- `PipelineOptionsFactory`: uses `HashSet` for usedDescriptors, `ImmutableSet` for IGNORED_METHODS
- `ExperimentContext`: uses `EnumSet` for experiment lookup
- `DataflowRunner.stageArtifacts`: uses `HashSet` for stagedNames dedup
- `CombineFns.checkUniqueness`: List.contains() but N is number of composed combiners (2-5)
## Keywords searched
No data-proportional linear scans inside loops found.
`.contains(`, `.indexOf(`, `ArrayList`, `List<`, `visited`, `seen`, `worklist`