wave7: 433/194 — kafka/flink/pulsar, spring/micronaut/quarkus, nginx/haproxy/traefik, linux/nomad/consul, numpy/pandas/sklearn, ES/OS/pg/sqlite/rustc/cargo
This commit is contained in:
parent
3735145aa5
commit
5fe6da7cc2
69 changed files with 6793 additions and 32 deletions
|
|
@ -0,0 +1,46 @@
|
|||
# flink-0002: RowTypeUtils.getUniqueName — List.contains() inside nested for+do-while
|
||||
|
||||
## Defect ID
|
||||
flink-0002
|
||||
|
||||
## File:Line
|
||||
`flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/RowTypeUtils.java:43,49`
|
||||
|
||||
## Description
|
||||
`getUniqueName(List<String> oldNames, List<String> checklist)` does:
|
||||
|
||||
```java
|
||||
for (String oldName : oldNames) { // O(N)
|
||||
if (checklist.contains(oldName) || result.contains(oldName)) { // O(M) + O(R)
|
||||
do {
|
||||
changedName = oldName + "_" + suffix++;
|
||||
} while (checklist.contains(changedName) || result.contains(changedName)); // O(M) + O(R) per iteration
|
||||
result.add(changedName);
|
||||
} else {
|
||||
result.add(oldName);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Both `checklist` and `result` are `List<String>`. Each `.contains()` is O(M) or O(R).
|
||||
The do-while iterates up to K times per name (until a unique suffix is found).
|
||||
Total complexity: O(N × (M + R) × K) which approaches O(N × M²) in the worst case
|
||||
when many names clash and checklist is large.
|
||||
|
||||
This method is called during SQL query planning whenever column names must be
|
||||
deduplicated (e.g., join projections, window aggregations) — it runs on every query.
|
||||
|
||||
## Complexity
|
||||
- Slow: O(N × M × K) — List.contains() = O(M)
|
||||
- Fast: O(N × K) — HashSet.contains() = O(1)
|
||||
|
||||
## Severity
|
||||
MEDIUM
|
||||
|
||||
## Speedup Estimate
|
||||
~M× improvement; at M=100 column names per relation, ~100x.
|
||||
|
||||
## Fix
|
||||
Convert `checklist` to `HashSet<String>` at call sites, or convert internally at
|
||||
method entry. Build a `HashSet<String> seen` from `checklist` plus accumulate
|
||||
`result` into a parallel `HashSet<String>` for O(1) membership checks.
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
# flink-0003: AggregateReduceGroupingRule — List<Integer>.contains() inside for loop
|
||||
|
||||
## Defect ID
|
||||
flink-0003
|
||||
|
||||
## File:Line
|
||||
`flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/AggregateReduceGroupingRule.java:88`
|
||||
|
||||
## Description
|
||||
In `onMatch()`:
|
||||
|
||||
```java
|
||||
List<Integer> newGroupingList = newGrouping.toList(); // List<Integer>
|
||||
for (int column : originalGrouping) { // O(G) iterations
|
||||
if (newGroupingList.contains(column)) { // O(G) linear scan each time
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`newGroupingList` is a `List<Integer>`. Each `.contains()` call does a linear scan
|
||||
O(G) where G = number of grouping columns. Total: O(G²).
|
||||
|
||||
This fires during query planning on every aggregate that has reducible grouping keys —
|
||||
i.e., every GROUP BY with functional dependencies. In queries with wide GROUP BY
|
||||
clauses (e.g., 50+ columns in analytics), this is O(2500) instead of O(50).
|
||||
|
||||
## Complexity
|
||||
- Slow: O(G²) — List.contains() = O(G)
|
||||
- Fast: O(G) — HashSet.contains() = O(1)
|
||||
|
||||
## Severity
|
||||
MEDIUM
|
||||
|
||||
## Speedup Estimate
|
||||
~G× improvement; at G=50 grouping columns, ~50x speedup in the planning phase.
|
||||
|
||||
## Fix
|
||||
Replace `newGroupingList` with `Set<Integer> newGroupingSet = new HashSet<>(newGrouping.toList())`.
|
||||
The list is only used for `.contains()` and `.size()` — both work identically on HashSet.
|
||||
Loading…
Add table
Add a link
Reference in a new issue