simplex-chat-0004: introduceToRemaining notElem O(N×M) member dedup; fix: Set.notMember O(log N)

This commit is contained in:
russell@unturf.com 2026-03-29 22:03:20 -04:00
parent eb534f0944
commit d9ca5b236f
12 changed files with 1006 additions and 0 deletions

View file

@ -0,0 +1,69 @@
# scylladb-0002: selection::from_selectors column dedup O(C²) via std::find
## Severity
MEDIUM
## Location
`cql3/selection/selection.cc:504-506``from_selectors` column dedup loop
## Pattern
SLOW: `std::find(defs.begin(), defs.end(), cv.col)` inside `for_each_expression` callback — O(C) per column
FAST: `std::unordered_set<const column_definition*> seen_defs` — O(1) per column
## Context
`selection::from_selectors` builds the list of unique `column_definition*` pointers referenced
by a CQL SELECT statement's selector list. It deduplicates using `std::find` on a growing vector:
```cpp
::shared_ptr<selection> selection::from_selectors(
data_dictionary::database db, schema_ptr schema, const sstring& ks,
const std::vector<prepared_selector>& prepared_selectors)
{
std::vector<const column_definition*> defs;
for (auto&& [sel, alias] : prepared_selectors) {
expr::for_each_expression<expr::column_value>(sel, [&] (const expr::column_value& cv) {
if (std::find(defs.begin(), defs.end(), cv.col) == defs.end()) { // O(C)
defs.push_back(cv.col);
}
});
}
// ...
}
```
For `SELECT *` on a wide table (C columns), `for_each_expression` is called C times, and each
call does `std::find` over a growing list of size 0..C-1. Total: O(C²/2).
This runs at statement prepare time and is called for every CQL SELECT operation. Wide tables
(time-series, IoT, event logs) commonly have 50500 columns, making this O(2,500125,000) pointer
comparisons per prepare.
`column_definition*` pointers are stable (schema is immutable per version), so pointer equality
is the correct dedup criterion — a hash set of raw pointers works directly.
## Speedup
250× at C=500 columns (wide time-series table SELECT *)
## Patch
```diff
--- a/cql3/selection/selection.cc
+++ b/cql3/selection/selection.cc
::shared_ptr<selection> selection::from_selectors(data_dictionary::database db, schema_ptr schema, const sstring& ks, const std::vector<prepared_selector>& prepared_selectors) {
std::vector<const column_definition*> defs;
+ std::unordered_set<const column_definition*> seen_defs;
for (auto&& [sel, alias] : prepared_selectors) {
expr::for_each_expression<expr::column_value>(sel, [&] (const expr::column_value& cv) {
- if (std::find(defs.begin(), defs.end(), cv.col) == defs.end()) {
+ if (seen_defs.insert(cv.col).second) {
defs.push_back(cv.col);
}
});
}
```
`seen_defs.insert(cv.col).second` returns `true` if the element was newly inserted (not a duplicate),
matching the previous semantics exactly while reducing complexity from O(C²) to O(C).