java-topology/docs/tickets/duckdb-0001-correlated-columns-vector-linear-dedup.md

duckdb-0001: CorrelatedColumns dedup via vector linear scan in Binder

Target: duckdb/duckdb File: src/planner/binder.cpp, src/planner/expression_binder/lateral_binder.cpp Severity: MEDIUM Pattern: CWE-407 — O(n) membership test inside loop → O(n²)

Description

CorrelatedColumns (in src/include/duckdb/planner/binder.hpp) is a wrapper around vector<CorrelatedColumnInfo>. Two dedup sites perform O(n) linear std::find membership tests on this vector before inserting a new entry:

Site 1 — Binder::AddCorrelatedColumn (binder.cpp)

void Binder::MergeCorrelatedColumns(CorrelatedColumns &other) {
    for (idx_t i = 0; i < other.size(); i++) {   // O(n) outer
        AddCorrelatedColumn(other[i]);             // O(n) inner std::find
    }
}
void Binder::AddCorrelatedColumn(const CorrelatedColumnInfo &info) {
    if (std::find(correlated_columns.begin(), correlated_columns.end(), info)
            == correlated_columns.end()) {        // O(n) scan
        correlated_columns.AddColumn(info);
    }
}

MergeCorrelatedColumns is called when flattening dependent joins. For a query with C correlated columns across S subquery levels, total work is O(C² × S).

Site 2 — LateralBinder::ExtractCorrelatedColumns (lateral_binder.cpp)

Called recursively for every expression node, with an O(n) std::find per BOUND_COLUMN_REF encountered. For a wide SELECT with many lateral references this is O(expr_nodes × correlated_columns).

Site 3 — HasCorrelatedExpressions::VisitReplace (has_correlated_expressions.cpp)

Inner loop: for each correlated_column → std::find on subquery binder's correlated_columns — O(C²) per subquery expression visit.

Fix

Add unordered_set<ColumnBinding, ColumnBindingHash> as a shadow set inside CorrelatedColumns (or replace the vector with a set+ordered vector pair) so that contains() is O(1) and insertion is O(1) amortized.

See defects/duckdb/patch/0001.patch.

Complexity

Operation	Before	After
AddCorrelatedColumn contains check	O(n)	O(1)
MergeCorrelatedColumns	O(n²)	O(n)
ExtractCorrelatedColumns per node	O(n)	O(1)