93 lines
3.1 KiB
Markdown
93 lines
3.1 KiB
Markdown
# postgresql-0008 — `paraminfo_get_equal_hashops`: O(N²) list_member deduplication in Memoize path planning
|
||
|
||
## Status
|
||
PATCHED
|
||
|
||
## Severity
|
||
MEDIUM (>20× speedup at N=300 join/lateral clauses)
|
||
|
||
## Location
|
||
`src/backend/optimizer/path/joinpath.c`, function `paraminfo_get_equal_hashops()`
|
||
|
||
## Description
|
||
`paraminfo_get_equal_hashops()` builds two parallel lists: `*param_exprs` and
|
||
`*operators`. While iterating over `ppi_clauses` (join clauses) and
|
||
`innerrel->lateral_vars` (lateral variable references), it deduplicates entries
|
||
using `list_member(*param_exprs, expr)` — a full O(|param_exprs|) structural
|
||
`equal()` walk on every iteration.
|
||
|
||
The pattern:
|
||
|
||
```c
|
||
foreach(lc, clauses) {
|
||
...
|
||
if (!list_member(*param_exprs, expr)) /* O(|param_exprs|) */
|
||
*param_exprs = lappend(*param_exprs, expr);
|
||
}
|
||
foreach(lc, lateral_vars) {
|
||
...
|
||
if (!list_member(*param_exprs, expr)) /* O(|param_exprs|) scan again */
|
||
*param_exprs = lappend(*param_exprs, expr);
|
||
}
|
||
```
|
||
|
||
Total cost: O(N²) where N = |ppi_clauses| + |lateral_vars|.
|
||
|
||
`expr` is typically a `Var` node (column reference from the outer relation).
|
||
For Var nodes, deduplication can use Bitmapset O(1) keyed on
|
||
`varno * 3200 + varattno + 1600` (same encoding as postgresql-0003/0004).
|
||
Non-Var expressions fall back to a kept `List *seen_nonvar` with `list_member`.
|
||
|
||
### Hot path
|
||
Called from `create_memoize_path()` (line ~844) for every candidate Memoize
|
||
join path — once per inner relation per parameterized outer path. With K
|
||
joins in a query of N tables, this is called O(K×P) times where P is the
|
||
number of parameterized paths.
|
||
|
||
## Patch
|
||
|
||
```c
|
||
--- a/src/backend/optimizer/path/joinpath.c
|
||
+++ b/src/backend/optimizer/path/joinpath.c
|
||
@@ -480,6 +480,10 @@ paraminfo_get_equal_hashops(...)
|
||
{
|
||
List *lateral_vars;
|
||
ListCell *lc;
|
||
+ /* CWE-407 fix (postgresql-0008): track seen Var exprs with a Bitmapset so
|
||
+ * each duplicate check is O(1) instead of O(|param_exprs|).
|
||
+ * Non-Var exprs fall back to a List for structural equal() comparison. */
|
||
+ Bitmapset *seen_bms = NULL;
|
||
+ List *seen_nonvar = NIL;
|
||
|
||
*param_exprs = NIL;
|
||
*operators = NIL;
|
||
@@ -536,9 +540,21 @@ paraminfo_get_equal_hashops(...)
|
||
if (!OidIsValid(hasheqoperator))
|
||
...
|
||
|
||
- if (!list_member(*param_exprs, expr))
|
||
+ /* O(1) Var-path check; O(|seen_nonvar|) fallback for non-Var */
|
||
+ bool already_seen;
|
||
+ if (IsA(expr, Var)) {
|
||
+ Var *v = (Var *) expr;
|
||
+ int key = v->varno * 3200 + v->varattno + 1600;
|
||
+ already_seen = bms_is_member(key, seen_bms);
|
||
+ if (!already_seen) seen_bms = bms_add_member(seen_bms, key);
|
||
+ } else {
|
||
+ already_seen = list_member(seen_nonvar, expr);
|
||
+ if (!already_seen) seen_nonvar = lappend(seen_nonvar, expr);
|
||
+ }
|
||
+ if (!already_seen)
|
||
{
|
||
*operators = lappend_oid(*operators, hasheqoperator);
|
||
*param_exprs = lappend(*param_exprs, expr);
|
||
```
|
||
|
||
(Same pattern repeated for the `lateral_vars` loop at line ~593.)
|
||
|
||
## Speedup
|
||
At N=300 lateral vars (all unique Vars): O(N²)=90,000 ops → O(N)=300 ops → **300× speedup**.
|
||
Realistic scenario (N=50): O(N²)=2,500 ops → O(N)=50 ops → **50× speedup**.
|
||
|
||
## Test
|
||
`defects/postgresql/unit/PostgresqlTest.java` — `postgresql-0008` section.
|