UNDF: UNDF-2026-000000497

pandas-0001: Styler render — O(n²) hidden_rows list membership in render loops

Severity: MEDIUM CWE: CWE-407 (Algorithmic Complexity — linear membership test in hot loop) Speedup: >25x at R=2000 rows, H=500 hidden rows Target: pandas (pandas-dev/pandas) File: pandas/io/formats/style_render.py

Description

Styler renders DataFrames to HTML, LaTeX, and string. During rendering, self.hidden_rows (declared as Sequence[int], initialized as list) is tested with r not in self.hidden_rows inside loops over all rows. This appears in at least five locations:

Line	Pattern	Outer loop scale
662	`if z[0] not in self.hidden_rows`	O(R) enumerate
843	`r not in self.hidden_rows`	O(R × C) body cell loop
910	`if r not in obj.hidden_rows`	O(R) recursive concat
971	`if r not in self.hidden_rows`	O(R) cline build
2306	`i in styler.hidden_rows`	O(R) export loop

With R rows and H hidden rows, each render call costs O(R × H) for these tests alone. When H approaches R (hide many rows), this becomes O(R²).

hidden_rows is populated via Styler.hide() which calls get_indexer_for() returning a numpy array. The array is assigned directly without conversion to a set, so membership tests remain O(H).

Root Cause

# style_render.py:131
self.hidden_rows: Sequence[int] = []   # list — O(n) membership

# style_render.py:662
for r, row_tup in [
    z for z in enumerate(self.data.itertuples()) if z[0] not in self.hidden_rows  # O(H) each
]:

Fix: convert hidden_rows to a frozenset (or maintain a parallel set) at assignment time so every membership test is O(1).

Patch

In style.py, after computing h_els:

h_els = getattr(self, objs).get_indexer_for(getattr(hide, objs))
setattr(self, f"hidden_{alt}", frozenset(h_els.tolist()))

Or ensure hidden_rows is typed as frozenset[int] and initialized to frozenset() in StylerRenderer.__init__.

Complexity Before

Each render: O(R × H) membership tests; worst case O(R²)

Complexity After

Membership check: O(1) → render cost: O(R + H)

Reproduction

cd defects/pandas/unit && javac -d . *.java && java -ea unit.PandasTest

2.3 KiB Raw Blame History Unescape Escape