java-topology/whitepaper/outreach/pinot.md

Apache Pinot — CWE-407 Disclosure Brief

2026-03-27 · Patch available — awaiting upstream merge

Finding

One O(F×S) defect in Apache Pinot's segment processor field specification path. The defect is in pinot-core/.../SegmentProcessorUtils.java — the getFieldSpecs() function that determines sort order for each field when processing segments. Called per field per segment processing invocation. Patched. Measured ratio: 46×.

The Defect

pinot-0001 (PATCHED — MEDIUM): pinot-core/.../SegmentProcessorUtils.java:66

// sortOrder is a List<String>
// Inside getFieldSpecs() — for (FieldSpec fieldSpec : schema.getAllFieldSpecs()) loop:
if (sortOrder.contains(fieldSpec.getName())) { ... }

sortOrder is a List<String>. .contains() performs an O(S) linear scan for each of F FieldSpec entries in the schema loop. O(F × S) per getFieldSpecs() call where F = number of fields in the schema and S = number of sort columns. Fixed: new HashSet<>(sortOrder) hoisted before the loop. Measured ratio: 46×.

Complexity Proof

Let:

• F = number of fields in the Pinot schema (the FieldSpec count from getAllFieldSpecs())
• S = number of sort columns in sortOrder

getFieldSpecs() iterates all F schema fields. For each field, it checks sortOrder.contains(fieldSpec.getName()) to determine if this field is part of the sort key:

• Cost: F × S string comparisons per call
• Fixed: build Set<String> sortSet = new HashSet<>(sortOrder) once before loop → F × O(1) = O(F)
• Overhead of building the set: O(S) — paid once

At F=46 fields, S=46 sort columns (wide table, all columns sorted): defective=2,116, fixed=92. 46× ratio.

The ratio scales with both F and S. Wide Pinot schemas with multi-column sort orders — the recommended pattern for optimizing range queries — hit the worst case. getFieldSpecs() is called during segment creation, merge, and processing in the ingestion pipeline. High-throughput ingestion paths invoking getFieldSpecs() per-segment multiply the overhead by the segment creation rate.

Impact

Every Apache Pinot deployment using sorted segments — which is the recommended configuration for range query performance and is used by default in most production deployments — hits this path during ingestion. Pinot is used for real-time OLAP at scale (LinkedIn, Uber, Stripe); segment processing throughput directly affects ingestion lag. Wide schemas (event tables with many dimensions and metrics) processing sorted segments with multi-column sort orders hit the O(F×S) worst case on every segment build. Offline table ingestion pipelines that process many segments in parallel multiply the overhead across all concurrent segment processor tasks.

The Fix

pinot-0001: Hoist HashSet<String> from sortOrder before the FieldSpec iteration loop:

// Before
for (FieldSpec fieldSpec : schema.getAllFieldSpecs()) {
    if (sortOrder.contains(fieldSpec.getName())) { ... }
}

// After
// CWE-407 fix: HashSet built once before loop for O(1) contains() instead of O(S) List scan.
Set<String> sortSet = new HashSet<>(sortOrder);
for (FieldSpec fieldSpec : schema.getAllFieldSpecs()) {
    if (sortSet.contains(fieldSpec.getName())) { ... }
}

String has standard hashCode() — no additional changes needed. The set is local to getFieldSpecs() and not retained after the call returns.

Patch

Fix available: defects/pinot/patch/pinot-0001-getfieldspecs-hashset.patch

Single-location, two-line change in SegmentProcessorUtils.java. No behavioral change — HashSet.contains() is semantically identical to List.contains() for String equality.

What We Ask

1. Confirm receipt and assign a GitHub issue reference (apache/pinot).
2. Validate the patch against the segment processor test suite and ingestion pipeline integration tests.
3. Coordinate a disclosure date — we are targeting 90 days from first contact.
4. We will credit the Apache Pinot team in the public disclosure. Preferred acknowledgment format welcome.

Contact: security@undefect.com. This brief is confidential until coordinated disclosure.

---

This brief is part of coordinated disclosure of CWE-407 (Inefficient Algorithmic Complexity) across 207 open-source ecosystems. Full report: https://undefect.com