Ray — CWE-407 Disclosure Brief

2026-03-27 · Patch available — awaiting upstream merge

Finding

One O(n²) defect in Ray's local node provider cluster reconciliation. ClusterState and OnPremCoordinatorState build a list_of_node_ips from a list, then scan it with if worker_ip not in list_of_node_ips in a loop — O(N²) cluster reconciliation. Patch ready for upstream review.

The Defects

ray-0001 (PATCHED — HIGH): python/ray/autoscaler/_private/local/node_provider.py:79-83,147-149

# ClusterState and OnPremCoordinatorState:
list_of_node_ips = list(worker_ips)  # builds a list

for worker_ip in workers:
    if worker_ip not in list_of_node_ips:  # O(N) list scan per worker
        ...
# O(N²) cluster reconciliation

if worker_ip not in list_of_node_ips performs O(N) list scan for each of N workers. O(N²) total cluster reconciliation. Measured ratio: 300×.

Complexity Proof

For N=300 worker nodes:

Per reconciliation: N iterations × O(N) scan = O(N²)
Fixed: set(worker_ips) → O(N)
At N=300: 90,000 comparisons vs 300 set lookups
300× measured ratio.

Impact

All Ray autoscaler deployments using the local/on-premise node provider. Cluster reconciliation runs continuously during autoscaling — scaling up (new workers) and scaling down (removing idle workers). Large Ray clusters with many worker nodes hit worst case on every reconciliation cycle. Ray is a widely used distributed Python framework for ML training, hyperparameter tuning, and data processing (used with PyTorch, HuggingFace, and similar).

The Fix

Replace list(worker_ips) with set(worker_ips):

# Before
list_of_node_ips = list(worker_ips)
if worker_ip not in list_of_node_ips:  # O(N) scan

# After
# CWE-407 fix: set for O(1) membership instead of O(N) list scan.
set_of_node_ips = set(worker_ips)
if worker_ip not in set_of_node_ips:  # O(1) set lookup

Patch

defects/ray/patch/ray-0001-node-provider-set.patch

What We Ask

Confirm receipt and assign a GitHub Security Advisory or issue reference.
Validate the patch against your autoscaler and node provider test suite.
Assess CVE eligibility — 300× overhead on every cluster reconciliation cycle.
Coordinate a disclosure date — we are targeting 90 days from first contact.

Contact: see cover email. This brief is confidential until coordinated disclosure.

2.4 KiB Raw Blame History Unescape Escape