Outreach docs (unblock intel page generation): - kdenlive: 10 defects (8 CWE-407 + 1 CWE-362 + 1 keyframe), C++ - libreoffice: 5 defects (Writer, Calc, SFX, Impress), C++ - maven: 7 defects (graph, lifecycle, sort-by-indexOf), Java - cpython: 7 defects (pkgutil, codegen, mock, pmerge MRO, pydoc), C/Python - blender: 4 defects (node runtime, USD skel, shader, anim), C++ Mastodon CWE-1333 benchmark: - test_mastodon_cwe1333.rb: validates (.+\.)? -> ([^@]+\.)? fix eliminates O(2^N) backtracking in email validator
145 lines
6.8 KiB
Markdown
145 lines
6.8 KiB
Markdown
# CPython — CWE-407 Disclosure Brief
|
|
**2026-04-13 · Patches available — awaiting upstream merge**
|
|
|
|
## Finding
|
|
|
|
Six algorithmic complexity defects in CPython across the package utility module, pattern-match compiler, C3 MRO linearization, unittest.mock, pydoc, turtle, and IDLE RPC. Two defects are O(N^2) linear-scan patterns; four are O(2^D) exponential diamond-inheritance traversals. All patched or documented with fixes.
|
|
|
|
## The Defects
|
|
|
|
**cpython-pkgutil (PATCHED — MEDIUM):** `Lib/pkgutil.py — extend_path()`
|
|
|
|
```python
|
|
# O(N^2) membership check — 'portion not in path' is O(N) on a list
|
|
for portion in portions:
|
|
if portion not in path: # O(N) linear scan per portion
|
|
path.append(portion)
|
|
```
|
|
|
|
`extend_path()` builds a namespace package path by checking `portion not in path` (O(N) list scan) per portion. Total: O(N^2) for N portions. Fix: shadow `set(path)` for O(1) membership.
|
|
|
|
**cpython-0001 (PATCHED — MEDIUM):** `Python/codegen.c — codegen_pattern_helper_store_name()`
|
|
|
|
```c
|
|
// O(S^2) duplicate check — PySequence_Contains on a PyList
|
|
int duplicate = PySequence_Contains(pc->stores, n); // O(S) per call
|
|
```
|
|
|
|
Pattern-match compiler checks for duplicate store names using `PySequence_Contains()` on `pc->stores` (a `PyList`). Called once per capture variable, giving O(S^2) total. At S=100 capture variables: ~5,000 comparisons. Fix: parallel `PySet` for O(1) membership. Also affects the mapping-pattern dedup loop.
|
|
|
|
**cpython-0001-mock (DOCUMENTED — MEDIUM):** `Lib/unittest/mock.py — reset_mock()`
|
|
|
|
```python
|
|
# O(N^2) cycle detection — 'id(self) in visited' is O(N) on a list
|
|
def reset_mock(self, visited=None, ...):
|
|
if visited is None:
|
|
visited = [] # list, not set
|
|
if id(self) in visited: # O(N) linear scan
|
|
return
|
|
visited.append(id(self))
|
|
```
|
|
|
|
`reset_mock` uses a `list` for cycle detection when traversing mock object trees. At N=1,000 mock nodes: ~500,000 comparisons. Fix: replace `list` with `set` for O(1) membership. **500x speedup at N=1,000.**
|
|
|
|
**cpython-0002 (PATCHED — LOW-MEDIUM):** `Objects/typeobject.c — pmerge()`
|
|
|
|
```c
|
|
// O(M^2 * K) C3 MRO linearization — tail_contains is O(M) per check
|
|
// Called M times in outer loop, K times in inner loop per merge list
|
|
```
|
|
|
|
The C3 MRO `pmerge()` function calls `tail_contains()` which performs O(M) linear scans across merge list tails. Total: O(M^2 \* K) where M = MRO length, K = number of direct bases. Fix: build a `PySet` of all tail-position classes, update as elements are consumed. Reduces each check from O(M) to O(1).
|
|
|
|
**cpython-0002-pydoc (DOCUMENTED — MEDIUM):** `Lib/pydoc.py — allmethods()`
|
|
|
|
```python
|
|
# O(2^D) diamond base traversal — no visited guard
|
|
def allmethods(cl):
|
|
for base in cl.__bases__:
|
|
methods.update(allmethods(base)) # unconditional recursion
|
|
```
|
|
|
|
With a diamond hierarchy at depth D, traversal visits 2^D nodes instead of the O(D) unique classes. Fix: add `_visited` set parameter, or use `inspect.getmro()` which already linearizes. **2,048x speedup at D=15.**
|
|
|
|
**cpython-0003 (DOCUMENTED — MEDIUM):** `Lib/turtle.py — __methodDict()`
|
|
|
|
```python
|
|
# O(2^D) diamond base traversal — fires at module import time
|
|
def __methodDict(cls, _dict):
|
|
for _super in baseList:
|
|
__methodDict(_super, _dict) # unconditional recursion
|
|
```
|
|
|
|
Called unconditionally at `import turtle` time via `__forwardmethods(ScrolledCanvas, TK.Canvas, '_canvas')`. Same O(2^D) diamond traversal pattern. Fix: add `_visited` set, or iterate `cls.__mro__` directly. **2,048x speedup at D=15.**
|
|
|
|
**cpython-0004 (DOCUMENTED — MEDIUM):** `Lib/idlelib/rpc.py — _getmethods()`
|
|
|
|
```python
|
|
# O(2^D) diamond base traversal — fires on every IDLE debug inspect
|
|
def _getmethods(obj, methods):
|
|
if isinstance(obj, type):
|
|
for super in obj.__bases__:
|
|
_getmethods(super, methods) # unconditional recursion
|
|
```
|
|
|
|
Triggered when IDLE debugger inspects a remote object. `dir(obj)` is called on each visit, making actual cost O(2^D \* M) where M = methods per class. Fix: add `_visited` set. **2,048x speedup at D=15.**
|
|
|
|
## Complexity Proof
|
|
|
|
**cpython-0001 (codegen):** At S=100 capture variables:
|
|
- Defective: ~5,000 `PySequence_Contains` calls
|
|
- Fixed: ~100 `PySet_Contains` calls
|
|
- **50x op reduction**
|
|
|
|
**cpython-0001-mock:** At N=1,000 mock nodes:
|
|
- Defective: 500,500 list membership checks
|
|
- Fixed: 1,000 set lookups
|
|
- **500x op reduction**
|
|
|
|
**cpython-0002 (pmerge):** At M=100 MRO entries, K=10 bases:
|
|
- Defective: ~100,000 pointer comparisons in `tail_contains`
|
|
- Fixed: ~1,000 set lookups
|
|
- **100x op reduction**
|
|
|
|
**cpython-0002/0003/0004 (diamond traversals):** At depth D=15:
|
|
- Defective: 32,767 node visits
|
|
- Fixed: 16 node visits
|
|
- **2,048x op reduction**
|
|
|
|
## Impact
|
|
|
|
CPython is the reference implementation of Python, running the vast majority of Python code worldwide. cpython-0001 affects the pattern-match compiler (PEP 634), which handles `match`/`case` statements in Python 3.10+. cpython-0001-mock affects every test suite that uses `unittest.mock` with complex mock trees. cpython-0002 affects C3 MRO computation, which fires on every class definition with multiple inheritance. The diamond-traversal defects (cpython-0002-pydoc, cpython-0003, cpython-0004) affect pydoc, turtle import, and IDLE debugging respectively.
|
|
|
|
## The Fix
|
|
|
|
**cpython-pkgutil:** Add `path_set = set(path)` before the loop, check `portion not in path_set`.
|
|
|
|
**cpython-0001:** Add `pc->stores_set` (`PySet`) alongside `pc->stores` (`PyList`); use `PySet_Contains()` for duplicate checks, `PySet_Add()` on insert.
|
|
|
|
**cpython-0001-mock:** Replace `visited = []` with `visited = set()` in `reset_mock()`.
|
|
|
|
**cpython-0002:** Build `tail_set = PySet_New(NULL)` of all tail-position classes in `pmerge()`; use `PySet_Contains()` instead of `tail_contains()` linear scan.
|
|
|
|
**cpython-0002-pydoc/0003/0004:** Add `_visited=None` parameter with `set()` guard at entry, or replace recursive `__bases__` traversal with `__mro__` iteration.
|
|
|
|
## Patch
|
|
|
|
Patches and documentation in `defects/cpython/patch/`:
|
|
- `0001-pkgutil-extend-path-set-dedup.patch`
|
|
- `cpython-0001-codegen-pattern-stores-list-contains.patch`
|
|
- `cpython-0001-mock-reset-visited-hashset.md`
|
|
- `cpython-0002-typeobject-pmerge-tail-contains.patch`
|
|
- `cpython-0002-pydoc-allmethods-diamond-bases.md`
|
|
- `cpython-0003-turtle-methoddict-diamond-bases.md`
|
|
- `cpython-0004-idlelib-rpc-getmethods-diamond-bases.md`
|
|
|
|
Language: C, Python
|
|
|
|
## What We Ask
|
|
|
|
1. Confirm receipt and assign a bugs.python.org reference (or GitHub issue on python/cpython).
|
|
2. Assess severity — cpython-0001 affects the pattern-match compiler; cpython-0002 affects MRO computation for all multiply-inherited classes.
|
|
3. Coordinate a disclosure date — we are targeting 90 days from first contact.
|
|
4. We will credit the CPython team in the public disclosure. Preferred acknowledgment format welcome.
|
|
|
|
Contact: see cover email. This brief is confidential until coordinated disclosure.
|