moad-sweep: 9 initial findings — MOAD-0005 (Hungry Regex) x1, MOAD-0006 (Thundering Herd) x8

MOAD-0005 candidates (ReDoS):
- bleach-MOADX-0001: sanitize_css O(2^N) — 71-char input causes 12s hang HIGH

MOAD-0006 candidates (Thundering Herd / cache stampede):
- hibernate-orm-MOADX-0001: QueryInterpretationCacheStandardImpl HQL plan cache HIGH
- elasticsearch-MOADX-0001: EnrichCache "intentionally non-locking" enrich search HIGH
- hadoop-MOADX-0001: FederationJCache/FederationCaffeineCache YARN default config HIGH
- celery-MOADX-0001: BaseBackend.get_task_meta chord fan-in Redis stampede HIGH
- traefik-MOADX-0001: CNAMEFlatten data race + N*30s DNS stampede HIGH
- kafka-MOADX-0001: CachedConnectors.lookup classloader scan MEDIUM
- spring-MOADX-0001: AbstractFallbackCacheOperationSource @Cacheable cold start MEDIUM
- django-MOADX-0001: cached.Loader.get_template template compile MEDIUM

CLEAN: Consul (singleflight), Kubernetes (RWMutex), Prometheus (per-scraper),
Pulsar (AsyncLoadingCache), Cassandra (LoadingCache), Flink (CAS loop),
ZooKeeper (synchronized), Airflow (hand-rolled singleflight)
This commit is contained in:
russell@unturf.com 2026-03-29 21:09:07 -04:00
parent 1f48d1b89c
commit 9325c98470
9 changed files with 1111 additions and 0 deletions

View file

@ -0,0 +1,144 @@
# UNDF: (pending)
# django-MOADX-0001: cached.Loader.get_template — stampede O(N) concurrent template compilations
## MOAD-0006 Candidate — The Thundering Herd
| Field | Value |
|-------|-------|
| ID | django-MOADX-0001 |
| Severity | MEDIUM |
| Ecosystem | Django |
| Language | Python |
| File | `django/template/loaders/cached.py` |
| Lines | 28–65 |
| Pattern | dict.get → None check → expensive compile → dict[key]=template (no lock) |
| Trigger | N concurrent WSGI threads all request the same template during startup / cache clear |
| Compute cost | File system read + template lexing + compilation to AST nodes |
## Defect
Django's `cached.Loader` wraps other template loaders and caches compiled `Template` objects in `self.get_template_cache` — a plain Python `dict`. The `Loader` instance is created once per engine alias (`EngineHandler.__getitem__`) and shared across all WSGI worker threads in the process.
```python
# django/template/loaders/cached.py:28-65
def get_template(self, template_name, skip=None):
key = self.cache_key(template_name, skip)
cached = self.get_template_cache.get(key) # <-- dict lookup, no lock
if cached:
# ... return or raise from cache
return cached
# NO LOCK AROUND THIS BLOCK:
try:
template = super().get_template(template_name, skip) # <-- file I/O + compile
except TemplateDoesNotExist as e:
self.get_template_cache[key] = (
copy_exception(e) if self.engine.debug else TemplateDoesNotExist
)
raise
else:
self.get_template_cache[key] = template # <-- dict write
return template
```
`super().get_template()` walks each configured loader, opens the template file from disk, and compiles the template string through Django's lexer and parser into a `Template` object tree. This is orders of magnitude more expensive than a cached dict lookup.
**CPython GIL consideration:** Individual `dict.get` and `dict[key] = value` operations are atomic under CPython's GIL. However, the three-step sequence (check → file load + compile → write) is not atomic. Threads can and do interleave between the miss check and the subsequent write:
- Thread A: `cache.get(key)` → `None`
- Thread B: `cache.get(key)` → `None` (before A has written)
- Thread A: `super().get_template(...)` → opens file, parses template
- Thread B: `super().get_template(...)` → opens file, parses template (duplicate work)
- Thread A: `cache[key] = template`
- Thread B: `cache[key] = template` (redundant write)
**Grower signal:** Harmless at low concurrency. Triggers on:
1. Application startup: all templates cold-cached, first request burst loads all at once.
2. `loader.reset()` called during configuration reload: cache cleared, all threads stampede on first access.
3. Templates with `skip` variants generate distinct cache keys, multiplying stampede surface.
For a Django app with 50 templates and 32 WSGI workers, a `loader.reset()` followed by a traffic burst can produce 1600 redundant file reads and parses (32 workers × 50 templates).
## Fix
**Option A — `threading.Lock` per template key (fine-grained):**
```python
import threading
class Loader(BaseLoader):
def __init__(self, engine, loaders):
self.get_template_cache = {}
self._cache_locks = {}
self._meta_lock = threading.Lock()
self.loaders = engine.get_template_loaders(loaders)
super().__init__(engine)
def _get_or_create_key_lock(self, key):
with self._meta_lock:
if key not in self._cache_locks:
self._cache_locks[key] = threading.Lock()
return self._cache_locks[key]
def get_template(self, template_name, skip=None):
key = self.cache_key(template_name, skip)
cached = self.get_template_cache.get(key)
if cached:
if isinstance(cached, type) and issubclass(cached, TemplateDoesNotExist):
raise cached(template_name)
elif isinstance(cached, TemplateDoesNotExist):
raise copy_exception(cached)
return cached
key_lock = self._get_or_create_key_lock(key)
with key_lock:
# Double-check after acquiring lock
cached = self.get_template_cache.get(key)
if cached:
if isinstance(cached, type) and issubclass(cached, TemplateDoesNotExist):
raise cached(template_name)
elif isinstance(cached, TemplateDoesNotExist):
raise copy_exception(cached)
return cached
try:
template = super().get_template(template_name, skip)
except TemplateDoesNotExist as e:
self.get_template_cache[key] = (
copy_exception(e) if self.engine.debug else TemplateDoesNotExist
)
raise
else:
self.get_template_cache[key] = template
return template
```
**Option B — coarse `threading.RLock` (simpler, acceptable given templates are rarely loaded):**
```python
self._lock = threading.RLock()
def get_template(self, template_name, skip=None):
key = self.cache_key(template_name, skip)
# Fast path without lock
cached = self.get_template_cache.get(key)
if cached:
...
with self._lock:
cached = self.get_template_cache.get(key)
if cached:
...
# compile under lock — acceptable since templates load once per key
...
```
**Option C — `functools.cached_property` pattern or `threading.local` per-thread cache:** Less correct since it defeats the purpose of a shared cache.
Option A is preferred: per-key locking means different templates compile concurrently, only the same template key serializes.
## Impact
- **Redundant work:** N threads × file read + template parse per cache miss burst.
- **Resource pressure:** Multiple simultaneous file handles to the same template file; parser AST allocations multiplied by N.
- **Fix:** After fix, first miss for each key compiles once; subsequent requests wait and receive the cached result. Net: 1 compile per unique (template_name, skip) key regardless of concurrency.