wave16: ghost-0001 UNDF-1302 (15x-184x ReferrersStats) + 6 clean-scan additions

Flagship: Ghost ReferrersStatsService.getReferrersHistory Array.find with
multi-key predicate per paid conversion (O(P*A) -> O(P+A)). Long-running
Ghost sites with 200+ referrers x year of dates hit 4 second dashboard
loads; Map<source|date,entry> hoist gives 184x speedup at A=110k P=1k.

Wave 16 honor roll: fastify, samtools, argo-workflows, cypress, bitcoin,
strapi. Cumulative: 80 projects.
This commit is contained in:
russell@unturf.com 2026-04-25 15:01:06 -04:00
parent 09b320386b
commit 878876c7af
No known key found for this signature in database
7 changed files with 433 additions and 0 deletions

View file

@ -0,0 +1,76 @@
# ghost-0001: ReferrersStatsService — O(P×A) Array.find merging paid conversions
**Target:** TryGhost/Ghost
**Severity:** HIGH
**CWE:** CWE-407 (Inefficient Algorithmic Complexity)
**MOAD:** MOAD-0001 (A Sedimentary Defect)
**File:** `ghost/core/core/server/services/stats/referrers-stats-service.js:147-160`
**Language:** JavaScript
**Status:** open
## Description
`ReferrersStatsService.getReferrersHistory()` builds the analytics dashboard's referrer history by merging paid-conversion events into a base list of signup events keyed by `(source, date)`. The merge does:
```js
paidConversionEntries.forEach(entry => {
const existing = allEntries.find(e =>
e.source === entry.source && e.date === entryDate
); // O(A) linear scan per conversion
...
});
```
`Array.find` with a multi-key predicate is an O(A) linear scan. Total cost: **O(P × A)** where P = paid-conversion count and A = total entries (sources × date range).
For long-running Ghost sites with 200+ referral sources tracked over a year of dates, A reaches 70k-100k entries. Hundreds of paid conversions per dashboard refresh produce 7M+ comparisons per page load.
## Root Cause
```js
// referrers-stats-service.js:147
paidConversionEntries.forEach((entry) => {
const entryDate = moment(entry.date).format('YYYY-MM-DD');
const existingEntry = allEntries.find(e => e.source === entry.source && e.date === entryDate);
if (existingEntry) {
existingEntry.paid_conversions = entry.paid_conversions;
} else {
allEntries.push({...entry, signups: 0, date: entryDate});
}
});
```
`Array.find` walks `allEntries` from index 0 each iteration — O(A) per call. Across P paid conversions: O(P × A).
## Fix
Build a `Map<"source|date", entry>` lookup once before the merge loop. Per-conversion lookup drops from O(A) to O(1). Total cost: O(P + A).
```js
const allEntriesByKey = new Map();
for (const e of allEntries) {
allEntriesByKey.set(`${e.source}|${e.date}`, e);
}
paidConversionEntries.forEach((entry) => {
const entryDate = moment(entry.date).format('YYYY-MM-DD');
const existingEntry = allEntriesByKey.get(`${entry.source}|${entryDate}`);
if (existingEntry) {
existingEntry.paid_conversions = entry.paid_conversions;
} else {
const newEntry = {...entry, signups: 0, date: entryDate};
allEntries.push(newEntry);
allEntriesByKey.set(`${entry.source}|${entryDate}`, newEntry);
}
});
```
The Map insertion path also caches new entries so subsequent matches against newly pushed items remain O(1).
## Severity Note
Hot path on every Ghost analytics dashboard load for any site that uses paid memberships and tracks referrer history. Bench (defects/ghost/bench/) shows 15× speedup at A=1.5k P=100 and 184× at A=110k P=1k.
## Complexity Gate
- A=10,000 entries × P=200 conversions: fixed must complete in <1ms
- k-scaling 5×: time ratio must be <17.5×