wave16: ghost-0001 UNDF-1302 (15x-184x ReferrersStats) + 6 clean-scan additions
Flagship: Ghost ReferrersStatsService.getReferrersHistory Array.find with multi-key predicate per paid conversion (O(P*A) -> O(P+A)). Long-running Ghost sites with 200+ referrers x year of dates hit 4 second dashboard loads; Map<source|date,entry> hoist gives 184x speedup at A=110k P=1k. Wave 16 honor roll: fastify, samtools, argo-workflows, cypress, bitcoin, strapi. Cumulative: 80 projects.
This commit is contained in:
parent
09b320386b
commit
878876c7af
7 changed files with 433 additions and 0 deletions
76
docs/tickets/ghost-0001-referrers-history-source-date-map.md
Normal file
76
docs/tickets/ghost-0001-referrers-history-source-date-map.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# ghost-0001: ReferrersStatsService — O(P×A) Array.find merging paid conversions
|
||||
|
||||
**Target:** TryGhost/Ghost
|
||||
**Severity:** HIGH
|
||||
**CWE:** CWE-407 (Inefficient Algorithmic Complexity)
|
||||
**MOAD:** MOAD-0001 (A Sedimentary Defect)
|
||||
**File:** `ghost/core/core/server/services/stats/referrers-stats-service.js:147-160`
|
||||
**Language:** JavaScript
|
||||
**Status:** open
|
||||
|
||||
## Description
|
||||
|
||||
`ReferrersStatsService.getReferrersHistory()` builds the analytics dashboard's referrer history by merging paid-conversion events into a base list of signup events keyed by `(source, date)`. The merge does:
|
||||
|
||||
```js
|
||||
paidConversionEntries.forEach(entry => {
|
||||
const existing = allEntries.find(e =>
|
||||
e.source === entry.source && e.date === entryDate
|
||||
); // O(A) linear scan per conversion
|
||||
...
|
||||
});
|
||||
```
|
||||
|
||||
`Array.find` with a multi-key predicate is an O(A) linear scan. Total cost: **O(P × A)** where P = paid-conversion count and A = total entries (sources × date range).
|
||||
|
||||
For long-running Ghost sites with 200+ referral sources tracked over a year of dates, A reaches 70k-100k entries. Hundreds of paid conversions per dashboard refresh produce 7M+ comparisons per page load.
|
||||
|
||||
## Root Cause
|
||||
|
||||
```js
|
||||
// referrers-stats-service.js:147
|
||||
paidConversionEntries.forEach((entry) => {
|
||||
const entryDate = moment(entry.date).format('YYYY-MM-DD');
|
||||
const existingEntry = allEntries.find(e => e.source === entry.source && e.date === entryDate);
|
||||
if (existingEntry) {
|
||||
existingEntry.paid_conversions = entry.paid_conversions;
|
||||
} else {
|
||||
allEntries.push({...entry, signups: 0, date: entryDate});
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
`Array.find` walks `allEntries` from index 0 each iteration — O(A) per call. Across P paid conversions: O(P × A).
|
||||
|
||||
## Fix
|
||||
|
||||
Build a `Map<"source|date", entry>` lookup once before the merge loop. Per-conversion lookup drops from O(A) to O(1). Total cost: O(P + A).
|
||||
|
||||
```js
|
||||
const allEntriesByKey = new Map();
|
||||
for (const e of allEntries) {
|
||||
allEntriesByKey.set(`${e.source}|${e.date}`, e);
|
||||
}
|
||||
paidConversionEntries.forEach((entry) => {
|
||||
const entryDate = moment(entry.date).format('YYYY-MM-DD');
|
||||
const existingEntry = allEntriesByKey.get(`${entry.source}|${entryDate}`);
|
||||
if (existingEntry) {
|
||||
existingEntry.paid_conversions = entry.paid_conversions;
|
||||
} else {
|
||||
const newEntry = {...entry, signups: 0, date: entryDate};
|
||||
allEntries.push(newEntry);
|
||||
allEntriesByKey.set(`${entry.source}|${entryDate}`, newEntry);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
The Map insertion path also caches new entries so subsequent matches against newly pushed items remain O(1).
|
||||
|
||||
## Severity Note
|
||||
|
||||
Hot path on every Ghost analytics dashboard load for any site that uses paid memberships and tracks referrer history. Bench (defects/ghost/bench/) shows 15× speedup at A=1.5k P=100 and 184× at A=110k P=1k.
|
||||
|
||||
## Complexity Gate
|
||||
|
||||
- A=10,000 entries × P=200 conversions: fixed must complete in <1ms
|
||||
- k-scaling 5×: time ratio must be <17.5×
|
||||
Loading…
Add table
Add a link
Reference in a new issue