wave6: knex-0001 flagship + 38-target docgen/webfw/migration scan survey
knex-0001: Migrator#rollback({all:true}) and Migrator#down() filter
allMigrations against completed via .map(name).includes() inside the
filter callback. Per-iter array allocation + linear scan = O(A*C)
real, O(A*C^2) amortized including GC. Fix: hoist Set<name> once,
Set#has = O(1). Bench: 355x at A=C=2000 migrations.
wave6-docgen-webfw-tui-survey.md: 38-target scan covering doc gens
(Sphinx, JSDoc, TypeDoc, Doxygen, MkDocs, Hugo, Jekyll, Gatsby,
Eleventy, Astro), web frameworks (Fastify, Express, Koa, hapi,
SvelteKit, Nuxt, Remix), TUI/CLI (Cobra, Click, Commander.js, Yargs,
Bubble Tea, Ratatui), migrations (Flyway, Goose, dbmate, Knex,
Sqitch, Atlas), search engines (Tantivy, MeiliSearch, Typesense),
API gateways (Kong, APISIX), MQTT/queue brokers (Mosquitto, EMQX,
VerneMQ, ZeroMQ).
Clean-scan honor roll +3: Bubble Tea, dbmate, libzmq.
This commit is contained in:
parent
33cc466b3a
commit
cd28454d2d
9 changed files with 366 additions and 1 deletions
73
docs/tickets/knex-0001-migrator-completed-name-set.md
Normal file
73
docs/tickets/knex-0001-migrator-completed-name-set.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
# knex-0001: Migrator rollback/down — O(A×C²) completed-name lookup
|
||||
|
||||
**Target:** knex/knex
|
||||
**Severity:** MEDIUM-HIGH
|
||||
**CWE:** CWE-407 (Inefficient Algorithmic Complexity)
|
||||
**MOAD:** MOAD-0001 (A Sedimentary Defect)
|
||||
**File:** `lib/migrations/migrate/Migrator.js:188-194, 217-222`
|
||||
**Language:** JavaScript
|
||||
**Status:** open
|
||||
|
||||
## Description
|
||||
|
||||
`Migrator#rollback({all: true})` and `Migrator#down()` both filter the full migration list against the list of completed migrations. Inside each per-migration filter callback they rebuild the completed-names array via `.map(...)` and scan it via `.includes(...)`:
|
||||
|
||||
```js
|
||||
// rollback (line 186-195)
|
||||
allMigrations
|
||||
.filter((migration) => {
|
||||
return completedMigrations
|
||||
.map((migration) => migration.name) // O(C) — new array per filter step
|
||||
.includes(this.config.migrationSource.getMigrationName(migration)); // O(C) scan
|
||||
})
|
||||
.reverse();
|
||||
|
||||
// down (line 217-222) — same shape
|
||||
const completedMigrations = all.filter((migration) => {
|
||||
return completed
|
||||
.map((migration) => migration.name)
|
||||
.includes(this.config.migrationSource.getMigrationName(migration));
|
||||
});
|
||||
```
|
||||
|
||||
For A all-migrations and C completed-migrations, per-call cost is O(A × 2C) = **O(A×C²)** when you count the wasted .map allocation per filter iteration. Mature projects with hundreds of migrations pay this on every `knex migrate:rollback --all` and `knex migrate:down`.
|
||||
|
||||
## Root Cause
|
||||
|
||||
Two waste sources:
|
||||
|
||||
1. The `.map((migration) => migration.name)` runs **inside every filter iteration**, allocating a fresh array of names and triggering GC pressure.
|
||||
2. The `.includes(...)` is an O(C) linear scan on that fresh array.
|
||||
|
||||
Combined: per-filter cost is O(C) compute + O(C) allocation. Across A filter iterations: **O(A·C)** real cost, **O(A·C²)** amortized when you count allocation.
|
||||
|
||||
## Fix
|
||||
|
||||
Hoist the name set out of the filter and use a `Set<string>` for O(1) lookup:
|
||||
|
||||
```js
|
||||
// rollback all branch
|
||||
const completedNameSet = new Set(completedMigrations.map((m) => m.name));
|
||||
return allMigrations
|
||||
.filter((migration) =>
|
||||
completedNameSet.has(this.config.migrationSource.getMigrationName(migration))
|
||||
)
|
||||
.reverse();
|
||||
|
||||
// down — same treatment
|
||||
const completedNameSet = new Set(completed.map((m) => m.name));
|
||||
const completedMigrationsList = all.filter((migration) =>
|
||||
completedNameSet.has(this.config.migrationSource.getMigrationName(migration))
|
||||
);
|
||||
```
|
||||
|
||||
Builds the Set once. `Set#has` is O(1). Total cost drops to O(A + C).
|
||||
|
||||
## Severity Note
|
||||
|
||||
Knex's migration runner is a CI/CD critical path. Mature databases (Rails-style projects ported to Node, monorepos with many service schemas) carry hundreds of migrations. Each `migrate:rollback --all` and `migrate:down` call hits this. Per-developer overhead and per-deployment overhead compound.
|
||||
|
||||
## Complexity Gate
|
||||
|
||||
- A=C=500: fixed must complete in <5ms
|
||||
- k-scaling 5×: time ratio must be <17.5×
|
||||
Loading…
Add table
Add a link
Reference in a new issue