russell@unturf.com
a150602100
The Gatsby authors annotated each of the three call sites in in-memory/indexing.ts with 'expensive at scale' comments. Their diagnosis is correct: nodeTypeNames.includes(node.internal.type) inside iterateNodes().forEach is O(N*T) per cache build. For N=100k+ nodes typical of mature content sites and T=10-30 declared types per query, this fires on every type-filtered query. gatsby develop in particular rebuilds caches per page render. Fix: hoist Set<string> once at the top of each function. O(1) per node lookup. Total cost O(N+T). Bench shows 8.4x at N=100k T=50; 2.7-4.7x at smaller scales. Three call sites patched: ensureIndexByElemMatch (line 326), ensureEmptyFilterCache (378), ensureIndexByElemMatchValue (504). Author 'expensive at scale' comments updated to record the fix.
7 lines
396 B
Text
7 lines
396 B
Text
=== gatsby-0001: in-memory indexing nodeTypeNames.includes vs Set.has ===
|
|
N=1000 T=5 : defective=0.743ms fixed=0.278ms speedup=2.7x
|
|
N=10000 T=10 : defective=4.695ms fixed=1.714ms speedup=2.7x
|
|
N=50000 T=20 : defective=45.698ms fixed=9.671ms speedup=4.7x
|
|
N=100000 T=20 : defective=92.455ms fixed=22.264ms speedup=4.2x
|
|
N=100000 T=50 : defective=198.740ms fixed=23.568ms speedup=8.4x
|
|
|