wave13: pyroscope-0001 UNDF-1301 (47x-438x GetBlockStats) + 5 clean-scan additions
Flagship: pyroscope PhlareDB.GetBlockStats slices.Contains per block (O(B*U) -> O(B+U)). Long-retention tenants with 5k-10k blocks pay 1M+ membership checks per block-stats RPC. Set hoist: 438x at B=10k U=1k. Wave 13 honor roll: lima, apollo-server, act, firecracker, nix. Cumulative: 59 projects.
This commit is contained in:
parent
1c9cb86381
commit
802e51430c
7 changed files with 359 additions and 0 deletions
55
docs/tickets/pyroscope-0001-getblockstats-ulids-set.md
Normal file
55
docs/tickets/pyroscope-0001-getblockstats-ulids-set.md
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
# pyroscope-0001: PhlareDB.GetBlockStats — O(B×U) ULID lookup per block
|
||||
|
||||
**Target:** grafana/pyroscope
|
||||
**Severity:** HIGH
|
||||
**CWE:** CWE-407 (Inefficient Algorithmic Complexity)
|
||||
**MOAD:** MOAD-0001 (A Sedimentary Defect)
|
||||
**File:** `pkg/phlaredb/phlaredb.go:597-613`
|
||||
**Language:** Go
|
||||
**Status:** open
|
||||
|
||||
## Description
|
||||
|
||||
`PhlareDB.GetBlockStats` walks three block sets (`heads`, `flushing`, `queriers`) and for each block calls `slices.Contains(req.Msg.GetUlids(), h.meta.ULID.String())`. `slices.Contains` is an O(U) linear scan over the requested ULID list. Per-request cost: **O(B × U)** where B = total blocks and U = requested ULID count.
|
||||
|
||||
Pyroscope tenants storing weeks of continuous profiles accumulate thousands of block queriers. Operators issuing block-stats queries with hundreds of ULIDs pay 1M+ membership checks per call. ULID.String() also re-formats per iteration, multiplying allocations.
|
||||
|
||||
## Root Cause
|
||||
|
||||
```go
|
||||
// pkg/phlaredb/phlaredb.go:597
|
||||
for _, h := range f.heads {
|
||||
if slices.Contains(req.Msg.GetUlids(), h.meta.ULID.String()) { // O(U) per call
|
||||
res.BlockStats = append(res.BlockStats, h.GetMetaStats().ConvertToBlockStats())
|
||||
}
|
||||
}
|
||||
for _, h := range f.flushing { ... } // same pattern
|
||||
for _, q := range f.blockQuerier.queriers { ... } // same pattern
|
||||
```
|
||||
|
||||
`slices.Contains` is a linear scan. Across B blocks total: O(B × U).
|
||||
|
||||
## Fix
|
||||
|
||||
Hoist `req.Msg.GetUlids()` into a `map[string]struct{}{}` once before the loops. Per-iter cost drops to O(1). Total cost: O(B + U).
|
||||
|
||||
```go
|
||||
requested := make(map[string]struct{}, len(req.Msg.GetUlids()))
|
||||
for _, u := range req.Msg.GetUlids() {
|
||||
requested[u] = struct{}{}
|
||||
}
|
||||
for _, h := range f.heads {
|
||||
if _, ok := requested[h.meta.ULID.String()]; ok {
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Severity Note
|
||||
|
||||
Hot path on every block-stats RPC. Long-retention tenants (Grafana Cloud Profiles, fleet-wide continuous profiling) routinely have 5k-10k blocks. Bench (defects/pyroscope/bench/) shows 47× speedup at B=500 U=100 and 438× at B=10k U=1k.
|
||||
|
||||
## Complexity Gate
|
||||
|
||||
- B=10,000 blocks × U=1000 ULIDs: fixed must complete in <2ms
|
||||
- k-scaling 5×: time ratio must be <17.5×
|
||||
Loading…
Add table
Add a link
Reference in a new issue