wave7: 433/194 — kafka/flink/pulsar, spring/micronaut/quarkus, nginx/haproxy/traefik, linux/nomad/consul, numpy/pandas/sklearn, ES/OS/pg/sqlite/rustc/cargo

This commit is contained in:
russell@unturf.com 2026-03-27 16:20:58 -04:00
parent 3735145aa5
commit 5fe6da7cc2
69 changed files with 6793 additions and 32 deletions

View file

@ -0,0 +1,68 @@
# nomad-0001: Bitmap.IndexesInRangeFiltered — O(range × filter) port allocation
## CWE
CWE-407: Inefficient Algorithmic Complexity
## Severity
HIGH
## Location
`nomad/structs/bitmap.go:94`
Called from `nomad/structs/network.go:682` (`getDynamicPortsPrecise`)
## Description
`IndexesInRangeFiltered` iterates over every port in [minDynamicPort, maxDynamicPort]
(default 2000060000 → up to 40 000 iterations) and for each port calls
`slices.Contains(filter, int(i))` — a linear scan over `portsInOffer`.
```go
for i := from; i <= to && i < b.Size(); i++ {
c := b.Check(i)
if c == set {
if len(filter) < 1 || !slices.Contains(filter, int(i)) { // O(|filter|)
indexes = append(indexes, int(i))
}
}
}
```
This executes on every job placement (scheduler hot path). With P ports already
offered and a port range of R:
| Complexity | Observed |
|------------|---------|
| Slow (current) | O(R × P) |
| Fast (patched) | O(R + P) |
At R=40 000 and P=100 already-offered ports this is 4 000 000 operations vs 40 100.
## Fix
Convert `filter` to a `map[int]struct{}` (or Go 1.21 `sets.Set`) before the loop.
```go
func (b Bitmap) IndexesInRangeFiltered(set bool, from, to uint, filter []int) []int {
filterSet := make(map[int]struct{}, len(filter))
for _, f := range filter {
filterSet[f] = struct{}{}
}
var indexes []int
for i := from; i <= to && i < b.Size(); i++ {
if b.Check(i) == set {
if len(filterSet) == 0 {
indexes = append(indexes, int(i))
} else if _, skip := filterSet[int(i)]; !skip {
indexes = append(indexes, int(i))
}
}
}
return indexes
}
```
## Impact
Every Nomad job placement that requests dynamic ports calls this function.
High-density clusters scheduling many allocations simultaneously are most
affected. Cumulative O(n²) slowdown degrades scheduler throughput.
## Speedup
~100x at R=40 000, P=100 (measured in unit test).

View file

@ -0,0 +1,34 @@
# nomad-0002: stream/subscription filter() — O(events × namespaces)
## CWE
CWE-407: Inefficient Algorithmic Complexity
## Severity
MEDIUM
## Location
`nomad/stream/subscription.go:142`
## Description
The `filter()` function iterates over all incoming events and for each event
calls `slices.Contains(req.Namespaces, event.Namespace)` — a linear scan over
the namespace allowlist.
```go
for _, event := range events {
if event.Namespace != "" && !slices.Contains(req.Namespaces, event.Namespace) {
continue
}
...
}
```
With E events per batch and N subscribed namespaces: O(E × N).
## Fix
Build a `map[string]struct{}` from `req.Namespaces` once before the loop
(or store it pre-built on `SubscribeRequest`).
## Speedup
~Nx where N = number of subscribed namespaces; worst case in large multi-tenant
clusters with many namespace subscriptions.

View file

@ -0,0 +1,35 @@
# nomad-0003: GetVaultConfigurations secrets dedup — O(tasks × secrets²)
## CWE
CWE-407: Inefficient Algorithmic Complexity
## Severity
MEDIUM
## Location
`nomad/structs/structs.go:5098-5102`
## Description
Three nested loops — task groups → tasks → secrets — with an inner
`slices.Contains(secrets, s.Provider)` scan to deduplicate providers.
The `secrets` slice grows as providers are appended, so each check
scans an O(P) growing accumulator.
```go
for _, tg := range j.TaskGroups {
secrets := []string{}
for _, task := range tg.Tasks {
for _, s := range task.Secrets {
if !slices.Contains(secrets, s.Provider) { // O(|secrets|)
secrets = append(secrets, s.Provider)
}
}
}
}
```
## Fix
Replace `secrets []string` accumulator with `map[string]struct{}`.
## Speedup
~Px where P = number of distinct secret providers per task group.

View file

@ -0,0 +1,30 @@
# nomad-0004: checkstore.shim.Difference — O(current × ids)
## CWE
CWE-407: Inefficient Algorithmic Complexity
## Severity
MEDIUM
## Location
`client/serviceregistration/checks/checkstore/shim.go:152-155`
## Description
`Difference` iterates over all stored check IDs for an allocation and for
each calls `slices.Contains(ids, id)` — a linear scan over the input slice.
```go
for id := range s.current[allocID] {
if !slices.Contains(ids, id) { // O(|ids|)
remove = append(remove, id)
}
}
```
With C stored checks and I input IDs: O(C × I).
## Fix
Build a `map[structs.CheckID]struct{}` from `ids` before the loop.
## Speedup
~Ix speedup where I = len(ids).