wave13: 514/239 — flink/nifi/artemis + K8s/cilium/linkerd2 + ES/OpenSearch/Solr + hadoop/hbase/spark

This commit is contained in:
russell@unturf.com 2026-03-27 17:50:40 -04:00
parent 424a2a7787
commit 8f0bc73afa
32 changed files with 3977 additions and 5 deletions

View file

@ -0,0 +1,96 @@
# cilium-0003: CWE-407 — Quadratic IP address deduplication in node manager
## Severity: HIGH
## Repository
github.com/cilium/cilium
Commit: (depth-1 clone, branch main)
## File
`pkg/node/manager/manager.go`
## Defective Lines
```
965: func (m *manager) removeNodeFromIPCache(oldNode nodeTypes.Node, resource ipcacheTypes.ResourceID,
966: ipsetEntries, nodeIPsAdded, healthIPsAdded, ingressIPsAdded, podCIDRsAdded []netip.Prefix,
967: ) {
977: for _, address := range oldNode.IPAddresses { // O(A) addresses
978: prefix := ip.IPToNetPrefix(address.IP)
979: if slices.Contains(nodeIPsAdded, prefix) { // O(A) linear scan
980: continue
981: }
...
990: if address.Type == addressing.NodeInternalIP &&
991: !slices.Contains(ipsetEntries, oldPrefixCluster.AsPrefix()) { // O(A) scan
...
1041: for entry := range m.podCIDREntries(...) { // O(CIDR) entries
1041: if slices.Contains(podCIDRsAdded, entry.Prefix.AsPrefix()) { // O(CIDR) scan
...
1058: if !prefix.IsValid() || slices.Contains(healthIPsAdded, prefix) { // O(H) scan
...
1073: if !prefix.IsValid() || slices.Contains(ingressIPsAdded, prefix) { // O(I) scan
```
## Call Chain
```
NodeUpdated(oldNode, newNode) →
NodeUpdated() [line 883] →
removeNodeFromIPCache(oldNode, ..., nodeIPsAdded, ...) →
for _, address := range oldNode.IPAddresses {
slices.Contains(nodeIPsAdded, prefix) // O(A) per address
```
`nodeIPsAdded` is built just before this call by appending one prefix per address
in `n.IPAddresses` (line 771). The same addresses that were appended to the slice
are then scanned with `slices.Contains` once per address removal — O(A²) total.
Five separate slices (`nodeIPsAdded`, `ipsetEntries`, `healthIPsAdded`,
`ingressIPsAdded`, `podCIDRsAdded`) are each scanned linearly.
## Complexity
O(A²) per node update event where:
- A = number of IP addresses on a node
In dual-stack clusters with multiple CIDRs, nodes can carry 10-30 addresses.
In large multi-cluster setups (ClusterMesh), every remote node update triggers
this path. With N remote nodes and A addresses each, reconciliation is O(N × A²).
## Impact
Every `NodeUpdated` call (triggered by any node label/address change in k8s) runs
`removeNodeFromIPCache` which performs five O(A) linear scans per address in a loop.
In large ClusterMesh deployments (500+ nodes, dual-stack) this causes measurable
ipcache update latency and CPU overhead in the node manager goroutine.
## Fix
Convert each `[]netip.Prefix` slice to `map[netip.Prefix]struct{}` before the removal loop.
```go
// Before (defective):
for _, address := range oldNode.IPAddresses {
prefix := ip.IPToNetPrefix(address.IP)
if slices.Contains(nodeIPsAdded, prefix) { // O(A) scan per iteration
continue
}
...
}
// After (fixed): pre-index all five slice parameters
nodeIPsSet := make(map[netip.Prefix]struct{}, len(nodeIPsAdded))
for _, p := range nodeIPsAdded { nodeIPsSet[p] = struct{}{} }
ipsetSet := make(map[netip.Prefix]struct{}, len(ipsetEntries))
for _, p := range ipsetEntries { ipsetSet[p] = struct{}{} }
// ... same for healthIPsAdded, ingressIPsAdded, podCIDRsAdded
for _, address := range oldNode.IPAddresses {
prefix := ip.IPToNetPrefix(address.IP)
if _, ok := nodeIPsSet[prefix]; ok { // O(1)
continue
}
...
}
```
## References
- CWE-407: Inefficient Algorithmic Complexity
- `pkg/node/manager/manager.go` lines 965-1080
- `removeNodeFromIPCache()` called from `NodeUpdated()` line 883

View file

@ -0,0 +1,104 @@
# cilium-0004: CWE-407 — Quadratic predecessor deduplication in eBPF CFG construction
## Severity: MEDIUM
## Repository
github.com/cilium/cilium
Commit: (depth-1 clone, branch main)
## File
`pkg/bpf/analyze/blocks.go`
## Defective Lines
```
64: func addPredecessors(ins *asm.Instruction, preds ...*asm.Instruction) {
65: l := setLeader(ins)
66: for _, pred := range preds { // O(P) new predecessors
67: if pred == nil {
68: continue
69: }
70: if !slices.Contains(l.predecessors, pred) { // O(E) linear scan
71: l.predecessors = append(l.predecessors, pred)
72: }
73: }
74: }
```
## Call Chain
```
buildCFG(insns) → [blocks.go line ~700]
second pass: for i.Next() { // O(I) instructions
targets.resolve(i.Offset, tgt, tgtPrev) →
for _, branch := range target.branches { // O(B) branches per target
setBranchTarget(branch, tgt, tgtPrev) →
addPredecessors(tgt, branch, prev) →
for _, pred := range preds { // O(P) = 1..2
slices.Contains(l.predecessors, pred) // O(E) scan
```
## Complexity
O(I × B × E) where:
- I = number of eBPF instructions in the program
- B = number of branches targeting a particular instruction
- E = number of existing predecessors accumulated per instruction
`l.predecessors` grows as instructions with multiple in-edges are encountered.
For heavily-branched programs (computed gotos, loop headers), E grows and the
deduplication check becomes O(E²) per instruction.
## Impact
eBPF program analysis is performed at program load time via `NewBlocks()`. Large,
heavily-branched eBPF programs (e.g., policy enforcement programs with many
conditional checks) trigger O(E²) predecessor deduplication. Cilium's datapath
programs can reach tens of thousands of instructions with complex CFGs.
This also affects `Backtracker.previousBlock()` at line 544:
```
544: if slices.Contains(bt.visited, pred) { // O(V) growing visited list
547: bt.visited = append(bt.visited, pred)
```
Each `previousBlock()` call during backtracking scans the entire visited list —
O(V²) total over a full backtrack traversal of V blocks.
## Fix
Replace `[]*Block` / `[]*asm.Instruction` slice with `map[*T]struct{}` for dedup.
```go
// Before (defective) — l.predecessors is []*asm.Instruction:
if !slices.Contains(l.predecessors, pred) {
l.predecessors = append(l.predecessors, pred)
}
// After (fixed) — use a companion map:
type leaderMeta struct {
predecessors []*asm.Instruction
predecessorSet map[*asm.Instruction]struct{} // add this field
}
func addPredecessors(ins *asm.Instruction, preds ...*asm.Instruction) {
l := setLeader(ins)
if l.predecessorSet == nil {
l.predecessorSet = make(map[*asm.Instruction]struct{})
}
for _, pred := range preds {
if pred == nil { continue }
if _, exists := l.predecessorSet[pred]; !exists { // O(1)
l.predecessorSet[pred] = struct{}{}
l.predecessors = append(l.predecessors, pred)
}
}
}
// Similarly for Backtracker.visited:
type Backtracker struct {
visited []*Block
visitedSet map[*Block]struct{} // add companion map
...
}
```
## References
- CWE-407: Inefficient Algorithmic Complexity
- `pkg/bpf/analyze/blocks.go` lines 64-74 (addPredecessors)
- `pkg/bpf/analyze/blocks.go` lines 520-555 (Backtracker.previousBlock)
- `pkg/bpf/analyze/util.go` line 88 (resolve → setBranchTarget call site)