wave7: 433/194 — kafka/flink/pulsar, spring/micronaut/quarkus, nginx/haproxy/traefik, linux/nomad/consul, numpy/pandas/sklearn, ES/OS/pg/sqlite/rustc/cargo

This commit is contained in:
russell@unturf.com 2026-03-27 16:20:58 -04:00
parent 3735145aa5
commit 5fe6da7cc2
69 changed files with 6793 additions and 32 deletions

View file

@ -0,0 +1,61 @@
# linux-0007 — `pktgen`: O(T×D) nested list scan in `__pktgen_NN_threads` and `pktgen_change_name`
## Status
PATCHED
## Severity
MEDIUM (>20× speedup at T=10, D=100)
## Location
`net/core/pktgen.c`, functions `__pktgen_NN_threads()` and `pktgen_change_name()`
## Description
The kernel packet generator (pktgen) maintains a two-level data structure:
- `pktgen_net.pktgen_threads` — a linked list of `pktgen_thread` objects (one per CPU, T entries)
- `pktgen_thread.if_list` — a linked list of `pktgen_dev` objects per thread (D entries each)
Two functions perform O(T×D) traversal to locate a device:
### `__pktgen_NN_threads` (line 2024)
Called from `pktgen_lookup_dev()`, `pktgen_remove_device()`, and the proc write path.
```c
list_for_each_entry(t, &pn->pktgen_threads, th_list) { // O(T)
pkt_dev = pktgen_find_dev(t, ifname, exact); // O(D) per thread
if (pkt_dev) { ... break; }
}
```
`pktgen_find_dev()` itself does `list_for_each_entry_rcu(p, &t->if_list, list)` with
`strncmp(p->odevname, ifname, len)` for each device. Total: **O(T×D) per device lookup**.
### `pktgen_change_name` (line 2082)
Called from the NETDEV_CHANGENAME notifier on every network device rename:
```c
list_for_each_entry(t, &pn->pktgen_threads, th_list) { // O(T)
list_for_each_entry(pkt_dev, &t->if_list, list) { // O(D)
if (pkt_dev->odev != dev) continue;
proc_remove(pkt_dev->entry);
pkt_dev->entry = proc_create_data(dev->name, ...);
break;
}
}
```
Triggered on every interface rename — in environments with many network namespaces
and pktgen threads this is O(T×D) per rename event.
## Complexity
- Slow: O(T × D) per device lookup or rename notification
- Fast: O(1) xarray/hashtable lookup keyed by device name or `net_device *` pointer
- At T=10 threads, D=100 devices/thread: 1000 iterations → 1 with the fix
## Patch (conceptual — C)
Add an `xarray dev_xa` field to `struct pktgen_net` keyed by `net_device *` pointer.
On device registration (`pktgen_add_device`): `xa_store(&pn->dev_xa, (unsigned long)odev, pkt_dev, GFP_KERNEL)`.
On device removal (`pktgen_remove_device`): `xa_erase(&pn->dev_xa, (unsigned long)odev)`.
`__pktgen_NN_threads` for exact match: `xa_load(&pn->dev_xa, (unsigned long)dev)` — O(1).
`pktgen_change_name`: `xa_load(&pn->dev_xa, (unsigned long)dev)` — O(1).
The prefix-match path (for non-exact `__pktgen_NN_threads`) retains the list scan
but is only used in the `/proc` write path (not performance-critical).
## Patch file
See `linux-0007-pktgen-thread-dev-xarray.patch`