russellballestrini/russell.ballestrini.net
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Clarify UPS setup: these servers lack battery backup, unlike other systems

Browse source
This commit is contained in:
Russell Ballestrini 2025-11-07 07:04:29 -05:00
parent bebd1573ce
commit cf89e77582
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
content/2025-11-06-nvidia-breaks-ubuntu-ethernet-linux-modules-extra-fix.rst
View file

@ -7,7 +7,7 @@ Brown Out Reboots Servers: NVIDIA Drivers Vanish, Ethernet Dies – Sneaker-Net
:tags: Code, DevOps
:status: published
Two identical Ubuntu 24.04.2 boxes, both running ML workloads on NVIDIA GPUs. A brownout rolls through the neighborhood—lights dim, UPS chirps, & upmon dutifully hard-reboots both machines. They come back up, login prompt intact, but ``nvidia-smi`` returns *nothing*. Drivers gone. No problem—``sudo ubuntu-drivers autoinstall``, reboot, and… oh no. Ethernet vanishes. ``ip link`` shows only ``lo``. No ``eth0``, no ``enp3s0``, no nothing. Both servers, perfectly synchronized in failure.
Two identical Ubuntu 24.04.2 boxes, both running ML workloads on NVIDIA GPUs. A brownout rolls through the neighborhood—lights dim & both machines abruptly power-cycle. Unlike my other systems with UPS protection, these servers lack battery backup & go down hard. They come back up, login prompt intact, but ``nvidia-smi`` returns *nothing*. Drivers gone. No problem—``sudo ubuntu-drivers autoinstall``, reboot, and… oh no. Ethernet vanishes. ``ip link`` shows only ``lo``. No ``eth0``, no ``enp3s0``, no nothing. Both servers, perfectly synchronized in failure.
The culprit? The NVIDIA install (or the kernel update it pulled) silently removed ``linux-modules-extra-6.14.0-35-generic``. Without it, the Intel ``igc`` NIC driver refuses to load. No network, no ``apt``, no recovery—classic isolation trap.