Context & Why: Surviving Kernel Upgrades with Out-of-Tree Drivers
During routine maintenance across our cluster of 45 edge servers, minor kernel security updates consistently triggered a frustrating failure: broken out-of-tree hardware drivers. Systems would reboot without their 2.5GbE Realtek NICs or specialized FPGA accelerators. That left our on-call engineers scrambling to manually recompile source trees against new kernel headers in the middle of the night.
This problem stems directly from how the Linux kernel handles internal interfaces. Unlike Windows or macOS, Linux deliberately does not maintain a stable internal Application Binary Interface (ABI).
When kernel developers alter internal data structures or function signatures between patch releases, loadable kernel modules (.ko files) built for kernel 5.15.0-80 fail to load on 5.15.0-84. If you run proprietary RAID controllers, custom network interfaces, ZFS, or VirtualBox drivers, your code lives outside the upstream kernel tree and needs recompilation for every single kernel update.
Dynamic Kernel Module Support (DKMS) automates this entire process. It hooks directly into your distribution package manager through post-install triggers. Whenever apt, dnf, or pacman pulls in a new kernel and its headers, DKMS takes over. It grabs the module source from /usr/src/<module>-<version>/, compiles it against the incoming kernel headers in roughly 15 to 30 seconds, and places the fresh .ko binary into the module tree.
Pair DKMS with modprobe—the standard utility for dependency resolution and driver loading—and you eliminate manual driver rebuilding entirely.
Installation: Preparing the Build Environment
DKMS requires a working C compiler toolchain and the kernel development headers matching your active or target kernel.
1. Debian and Ubuntu Systems
Update your local package lists and install the build tools, DKMS, and current kernel headers:
sudo apt update
sudo apt install -y build-essential dkms linux-headers-$(uname -r)2. RHEL, Rocky Linux, and AlmaLinux
Enable the EPEL repository first on RPM-based enterprise distributions, then pull down the required development packages:
sudo dnf install -y epel-release
sudo dnf install -y gcc make dkms kernel-devel-$(uname -r) kernel-headersCheck that the build symlink points to your installed headers directory:
ls -ld /lib/modules/$(uname -r)/buildIf this resolves to a valid directory under /usr/src/kernels/ or /usr/src/linux-headers-*/, your build toolchain is ready.
Configuration: Registering Modules and Fine-Tuning modprobe
Getting an automated module running involves two steps: writing a dkms.conf manifest and defining runtime loading rules with modprobe.
Step 1: Setting up DKMS Source and dkms.conf
DKMS looks for source trees inside /usr/src/<module_name>-<module_version>/. A configuration file named dkms.conf in that directory defines build commands and binary target paths.
For example, to package a custom network driver called custom_nic at version 1.0.0, create the target source path:
sudo mkdir -p /usr/src/custom_nic-1.0.0Place your C source files and Makefile into that directory, then create /usr/src/custom_nic-1.0.0/dkms.conf:
PACKAGE_NAME="custom_nic"
PACKAGE_VERSION="1.0.0"
CLEAN="make clean"
MAKE[0]="make -C /lib/modules/${kernelver}/build M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build modules"
BUILT_MODULE_NAME[0]="custom_nic"
DEST_MODULE_LOCATION[0]="/updates/dkms"
AUTOINSTALL="yes"Key directives to note:
AUTOINSTALL="yes": Instructs DKMS to rebuild this module automatically whenever a package manager installs a new kernel.${kernelver}: An environment variable passed by DKMS that holds the target kernel version string (e.g.,5.15.0-88-generic).DEST_MODULE_LOCATION[0]: Specifies where the compiled.kobinary will install under/lib/modules/<kernel_version>/.
Now register, compile, and install the module with three quick commands:
sudo dkms add -m custom_nic -v 1.0.0
sudo dkms build -m custom_nic -v 1.0.0
sudo dkms install -m custom_nic -v 1.0.0Step 2: Configuring Persistent Loading and Options with modprobe
After DKMS drops the binary into /lib/modules/$(uname -r)/updates/dkms/, generate fresh module dependency maps:
sudo depmod -aTo load the module automatically upon system boot, register it in /etc/modules-load.d/:
echo "custom_nic" | sudo tee /etc/modules-load.d/custom_nic.confIf your driver needs specific hardware parameters or must override a conflicting module, configure them in /etc/modprobe.d/:
sudo tee /etc/modprobe.d/custom_nic.conf << 'EOF'
# Runtime parameters for custom_nic
options custom_nic debug=1 rx_buffer_size=4096
# Prevent an in-tree conflicting driver from loading
blacklist generic_legacy_nic
EOFOn our Ubuntu 22.04 LTS servers, this automation cut our post-kernel maintenance downtime to zero. Background security patches rebuild and register the driver during the package upgrade step, before any reboot occurs.
Verification & Monitoring: Managing Driver Lifecycle
Once configured, verify your module status across installed kernels and audit logs under /var/lib/dkms/.
1. Inspecting DKMS Registration Status
Run dkms status to list every registered module and its build state across all local kernels:
dkms statusA properly installed module prints output like this:
custom_nic/1.0.0, 5.15.0-84-generic, x86_64: installed
custom_nic/1.0.0, 5.15.0-88-generic, x86_64: installedIf the state shows added or built instead of installed, the driver binary has not been copied to the module directory and modprobe will fail to find it.
2. Verifying and Loading Modules with modprobe
Load the module into the active kernel with verbose output enabled:
sudo modprobe -v custom_nicInspect the compiled module details, author metadata, and license string:
modinfo custom_nicConfirm the kernel loaded the module and check its active memory footprint:
lsmod | grep custom_nicReview the kernel ring buffer to confirm initialization parameters were accepted:
sudo dmesg | grep -i custom_nic3. Testing Unload and Clean-up Procedures
When releasing version 1.0.1 of your driver, unload the active module from memory and delete the old DKMS build artifacts:
# Safely remove the running module from memory
sudo modprobe -r custom_nic
# Deregister version 1.0.0 from all installed kernels
sudo dkms remove -m custom_nic -v 1.0.0 --allIf a build ever fails during a kernel upgrade, check /var/lib/dkms/custom_nic/1.0.0/build/make.log. That log will pinpoint the exact C header file, struct member, or deprecated function call that caused the compiler error.
