Problem Statement: Surviving the 2 AM Bare-Metal Outage
Picture this: it is 2:15 AM on a Sunday. A storage failure just wiped the boot drives across a rack of thirty-two compute nodes. Datacenter remote hands swapped in fresh NVMe drives thirty minutes ago. Now comes the hard part: thirty-two bare-metal servers sitting at blank BIOS prompts, waiting for an operating system.
Walking the datacenter aisle with a handful of USB drives takes hours you do not have. Legacy PXE setups will not help you either. Traditional network boots rely entirely on TFTP (Trivial File Transfer Protocol) to push multi-hundred-megabyte kernel and initrd images over UDP. That works fine for one machine. Put twenty hosts on it at once, however, and UDP packet loss spikes. Timeouts trigger, transfers stall, and installations crawl along for forty minutes before failing entirely.
A two-stage boot architecture fixes this bottleneck. First, a lightweight service hands out network leases and serves a tiny intermediate bootloader. Second, that bootloader switches protocols to pull heavy ISOs and kernel images over plain HTTP. Pairing dnsmasq for DHCP and architecture detection with Nginx for HTTP delivery lets you provision entire server racks in under five minutes.
Core Concepts: Why Traditional PXE Fails and How iPXE Fixes It
Before editing configuration files, let’s examine how network firmware, DHCP options, and transfer protocols interact during boot.
The TFTP Bottleneck vs. HTTP Streaming
Standard PXE ROMs burned onto motherboard NICs are deliberately minimal. They speak basic DHCP and rudimentary TFTP—and nothing else. TFTP uses a lock-step acknowledgment model: send one 512-byte block, wait for an ACK packet, then send the next block. Even on a 10Gbps switch fabric, packet round-trip latency caps single-stream TFTP throughput at roughly 1.5 to 2.5 MB/s. Pulling a 1.8GB Ubuntu 24.04 live-server image this way takes twenty to thirty minutes per node.
iPXE solves this by upgrading the bootloader directly in RAM. It is an open-source network bootloader equipped with a full TCP/IP stack supporting HTTP, HTTPS, DNS, and iSCSI. The native PXE ROM downloads just a tiny iPXE binary (around 350KB for UEFI, 65KB for BIOS) over TFTP in under a second. Once running, iPXE takes control of the NIC and fetches the kernel and ramdisk over standard HTTP from Nginx, saturating 1Gbps or 10Gbps links effortlessly.
Legacy BIOS vs. UEFI Boot Modes
Production environments frequently run a mix of older legacy BIOS hosts alongside 64-bit UEFI hardware. Serving a single hardcoded filename will fail half your inventory. A boot server must inspect DHCP Option 93 (Client System Architecture Type, RFC 4578) to supply the matching binary:
undionly.kpxe: For Legacy BIOS machines (Architecture00:00), using standard UNDI driver calls.ipxe.efi: For 64-bit UEFI systems (Architecture00:07,00:09, and00:10for HTTP-boot).
The Role of dnsmasq as a Boot Coordinator
Rather than stitching together isc-dhcp-server and tftpd-hpa, dnsmasq integrates DNS, flexible DHCP tagging, and a built-in TFTP engine inside one lightweight daemon. It parses vendor-class identifiers on the fly. This lets you write conditional boot rules based on whether a machine is booting for the first time or returning with the iPXE user class.
Hands-On Practice: Building the iPXE Deployment Engine
This setup runs on an Ubuntu Server 22.04 or 24.04 LTS host with a dedicated provisioning interface (eth1) assigned static IP 192.168.10.10/24 on the deployment subnet 192.168.10.0/24.
Step 1: Install Required Packages
Install dnsmasq, Nginx, and pre-built iPXE binaries directly from the package repositories:
sudo apt update
sudo apt install -y dnsmasq nginx ipxe syslinux pxelinux
Step 2: Prepare the TFTP Directory Structure
Create the TFTP root directory and copy the standard BIOS and UEFI iPXE binaries:
sudo mkdir -p /srv/tftp
# Copy Legacy BIOS iPXE image
sudo cp /usr/lib/ipxe/undionly.kpxe /srv/tftp/
# Copy UEFI 64-bit iPXE binary
sudo cp /usr/lib/ipxe/ipxe.efi /srv/tftp/
sudo chown -R nobody:nogroup /srv/tftp
sudo chmod -R 755 /srv/tftp
Step 3: Configure dnsmasq for Dynamic Chainloading
Create a dedicated configuration file at /etc/dnsmasq.d/ipxe.conf. Notice the port=0 directive: this disables the DNS listener so dnsmasq never conflicts with Ubuntu’s local systemd-resolved stub listener on port 53.
sudo bash -c 'cat > /etc/dnsmasq.d/ipxe.conf' << "EOF"
# Bind exclusively to the provisioning interface
interface=eth1
bind-interfaces
port=0
# DHCP IP pool and gateway
dhcp-range=192.168.10.100,192.168.10.200,255.255.255.0,12h
dhcp-option=option:router,192.168.10.1
dhcp-option=option:dns-server,1.1.1.1,8.8.8.8
# Built-in TFTP configuration
enable-tftp
tftp-root=/srv/tftp
# Match client architecture (Option 93)
dhcp-match=set:bios,option:client-arch,0
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-match=set:efi-x86_64,option:client-arch,9
# Detect if client is already running iPXE
dhcp-userclass=set:ipxe,iPXE
# STAGE 1: Bare client boots native PXE -> serve iPXE loader via TFTP
dhcp-boot=tag:!ipxe,tag:bios,undionly.kpxe
dhcp-boot=tag:!ipxe,tag:efi-x86_64,ipxe.efi
# STAGE 2: Client re-enters with iPXE -> hand off HTTP boot script
dhcp-boot=tag:ipxe,http://192.168.10.10/boot.ipxe
EOF
Verify syntax and restart the service:
sudo dnsmasq --test
sudo systemctl restart dnsmasq
sudo systemctl status dnsmasq
Step 4: Configure Nginx to Serve Boot Assets
Create directories for ISO images, extracted kernels, and cloud-init automation files:
sudo mkdir -p /var/www/netboot/os/ubuntu24
sudo mkdir -p /var/www/netboot/scripts
Configure an optimized Nginx server block at /etc/nginx/sites-available/netboot:
server {
listen 80;
server_name 192.168.10.10;
root /var/www/netboot;
access_log /var/log/nginx/netboot_access.log;
error_log /var/log/nginx/netboot_error.log;
location / {
autoindex on;
sendfile on;
sendfile_max_chunk 2m;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
}
}
Activate the site, drop the default configuration, and reload Nginx:
sudo ln -s /etc/nginx/sites-available/netboot /etc/nginx/sites-enabled/
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t && sudo systemctl reload nginx
Step 5: Extract OS Kernel and Build the iPXE Menu
Mount your Ubuntu 24.04 Server ISO. Extract the kernel (vmlinuz) and initial ramdisk (initrd), then move the full ISO to the web root so Subiquity can mount it remotely:
sudo mount -o loop /root/ubuntu-24.04-live-server-amd64.iso /mnt
sudo cp /mnt/casper/vmlinuz /var/www/netboot/os/ubuntu24/
sudo cp /mnt/casper/initrd /var/www/netboot/os/ubuntu24/
sudo cp /root/ubuntu-24.04-live-server-amd64.iso /var/www/netboot/os/ubuntu24/
sudo umount /mnt
Create dummy metadata files in /var/www/netboot/scripts/ so the Ubuntu installer does not fail during its cloud-init check:
sudo touch /var/www/netboot/scripts/meta-data
sudo touch /var/www/netboot/scripts/user-data
Now write the boot script at /var/www/netboot/boot.ipxe. Note the explicit initrd=initrd argument on the kernel line, which ensures the EFI stub loader discovers the ramdisk properly:
sudo bash -c 'cat > /var/www/netboot/boot.ipxe' << "EOF"
#!ipxe
set server_ip 192.168.10.10
set base_url http://${server_ip}
set menu-timeout 10000
:start
menu Network Boot System - Bare Metal Provisioning
item --gap -- ---------------- Operating Systems ----------------
item ubuntu24 Install Ubuntu Server 24.04 LTS (Automated)
item --gap -- ---------------- Recovery & Tools -----------------
item shell Drop to iPXE Interactive Shell
item reboot Reboot Host
item exit Boot from Local Hard Drive
choose --timeout ${menu-timeout} --default ubuntu24 target && goto ${target}
:ubuntu24
echo Fetching Ubuntu 24.04 Kernel and Initrd via HTTP...
kernel ${base_url}/os/ubuntu24/vmlinuz initrd=initrd ip=dhcp url=${base_url}/os/ubuntu24/ubuntu-24.04-live-server-amd64.iso autoinstall ds=nocloud-net;s=${base_url}/scripts/ cloud-config-url=/dev/null
initrd ${base_url}/os/ubuntu24/initrd
echo Booting Linux kernel...
boot
:shell
shell
:reboot
reboot
:exit
exit
EOF
Verification and Real-Time Troubleshooting
Boot a client node with PXE enabled. Open two terminal windows on the server to observe the two-stage boot in action.
In the first terminal, tail the dnsmasq log:
sudo journalctl -u dnsmasq -f
You should see two sequential DHCP requests from the client’s MAC address:
- An initial
DHCPDISCOVERwith no user class. Dnsmasq matches Option 93 and offersipxe.efiorundionly.kpxeover TFTP. - A follow-up
DHCPDISCOVERcarryingUser-Class: iPXE. Dnsmasq recognizes the tag and hands offhttp://192.168.10.10/boot.ipxe.
In the second terminal, watch Nginx stream the OS files:
sudo tail -f /var/log/nginx/netboot_access.log
You will see GET /os/ubuntu24/vmlinuz and GET /os/ubuntu24/initrd return 200 OK. On a standard 1Gbps link, pulling the 90MB initrd takes under one second—compared to nearly two minutes over traditional TFTP.
Conclusion: Production-Ready Automation
Replacing pure TFTP boots with an iPXE and HTTP pipeline turns bare-metal provisioning from a slow chore into a fast, reliable workflow. Dnsmasq reconciles differences between Legacy BIOS and modern UEFI clients without manual intervention, while Nginx handles the heavy file transfers at line speed.
With this foundation in place, you can hook the iPXE menu directly into custom cloud-init configs, automated Kickstart recipes, or hardware inventory scrapers. The result is a true zero-touch pipeline: plug in power and network, and nodes provision themselves automatically.
