Ubuntu Dual WAN Load Balancing and Failover with IPRoute2 and Netplan

Table of Contents

The 2 AM Crisis: Why Single WAN Isn’t Enough

It was 2 AM on a Tuesday when my phone started screaming. The primary fiber line at our main branch had been severed by a construction crew working late. Despite having a backup LTE connection, the server was blind. Why? Because the routing table was still stubbornly trying to send packets out of a dead interface. I had to manually log in via a console, delete the default route, and add a new one while half-asleep. That was the night I decided to never rely on manual intervention again.

Most small-to-medium businesses think they need an expensive Cisco or Juniper router to handle two ISPs. The reality is that your Ubuntu Server is more than capable of handling Dual WAN scenarios using tools that are already built into the kernel: IPRoute2 and Netplan. I have applied this approach in production and the results have been consistently stable, providing both increased bandwidth through load balancing and peace of mind through automated failover.

Core Concepts: Policy-Based Routing (PBR)

Standard routing works by looking at the destination IP. If you want to reach 8.8.8.8, the kernel looks at the routing table and sends it out the default gateway. In a Dual WAN setup, this isn’t enough. We need Policy-Based Routing (PBR).

PBR allows us to make routing decisions based on the source address or other criteria. If a packet comes in from ISP1, the response must go back through ISP1. If we just had one routing table, the server might try to send ISP1’s response through ISP2, which the ISP would promptly drop as a spoofed packet. To solve this, we create separate routing tables for each ISP and a main table for load balancing.

Hands-on: Configuring Dual WAN on Ubuntu

For this setup, let’s assume the following environment:

Interface eth0 (ISP1): IP 1.1.1.2, Gateway 1.1.1.1
Interface eth1 (ISP2): IP 2.2.2.2, Gateway 2.2.2.1
Interface eth2 (LAN): IP 192.168.1.1

Step 1: Network Interface Configuration with Netplan

First, we define our interfaces. Ubuntu uses Netplan, so we edit /etc/netplan/01-netcfg.yaml. Note that we do not set a global default gateway here. Instead, we define the routes manually to prevent the kernel from getting confused.

network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      addresses:
        - 1.1.1.2/24
    eth1:
      addresses:
        - 2.2.2.2/24
    eth2:
      addresses:
        - 192.168.1.1/24

Apply the changes with sudo netplan apply. At this point, your server has no default gateway. It can talk to the local subnets of the ISPs but cannot reach the internet.

Step 2: Defining Custom Routing Tables

We need to tell the system that two new routing tables exist. Edit /etc/iproute2/rt_tables and add two entries at the bottom:

# sudo nano /etc/iproute2/rt_tables
100 isp1
200 isp2

Step 3: Creating the Routing Logic

Now we populate these tables. We want each table to have its own default gateway and ensure that traffic originating from a specific interface stays on that interface. I usually wrap these commands in a script or a systemd service to ensure they persist after a reboot.

# Setup Table ISP1
ip route add 1.1.1.0/24 dev eth0 src 1.1.1.2 table isp1
ip route add default via 1.1.1.1 table isp1

# Setup Table ISP2
ip route add 2.2.2.0/24 dev eth1 src 2.2.2.2 table isp2
ip route add default via 2.2.2.1 table isp2

# Routing rules to ensure traffic returns the way it came
ip rule add from 1.1.1.2 table isp1
ip rule add from 2.2.2.2 table isp2

Step 4: Implementing Load Balancing

This is where the magic happens. We use the nexthop parameter to distribute traffic across both gateways. You can assign weights if one ISP is faster than the other (e.g., weight 2 for a 200Mbps line and weight 1 for a 100Mbps line).

sudo ip route add default scope global \
    nexthop via 1.1.1.1 dev eth0 weight 1 \
    nexthop via 2.2.2.1 dev eth1 weight 1

The Linux kernel will now perform per-flow load balancing. This means a single connection (like a file download) stays on one WAN to avoid packet reordering issues, but multiple users or applications will be spread across both links.

Automating Failover with a Health Check Script

Load balancing is great, but if ISP1 goes down, the kernel might still try to send 50% of the traffic into a black hole. We need a way to detect a failure and update the routing table. While there are complex tools like keepalived, a simple bash script is often more reliable and easier to debug for this specific task.

#!/bin/bash
# Basic Failover Script

ISP1_GW="1.1.1.1"
ISP2_GW="2.2.2.1"
CHECK_IP="8.8.8.8"

check_alive() {
    ping -I $1 -c 3 -W 2 $CHECK_IP > /dev/null 2>&1
    return $?
}

while true; do
    check_alive eth0
    ISP1_STAT=$?
    
    check_alive eth1
    ISP2_STAT=$?

    if [ $ISP1_STAT -ne 0 ] && [ $ISP2_STAT -eq 0 ]; then
        # ISP1 down, ISP2 up - Switch all to ISP2
        ip route replace default via $ISP2_GW dev eth1
    elif [ $ISP1_STAT -eq 0 ] && [ $ISP2_STAT -ne 0 ]; then
        # ISP2 down, ISP1 up - Switch all to ISP1
        ip route replace default via $ISP1_GW dev eth0
    elif [ $ISP1_STAT -eq 0 ] && [ $ISP2_STAT -eq 0 ]; then
        # Both up - Restore Load Balancing
        ip route replace default scope global \
            nexthop via $ISP1_GW dev eth0 weight 1 \
            nexthop via $ISP2_GW dev eth1 weight 1
    fi
    sleep 5
done

Run this script as a background service. It pings Google’s DNS through specific interfaces. If one fails, it modifies the default route in the main table. Once the ISP recovers, it automatically restores the load-balanced state.

Validation and Monitoring

To verify which path your traffic is taking, don’t just use ping. Use traceroute with the interface flag:

traceroute -i eth0 google.com
traceroute -i eth1 google.com

To see your current active routes and the weights being used, run:

ip route show

Check the policy rules to ensure your source-based routing is active:

ip rule show

Final Thoughts

Setting up Dual WAN on Ubuntu doesn’t require proprietary software or expensive hardware. By leveraging IPRoute2’s multiple routing tables and a simple health check script, you create a system that is both resilient and performant.

My experience with this setup has shown that it handles ISP hiccups far more gracefully than many consumer-grade “multi-WAN” routers. Just remember to keep your scripts simple and your monitoring active. If you ever find yourself troubleshooting at 2 AM again, you’ll be glad the server handled the failover for you.