Moving Beyond the Default Gateway
Most Linux servers operate on a “one exit” policy. If a packet lacks a specific destination in the routing table, the kernel tosses it to the default gateway. This works for basic setups. But the moment you add a 10Gbps fiber line alongside a backup LTE link, or try to force a specific Docker container through a WireGuard tunnel, the single gateway becomes a liability. You need logic that says: “Database backups use the slow link, but web traffic stays on the fiber.”
Policy Based Routing (PBR) solves this. Instead of routing based solely on where a packet is going, PBR lets you decide based on who is sending it or what protocol it uses. In production edge environments, this is the difference between an efficient hybrid cloud and a networking nightmare.
How PBR Differs from Standard Routing
To use PBR effectively, you have to understand the shift from destination-only logic to rule-based selection.
Standard Destination-Based Routing
This is the default. The kernel inspects the destination IP, matches the most specific prefix in the main table, and ships it out. It is blind to the source application. Whether it’s a system update or a critical API call, they all queue for the same exit. It is simple but inflexible.
Policy Based Routing (PBR)
PBR adds a “Rules” layer (ip rule) on top of your tables. Before the kernel even touches the main routing table, it checks these rules. A rule acts as a filter: “If this packet originated from 192.168.1.50, ignore the main table and use Table 100.” Table 100 can have its own unique default gateway, effectively creating isolated routing lanes on a single machine.
The Trade-offs
Complexity is the price of control. Consider these factors before deploying.
- Advantages:
- Cost Optimization: Move 500GB of nightly backups over a cheap, high-latency 100Mbps link while reserving your 1Gbps low-latency line for users.
- VPN Isolation: Route specific users or namespaces through a VPN (like OpenVPN or Tailscale) without leaking the rest of your system traffic to the tunnel.
- Eliminate Asymmetric Routing: Ensure traffic entering via ISP B also leaves via ISP B, preventing firewalls from dropping “out-of-state” packets.
- Drawbacks:
- Hidden Logic: Troubleshooting is harder. A simple
ip route showdoesn’t explain why a packet is failing if a rule is overriding it. - Maintenance: You must manually keep custom tables synchronized with local network changes.
- Hidden Logic: Troubleshooting is harder. A simple
The Scenario
We will use a typical dual-homed setup for this guide. We assume you are on Ubuntu 22.04 or 24.04 with iproute2.
The Setup:
- eth0 (Primary): 192.168.1.10 (Gateway: 192.168.1.1)
- eth1 (Secondary/VPN): 10.0.0.5 (Gateway: 10.0.0.1)
Our goal: Force traffic from local IP 192.168.1.100 to exit via eth1 (10.0.0.1), while the rest of the system keeps using eth0.
Implementation Steps
1. Define a Custom Routing Table
Linux allows up to 252 custom tables. Avoid using IDs above 253 as they are reserved for system use. We’ll register a name in /etc/iproute2/rt_tables to make our commands readable.
# Open the table definitions
sudo nano /etc/iproute2/rt_tables
# Append our new table
200 secondary_link2. Populate the Custom Table
The secondary_link table is currently empty. We must give it a default route and a route to the local network so it doesn’t try to send local traffic out to the ISP.
# Set the gateway for our new table
sudo ip route add default via 10.0.0.1 dev eth1 table secondary_link
# Ensure internal traffic still works
sudo ip route add 192.168.1.0/24 dev eth0 table secondary_link3. Apply the Routing Rule
This is where the magic happens. We tell the kernel to trigger the secondary_link table for any traffic originating from our target IP.
# Create the rule
sudo ip rule add from 192.168.1.100 table secondary_link
# Audit the rule hierarchy
ip rule showThe output shows the priority. Lower numbers win:
0: from all lookup local
32765: from 192.168.1.100 lookup secondary_link
32766: from all lookup main
32767: from all lookup default4. Configure Source NAT
If eth1 is an ISP or VPN, its gateway expects packets to match its own subnet. Use iptables to masquerade the traffic so it appears to come from 10.0.0.5.
# Masquerade outgoing traffic on eth1
sudo iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE5. Verify the Path
Test the routing by binding curl to the specific source IP. Compare the results to see if the traffic actually takes the alternate path.
# Default route check (should be ISP A)
curl -s icanhazip.com
# PBR route check (should be ISP B/VPN)
curl -s --interface 192.168.1.100 icanhazip.comAdvanced Use: Routing by Service (FWMARK)
Sometimes you need to route by protocol—like sending all HTTPS traffic (port 443) through a faster link. You can’t do this with ip rule alone. You must “mark” the packets using iptables first.
# 1. Mark TCP 443 packets with ID "1"
sudo iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark 1
# 2. Route any packet with mark "1" using our custom table
sudo ip rule add fwmark 1 table secondary_linkPersisting Changes with Netplan
The ip commands are volatile and disappear after a reboot. On Ubuntu, use Netplan to make these rules permanent. Update your YAML config in /etc/netplan/:
network:
version: 2
ethernets:
eth1:
addresses: [10.0.0.5/24]
routes:
- to: default
via: 10.0.0.1
table: 200
routing-policy:
- from: 192.168.1.100
table: 200Run sudo netplan apply to lock in the configuration.
Wrapping Up
Policy Based Routing shatters the rigid “one-way-out” model of networking. While it adds a layer of complexity to your debugging process, the flexibility it provides for multi-homed servers and VPN gateways is unmatched. Once you grasp the flow—Rules first, Tables second—you can precisely steer traffic across any number of interfaces.
