Last November, an upstream fiber cut severed our primary link between Austin and Chicago at 2:14 AM. Twenty engineers working on an overnight PostgreSQL migration suddenly lost access. Worse, our static IPsec tunnels sat idle while engineers scrambled across three gateways to manually rewrite routing tables. That four-hour outage cost us roughly $18,000 in delayed deployment downtime. It also convinced our infrastructure team to ditch static routes for good.
WireGuard pairs cleanly with FRRouting (FRR) running Open Shortest Path First (OSPF). Together, they discover adjacent links, monitor link health with sub-second heartbeats, and re-route traffic without manual intervention. We run this exact setup across our hybrid AWS and bare-metal colocation sites. Failover now happens cleanly in under four seconds with zero packet loss on active TCP sessions.
Quick Start: Minimal Two-Site Setup in Under 10 Minutes
This baseline setup connects two Ubuntu 24.04 gateways (Site A: Austin HQ and Site B: Chicago Branch) over an encrypted point-to-point tunnel with OSPF route exchange.
1. Network Topology and Addressing
- Site A (HQ): Public WAN
203.0.113.10, Local LAN10.10.0.0/24, Tunnel IP10.255.255.1/30 - Site B (Branch): Public WAN
198.51.100.20, Local LAN10.20.0.0/24, Tunnel IP10.255.255.2/30 - Operating System: Ubuntu 22.04 or 24.04 LTS (kernel 5.15+)
2. Configure the WireGuard Tunnels
First, install WireGuard and generate cryptographic keys on both hosts:
sudo apt update && sudo apt install -y wireguard
umask 077
wg genkey | tee privatekey | wg pubkey > publickey
On Site A (HQ), save the following to /etc/wireguard/wg0.conf. Notice Table = off: this setting prevents wg-quick from overriding your system default gateway when handling catch-all peer definitions.
[Interface]
PrivateKey = <Site_A_PrivateKey>
Address = 10.255.255.1/30
ListenPort = 51820
Table = off
MTU = 1420
[Peer]
PublicKey = <Site_B_PublicKey>
Endpoint = 198.51.100.20:51820
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 15
On Site B (Branch), mirror the configuration inside /etc/wireguard/wg0.conf:
[Interface]
PrivateKey = <Site_B_PrivateKey>
Address = 10.255.255.2/30
ListenPort = 51820
Table = off
MTU = 1420
[Peer]
PublicKey = <Site_A_PublicKey>
Endpoint = 203.0.113.10:51820
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 15
Enable kernel packet forwarding and bring up the tunnels:
echo "net.ipv4.ip_forward = 1" | sudo tee /etc/sysctl.d/99-router.conf
sudo sysctl --system
sudo systemctl enable --now wg-quick@wg0
3. Spin Up FRRouting OSPF
Install the FRR suite and enable the OSPF routing daemon:
sudo apt install -y frr frr-pythontools
sudo sed -i 's/ospfd=no/ospfd=yes/' /etc/frr/daemons
sudo systemctl restart frr
Launch the FRR integrated shell on Site A using sudo vtysh, then enter these configuration directives:
configure terminal
interface wg0
ip ospf network point-to-point
ip ospf area 0
exit
router ospf
ospf router-id 10.255.255.1
network 10.10.0.0/24 area 0
exit
do write
exit
Now apply the corresponding configuration on Site B inside its own sudo vtysh prompt:
configure terminal
interface wg0
ip ospf network point-to-point
ip ospf area 0
exit
router ospf
ospf router-id 10.255.255.2
network 10.20.0.0/24 area 0
exit
do write
exit
Confirm neighbor establishment directly from the shell:
sudo vtysh -c "show ip ospf neighbor"
Your neighbor state will transition from Init to Full/ - in under three seconds. Zebra will immediately install the remote subnet (10.20.0.0/24) into the Linux kernel routing table.
Deep Dive: Why WireGuard Breaks Traditional Dynamic Routing
WireGuard does not act like a standard dummy TUN or GRE interface. It operates as a Cryptokey Routing engine. Outgoing packets must match an explicit subnet inside AllowedIPs before WireGuard encrypts them and delivers them to a peer.
The Multicast Problem
OSPF relies heavily on link-local multicast traffic sent to 224.0.0.5 (All OSPF Routers) and 224.0.0.6 (Designated Routers). Many engineers attempt to lock down AllowedIPs to explicit subnets like 10.255.255.2/32, 10.20.0.0/24. WireGuard silently drops all multicast packets in that setup.
The cryptographic handshake succeeds. Counters in wg show increment normally. Yet OSPF neighbors never form. Setting AllowedIPs = 0.0.0.0/0 alongside Table = off solves this hurdle. It allows multicast packets through the crypto-check while delegating actual forwarding decisions to FRR.
Why Point-to-Point Mode Is Non-Negotiable
OSPF defaults to broadcast network semantics on non-serial interfaces. In broadcast mode, OSPF elects a Designated Router (DR) and Backup Designated Router (BDR). Over VPN tunnels, these elections add 40 seconds of unnecessary latency. Any packet jitter can trigger election flapping.
Running ip ospf network point-to-point tells OSPF that exactly two nodes share the link. It skips DR/BDR elections completely. Adjacencies form instantly, and recovery times drop from 40 seconds down to sub-second thresholds.
Advanced Multi-Branch Mesh with Cost-Based Failover
Production environments demand route redundancy. Consider a triangular topology connecting Austin HQ, Chicago Branch 1, and Denver Branch 2. Austin handles heavy traffic across 1 Gbps fiber. A secondary 200 Mbps broadband tunnel directly bridges Chicago and Denver for emergency redundancy.
+-----------------------+
| Austin HQ (HQ) |
| 10.10.0.0/24 |
+-----------------------+
/ \
wg0 (Cost 10) wg1 (Cost 10)
/ \
+-------------------+ wg2 (Cost 50) +-------------------+
| Chicago (Branch) |=================| Denver (Branch) |
| 10.20.0.0/24 | | 10.30.0.0/24 |
+-------------------+ +-------------------+
Tuning Timers and Metric Costs
We assign lower metrics to primary fiber links and higher metrics to the cross-branch tunnel. We also lower hello and dead intervals to speed up failure detection.
On the Chicago gateway, configure interface parameters in vtysh:
configure terminal
interface wg0
description Uplink-to-Austin-HQ
ip ospf cost 10
ip ospf dead-interval 4
ip ospf hello-interval 1
exit
interface wg2
description Cross-Branch-Backup-to-Denver
ip ospf cost 50
ip ospf dead-interval 4
ip ospf hello-interval 1
exit
do write
With 1-second hello intervals and a 4-second dead timer, FRR detects link drops in four missed heartbeats. If Austin drops offline, FRR immediately recalculates Shortest Path First (SPF) routes. Traffic routes through wg2 into Denver in 4.2 seconds, avoiding broken TCP sessions on long-lived database queries.
Production Hardening and Troubleshooting
1. Prevent DBD Packet Fragmentation with Safe MTU Sizing
Encapsulating packets inside WireGuard adds 60 bytes of overhead over IPv4 (20-byte IP header + 8-byte UDP header + 32-byte WireGuard header) and 80 bytes over IPv6. When your WAN MTU is 1500, setting a 1500 MTU on wg0 forces packet fragmentation.
Large OSPF Database Description (DBD) packets will fragment during synchronization. Many cloud transit firewalls drop fragmented UDP packets outright. When this happens, OSPF neighbors hang indefinitely in the ExStart or Exchange state. Set your tunnel MTU explicitly to 1420 inside [Interface] to guarantee clean passage across any upstream carrier.
2. Keep NAT Sessions Alive
Gateways running behind AWS NAT Gateways or local enterprise firewalls drop idle UDP session mappings after 30 to 60 seconds. Add PersistentKeepalive = 15 to all remote peer definitions. This setting emits empty encrypted packets every 15 seconds, holding the stateful firewall port open around the clock.
3. Three-Step Incident Triage
When an automated route alert fires, work through your network stack in this exact sequence:
# 1. Verify cryptographic transport and handshake age
sudo wg show wg0 latest-handshakes
# 2. Check routing adjacency and state machines
sudo vtysh -c "show ip ospf neighbor"
# 3. Verify kernel routing table population
ip route show proto ospf
If the handshake is older than 120 seconds, troubleshoot UDP port 51820 reachability first. If the handshake is recent but neighbors stay in ExStart, verify that MTUs match on both sides. When neighbor status reads Full/ -, check ip route show proto ospf to verify zebra has properly populated your Linux kernel tables.
