Quick Start (5 Min): Immediate VM Port Isolation
Six months ago, our public hosting cluster ran into a security headache. Over 300 customer VMs sat inside shared /24 broadcast domains. Rogue tenants could sniff unencrypted ARP queries, run port scans against neighboring instances, and launch spoofing attacks.
Carving up the space into micro-subnets was off the table—public IPv4 addresses cost over $35 each on the secondary market, and slicing subnets would waste valuable network and broadcast IPs. The quickest remedy? Toggling the native isolated flag directly on our Linux bridges.
Linux bridge port isolation enforces one simple rule: isolated ports cannot transmit Ethernet frames to each other. They can only exchange frames with non-isolated uplink ports, such as the physical router interface.
Take a hypervisor with bridge br0, physical uplink eth0, and two guest tap devices (vnet0 and vnet1). You can lock down east-west traffic in seconds:
# 1. Inspect existing bridge port states
bridge link show
# 2. Flag both guest VM tap devices as isolated
sudo bridge link set dev vnet0 isolated on
sudo bridge link set dev vnet1 isolated on
# 3. Explicitly verify that your gateway uplink remains non-isolated
sudo bridge link set dev eth0 isolated off
# 4. Confirm the isolation flags in the detailed link table
bridge -d link showWith those flags active, vnet0 and vnet1 can no longer exchange a single packet at Layer 2. North-south traffic flows out through eth0 without interruption. It is one of the most effective, lightweight tricks available for securing multi-tenant Linux virtualization.
Deep Dive: Under the Hood of Linux Bridge Filtering & PVLAN
Running this setup across 48 dual-socket hypervisors taught us that understanding the packet flow saves hours when hunting edge-case drops.
How Port Isolation Works in the Kernel
When an ingress frame arrives on a port flagged with BR_ISOLATED, the kernel evaluates the destination port in the bridge Forwarding Database (FDB). If the target port also carries the BR_ISOLATED flag, the kernel immediately discards the frame inside net/bridge/br_forward.c via br_forward_finish() before it ever reaches the egress queue.
This drop behavior shuts down common lateral vectors:
- Direct Port Scanning: An aggressive
nmap -sS -p 1-65535from a compromised tenant VM receives zero responses and detects no live neighbors. - ARP Poisoning: Gratuitous ARP broadcasts and poisoned ARP replies originating from an isolated port never reach sibling VM ports.
- Lateral Malware Propagation: Worms like Mirai or ransomware scanning internal subnets hit an immediate dead end at Layer 2.
Extending Beyond Basic Isolation: Private VLANs (PVLAN)
Basic port isolation lumps all isolated ports into a single flat bucket. Complex multi-tier environments, however, require dedicated segmentation modeled after hardware Private VLANs (RFC 5517):
- Promiscuous: Gateway routers, firewalls, and central syslog collectors that must talk to every port.
- Isolated: Single-tenant or DMZ VMs that should only talk to the promiscuous gateway.
- Community: Clustered nodes—like three load-balanced Nginx web nodes and their shared Redis cache—that must talk among themselves, but remain blind to other tenants.
You do not need Open vSwitch (OVS) to achieve standard PVLAN semantics. Combine Linux Bridge VLAN filtering with ebtables or nftables MAC policies:
# Enable VLAN filtering on the existing bridge
sudo ip link set dev br0 type bridge vlan_filtering 1
# Gateway uplink on eth0: Primary promiscuous VLAN 100
sudo bridge vlan add dev eth0 vid 100 pvid untagged
# VM 1 on vnet0: Secondary isolated VLAN 101
sudo bridge vlan add dev vnet0 vid 101 pvid untagged
# VM 2 on vnet1: Secondary isolated VLAN 102
sudo bridge vlan add dev vnet1 vid 102 pvid untaggedAdvanced Usage: Production Persistence and Libvirt Integration
Interface recreation bit us almost immediately during early testing. Every time a tenant rebooted their VM, QEMU and libvirt tore down the tap device and recreated it from scratch, stripping all dynamic bridge link flags in the process.
Automating Isolation via Libvirt Hook Scripts
To ensure every newly provisioned or rebooted tap device inherits isolation instantly, install a libvirt QEMU hook script at /etc/libvirt/hooks/qemu:
#!/bin/bash
# /etc/libvirt/hooks/qemu
OBJECT="$1"
OPERATION="$2"
SUB_OPERATION="$3"
if [ "$OPERATION" = "started" ] || [ "$OPERATION" = "attach" ]; then
# Enumerate all virtual tap interfaces attached to this domain
for nic in $(virsh domiflist "$OBJECT" 2>/dev/null | awk '/vnet/ {print $1}'); do
# Confirm interface belongs to our multi-tenant bridge br0
BR=$(cat "/sys/class/net/$nic/brport/bridge/uevent" 2>/dev/null | grep 'INTERFACE=' | cut -d= -f2)
if [ "$BR" = "br0" ]; then
bridge link set dev "$nic" isolated on
logger -t libvirt-hook "Applied port isolation to $nic on $BR for domain $OBJECT"
fi
done
fiSet execution permissions with sudo chmod +x /etc/libvirt/hooks/qemu and restart libvirtd. The hook executes in under 15 milliseconds, enforcing the isolation flag long before the guest kernel boots and issues its first DHCP request.
Preventing ARP Floods with Kernel Hairpin Mode Control
If you implement proxy-ARP so isolated VMs can communicate strictly through an external perimeter firewall on the same subnet, make sure hairpin mode is explicitly disabled on guest ports:
# Disable hairpin mode to eliminate reflective broadcast loops
sudo bridge link set dev vnet0 hairpin off
sudo bridge link set dev vnet1 hairpin offPractical Tips: Lessons from 6 Months in Production
These four operational insights come directly from production metrics, incident reports, and system telemetry:
1. The Rogue DHCP Mitigation Bonus
Because isolated ports drop sibling traffic, DHCP Discover and Request broadcasts sent from an isolated VM reach the physical gateway, but never reach another tenant. If a malicious or misconfigured tenant starts a rogue DHCP server inside their VM, its DHCP Offer packets never leak to peer guests. That eliminated accidental IP hijacks across our fleet without needing extra ebtables filters or dedicated DHCP snooping daemons.
2. Script Inspections via Sysfs
While bridge link show is great for interactive terminal debugging, custom health checks and Ansible playbooks should read kernel state directly from sysfs:
# Returns 1 when isolated, 0 for standard forwarding
cat /sys/class/net/vnet0/brport/isolated3. IPv6 Neighbor Discovery (NDP) Traps
Port isolation blocks all Layer 2 multicast between guests, which breaks IPv6 Neighbor Solicitations (NS) and Neighbor Advertisements (NA). If two VMs attempt to talk via their assigned /64 global unicast addresses on the same bridge, NDP fails immediately. To resolve this, configure your upstream edge router to advertise itself as the default hop for the entire prefix, or run ndppd (Neighbor Discovery Protocol Proxy Daemon) on the host.
4. Negligible CPU Overhead
Across sustained peak loads of 14 Gbps per hypervisor, packet evaluation CPU overhead was negligible. The kernel implements the isolation check as a single bitwise AND operation on port flags inside br_forward.c. Compared to Open vSwitch with conntrack rules, our hypervisors averaged 4% lower per-core CPU utilization during heavy east-west traffic surges.
