Why Traditional Tools Fail at High Speeds
Standard Linux tools like iperf3 or netperf usually choke long before your hardware does. These utilities rely on the Linux kernel’s networking stack. While robust, the kernel introduces massive overhead through interrupt handling and constant context switching. When I tried pushing a 10Gbps link with a single iperf3 stream, my CPU cores hit 100% utilization while the throughput stalled at a measly 3.8Gbps.
If you need to validate a high-end router or a Next-Gen Firewall (NGFW) before a major rollout, you need line-rate traffic. You cannot afford to be limited by the operating system. That is why I switched to TRex. After benchmarking data center interconnects for six months, I’ve found it is the only open-source tool that reliably generates millions of packets per second without flinching.
The DPDK Advantage
TRex runs on top of the Data Plane Development Kit (DPDK). Instead of asking the Linux kernel to move packets, TRex takes direct ownership of the Network Interface Card (NIC). It bypasses the kernel entirely. This architectural shift allows a single commodity server to generate hundreds of gigabits of traffic.
Here is what makes TRex different from your average traffic generator:
- True Scalability: It can push 14.88 million packets per second (pps) on a 10GbE link—the theoretical maximum for 64-byte packets.
- IMIX Traffic: It simulates “Internet Mix” patterns, blending 64B, 594B, and 1518B packets to mirror real-world usage rather than just sending a flat stream of data.
- Protocol Depth: You can test simple L2/L3 throughput or simulate complex L7 application flows like HTTPS and DNS.
My team uses these metrics to find the exact point where a firewall’s silicon starts to fail, providing data that standard tools simply cannot reach.
Hardware Selection and Preparation
Hardware compatibility is the most common pitfall. Because TRex relies on DPDK, your NIC must be supported. Intel X520, X710, and Mellanox ConnectX-5 cards are the industry gold standards. For this guide, I’m using an Ubuntu 22.04 LTS server with an Intel X722 dual-port 10G NIC.
Start by installing the base dependencies and identifying your PCI bus addresses:
sudo apt update
sudo apt install -y python3 python3-distutils zlib1g-dev pciutils
# Identify your NIC's PCI address
lspci | grep EthernetLook for addresses like 0000:03:00.0. You will need these to map the ports in the configuration file later.
Installation and Setup
TRex does not use a typical make install workflow. You download the binary package, extract it, and run it directly. This portable approach is great for moving the tool between different test rigs in the lab.
mkdir -p /opt/trex
cd /opt/trex
wget https://trex-tgn.cisco.com/trex/release/latest
tar -xzvf latest
cd v3.0* # Enter the current version directoryYou must create a /etc/trex_cfg.yaml file to define your environment. This file maps the physical ports on your NIC to the TRex software. Here is a basic configuration for a dual-port setup:
- version: 2
interfaces: ["03:00.0", "03:00.1"]
port_info:
- dest_mac: "00:11:22:33:44:55" # MAC of your Router Port 1
src_mac: "00:55:44:33:22:11" # MAC of TRex Port 1
- dest_mac: "00:11:22:33:44:66" # MAC of your Router Port 2
src_mac: "00:55:44:33:22:22" # MAC of TRex Port 2Executing Your First Stress Test
TRex uses a server-client model. You launch the engine first, then use a console to inject traffic.
Step 1: Fire up the Engine
sudo ./t-rex-64 -iThe -i flag enables interactive mode. Once the ports initialize, the server sits and waits for your instructions.
Step 2: Connect the Console
Open a second terminal and launch the controller:
./trex-consoleStep 3: Saturate the 10G Link
Now, let’s load a profile. We will use a standard IMIX profile to push 10Gbps of traffic through port 0:
start -f stl/imix.py -p 0 -m 10gbpsType tui to open the text user interface. Watch the opackets (output) and ipackets (input) carefully. If the output is 10Gbps but the input is only 8Gbps, you have just discovered that your device under test is dropping 20% of its traffic.
Simultaneous Throughput and Latency Analysis
High throughput is useless if latency spikes to unplayable levels. Most routers introduce jitter long before they start dropping packets. TRex can measure this by interleaving high-speed traffic with specialized latency probes.
To test this, run a benchmark at 90% load with latency tracking enabled:
start -f stl/bench.py -p 0 -m 90% --latencyI recently used this to troubleshoot a core switch. While it handled 9Gbps easily, the latency jumped from 15μs to 450μs at that load. This revealed a bufferbloat issue that a simple throughput test would have missed.
Hard-Won Lessons from the Lab
Over the last year, I have learned that the environment matters as much as the software. Here are three rules to live by:
- The Back-to-Back Rule: Always connect TRex ports directly to each other first. If you cannot hit 10Gbps in a loopback, your server or PCI settings are the problem, not your router.
- Respect NUMA: On multi-socket servers, ensure your NIC and TRex processes stay on the same CPU socket. Crossing the QPI/UPI bus can slash your performance by 30%.
- Watch the CPS: When testing stateful firewalls, focus on Connections Per Second (CPS). Many firewalls crash while trying to set up 50,000 new sessions per second, even if the total bandwidth is low.
Final Thoughts
Moving from basic tools to TRex changed how we plan our network capacity. We no longer guess if a firewall can handle peak load; we have the hard data to prove it. While setting up DPDK requires a bit of a learning curve, the results are professional-grade. You effectively turn a standard Linux box into a high-end traffic generator that would otherwise cost tens of thousands of dollars in proprietary hardware.
