The Anatomy of Automated SSH Scans and Authentication Abuse
Spin up an unmanaged VPS, expose port 22 to the public internet, and watch your logs. Within fifteen minutes, automated bots will start knocking. A typical public-facing server routinely absorbs 1,000 to 5,000 rogue authentication attempts every single day, with scanners cycling through credential dumps for common accounts like root, admin, deploy, and test.
The core bottleneck lies in how SSH works. OpenSSH establishes a TCP connection, completes a cryptographic key exchange, and calculates session keys before it ever checks a password. That handshake is computationally expensive. Multiply it by dozens of requests per minute, and rogue scanners burn CPU cycles, flood your system logs (/var/log/auth.log or the systemd journal), and steadily search for accounts with weak credentials.
Key-based authentication and non-standard ports help, but they don’t eliminate unsolicited traffic. You need an automated boundary defense that drops hostile packets at the kernel level before userland daemons ever touch them.
Filtering Engine Architectures Compared
Bridging log monitors to packet filters isn’t new. For years, administrators leaned on iptables or application-level wrappers. Today, nftables offers a fundamentally better approach.
Here is how the three main packet-blocking strategies behave under heavy attack traffic:
- Fail2ban with Legacy Iptables: Adds an individual
-j DROPrule for each banned IP. When a coordinated botnet hits and your blocklist swells past 5,000 addresses, the Linux kernel must evaluate incoming packets against every rule sequentially. That linear O(N) evaluation adds measurable latency to legitimate connections. - TCP Wrappers (
hosts.deny) / Application Deny Lists: Blocks traffic entirely in userspace. The kernel still completes the three-way TCP handshake and wakes up the daemon process, wasting RAM, socket handles, and context switches. - Fail2ban with Nftables Sets: Writes banned addresses directly into atomic kernel sets. Lookup overhead drops from linear O(N) scans to O(1) hash table lookups. Whether you ban 15 IPs or 50,000, filtering speed stays instantaneous.
Pros and Cons of Nftables Integration
Before standardizing on nftables across your infrastructure, consider the operational tradeoffs.
Advantages
- Predictable CPU and Memory Overhead: Storing thousands of blocked IPs inside native hash sets keeps packet filtering lightweight. On a modest 2 vCPU cloud instance with 4 GB RAM facing a sustained wave of 20,000 bans, kernel CPU usage during packet checks remains virtually flat—unlike iptables, which can push softirq load over 15%.
- Atomic Set Operations: Fail2ban adds and evicts elements in memory dynamically without flushing or rebuilding the active firewall ruleset.
- Unified IPv4 and IPv6 Handling: You can manage dual-stack filtering inside a single
inetfamily table instead of running separateiptablesandip6tablescommands.
Disadvantages
- Tooling Muscle Memory: Familiar commands like
iptables -L -nwon’t show active bans stored inside nftables sets unless your system uses compatibility shims. - Package Requirements: Reliable nftables set actions require Fail2ban 0.11.x or newer. If you maintain legacy systems on ancient distributions, you’ll need backports.
Recommended Environment
This deployment targets current distributions running Linux kernel 5.4 or later—including Debian 11/12, Ubuntu 22.04/24.04, AlmaLinux 9, and Rocky Linux 9. We will pair Fail2ban 1.0+ with native Nftables sets using the nftables-multiport action and feed authentication events straight from the systemd journal.
Implementation Guide
1. Install Required Packages
Install both packages using your system’s package manager:
# On Ubuntu / Debian
sudo apt update
sudo apt install -y fail2ban nftables
# On AlmaLinux / Rocky Linux / RHEL 9
sudo dnf install -y epel-release
sudo dnf install -y fail2ban nftables
Enable and start the nftables packet engine:
sudo systemctl enable --now nftables
2. Configure Fail2ban Global Defaults
Never modify /etc/fail2ban/jail.conf directly; vendor package updates will overwrite your edits. Instead, place your custom rules in a dedicated override file at /etc/fail2ban/jail.local:
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local
Find the [DEFAULT] section and apply these settings to configure trusted IP ranges, ban timers, and the nftables action:
[DEFAULT]
# Exclude localhost and internal VPN/office subnets
ignoreip = 127.0.0.1/8 ::1 10.8.0.0/24 192.168.1.50
# Time window and trigger thresholds
findtime = 10m
maxretry = 5
bantime = 1h
# Exponential backoff: progressively penalize persistent bots
bantime.increment = true
bantime.factor = 2
bantime.maxtime = 4w
# Direct traffic drops into kernel-level nftables sets
banaction = nftables-multiport
banaction_allports = nftables-allports
3. Define the SSH Jail
Keep service jails modular by defining them in separate drop-in files. Create /etc/fail2ban/jail.d/sshd.local:
[sshd]
enabled = true
port = ssh
backend = systemd
maxretry = 3
findtime = 10m
bantime = 24h
4. Verify and Test Filter Rules
Fail2ban parses log streams against regex patterns defined in /etc/fail2ban/filter.d/. Before activating any jail in production, confirm that the regex matches your distribution’s actual log format:
sudo fail2ban-regex systemd-journal /etc/fail2ban/filter.d/sshd.conf
Look at the output summary. If the rule matches failed password attempts and invalid user requests in your journal, you are ready to enforce bans.
5. Start Fail2ban and Inspect Kernel State
Apply your configuration and start the daemon:
sudo systemctl enable --now fail2ban
Verify jail health with fail2ban-client:
sudo fail2ban-client status sshd
Once attackers trigger the threshold, inspect how nftables handles the drops. Instead of a long list of individual rules, nftables manages bans inside a compact named set:
sudo nft list table inet f2b-table
The resulting output shows active elements stored in a fast kernel set:
table inet f2b-table {
set addr-set-sshd {
type ipv4_addr
elements = { 198.51.100.23, 203.0.113.88 }
}
chain f2b-chain-sshd {
type filter hook input priority filter - 1; policy accept;
meta l4proto tcp ip saddr @addr-set-sshd tcp dport 22 drop
}
}
6. Managing Manual Bans and Emergency Unbans
If an engineer accidentally triggers a ban from home or you need to block a scanner manually, use the client directly without touching configuration files:
# Remove an accidental ban
sudo fail2ban-client set sshd unbanip 198.51.100.23
# Immediately ban an abusive IP address
sudo fail2ban-client set sshd banip 203.0.113.99
