Context & Why PAM Matters
Every time you log into a Linux box via SSH, invoke sudo, or switch users with su, PAM (Pluggable Authentication Modules) handles the gatekeeping. Instead of forcing every daemon to write custom authentication routines, Linux hands the job over to PAM. It acts as a modular broker between user-facing services and the underlying security backends.
PAM organizes authentication workflows into four distinct facilities:
- auth: Confirms identity, typically by prompting for a password or TOTP verification token.
- account: Checks access eligibility. It verifies password expiration, active time windows, and account lock states.
- password: Manages credential updates, including token rotation and complexity enforcement.
- session: Handles setup and teardown tasks around a shell session, like mounting user storage, applying ulimits, and updating audit logs.
Relying exclusively on SSH keys is standard practice. However, an unpassphrased key on a lost laptop leaves your infrastructure wide open. Adding a Time-based One-Time Password (TOTP) module into PAM’s auth stack solves this. Attackers then need both the private key file and an ephemeral six-digit code from a mobile device.
On a fleet of 2-vCPU Ubuntu nodes running edge workloads, this setup verified local tokens in roughly 8 to 12 milliseconds. That shaved off the 250ms latency spikes we previously saw when querying remote directory servers on every handshake.
Installation
We will use the open-source Google Authenticator PAM package. It implements RFC 6238 and pairs reliably with standard authenticator apps such as Aegis, 1Password, Google Authenticator, and Bitwarden.
Debian and Ubuntu
Refresh your repository index and pull down the PAM module:
sudo apt update
sudo apt install -y libpam-google-authenticatorRHEL, Rocky Linux, and AlmaLinux
Enable the EPEL repository first, then install the package:
sudo dnf install -y epel-release
sudo dnf install -y google-authenticatorConfiguration
Locking down SSH requires three concrete phases: generating your user secret, updating the PAM service definition, and reconfiguring the OpenSSH daemon.
Step 1: Generate the TOTP Secret Key
Log in as the regular non-root user who needs SSH access. Run the generator wizard directly in your terminal:
google-authenticator -t -d -f -r 3 -R 30 -qHere is what these flags do under the hood:
-t: Configures standard time-based tokens (TOTP) rather than counter-based tokens (HOTP).-d: Blocks token reuse to neutralize replay attempts.-f: Writes state directly to~/.google_authenticatorwithout prompting for file paths.-r 3 -R 30: Enforces rate-limiting. Users get at most 3 authentication attempts every 30 seconds.-q: Runs quietly, skipping interactive confirmations while generating the secret.
The utility prints a base32 secret, an ASCII QR code, and five emergency backup codes. Add the secret to your mobile authenticator app. Store the 8-digit emergency codes in an offline password manager like KeePassXC; they are your only backdoor if your phone dies.
Step 2: Configure PAM for SSH
Open the SSH PAM profile in your text editor:
sudo nano /etc/pam.d/sshdMost default configurations include a standard password prompt. On Debian and Ubuntu, look for @include common-auth. On RHEL, look for auth substack password-auth. If your goal is SSH key plus TOTP—without prompting for a Linux user password—comment out that include line:
# Disable fallback to standard UNIX passwords
# @include common-authNext, append the authenticator module to the file:
auth required pam_google_authenticator.so nullokThe nullok parameter lets users log in normally before they have generated a TOTP token. It prevents lockouts during rollout. Once your entire team scans their QR codes, drop nullok to enforce MFA across the board.
Step 3: Enforce Key + MFA in OpenSSH
Now, adjust the daemon configuration in /etc/ssh/sshd_config:
sudo nano /etc/ssh/sshd_configSet these directives to coordinate OpenSSH with PAM:
# Enable PAM and interactive input prompts
UsePAM yes
KbdInteractiveAuthentication yes
PasswordAuthentication no
# Demand a public key challenge FIRST, then the PAM TOTP prompt
AuthenticationMethods publickey,keyboard-interactiveNote: If you are running OpenSSH versions older than 8.7, use ChallengeResponseAuthentication yes instead of KbdInteractiveAuthentication yes.
The AuthenticationMethods line does the heavy lifting. OpenSSH must validate your public key before it exposes the keyboard-interactive prompt for your authenticator code.
Test the syntax before applying changes. If valid, reload the service:
sudo sshd -t && sudo systemctl restart sshdStep 4: Stop Brute-Force Attacks with Account Lockouts
MFA protects individual credentials, but noisy brute-force attempts waste CPU cycles. You can throttle them using pam_faillock. Open /etc/security/faillock.conf:
# Lock users out after 4 failed attempts within 15 minutes
deny = 4
fail_interval = 900
unlock_time = 600
even_deny_rootWith these values, 4 consecutive failed logins freeze the account for 10 minutes (600 seconds). Botnets running credential-stuffing tools hit a brick wall immediately.
Verification & Troubleshooting
Keep your current root terminal session open. Open a fresh terminal tab on your workstation to test your new configuration. If anything breaks, you can fix typos in the first shell without getting locked out.
Step 1: Test the Login Sequence
Initiate an SSH connection with verbose output:
ssh -v [email protected]Your client will offer its private key, receive a partial success signal, and then prompt for the dynamic token:
Authenticated with partial success.
Verification code:Type your 6-digit authenticator code and hit enter. Your shell should launch immediately.
Step 2: Inspect Live PAM Logs
Track PAM events as logins occur. On Ubuntu and Debian systems, tail the auth log:
sudo tail -f /var/log/auth.log | grep -E "sshd|pam"On RHEL, Fedora, or Rocky Linux, query the systemd journal:
sudo journalctl -u sshd -fWhen someone mistypes a code, PAM logs the explicit failure context:
pam_google_authenticator(sshd:auth): Invalid verification code for user deployer
Failed keyboard-interactive/pam for deployer from 198.51.100.24 port 48122 ssh2Step 3: Inspect and Reset Lockouts
If an engineer mistypes their token and triggers a lockout, inspect their fail status:
sudo faillock --user deployerYou will see a timestamped log of each failed attempt. Clear the lock to restore access immediately:
sudo faillock --user deployer --resetPairing pam_faillock with mandatory TOTP gives you solid defense-in-depth on Linux hosts without requiring third-party SaaS identity brokers.
