Quick Start: Deploying your first SSH Certificate in 5 minutes
Wrestling with raw SSH keys across 50+ servers isn’t just a chore—it’s a security disaster in the making. After a brute-force attack slammed my root account 4,500 times in a single hour, I realized that traditional key management is too slow and too risky. If you want to see how SSH certificates fix this immediately, we can set up a minimal Certificate Authority (CA) using Smallstep right now.
First, grab the step CLI and step-ca for your control machine. I’m using Ubuntu 22.04 here:
wget https://dl.step.sm/gh-release/cli/gh-release-header/v0.24.4/step-cli_0.24.4_amd64.deb
sudo dpkg -i step-cli_0.24.4_amd64.deb
wget https://dl.step.sm/gh-release/ca/gh-release-header/v0.24.2/step-ca_0.24.2_amd64.deb
sudo dpkg -i step-ca_0.24.2_amd64.debNext, initialize your CA. This command generates the configuration and the root keys that will sign your identity:
step ca init --name="My-Internal-CA" \
--provisioner="[email protected]" \
--dns="localhost" \
--address=":9000"Fire up the CA server in a separate terminal window:
step-ca $(step path)/config/ca.jsonTo sign a user key, run this command:
step ssh certificate [email protected] id_rsaThis creates id_rsa-cert.pub. It’s that simple. Instead of copying public keys to every single node, you tell your servers to trust your CA once and never touch authorized_keys again. It’s a fundamental shift in how you handle infrastructure access.
The Real Cost of Traditional SSH Keys
We’ve all done it: generate an RSA key, run ssh-copy-id, and hope we never have to touch it again. This works for a solo project. But as your team grows, you hit a wall. When a DevOps engineer leaves the company, you have to manually scrub their public key from every production instance. Miss one? You’ve left a permanent backdoor open.
Then there is the “Trust on First Use” (TOFU) headache. You know that scary warning when you SSH into a new server? Most engineers just type “yes” without a second thought. That habit is a gift to attackers looking to launch Man-in-the-Middle (MITM) strikes.
Why Certificates are Superior
SSH Certificates introduce a Third Party: the Certificate Authority. Think of it like a digital passport system. Your servers don’t need to know you personally. They just need to trust the government (the CA) that issued your passport (the Certificate).
- Empty authorized_keys: Servers only need the CA’s public key. If the certificate is signed by that CA, the door opens.
- Role-Based Access: You can embed specific usernames or roles directly into the certificate.
- Built-in Expiration: Issue certificates that last only 8 or 12 hours. If a laptop is stolen at 6 PM, the thief’s access is gone by morning.
The Smallstep Advantage
Smallstep is the engine that makes this practical. It provides step-ca, a lightweight, secure CA designed for automation. Forget manual signing. You can link Smallstep to your existing Identity Provider like Okta, Google, or GitHub. A developer logs in via SSO, and Smallstep hands them a temporary, valid-for-today-only certificate automatically.
Advanced Setup: Automating Host and User Trust
To kill SSH warnings for good, you must configure trust in both directions.
1. Teaching Servers to Trust the CA
Your target Linux servers need to know your CA is legitimate. Grab the CA’s public SSH key first:
step ssh config --ssh-ca-userSave this key to /etc/ssh/trusted-user-ca-keys.pem on the server, then update /etc/ssh/sshd_config:
# Point SSH to your CA key
TrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pemRestart the service with sudo systemctl restart ssh. Now, any user with a CA-signed certificate can jump in. No manual key synchronization required.
2. Killing “Unknown Host” Warnings
Servers need certificates too. You can sign host certificates during provisioning via Terraform or Ansible. In the server’s /etc/ssh/sshd_config, add:
HostKey /etc/ssh/ssh_host_ecdsa_key
HostCertificate /etc/ssh/ssh_host_ecdsa_key-cert.pubOn your local machine, add the CA’s host key to ~/.ssh/known_hosts:
@cert-authority * ecdsa-sha2-nistp256 AAAAE... (your CA public key)Now your SSH client trusts every server the CA signed. No more prompts. No more MITM risk.
3. SSO Integration with ‘step ssh login’
The gold standard is the step ssh login workflow. When your team starts their day, they run one command:
step ssh login [email protected]A browser pops up, they authenticate with their company SSO, and Smallstep injects a 12-hour certificate into their SSH agent. The user never even sees a private key file. It’s seamless and incredibly secure.
Production Realities: Lessons from the Field
Moving to certificates requires a change in strategy. Here is what I’ve learned deploying this in high-traffic environments.
Revocation Strategies
Short-lived certificates (under 16 hours) usually don’t need a Revocation List. If a key is compromised, you just wait for it to expire. For longer-lived credentials, Smallstep supports Key Revocation Lists (KRLs). If a breach happens, you update the KRL on your servers to block a specific ID instantly.
Reliability is King
Do not run step-ca in a random terminal or screen session. Use a systemd unit file. Since the CA is now the heart of your infrastructure, run it in a High Availability (HA) setup or behind a reliable load balancer once your team grows beyond a few people.
The “Break-Glass” Protocol
When you first roll out Smallstep, keep one traditional SSH key in an admin’s authorized_keys. If you misconfigure the CA trust, you don’t want to be locked out of your own data center. Once the automation is rock solid, you can phase out these static keys for good.
Auditing and Visibility
Smallstep logs every single certificate request. This creates a perfect audit trail. You can see exactly who accessed what and when. Pipe these logs into Grafana or an ELK stack for real-time visibility. It’s a massive upgrade over hunting through .ssh folders across a thousand nodes.
Switching to SSH certificates is an investment that pays off in peace of mind. You’re replacing manual, error-prone file management with an identity-driven system that scales. No more midnight security anxiety—just clean, automated access.
