The Hidden Risk of Infrastructure Drift
Anyone who has managed cloud infrastructure knows the 3 AM sinking feeling. You spent weeks perfecting your Terraform modules, ensuring every security group and S3 bucket is defined in code. Then, a production incident hits. A teammate logs into the AWS Console, manually adds an inbound rule to a security group to restore service, and forgets to update the code the next morning.
This is Infrastructure Drift. It is a persistent headache for DevOps teams. Over time, your real-world environment and your Git repository stop matching. When you finally run terraform apply two weeks later, you might accidentally overwrite a critical manual fix. Even worse, you might fail to notice that a rogue developer launched an expensive p3.16xlarge instance that is now burning $24 per hour.
I have implemented drift detection in several production environments. The results were immediate. By using Driftctl, we moved from reactive firefighting to proactive governance.
Why Terraform Plan Often Misses the Full Picture
A common mistake is assuming terraform plan catches everything. It does not. Terraform only tracks resources listed in its state file. If a user manually creates a new RDS database or an IAM user through the console, terraform plan will ignore it completely. The tool simply lacks visibility into resources it didn’t create.
Driftctl fills this visibility gap. It scans your cloud provider (AWS, Azure, or GCP) and compares every discovered resource against your Terraform state. It categorizes findings into three buckets:
- Managed: Resources in your code that match the cloud exactly.
- Drifted: Resources in your code that someone modified manually.
- Unmanaged: Resources existing in the cloud that are missing from your code. This is usually where security vulnerabilities hide.
Setting Up Driftctl
Installation is simple since Driftctl is a Go-based binary. If you are on macOS or Linux using Homebrew, run this command:
brew install driftctl
For CI/CD pipelines, grab the binary directly from the Snyk/Driftctl GitHub releases. The tool uses your existing cloud credentials. If your AWS CLI is already configured with a profile, Driftctl will use it automatically without extra setup.
Scanning Your Infrastructure
The driftctl scan command is your primary tool. To get an accurate result, point the tool toward your remote state file, which is typically stored in an S3 bucket.
# Scan AWS and compare against a remote S3 state file
driftctl scan --from tfstate+s3://my-terraform-state-bucket/project/terraform.tfstate
The first scan is often a wake-up call. In one project, our initial report revealed 42 unmanaged resources. These included forgotten default VPCs, old IAM roles from a 2021 migration, and experimental Lambda functions that were still active.
Filtering Noise with .driftignore
Not every unmanaged resource requires your attention. You might have legacy systems or resources managed by other tools like Kubernetes. To keep your reports clean, create a .driftignore file in your project root.
# .driftignore
# Ignore all default AWS resources that we don't manage
aws_default_vpc.*
aws_default_security_group.*
# Ignore a specific legacy bucket used by the data team
aws_s3_bucket.legacy-archive-2020
# Ignore resources with a specific tag
*::tags.Environment: development
Meaningful alerts are the key to success. If your tool reports 100 false positives every day, your team will eventually stop checking the logs.
Automating Detection in CI/CD
Manual scans are better than nothing, but automation provides real security. I recommend scheduling a drift scan every four hours. This frequency ensures you catch console changes quickly, even if no one has touched the Terraform code in days.
Here is a streamlined GitHub Actions workflow for automated scanning:
name: Infrastructure Drift Detection
on:
schedule:
- cron: '0 */4 * * *'
workflow_dispatch:
jobs:
scan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Install Driftctl
run: |
curl -L https://github.com/snyk/driftctl/releases/latest/download/driftctl_linux_amd64 -o driftctl
chmod +x driftctl
sudo mv driftctl /usr/local/bin/
- name: Run Driftctl Scan
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: "us-east-1"
run: |
driftctl scan --from tfstate+s3://my-prod-bucket/terraform.tfstate --output json://drift-report.json
- name: Alert on Drift
if: failure()
run: |
echo "Drift detected in production! Sending summary to Slack..."
# Insert Slack webhook or SNS notification logic here
Driftctl returns a non-zero exit code when it finds a discrepancy. This behavior makes it easy to trigger the if: failure() condition and notify your team immediately.
Lessons from the Field
After running Driftctl across multiple AWS accounts, I have identified several practices that improve the experience:
- Start Small: Avoid scanning your entire AWS Organization on day one. Focus on a single state file or a specific region first. Clean up that noise before expanding your scope.
- Standardize Tagging: Use consistent tags across all resources. Driftctl can filter by tags, which helps exclude shared resources or assets managed by third-party vendors.
- Prioritize Visibility: A report buried in a CI/CD log is useless. Use the JSON output to push a summary to Slack or PagerDuty. Knowing exactly what changed within hours of a manual edit is a massive advantage.
- Enforce a “Drift Fix” Policy: When drift appears, you have two choices. You can update the Terraform code to match the new reality, or you can revert the manual change. Do not let drift persist for more than 24 hours.
Enforcing the Source of Truth
Infrastructure as Code only provides value if the code accurately reflects production. Without a tool like Driftctl, your Terraform manifests are just a suggestion rather than a definitive record. Automated scanning enforces a “Code First” culture where manual changes are visible and discouraged.
Cleaning up initial findings takes effort, but the peace of mind is worth it. You will prevent “ghost” resources from inflating your bill. More importantly, you ensure that your disaster recovery plans—which depend entirely on your code—will actually work when you need them most.

