Posted inDevOps

Kubernetes Autoscaling with Karpenter: Cut EKS Provisioning Times from 5 Minutes to 45 Seconds

The Problem with Traditional EKS Autoscaling

Traffic spikes on AWS Elastic Kubernetes Service (EKS) expose an annoying problem: slow node spin-up times. When sudden traffic bursts hit, pods sit in a Pending state for 3 to 7 minutes waiting for compute capacity. If you run latency-sensitive APIs, those minutes of lag mean dropped connections, 504 errors, and frustrated users.

The Bottleneck: Auto Scaling Group Overhead

Kubernetes Cluster Autoscaler (CAS) served as the standard scaling engine for years. But CAS cannot launch compute instances on its own. Instead, it relies on an intermediary: AWS EC2 Auto Scaling Groups (ASGs).

That extra layer introduces noticeable friction. When pods cannot find an available node, CAS scans your static node groups, updates the desired capacity on a matching ASG, and waits for EC2 to respond. If your cluster runs mixed workloads—such as GPU nodes for machine learning, ARM64 Graviton instances for web workers, and Spot instances for batch processing—you quickly end up managing 15 to 25 separate ASGs. This setup causes two main issues:

  • High Provisioning Latency: Waiting for ASG lifecycle signals, EC2 instance initialization, and kubelet bootstrapping takes between 3 and 6 minutes.
  • Wasted Cloud Spend: Pods rarely fill fixed instance sizes cleanly. A service requesting 3.2 vCPU on a fixed 4-vCPU instance leaves 20% of the box idle, yet AWS bills you for the full server.

How Karpenter Solves This

Karpenter removes Auto Scaling Groups from the equation. It monitors pending pods directly through the Kubernetes API server and calls ec2:CreateFleet to launch instances matching your exact pod requirements. Nodes register and become ready in roughly 45 seconds. By packing pods densely onto optimal instance types, Karpenter routinely cuts compute bills by 30% to 50%.

Quick Start (5 min)

Follow these steps to deploy Karpenter on an existing EKS cluster with Helm and configure dynamic node provisioning.

1. Install Karpenter via Helm

Confirm that your EKS cluster has OpenID Connect (OIDC) or EKS Pod Identity enabled, along with the required IAM roles. Then install the Karpenter chart:

export KARPENTER_VERSION="1.0.0"
export CLUSTER_NAME="production-eks-cluster"

helm install karpenter oci://public.ecr.aws/karpenter/karpenter \
  --version "${KARPENTER_VERSION}" \
  --namespace karpenter \
  --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "serviceAccount.annotations.eks\.amazonaws\.com/role-arn=arn:aws:iam::123456789012:role/KarpenterControllerRole" \
  --wait

2. Define the EC2NodeClass and NodePool

Karpenter v1 relies on two primary Custom Resource Definitions (CRDs):

  • EC2NodeClass: Configures AWS-level settings, including subnets, security groups, AMI family, and storage volumes.
  • NodePool: Sets pod scheduling boundaries, instance architectures, purchase options, and disruption policies.

Apply this manifest to enable dynamic node provisioning:

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2023
  role: KarpenterNodeInstanceRole
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: production-eks-cluster
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: production-eks-cluster
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-compute
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
  limits:
    cpu: 1000
    memory: 2000Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

3. Test Dynamic Scaling

Create a test deployment that requests 10 replicas with substantial CPU allocations:

kubectl create deployment inflate --image=public.ecr.aws/eks-distro/kubernetes/pause:3.7 --replicas=10
kubectl set resources deployment inflate --requests=cpu=1,memory=1Gi

Tail the Karpenter controller logs to watch it evaluate pending requests, query the AWS Fleet API, and spin up an optimal node right away:

kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

Deep Dive: Karpenter vs. Cluster Autoscaler

Why is Karpenter so much faster than Cluster Autoscaler? The key difference lies in the scheduling model.

How Karpenter Schedules Nodes

Cluster Autoscaler works reactively within rigid group boundaries. When a pod is pending, CAS checks whether an existing ASG can host it. If a group matches, CAS increments that ASG’s capacity and waits for AWS EC2 Auto Scaling to complete its launch lifecycle.

Karpenter acts as a proactive scheduling assistant. Its control loop runs in four clear stages:

  1. Grouping: Karpenter batches pending pods based on affinities, taints, tolerations, and requested resources.
  2. Instance Selection: The controller checks EC2 pricing and instance capabilities in real time. It calculates the most cost-effective instance configuration that fits the entire batch.
  3. Direct Provisioning: Karpenter triggers ec2:CreateFleet directly. It bypasses ASG launch templates, warm pool delays, and polling overhead.
  4. Automatic Registration: The EC2 instance joins the EKS control plane, and pods bind as soon as the node becomes ready.

Automated Bin-Packing and Node Consolidation

Traditional node groups often run at only 30% to 45% CPU utilization. Reclaiming slack capacity usually requires custom draining scripts or separate third-party controllers.

Karpenter solves this through active consolidation. With consolidationPolicy: WhenEmptyOrUnderutilized, it constantly monitors cluster density. For example, if traffic drops at night and three pods on a $0.38/hr m6i.2xlarge can fit onto a $0.096/hr c6i.large, Karpenter executes a rolling migration:

  1. It launches the replacement c6i.large node.
  2. It drains and cordons the m6i.2xlarge safely.
  3. It terminates the larger node once all pods run on the new instance.

Advanced Usage

1. Prioritizing Cost with Spot and Graviton

Pairing AWS Graviton (ARM64) with EC2 Spot instances can lower compute expenses by 60% to 80% compared to On-Demand x86 nodes. For instance, a c7g.xlarge Spot instance often costs around $0.05 per hour versus $0.17 per hour for an On-Demand c6i.xlarge.

You can enforce this strategy directly inside a NodePool:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: cost-optimized
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
      expireAfter: 720h
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s
    budgets:
      - nodes: "20%"

2. Drift Detection and Automated Node Upgrades

Updating worker node AMIs in standard EKS setups requires updating launch templates, tweaking ASGs, and manually cycling instances. Karpenter handles configuration drift natively. When you update the AMI tag or security group in your EC2NodeClass, Karpenter marks older nodes as drifted. It replaces them gradually while honoring your disruption budget, keeping your workloads online throughout the rollout.

Production Best Practices

Here are four operational safeguards learned from managing high-throughput EKS clusters:

  • Diversify Spot Instance Types: Avoid locking Karpenter to a single instance family like c5.xlarge. Allow multiple families such as c6i, c7g, m6i, and m7g. A broad selection prevents launch failures when AWS availability zones experience Spot capacity crunches.
  • Enforce Pod Disruption Budgets (PDB): Karpenter consolidates aggressively. Without PDBs, quick node drains can cause brief service interruptions. Always configure a PDB with minAvailable: 1 or maxUnavailable: 20% on mission-critical services.
  • Isolate the Controller on On-Demand Nodes: Run Karpenter’s controller pods on an On-Demand managed node group or AWS Fargate. If Karpenter runs on a Spot node that gets reclaimed, the cluster cannot provision replacement compute until the controller restarts elsewhere.
  • Audit Subnet and Security Group Tags: Karpenter discovers VPC subnets and security groups through AWS tags like karpenter.sh/discovery: <cluster-name>. Make sure these tags exist on all private subnets and target security groups in your Terraform or OpenTofu manifests.
Share:
Tags: