Quick Start: Deploying Kyverno in 5 Minutes
Six months ago, our platform team needed admission control across 14 production clusters running roughly 600 microservices. Our engineers already juggled Helm, Kustomize, and Terraform. Forcing them to learn Open Policy Agent’s Rego DSL triggered immediate pushback. We turned to Kyverno, an open-source policy engine that treats policies as standard Kubernetes manifests.
Deploying Kyverno takes under two minutes using Helm:
# Add the official Kyverno Helm repository
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
# Install high-availability admission controllers
helm install kyverno kyverno/kyverno \
--namespace kyverno \
--create-namespace \
--set admissionController.replicas=3Once all three controller pods show Running, you can enforce your first security rule. Here is a cluster-wide policy blocking root containers:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-run-as-non-root
spec:
validationFailureAction: Enforce
background: true
rules:
- name: check-run-as-non-root
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Root containers are forbidden. Set securityContext.runAsNonRoot to true."
pattern:
spec:
securityContext:
runAsNonRoot: trueSave this manifest as require-non-root.yaml and apply it with kubectl apply -f require-non-root.yaml. Kyverno will reject any deployment missing that field straight at the API server admission phase.
Architecture: Why YAML-Native Policies Win
Gatekeeper treats Kubernetes resources as arbitrary JSON documents and parses them through Rego queries. Kyverno takes a different route. It models policies as Kubernetes Custom Resource Definitions (CRDs) matching standard workload schemas.
How Admission Webhooks Work
Kyverno hooks directly into the kube-apiserver admission lifecycle via dynamic webhooks:
- Mutating Webhook: Injects defaults before schema validation. It can add required labels, inject sidecar proxies, or set safe security contexts automatically.
- Validating Webhook: Evaluates incoming payloads against active rules. It admits compliant objects or returns actionable error strings directly to your developers.
- Background Controller: Scans existing cluster state against newly applied rules every 60 minutes. It generates
PolicyReportobjects so security teams can audit drift without breaking running workloads.
The 4 Core Rule Types
Kyverno handles four distinct operational scenarios:
- Validate: Checks resource fields using overlay patterns or Common Expression Language (CEL) expressions.
- Mutate: Modifies objects via Strategic Merge Patches or RFC 6902 JSON patches.
- Generate: Clones or creates supporting resources—such as default
NetworkPolicyorResourceQuotamanifests—when a trigger resource appears. - Verify Images: Validates Cosign and Sigstore supply-chain signatures against trusted public keys before container runtime pull.
Advanced Automation: Mutation and Resource Generation
Blocking broken manifests is only half the battle. Good platform engineering fixes small misconfigurations automatically so developers can ship without friction.
Auto-Injecting Secure Defaults
Don’t force developers to remember every Linux capability flag. Use a mutation rule to strip dangerous privileges automatically:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: mutate-pod-security-defaults
spec:
rules:
- name: drop-all-capabilities
match:
any:
- resources:
kinds:
- Pod
mutate:
patchStrategicMerge:
spec:
containers:
- (name): "*"
securityContext:
capabilities:
drop:
- ALLAutomating Multi-Tenant Isolation
Whenever an engineer spins up a new namespace, our clusters automatically provision isolation barriers. Kyverno generates a default-deny network policy without requiring custom Go operators:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: auto-generate-namespace-security
spec:
rules:
- name: generate-default-deny-networkpolicy
match:
any:
- resources:
kinds:
- Namespace
generate:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
name: default-deny-ingress
namespace: "{{request.object.metadata.name}}"
synchronize: true
data:
spec:
podSelector: {}
policyTypes:
- IngressBecause synchronize: true is active, Kyverno acts as a self-healing controller. If someone deletes or edits this generated NetworkPolicy, Kyverno restores the baseline within seconds.
4 Hard-Won Lessons from Production
Rolling out admission controllers across active clusters can trigger major outages if handled carelessly. Here is how we rolled out 45 policies safely across 14 clusters:
1. Start in Audit Mode for 14 Days
Never roll out a policy in Enforce mode on day one. Always set validationFailureAction: Audit first:
# Inspect violations surfaced by the background scanner
kubectl get clusterpolicyreport -o wide
kubectl describe clusterpolicyreport cpolr-4bf9aAuditing gives product teams two full sprint cycles to update their manifests before Kyverno blocks live CI/CD deployments.
2. Protect System Namespaces
A policy error in kube-system can break node initialization, CNI plugins, or CoreDNS. Always exempt critical namespaces explicitly:
spec:
rules:
- name: require-labels
match:
any:
- resources:
kinds:
- Pod
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
- monitoring
- ingress-nginx3. Shift Left with the Kyverno CLI
Catching policy failures at the cluster level frustrates engineers. Move policy evaluation into pre-commit hooks and GitHub Actions using the standalone CLI:
# Run policies against local manifests
kyverno apply ./policies/ --resource ./deployments/service.yamlIn our CI pipeline, this check takes under 400ms and catches 95% of policy issues before code merges to main.
4. Monitor Webhook Overhead Closely
Every admission request adds network round-trips to the API server. In our clusters, baseline admission latency sits under 15ms per request. Set Prometheus alerts on kyverno_admission_review_duration_seconds. If p99 latency spikes above 200ms, check for unoptimized wildcard pattern matches and scale your admission replicas.
