Time Synchronization Approaches: Chrony vs. NTPd vs. systemd-timesyncd
Unchecked clock drift wreaks havoc across distributed infrastructure. A gap of just 200 milliseconds can fracture Raft consensus in etcd clusters, invalidate short-lived JSON Web Tokens (JWTs), and garble distributed tracing in Jaeger. When choosing a time daemon for Linux, you generally have three routes:
- systemd-timesyncd: A minimalist SNTP client baked directly into systemd. It only pulls time from remote sources and adjusts local clocks periodically. It cannot act as a local time server.
- Legacy NTPd: The venerable reference daemon that powered early UNIX systems. While battle-tested and feature-dense, it takes hours to stabilize after boot. It also struggles when network latency spikes on congested cloud links.
- Chrony (chronyd): A modern, high-performance NTP engine built for virtual machines, cloud instances, and edge devices. It adapts quickly when VM snapshots pause execution, network interfaces drop, or clock frequencies fluctuate.
Comparing the Contenders: Trade-offs in Real Deployments
Each tool targets a different operational profile. Here is how they stack up in everyday production:
1. systemd-timesyncd
Pros: Zero extra packages to install on modern distros; tiny memory footprint (~3MB RSS); zero maintenance for simple standalone boxes.
Cons: Cannot serve time to internal VPC subnets; lacks hardware timestamping support; cannot negotiate Network Time Security (NTS) keys; recovers poorly if a clock starts with large offsets.
2. NTPd (Network Time Protocol Daemon)
Pros: Decades of real-world history; broad driver support for specialized hardware reference clocks like radio receivers or atomic standards.
Cons: Very slow convergence—often needing 30 to 60 minutes to pull clock offset under 10 milliseconds; fragile over jittery WAN links; historically prone to plaintext packet spoofing without cumbersome autokey setups.
3. Chrony
Pros: Locks system time within sub-millisecond accuracy in just seconds; manages clock frequency drift with precision; operates seamlessly as a client and an enterprise-grade server; natively supports Network Time Security (NTS) over TLS.
Cons: Replaces the distribution default daemon, meaning older shell scripts that scrape ntpq output must be updated to use chronyc.
A Production-Tested Architecture
For high-availability fleets, avoid letting hundreds of instances poll external public pool servers independently. Instead, build a two-tier synchronization model:
- VPC Edge Time Relays: Deploy at least two internal Chrony relay nodes per cloud region. Configure them to pull time from public NTS-secured Stratum-1 and Stratum-2 servers over TLS-authenticated channels.
- Internal Workload Nodes: Configure database servers, Kubernetes nodes, and microservice hosts as Chrony clients targeting your internal relays over private subnets. Include a fallback pool for disaster recovery.
- Strict Drift Policies: Configure
makesteponly for initial machine boots. Enforce gradual clock slewing during runtime. Stepping backwards during normal operations can corrupt write sequences in PostgreSQL, Cassandra, or Prometheus.
Step-by-Step Implementation Guide
Step 1: Install Chrony and Disable Competing Daemons
First, clear out default daemons so nothing fights over UDP port 123:
# On Ubuntu/Debian
sudo apt update && sudo apt install -y chrony
sudo systemctl stop systemd-timesyncd 2>/dev/null || true
sudo systemctl disable systemd-timesyncd 2>/dev/null || true
# On RHEL/Rocky/AlmaLinux
sudo dnf install -y chrony
Step 2: Configure Chrony as an NTS-Secured Client
Network Time Security (NTS) pairs NTP with TLS. It verifies your upstream sources cryptographically and stops man-in-the-middle attacks cold. Edit your configuration file at /etc/chrony/chrony.conf (or /etc/chrony.conf on RHEL-based systems):
# Query diverse, geographically close NTS-enabled Stratum-1/2 servers
server time.cloudflare.com iburst nts
server nts.netnod.se iburst nts
server ptbtime1.ptb.de iburst nts
# Allow the clock to step only during the first 3 updates if offset > 1s
# After that, force smooth slewing
makestep 1.0 3
# Track oscillator frequency drift across reboots
driftfile /var/lib/chrony/drift
# Sync kernel RTC (Real Time Clock) every 11 minutes
rtcsync
# Log clock adjustments
logdir /var/log/chrony
Apply changes and enable the service on boot:
sudo systemctl restart chrony
sudo systemctl enable chrony
Step 3: Serve Time to Your Internal Network
Turn a gateway or bastion host into a local NTP server for your VPC. Open the configuration file and add subnet permissions along with a local fallback stratum:
# Permit sync requests from your private VPC subnet
allow 10.0.0.0/16
# Serve local time even if upstream internet drops temporarily
local stratum 10
# Bind exclusively to the internal interface (e.g., eth1 IP)
bindaddress 10.0.1.5
Activate the server configuration without interrupting active sessions:
sudo systemctl reload chrony
Step 4: Verify NTS Encryption and Monitor Drift
In a test run on an AWS c6i.large instance running Ubuntu 22.04 LTS under a 10,000 req/sec load, Chrony kept clock offset within 18 microseconds of UTC. Memory usage settled under 6MB RSS, while CPU consumption remained beneath 0.1%.
Check your upstream synchronization status and peer health:
chronyc sources -v
Look for an asterisk (*) next to the best source, which shows Chrony has locked on as its primary reference. Then verify NTS authentication:
chronyc authdata
A non-zero cookie count and verified key exchange indicate that cryptographic authentication is functioning properly.
Troubleshooting Production Drift and Network Dropouts
1. Source Unreachable (Reference ID 00000000 / Stratum 0)
Run chronyc tracking to review core metrics:
chronyc tracking
# Output indicating missing upstream link:
# Reference ID : 00000000 ()
# Stratum : 0
# Leap status : Not synchronised
When the Reference ID is blank, network firewalls are usually blocking traffic. NTS requires two open ports: TCP port 4460 for the initial TLS key exchange handshake and UDP port 123 for time packets. Test connectivity using nc:
# Test the NTS key exchange port (TCP 4460)
nc -zvw3 time.cloudflare.com 4460
# Test the NTP packet port (UDP 123)
nc -zuvw3 time.cloudflare.com 123
2. Recovering from Severe VM Pauses
Hypervisor maintenance or live migrations can freeze a guest VM for seconds or even minutes. Once the VM resumes, Chrony by default will slew the clock slowly to avoid breaking running applications. If your workloads require immediate alignment, force a manual step:
sudo chronyc makestep
This skips the gradual slew rate and synchronizes system time immediately.
3. Auditing Jitter and Removing Flaky Peers
Track network jitter and clock stability across peers over time:
chronyc sourcestats -v
Pay close attention to the Std Dev (estimated standard deviation) and Est offset columns. Upstream servers showing standard deviations above 10 milliseconds or high packet loss should be dropped from your configuration in favor of lower-jitter alternatives.
