Real-Time SQLite Backup to S3 with Litestream: Step-by-Step Guide for Zero-RPO Recovery

Running web applications on a single $5/month VPS with SQLite has become one of my favorite deployment patterns. You drop network latency down to 20 microseconds, slice hosting bills to pennies, and shrink the entire stack down to a single binary. Still, many engineers hesitate. The nagging concern is always data safety: what happens if the host’s NVMe drive bricks, or the cloud provider loses your instance?

Scheduled cron snapshot backups leave a dangerous Recovery Point Objective (RPO) gap. Take an app with 6-hour dump intervals. If the server dies at 5 hours and 50 minutes, you wipe out nearly six hours of user purchases, comments, or account creations. Litestream solves this cleanly. It streams SQLite’s Write-Ahead Log (WAL) frames straight to Amazon S3 or S3-compatible object stores in real time. Here is how to configure Litestream from scratch to achieve sub-second RPO recovery.

Table of Contents

Quick Start (5 Min): Streaming SQLite to S3

Getting Litestream running on Linux takes three minutes and a handful of commands. We will install the binary, set up an S3 bucket target, and verify our first live replication stream.

1. Install Litestream

On Ubuntu or Debian, grab the release package directly from GitHub:

# Download and install Litestream deb package
wget https://github.com/benbjohnson/litestream/releases/download/v0.3.13/litestream-v0.3.13-linux-amd64.deb
sudo dpkg -i litestream-v0.3.13-linux-amd64.deb

# Confirm the installation
litestream version

Deploying on Alpine Linux or Fedora? Download the compiled tarball instead, unpack it, and move the binary to /usr/local/bin/litestream.

2. Configure AWS S3 Credentials and Bucket

Create a dedicated Amazon S3 bucket (for instance, my-sqlite-backups-bucket). Set up an IAM user or role scoped tightly to that bucket. You need four permissions: s3:PutObject, s3:GetObject, s3:ListBucket, and s3:DeleteObject. Export the keys in your shell session:

export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
export AWS_REGION="us-east-1"

3. Replicate and Test Data Restoration

Set up a test database. Enable WAL mode immediately, because Litestream depends on WAL frames to track state changes:

mkdir -p /var/lib/myapp
sqlite3 /var/lib/myapp/app.db "PRAGMA journal_mode = WAL; CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT); INSERT INTO users (name) VALUES ('Alice');"

Kick off replication directly from the terminal:

# Replicate app.db to S3
litestream replicate /var/lib/myapp/app.db s3://my-sqlite-backups-bucket/app.db

Litestream pushes a baseline snapshot to S3, then idles while listening for WAL commits. Open a second terminal window to run a restore drill:

# Restore the latest state to a new file location
litestream restore -o /tmp/restored-app.db s3://my-sqlite-backups-bucket/app.db

# Verify your data survived
sqlite3 /tmp/restored-app.db "SELECT * FROM users;"

Deep Dive: How Litestream Achieves Sub-Second RPO

Before putting Litestream in front of production workloads, you should understand how its synchronization engine actually moves data. Relational engines like PostgreSQL rely on dedicated replication daemons streaming write-ahead logs to read-replicas over TCP. SQLite is fundamentally different. It is an in-process C library writing directly to local disk files.

The Role of SQLite WAL Mode

By default, SQLite runs in rollback-journal mode, locking the database during write operations. Enabling Write-Ahead Logging (PRAGMA journal_mode = WAL;) completely reshapes that I/O pattern. Database writes no longer modify the primary .db file directly. Instead, they append sequentially to a sibling file named .db-wal.

Readers continue querying historical pages from .db without blocking writers, while writers append new pages to the log. At routine intervals, SQLite runs a checkpoint. This flushes pages from .db-wal back into the base file and resets the log. Standard cron backup utilities struggle here: they often copy files mid-checkpoint, leading to torn pages and corrupted backup files.

Continuous Frame Shadowing

Litestream runs as a lightweight daemon beside your app. It inspects SQLite’s WAL index file (.db-shm) using shared-memory lock counters. Whenever a transaction commits, Litestream reads the newly written 4KB frames immediately. Rather than waiting for a checkpoint, it bundles new frames into compressed Litestream Snapshot (LSS) files. It then pushes them to S3 within roughly 100 to 250 milliseconds.

[Application Writes] 
        │
        ▼
  app.db-wal (Local NVMe)
        │
  [Litestream monitors WAL frames]
        │
        ├───────────────┬───────────────┐
        ▼               ▼               ▼
  Snapshot (Base)    WAL Frame 1    WAL Frame 2   --> S3 Storage

This continuous shipping narrows your exposure window from hours down to sub-second durations. Even during bulk operations, Litestream streams changes as fast as disk I/O permits. When testing seed scripts or preparing raw imports (such as piping browser-converted JSON datasets into SQLite tables), every record committed to the WAL makes it to S3 seconds after ingestion completes.

Advanced Usage: Docker, Systemd, and Multi-Replica Setups

A manual terminal command works for local testing, but production servers need systemd supervisors, multi-cloud redundancy, and clean container entrypoints.

1. Configuration via litestream.yml

Manage your databases and replica targets through /etc/litestream.yml:

dbs:
  - path: /var/lib/myapp/app.db
    replicas:
      - type: s3
        bucket: my-sqlite-backups-bucket
        path: app.db
        region: us-east-1
        sync-interval: 1s
        retention: 72h
      # Secondary off-site replica (Cloudflare R2)
      - type: s3
        endpoint: https://<account-id>.r2.cloudflarestorage.com
        bucket: my-dr-backup
        path: app.db
        access-key-id: ${R2_ACCESS_KEY_ID}
        secret-access-key: ${R2_SECRET_ACCESS_KEY}

Litestream natively streams to multiple endpoints simultaneously. In this configuration, committed transactions fan out in parallel to AWS S3 and Cloudflare R2, shielding you from single-provider outages.

2. Managing Litestream with Systemd

Create a persistent service file at /etc/systemd/system/litestream.service:

[Unit]
Description=Litestream SQLite Replication Service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=www-data
Group=www-data
EnvironmentFile=/etc/default/litestream
ExecStart=/usr/local/bin/litestream replicate -config /etc/litestream.yml
Restart=always
RestartSec=5s
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

Put bucket credentials in /etc/default/litestream, lock permissions down to root (chmod 600), reload systemd, and start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now litestream
sudo systemctl status litestream

3. The “Exec” Pattern for Docker Containers

Running Litestream inside Docker does not require complex multi-container orchestration. Litestream provides a process supervisor mode via litestream exec. When the container starts, it downloads the latest database backup if the local volume is empty, executes your application as a child process, and streams WAL frames until shutdown.

Here is a production container entrypoint (run.sh):

#!/bin/sh
set -e

DB_PATH="/data/app.db"

# Restore from S3 if local database file is missing
if [ ! -f "$DB_PATH" ]; then
  echo "No local database found. Restoring from S3..."
  litestream restore -if-replica-exists -o "$DB_PATH" s3://$S3_BUCKET/app.db
fi

# Run Litestream as a supervisor around your main application command
exec litestream exec -config /etc/litestream.yml -- ./my-web-server -db "$DB_PATH"

If the underlying host crashes, spin up a replacement container anywhere. The script spots the empty directory, pulls the database state from S3 down to the final second, and boots your application.

Practical Tips and Troubleshooting from Production

Operating Litestream under sustained traffic requires careful attention to WAL file behavior, monitoring, and cloud storage retention.

Keep SQLite Checkpointing Under Control

SQLite normally triggers an automatic checkpoint once the WAL file hits 1,000 pages (~4MB). Litestream tracks this without issues. However, long-running read queries (such as a 30-second analytical report) hold shared locks, blocking checkpoints. When this happens, the WAL file can balloon past several gigabytes. Watch the file size directly:

# Check current WAL sizes
ls -lh /var/lib/myapp/app.db*

Always configure SQLite connection pools with a busy timeout (such as PRAGMA busy_timeout = 5000;). This prevents transient lock contention from delaying frame processing.

Monitor Sync Health

Verify whether Litestream is tracking writes properly using the generations command:

litestream generations -config /etc/litestream.yml /var/lib/myapp/app.db

This lists backup generations stored in S3 alongside their creation dates. If the list is empty or shows stalled sync timestamps, inspect the daemon logs immediately with journalctl -u litestream -n 100 to catch auth errors or network timeouts.

Set Up S3 Lifecycle Rules

Litestream deletes expired snapshots according to the retention setting in your config (like retention: 72h). Even so, interrupted multipart uploads can leave unreferenced chunk artifacts behind in AWS. Configure an S3 Lifecycle Rule to abort incomplete multipart uploads after 3 days. This stops ghost upload fragments from inflating your monthly AWS bill.

Pairing SQLite with Litestream gives you the sub-millisecond read latency of local NVMe storage along with the disaster recovery guarantees of distributed cloud platforms. You get bulletproof durability without managing the operational overhead of a multi-node database cluster.