Agent skill

litestream

SQLite disaster recovery and streaming replication to cloud storage (S3, GCS, Azure, SFTP, NATS). Use this skill for configuring Litestream, deploying to cloud platforms, troubleshooting WAL replication issues, implementing point-in-time recovery, and setting up VFS read replicas.

View SKILL.md on GitHub Repository
Stars 163
Forks 31

Install this agent skill to your Project

npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/development/litestream-benbjohnson-litestream-skills

SKILL.md

Litestream

Litestream is a disaster recovery tool for SQLite that runs as a background process, continuously replicating database changes to cloud storage. It monitors the SQLite Write-Ahead Log (WAL), converts changes to immutable LTX files, and uploads them to your chosen storage backend.

Key Concepts

  • WAL Monitoring: Watches SQLite Write-Ahead Log for changes at configurable intervals (default 1s)
  • LTX Files: Immutable files containing database page changes, never modified after creation
  • Level-Based Compaction: Multi-level system (L0-L9) that merges LTX files to reduce storage overhead
  • Point-in-Time Recovery: Restore to any transaction using TXID or timestamp
  • VFS Support: Read replicas directly from cloud storage without local database copy
  • MCP Server: AI tool integration for automated database management

Quick Start

Installation

bash
# macOS
brew install litestream

# Linux (Debian/Ubuntu)
wget https://github.com/benbjohnson/litestream/releases/download/v0.5.5/litestream-v0.5.5-linux-amd64.deb
sudo dpkg -i litestream-v0.5.5-linux-amd64.deb

# Docker
docker pull litestream/litestream:0.5

Basic Replication

bash
# Command line (single database)
litestream replicate /path/to/db.sqlite s3://bucket/db

# With config file
litestream replicate -config /etc/litestream.yml

Restore Database

bash
# Restore latest backup
litestream restore -o /path/to/restored.db s3://bucket/db

# Point-in-time recovery
litestream restore -timestamp 2024-01-15T10:30:00Z -o /tmp/db s3://bucket/db

# Restore to specific transaction
litestream restore -txid 1000 -o /tmp/db s3://bucket/db

Check Status

bash
# View all databases and their replication status
litestream status

# List available LTX files
litestream ltx /path/to/db.sqlite

Critical Rules

These rules are essential for correct Litestream operation:

  1. Lock Page at 1GB: SQLite reserves the page at offset 0x40000000 (1GB). This page must always be skipped during replication and compaction. The page number varies by page size:

    • 4KB pages: page 262145
    • 8KB pages: page 131073
    • 16KB pages: page 65537

  2. LTX Files are Immutable: Once created, LTX files are never modified. New changes create new files.

  3. Single Replica per Database: Each database can only replicate to one destination. Use multiple Litestream instances for multiple destinations.

  4. WAL Mode Required: SQLite must be in WAL mode (PRAGMA journal_mode=WAL;)

  5. Use litestream ltx: The litestream wal command is deprecated.

Storage Backends

Backend URL Scheme Example
AWS S3 s3:// s3://bucket/path
S3-Compatible (R2, Tigris, MinIO) s3:// s3://bucket/path?endpoint=host:port
Google Cloud Storage gs:// gs://bucket/path
Azure Blob Storage abs:// abs://container@account/path
SFTP sftp:// sftp://user@host:22/path
NATS JetStream nats:// nats://host:4222/bucket
WebDAV webdav:// webdav://host/path
Alibaba OSS oss:// oss://bucket/path
Local File (path) /var/backups/db

Configuration File Example

yaml
dbs:
  - path: /data/app.db
    replica:
      url: s3://my-bucket/app-backup
      access-key-id: ${AWS_ACCESS_KEY_ID}
      secret-access-key: ${AWS_SECRET_ACCESS_KEY}
      region: us-east-1
      sync-interval: 1s

When to Use This Skill

Use Litestream skill when:

  • Configuring SQLite database replication to cloud storage
  • Setting up disaster recovery for SQLite applications
  • Troubleshooting replication issues (WAL growth, sync failures)
  • Implementing point-in-time recovery procedures
  • Deploying Litestream with Docker, Fly.io, Kubernetes, or systemd
  • Understanding WAL-based replication concepts

Skill Contents

  • concepts/ - Architecture, replication mechanics, LTX format, SQLite WAL, VFS support, compaction
  • configuration/ - Storage backend configurations (S3, GCS, Azure, SFTP, NATS, OSS, WebDAV, File)
  • commands/ - CLI command reference (replicate, restore, status, ltx, databases)
  • operations/ - Monitoring, troubleshooting, recovery, and heartbeat monitoring
  • deployment/ - Docker, Fly.io, Kubernetes, and systemd deployment guides
  • integrations/ - MCP server integration for AI tools
  • scripts/ - Validation and diagnostic helper scripts

Common Issues

WAL Growing Too Large

  • Check sync interval (default 1s may be too slow for write-heavy workloads)
  • Verify storage backend connectivity
  • Check for checkpoint blocking (long-running transactions)

Replication Lag

  • Monitor with litestream status
  • Check network connectivity to storage backend
  • Review sync interval configuration

Restore Failures

  • Verify backup exists: litestream ltx /path/to/db
  • Check storage backend credentials
  • Ensure target directory is writable

Links

Didn't find tool you were looking for?

Be as detailed as possible for better results