Skip to Content
DeploymentRunning topics

Running topics

topics runs as one self-contained server: the complete /v0 API backed by a write-ahead log on local disk, with no external dependencies — no database, no broker, no sidecar. You build it once, point it at a data directory, and run it. This page covers what the process does on start, how it binds and authenticates, and how it shuts down.

Build and run

The binary is named topics. Build it with a Rust toolchain, then run it. On a loopback bind with no keys configured it starts in dev mode (auth disabled) — the default — so the Quickstart commands work verbatim.

# Build the single binary. cargo build --release # Run it. Defaults: bind 127.0.0.1:4000 (loopback), auth disabled (dev mode); # WAL + segments + snapshots live under ./topics-data. ./target/release/topics

If you’d rather not build from source, run the published container image instead — see Docker.

What happens on start

The process is a state machine that recovers before it serves. On start it:

Opens the data directory

It opens TOPICS_DATA_DIR (default ./topics-data), which holds the WAL, segment files, and snapshots. A missing or empty directory is a clean first start.

Loads the latest snapshot

A snapshot is a checkpointed image of all topic state. Loading it lets recovery skip replaying the entire WAL from the beginning.

Replays the WAL forward

From the snapshot’s checkpoint, it replays every WAL frame in order — re-applying appends, deletes, and watermark moves — and truncates any torn tail (a frame whose length runs past end-of-file, or whose XXH3-64 checksum doesn’t match). A complete, checksum-valid frame for an acknowledged durable write is never lost.

Gates readiness until replay completes

Until recovery finishes, GET /v0/ready returns 503 not_ready with a Retry-After header and a detail.replay_progress value in [0.0, 1.0]. Once recovery completes it returns 200 { "status": "ready", "wal_replay_complete": true, "topics": N }.

Put GET /v0/ready (alias /readyz) in front of any load balancer or orchestrator readiness check. It returns 503 during WAL replay and only flips to 200 once an acknowledged durable write is fully recovered — so traffic never hits a partially-recovered server. GET /v0/health (alias /healthz) is a liveness probe and returns 200 as soon as the process is serving.

Binding and dev-mode auth

The default bind is loopback (127.0.0.1:4000), so an unconfigured server is never accidentally a public, unauthenticated event store. Two rules follow from that:

  • On a loopback bind with no TOPICS_API_KEYS, the server runs in dev mode with auth disabled. This is the local-development default.
  • Binding a non-loopback address (anything other than 127.0.0.1) with no keys makes the server refuse to start — it logs the reason loudly — unless you set TOPICS_ALLOW_INSECURE_NO_AUTH=1. That flag is for local/dev only; never set it on a network-exposed deployment.

Set TOPICS_HOST, TOPICS_PORT, and TOPICS_API_KEYS to bind elsewhere and turn auth on. See Configuration for every variable and Security for the key, scope, and prefix-allowlist model.

topics speaks plain HTTP — it does not terminate TLS, by design. For any non-loopback exposure, run it behind a TLS-terminating reverse proxy, or bind loopback and tunnel. Native TLS is out of scope — terminate it at the proxy. See Security.

Graceful shutdown

On SIGINT or SIGTERM, the server drains gracefully and writes a final snapshot, so a clean restart starts from a current checkpoint with minimal WAL to replay. While draining, GET /v0/ready returns 503 shutting_down.

Because durability is WAL-first, you don’t depend on the shutdown snapshot for correctness: an acknowledged durable (disk/fsync) write is already in the WAL and is recovered by replay even after an abrupt kill. The final snapshot is a recovery-speed optimization, not a correctness requirement. (A memory-class topic takes the same group-committed WAL path as disk but is best-effort — after a restart its records may survive or be lost, with no guarantee either way; its config always survives — see Durability.)

Where next

Last updated on