Skip to Content
API ReferenceWriting Records

Writing Records

Append records to a topic. A single POST /v0/topics/:topic carries one or many records, the server assigns each a monotonic $seq and a $ts, and the whole batch commits atomically — all of it or none of it. The same endpoint can lazily create the topic and carry an idempotency key, so the common write path needs no separate calls.

This page covers the write request body, atomicity, full-topic behavior, the response shape, the hard batching limits, and the errors. For how durable a write actually is, see Durability; for retry-safe writes, see Idempotency.

POST /v0/topics/:topic — append records

POST/v0/topics/:topic

Append one or many records. The server assigns the seqs and returns them. Inline config (applied only on a lazy create) means the common case needs no separate PUT.

Path:topic, name ^[A-Za-z0-9][A-Za-z0-9._:-]{0,254}$ (1–255 chars, starts alphanumeric, allows . _ : -). Case-sensitive, byte-exact.

Request body

curl -X POST $TOPICS/v0/topics/orders \ -H 'content-type: application/json' \ -d '{ "node": "checkout-1", "idempotency_key": "client-batch-7f3a", "records": [ { "data": { "sku": "AEROPRESS-GO", "qty": 1, "total": 3499 }, "tag": "order-7731", "meta": { "trace": "abc123" } }, { "data": "raw-opaque-or-base64", "tag": "order-7732" } ] }'
# → 200 OK (or 201 if this write created the topic) { "topic": "orders", "first_seq": 480232, "last_seq": 480233, "seqs": [480232, 480233], "head_seq": 480233, "count": 2, "created": false, "deduped": false, "performance": { "server_total_ms": 0.62, "wal_append_ms": 0.10, "fsync_ms": 0.39 } }
FieldTypeReq?DefaultMeaning
recordsarrayyes1..=MAX_BATCH_RECORDS. Seqs assigned in array order, contiguously.
records[].dataany JSONyesOpaque payload, stored and returned verbatim. May be JSON null.
records[].tagstringnoMatch key for deletion. ≤ MAX_TAG_BYTES.
records[].nodestringnobatch nodePer-record origin node; overrides the batch-level node. ≤ MAX_NODE_BYTES.
records[].metaobjectnoSmall headers, returned verbatim. ≤ MAX_META_BYTES, ≤ 64 keys.
nodestringnoBatch-level default origin node (the common one-writer case). Recorded as $node.
idempotency_keystringnoDedupe key scoped to this topic. ≤ 256 chars. May also be sent as header Idempotency-Key: (the body field wins).
createboolnotopic’s auto_createOverrides auto_create for this write only. true ⇒ auto-create if absent; false404 topic_not_found if absent (the Redis NOMKSTREAM analog — prevents typo-topics).
configTopic confignoApplied only if this write creates the topic. Ignored if the topic already exists (config changes go through PUT).
disable_backpressureboolnofalseIf true, opt out of 429 throttled for this write; the server may queue it instead, trading latency for not failing (a trusted-loader opt-out).

node is the load-bearing field for fan-out. A record’s $node is recorded at write time, and on later reads or SSE presenting that same node, these records are filtered out — so N symmetric nodes can write to and watch a shared topology without echo. See node loop-prevention.

Atomicity

A write is all-or-nothing. All N records in records commit with contiguous seqs, or none do — there is no partial append. first_seq and last_seq bound the assigned range, and seqs[i] is the seq of records[i].

For a durable topic the response is held until the write is committed per the topic’s durability class: an fsync topic holds the ack until the WAL frame is durably synced; a disk topic acks on group-commit enqueue; a memory topic acks the same way as disk (on WAL-frame enqueue) but is best-effort — not fsync-gated and with no durability guarantee. A non-fsync topic never blocks on fsync.

Full-topic behavior

When the topic is at its cap_records / cap_bytes limit, the discard policy decides what happens to the incoming write:

discardOn a full topicWriter sees
"old" (default)Evicts the oldest records to make room; the write succeeds.200/201 as normal. The eviction may later surface to a lagging consumer as a tombstone.
"reject"Refuses the write; nothing is appended (all-or-nothing).422 topic_full synchronously — learned at write time, never ack-then-drop.

discard:"reject" is the durable-queue choice: unconsumed work fails the producer loudly rather than being silently dropped. discard:"old" is the pub/sub choice: the newest data always lands, and any consumer that fell below the eviction floor gets an explicit tombstone on its next read.

Response

{ "topic": "orders", "first_seq": 480232, "last_seq": 480234, "seqs": [480232, 480233, 480234], "head_seq": 480234, "count": 3, "created": false, "deduped": false, "performance": { "server_total_ms": 0.62, "wal_append_ms": 0.10, "fsync_ms": 0.39 } }
FieldTypeMeaning
topicstringThe topic name.
first_sequ64Seq of the first record in this batch.
last_sequ64Seq of the last record in this batch.
seqsarrayPer-record assigned seqs, in records order. Suppressed with ?return_seqs=false (then only first_seq/last_seq are returned — useful for huge batches).
head_sequ64Highest assigned seq in the topic after this write (the log end).
countintNumber of records appended by this call.
createdbooltrue only when this call brought the topic into existence (paired with 201).
dedupedbooltrue when an idempotency_key matched a prior in-window write; seqs are then the original seqs and no new append happened.
performanceobjectTiming block. fsync_ms is 0.0 for non-durable topics; clients must tolerate any subset of its fields.

deduped: true is the safe-retry signal. If you replay a write with the same idempotency_key inside the topic’s idempotency_window_ms (default 120000), you get the original seqs back with deduped: true and the log is not double-appended. A log has no “previous value,” so a dedupe key — not a CAS condition — is the right append-retry primitive. See Idempotency.

Batching limits

Hard, documented limits. Design your batches within them; exceeding one is a 400 (batch_too_large / record_too_large) or, for the raw body, a 413. These are fixed protocol limits — only the raw request-body cap is tunable, via TOPICS_MAX_BODY_BYTES (default 64 MiB); see Configuration.

LimitValue
Max records per write10000
Max single record data+meta1 MiB
Max total request body64 MiB (env-tunable: TOPICS_MAX_BODY_BYTES)
Max meta per record16 KiB, ≤ 64 keys
Max tag length256 bytes
Max node length128 bytes

Errors

error.codeHTTPWhen
invalid_request400Bad JSON, bad field type, or a value out of range.
batch_too_large400records exceeds MAX_BATCH_RECORDS.
record_too_large400A single record’s data+meta exceeds MAX_RECORD_BYTES, or a per-field cap (meta/tag/node) is exceeded.
topic_not_found404create:false (or auto_create:false) and the topic does not exist.
payload_too_large413The raw request body exceeds TOPICS_MAX_BODY_BYTES (rejected pre-parse).
unsupported_media_type415Missing or wrong Content-Type (must be application/json).
topic_full422Write to a full discard:"reject" topic. Nothing was appended.
throttled429Elastic throttle under CPU pressure, or a resource cap reached (carries Retry-After). Opt out per-write with disable_backpressure. See Errors.

See also

  • Reading (diff) — consume what you wrote with POST /v0/topics/:topic/diff.
  • Durability — what an acknowledged write survives, per class.
  • Idempotencyidempotency_key, the dedupe window, and deduped.
  • Topics — create and configure a topic with PUT, and the full config object.
Last updated on