IronStratum

Idempotency

Retries are safe here. Send an Idempotency-Key header. The same logical request returns the same logical response. There is no duplicate generation and no double billing. We shipped this for inference because nobody else has it, including OpenAI.

The problem it solves

Inference is not a read. Say a network timeout makes your client retry a chat completion. The retry runs a second generation. You pay twice, and your user may see the answer twice. Payment processors solved this decades ago with idempotency keys. We ported the model to inference.

Semantics

Send a unique string of 8 to 256 bytes as the Idempotency-Key header:

  • Same key, identical request body → the stored original response is replayed. The replay carries the original x-request-id (the ledger row the first execution wrote stays the correlation point) and an Idempotency-Replayed: true header so you can tell a replay from a fresh execution.
  • Same key, different request body → 409 idempotency_key_reused. A key names one request; reusing it for a different one is a client bug we refuse to paper over.
  • Keys are cached for 10 minutes today. That window only ever grows; it will never shrink under a live integration.
  • The store is single-instance today; multi-instance replay with a longer window is scheduled next sprint. Same-key requests that race the cache simply execute: at-least-once, never double-billed beyond the executed turn.

Replay: run it twice

The second curl replays the stored response (watch for Idempotency-Replayed: true and the original x-request-id in the headers; -i shows them):

curl -sS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: original"}],
    "max_tokens": 24
  }'
curl -isS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: original"}],
    "max_tokens": 24
  }'

Both calls return 200 with the same response body. The second one never touched a model.

Conflict: same key, different body

Reusing a key with a different body is a 409:

curl -sS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: different"}],
    "max_tokens": 24
  }'
# expect: 409

What to do

  • Generate a key per logical operation (a UUID or a hash of the user action), not per session, not a constant.
  • Retry the same request (same body, same key) on any timeout, connection reset, or 5xx.
  • Treat 409 idempotency_key_reused as a client bug: your key generation collided across different payloads.