Saving with cache

Why input dwarfs output

Every call resends the entire context: system prompt, conversation history, attached documents. The model remembers nothing between calls — it only reads what you send this time. So a long conversation grows its input steadily while output stays one answer long.

That is why the bill sits on the input side.

What cache does

When you resend a stretch of context byte-identical to last time, it is read from cache instead of reprocessed. In models, the Cache read column is far below Input — for Claude, roughly one tenth.

Caching works automatically, including through /v1/chat/completions. There is nothing to enable.

How to benefit

1. Put the fixed part first. System prompt, reference docs, few-shot examples — keep them together and do not inject a timestamp or random value in the middle. One changed character invalidates everything after it.
2. Keep the prefix stable across turns. Append new messages at the end; don't edit what came before.
3. Trim history once it is long. A 50-turn conversation usually needs the last 10 turns plus a summary. Many tools call this compact.
4. Set max_tokens. It caps the output side, which is the expensive side on Claude models.

Verify it

GET /v1/usage reports input, output and cached tokens per call. If you restructure a prompt and the cached number does not rise, something in the prefix is still changing.

curl https://revidapi.com/v1/usage -H "X-API-Key: sk_YOUR_KEY"