What prompt caching actually does and when it saves money
"Prompt caching" sounds like it should mean the API remembers your conversation so you don't have to resend it. That's not quite it — you always resend the full conversation history on every request (these APIs are stateless). What caching actually does is let the provider skip re-processing the parts of that resent text it's already seen, and charge less for the parts it skips.
The mechanism: prefix matching
Caching works on an exact-prefix basis. If a request's initial content (system prompt, tool definitions, and however much of the conversation you mark) is byte-for-byte identical to a request sent recently, the provider reuses the internal computation for that shared prefix instead of redoing it. Change a single character anywhere in that prefix — a timestamp, a reordered field, a different tool list — and everything after that point in the prefix stops matching, cache or no cache.
This is why caching quietly fails on setups that look fine: a system prompt that
interpolates datetime.now(), or JSON serialized without sorted keys, produces a slightly
different prefix on every single request, so nothing ever matches and every request pays
full price with no visible error — it just silently never hits.
What it actually costs and saves
Caching isn't free — writing something into the cache costs more than a normal token (roughly 1.25x for a short-lived cache, more for a longer-lived one), and reading from cache costs a small fraction of normal price (roughly a tenth). So the economics are: pay a premium once to write, then pay much less on every subsequent request that reuses it. It only pays off once you're making more than a couple of requests against the same prefix — a one-off request that will never be repeated gains nothing from caching and just eats the write premium.
Where it actually helps
- A long, mostly-static system prompt reused across many separate requests (a support bot, a coding agent with a large fixed instruction set) — the system prompt gets written to cache once, then read cheaply on every subsequent call.
- Long multi-turn conversations — each new turn can reuse everything before it, so a long agent session gets cheaper per-turn as it goes, not more expensive.
- A large shared document multiple different questions get asked against — upload/cache it once, ask many things about it for a fraction of the cost each time.
Where it doesn't help
- Requests whose content is different from the start every time — there's no shared prefix to cache.
- A single one-off request that's never repeated — you'd just be paying the write premium for nothing.
- Content placed before the part that actually varies, if the varying part comes first in the request — caching only protects what comes before the first change, so volatile content needs to go at the end, not the beginning.
How to tell if it's actually working
Usage responses report this directly: a field for tokens written to cache and a separate field for tokens read from cache. If the "read from cache" number stays at zero across requests that should share a prefix, something in that prefix is changing between calls — that's the signal to go looking for a timestamp, a random ID, or non-deterministic serialization sitting somewhere it shouldn't be.
Sources: Anthropic's Claude API documentation on prompt caching (prefix-match behavior, write/read pricing multipliers, minimum cacheable prefix length, and the diagnostic usage fields).