--- title: "Where prompt caching quietly misses: TTL, prefix order, and what invalidates the cache" canonical: https://maxtokens.ai/posts/prompt-cache-misses/ date: 2026-06-19 tags: [llm, cost] description: "How prompt-cache savings are really computed at Anthropic and OpenAI, and the specific places the cache silently stops working: TTL, prefix order, invalidation." --- OpenAI's headline is "up to 90% cheaper input tokens." It's accurate, and it's the most misread number in the pricing docs. The discount is real, but it only lands on cache hits. Those misses happen more often, and more silently, than the pitch suggests. Anthropic and OpenAI both ship prompt caching with the same promise and different rules: different lifetimes, different ways the prefix is matched, different things that wipe the whole thing without telling you. This is about how the savings are really computed, and the specific places the cache stops working. Every number here comes from Anthropic's and OpenAI's own docs. Nothing is extrapolated. ## 1. The discount is a set of multipliers, and a write costs more than base On Anthropic the whole economy is three numbers relative to the base input price: a 5-minute cache write is 1.25× the base input tokens price, a 1-hour write is 2×, and a cache read is 0.1×. Two of those are above 1.0. Writing to the cache costs 25% more than sending the same tokens uncached, and the one-hour variant costs double. The only cheap number is the read: 0.1× is 90% off, and that discount only ever applies to reads. So the cache saves money only if you read it enough to pay back the write premium. The arithmetic, for a prefix sent once and reused across R requests on the 5-minute tier: ``` no cache: R × 1.0 = R 5-min cache: 1.25 + (R−1)×0.1 = 1.15 + 0.1R → cheaper once R > 1.28 1-hour cache: 2.0 + (R−1)×0.1 = 1.90 + 0.1R → cheaper once R > 2.11 ``` The 5-minute tier is ahead on the very first reuse. The 1-hour tier needs two reuses before it beats sending the tokens raw, because the 2× write is a steeper hole to climb out of. Single-use caching is the trap: if you write the cache and never read it before expiry, you pay 1.25× or 2× for nothing, which is worse than not caching. Whether you read it in time is a TTL question, which is the next problem. ## 2. TTL: the cache is colder than you think Anthropic's default cache lives 5 minutes, and it is refreshed for no additional cost each time the cached content is used. That "refreshed on use" detail is the part people miss: the lifetime is a sliding window, not a fixed countdown. A prefix that gets hit every few minutes stays warm indefinitely at no extra cost; a prefix used once an hour is cold every single time. There's a paid 1-hour option for when 5 minutes isn't enough. OpenAI describes its cache lifetime differently: cached prefixes generally remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour. "Generally" and "up to" are load-bearing words: it's best-effort behavior, not a guarantee you can build a billing model on. Same shape on both sides: caching rewards traffic. A high request rate on a shared prefix means the write amortizes across many reads and the cache stays warm. On Anthropic, low traffic means you keep paying the write premium and rarely collect the read discount, so the economics from section 1 invert at low volume. ## 3. Anthropic matches a prefix hierarchy; changing one level loses the rest Anthropic builds cache prefixes in a fixed order: tools, then system, then messages. That order is a hierarchy where each level builds on the previous ones. The cache stores the full prefix — tools, system, and messages, in that order — up to and including the block you marked with `cache_control`. The invalidation rule falls straight out of the order: a change at any level invalidates that level and all subsequent levels. Edit a tool definition and you don't just lose the tools cache, you lose system and messages too, because they sit downstream of tools in the hierarchy. Anthropic states it plainly: modifying tool definitions (names, descriptions, parameters) invalidates the entire cache. Tools are the top of the hierarchy, so touching them is the most expensive change you can make. Anthropic cache prefix hierarchy tools, system, messages; an edit at one level invalidates that level and every level below it Order your prompt by rate of change. Anything that mutates often belongs late — in messages, near the end — so an edit invalidates as little of the prefix as possible. Anything stable (tool schemas, the system core) belongs first and should change rarely. A single reworded tool description on every request means you cache nothing. ## 4. OpenAI matches only an exact prefix OpenAI's caching turns on automatically for prompts that are 1024 tokens or longer, and the discount is applied to shared prefixes without requiring any change to your API integration. Nothing to mark, no parameter to set. The constraint is one sentence: cache hits are only possible for exact prefix matches within a prompt. The API caches the longest prefix of a prompt that has been previously computed, starting at 1024 tokens and increasing in 128-token increments. So the match is greedy from the front and quantized — it finds the longest run from the very start of the prompt that it has seen before, rounded down to a 128-token boundary. The start of the prompt determines the cache hit. One changed token near the top — a timestamp, a per-user id, a session uuid dropped into the system message — breaks the exact-prefix match at that point, and everything after it is uncached, no matter how much of it is byte-identical to last time. The fix is the same discipline as the Anthropic hierarchy: static content first, anything per-request last. ## 5. The silent failure: too short to cache, no error Both providers have a floor, and Anthropic's is the one that bites quietly. Shorter prompts cannot be cached, even if marked with `cache_control`; any request to cache fewer than the minimum number of tokens is processed without caching, and no error is returned. You set `cache_control`, you get a clean response, and you pay full price — nothing tells you the cache didn't take. The floor depends on the model: 1,024 tokens for Claude Opus 4.8, Claude Sonnet 4.6, and Claude Sonnet 4.5. OpenAI draws the same line from the other direction — caching simply doesn't engage below 1024 tokens. Caching a small system block fails on two counts: it's below the floor so it may not cache at all, and even if it did, section 1 says a write under the break-even reuse count loses money. Cache big stable prefixes, not small ones. ## The checklist - A cache read is 0.1× base input (90% off); a 5-minute write is 1.25×, a 1-hour write is 2×. Writes cost more than not caching. - The 5-minute tier breaks even on the first reuse; the 1-hour tier needs two. A cache written and never read before it expires is strictly worse than no cache. - Anthropic's TTL is 5 minutes, refreshed for free on every hit — a sliding window. A paid 1-hour option exists. - OpenAI's TTL is 5–10 minutes idle, one hour max, best-effort. - Anthropic caches a tools → system → messages hierarchy; a change at one level invalidates it and everything downstream. Editing a tool definition invalidates the entire cache. - OpenAI matches the longest exact prefix from the start, with a 1024-token floor and 128-token steps. One changed token near the top breaks the rest. - Order both prompts by rate of change: stable first, per-request last. - Below ~1024 tokens nothing caches, and Anthropic returns no error — you just pay full price. The "up to 90%" is real. It's a property of the read, not of your integration — you collect it only when the prefix is long, stable, and reused often enough to stay warm. If those conditions aren't met, the discount silently never shows up. And on Anthropic you also pay the write premium for nothing. ## Sources - Anthropic — Prompt caching: [platform.claude.com/docs/en/build-with-claude/prompt-caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) - OpenAI — Prompt caching guide: [developers.openai.com/api/docs/guides/prompt-caching](https://developers.openai.com/api/docs/guides/prompt-caching) - OpenAI — Prompt Caching in the API: [openai.com/index/api-prompt-caching](https://openai.com/index/api-prompt-caching/)