Prompt caching: the cheapest LLM cost cut most apps miss

Prompt caching cuts the price of repeated input by 90% on OpenAI's current models, and on those same models it can also make a request 25% more expensive. GPT-5.6 Sol charges $4.00 per million input tokens, $0.40 per million cached input tokens and $5.00 per million cache-write tokens. Which of those rates you pay depends almost entirely on how your application orders its prompt. This guide covers how the cache works, the ordering rules that make it hit, how to read OpenAI's new dashboard and diagnostics, and a worked saving built only from published prices.

Most of what follows applies to any provider that caches prompt prefixes. The numbers and parameter names are OpenAI's, as of September 2026.

What is prompt caching?

Prompt caching is the reuse of work the model already did on the start of a prompt. When a model processes input it computes key-value (KV) states, and the cache stores those states for a reusable prefix: the unchanged tokens at the beginning of a request. When a later request starts with the same prefix and finds a matching entry, the model skips recomputing it and only processes what is new.

Three consequences follow from that definition. The cache stores tensors, not text, so there is nothing to look up or invalidate by hand. Only a prefix can be reused, so a single changed token early in the prompt breaks reuse for everything after it. And OpenAI caches the full rendered context, including tool definitions, developer messages, conversation history, images and documents, so all of it counts towards the prefix.

Caching is on by default for supported models. You do not opt in; you either build prompts that let it work, or you pay full price without noticing.

How much does prompt caching save?

On GPT-5.6 and later, cache reads cost 0.1× the uncached input rate and cache writes cost 1.25×. On older models the discount is model-specific and there is no write charge. The published standard rates per million input tokens:

Model	Input	Cached input	Cache writes	Discount on cached input
gpt-6-astra	$10.00	$1.00	$12.50	90%
gpt-5.6-sol	$4.00	$0.40	$5.00	90%
gpt-5.6-luna	$0.20	$0.02	$0.25	90%
gpt-5.4	$2.50	$0.25	none	90%
gpt-4.1	$2.00	$0.50	none	75%
gpt-4o	$2.50	$1.25	none	50%

Figures are short-context standard rates from OpenAI's pricing page. Two details matter. Sol's $4.00 input price is promotional, available at least through 21 November 2026 according to the changelog entry of 21 August. And if you are still on gpt-4o, moving to a newer model changes your cached-input rate from half the input price to a tenth of it, before any change to your prompts.

The write charge is where caching can cost you

The cache-write rate is not an extra fee on top of input; each input token is billed at exactly one of the uncached, cached or cache-write rates. OpenAI's own arithmetic shows why the write is usually worth it: writing a prefix once and reusing it once costs 1.35× its ordinary input cost, against 2× without caching; one write and nine reads cost 2.15×, against 10×.

The inverse case is the one teams miss. A prefix written and never read costs 1.25× what it would have uncached. In implicit mode, OpenAI places a breakpoint at the end of the latest eligible message, which means content that changes on every request can end up written to the cache and never reused. Explicit-only mode exists for that: content after your last chosen breakpoint is charged at the plain input rate with no write charge.

How does OpenAI prompt caching work?

Five mechanics decide whether a request hits. All of them are in the prompt caching guide.

• Minimum length. A prefix must be at least 1,024 visible input tokens on GPT-5.6 and later before it can be cached. On earlier models the minimum varies with tools, images, schemas, reasoning effort and verbosity.
• Breakpoints. A breakpoint marks the end of a prefix that can be saved. On GPT-5.6 and later you can place up to four cache writes per request explicitly with prompt_cache_breakpoint, or let implicit mode choose.
• Lifetime. On GPT-5.6 and later a cached prefix stays eligible for at least 30 minutes after its latest write or reuse, and each reuse refreshes it without another write charge. Earlier models use prompt_cache_retention with in_memory or 24h.
• Location. Cached states live on individual machines, and traffic above about 15 requests per minute can overflow to machines that do not hold the entry. Caches are not shared across organisations or across regional processing boundaries.
• Settings. Changing model, tools, text.format, reasoning.effort, text.verbosity or parallel_tool_calls can change the rendered prefix and break reuse, even if your messages are identical.

The API you call matters too. OpenAI reports that the Responses API improved cache utilisation by 40% to 80% compared with Chat Completions in its internal tests. That is a vendor benchmark rather than an independent one, but if you are still on Chat Completions it is a reason to measure.

Prompt ordering: how to improve your prompt cache hit rate

The fixes that matter are in how your code assembles the prompt, not in provider settings. The rule is one sentence: stable content first, changing content last, and never rewrite what came before.

Put stable content at the front

System instructions, tool definitions, output schemas and shared reference material go first, in a fixed order. A timestamp, request ID or user name in the system prompt puts a changing token at the very start of the prefix and breaks reuse for every request. OpenAI's guide to moving prompts into code makes the same point: keep static content first and dynamic content later, because cache hits depend on exact prefix matches.

Keep tool definitions identical

Adding, removing or reordering a tool changes the prefix. To switch tools off for one request, set tool_choice: "none"; to restrict them, use allowed_tools. Both keep the supplied tools list byte-for-byte the same.

Append conversation history, do not rewrite it

In multi-turn apps the growing conversation is usually the largest reusable prefix. Summarising, truncating or compacting earlier turns changes that prefix. Sometimes that is still the right call, because fewer input tokens can cost less than a higher hit rate, but make the trade-off on purpose.

Escape the minimum-length trap

A shared prefix just under 1,024 tokens is never cached. OpenAI's guide works through the break-even: across 10 requests, expanding a prefix of at least 221 tokens to 1,024 tokens is cheaper than leaving it short, provided the added material is useful, such as examples or reference text, and your evals stay stable.

Prompt caching cost savings example: a support bot

The inputs below are published GPT-5.6 Sol standard prices. Everything else is a labelled assumption; change them to match your traffic.

• Assumption: a support bot answers single questions against a fixed 4,000-token prefix (instructions, tool definitions and policy text).
• Assumption: each request adds 600 tokens of question and context after the prefix.
• Assumption: 100,000 requests a month.
• Assumption: one cache write for every nine cache reads, the same ratio as OpenAI's ten-request example.
• Assumption: explicit-only mode with a breakpoint after the prefix, so the 600-token tail is billed at the plain input rate.

Input line	Tokens per month	Rate per 1M	Cost
Prefix, no caching	400M	$4.00	$1,600
Prefix, cache writes (10%)	40M	$5.00	$200
Prefix, cache reads (90%)	360M	$0.40	$144
Per-request tail, either way	60M	$4.00	$240

Without caching, monthly input costs $1,600 + $240 = $1,840. With caching it costs $200 + $144 + $240 = $584, a 68% cut in input spend. Output tokens are unaffected and are not included. If the same bot ran in implicit mode and the 600-token tail were written on every request without ever being read, that tail would cost 60M × $5.00 = $300 instead of $240. That $60 is small here, and it grows with longer, more variable tails.

The weak assumption is the write-to-read ratio. Low traffic spread across long gaps lets entries expire, and very high traffic can overflow onto machines without the entry. Measure your real ratio before you budget on it; the next section is how.

Why is my prompt cache not hitting?

OpenAI has shipped two tools since 20 August that answer this directly. On 20 August it released the Prompt Caching dashboard, which tracks hit rate over time, cache reads per write, and the split between cache-read, cache-write and uncached tokens, filterable by model and service tier. On 8 September Prompt Cache Diagnostics became generally available in the Responses API for GPT-5.6 and later.

Diagnostics compare a request against an earlier response. You set prompt_cache_options.comparison_response_id to a baseline response's ID, usually the previous turn, and read prompt_cache_diagnostics on the new response. A miss comes back with a reason: model_changed, tools_changed, text_format_changed, reasoning_effort_changed, verbosity_changed, service_tier_changed, prompt_cache_key_changed, context_compacted or input_changed. Diagnostics have no additional cost, work with Zero Data Retention, and never change the response. They report only the first reason, so fix it and compare again.

Two of those reasons tend to surprise people. model_changed catches A/B tests and fallback routing that quietly send a request to another model. service_tier_changed matters because the tier that processed a request can differ from the one you asked for. For day-to-day tracking, log usage.input_tokens_details.cached_tokens and cache_write_tokens per request and compute hit rate as cached tokens over input tokens, as the caching guide recommends. We cover wider tracing and spend caps in our post on AI agent observability and cost controls.

When prompt caching will not help much

Caching is not a universal fix, and it is worth being clear about when it does little. If your prompts are short and unique, below 1,024 tokens with nothing shared, there is no prefix to cache. If output dominates your bill, a 90% cut on input moves the total less than the headline suggests: on Sol, output is $20.00 per million against $4.00 for input. And if traffic is so sparse that entries expire between requests, you mostly pay write rates.

In those cases the bigger levers are model choice and routing, which we compared in our LLM cost optimization analysis, or the hosting decision covered in our look at self-hosted LLM cost.

What to do this week

1. Open the Prompt Caching dashboard and note your current hit rate and reads per write, by model.
2. Remove timestamps, IDs and user data from system prompts and move them to the end.
3. Freeze tool definitions and their order; use tool_choice and allowed_tools instead of editing the list.
4. On GPT-5.6 and later, add an explicit breakpoint after the stable prefix and consider explicit-only mode for variable tails.
5. Turn on diagnostics for a sample of traffic and fix the first miss reason it reports.

Prompt caching is usually the cheapest cost cut available to an existing LLM feature because it needs no model change and no quality trade-off, only discipline in how prompts are assembled. If you want a second pair of eyes on that assembly, it is routine work for our AI engineering team.