Prompt Caching Cost for AI Agents: Anthropic, OpenAI, Gemini

A 12-step agent run on Claude Sonnet 5 costs about $0.39 without caching and about $0.14 with it. On GPT-5.4 the same run is about $0.50 uncached and about $0.17 cached. The reason is simple: an agent re-sends its whole context on every step, and cached input reads are billed at roughly 10% of the list input rate on Anthropic and OpenAI, roughly 25% on Gemini. Caching does nothing for output tokens.

Those numbers use a realistic trace: a 4,000-token prefix (system prompt plus tool definitions), a 500-token task, and twelve steps where each step returns a 300-token tool call and gets back a 1,500-token tool result. The full arithmetic is below.

Every step in a tool-calling loop sends the entire conversation so far: the system prompt, every tool schema, the task, every earlier tool call and every earlier tool result. Step 1 sends 4,500 tokens. Step 12 sends 4,500 + 11 × 1,800 = 24,300 tokens. Add the twelve steps together and the model has read 172,800 input tokens to produce only 4,800 output tokens.

Almost all of that 172,800 is repeated text. The prefix is identical on every call and the conversation only grows at the end. That is exactly the shape a prefix cache is built for, which is why the per-step re-send that makes agents expensive is also what makes caching so effective. The general shape of agent bills is covered in [how much an AI agent costs to run](/guides/how-much-does-an-ai-agent-cost-to-run); this article is only about the cache.

Anthropic renders the request in the order tools, then system, then messages, and the cache is a strict prefix match. A breakpoint caches everything before it, so one breakpoint on the last tool definition and one on the system prompt covers the stable part. Add a third on the latest user message and each step reads the previous conversation from cache too. Anthropic's minimum cacheable prefix is model dependent (around 1,000 tokens or more), so short prompts silently do not cache. See the per-vendor detail in [Claude agent API cost](/guides/claude-agent-api-cost), [OpenAI Agents API cost](/guides/openai-agents-api-cost) and [Gemini agent API cost](/guides/gemini-agent-api-cost).

Put the things that never change during the run at the front, in a fixed order:

• The system prompt, with no date, no user name and no run ID inside it
• Tool definitions, sorted deterministically and never added to mid-run
• Retrieved documents, style guides or schemas the agent consults on every step
• Few-shot examples

Put everything that varies after the last breakpoint: the task, the user's question, per-request metadata. On Anthropic, if you want a timestamp or a run ID in context, put it in the first user message rather than the system prompt, so it sits after the cached prefix instead of inside it.

Uncached, the run reads 172,800 input tokens and writes 4,800 output tokens. Cached, we split the input into three buckets. New tokens are written to the cache once: 4,500 at step 1 and 1,800 at each of the next eleven steps, 24,300 in total. Everything else, 148,500 tokens, is a cache read.

All per-million rates; cache rates are the "roughly 10% / 25%" rule applied to list and the Anthropic write premium is roughly 125%. The write premium is small because each token is written once and read up to eleven times. For a 40-step run the saving climbs past 80%, because the share of the input that is repeated grows with every step. Try your own trace in the [calculator](/calculator).

Anthropic, explicit breakpoints on the last tool and the system prompt, plus one on the latest message so the conversation is read from cache:

from anthropic import Anthropic
client = Anthropic()
tools[-1]["cache_control"] = {"type": "ephemeral"}      # caches every tool before it too
messages[-1]["content"][-1]["cache_control"] = {"type": "ephemeral"}
resp = client.messages.create(
    model="claude-sonnet-5", max_tokens=2048,
    system=[{"type": "text", "text": SYSTEM_PROMPT,
             "cache_control": {"type": "ephemeral"}}],
    tools=tools, messages=messages,
)
u = resp.usage
# Sonnet 5 per 1M: input $2, write ~$2.50, read ~$0.20, output $10
cost = (u.input_tokens * 2 + u.cache_creation_input_tokens * 2.5
        + u.cache_read_input_tokens * 0.2 + u.output_tokens * 10) / 1_000_000

OpenAI, where caching is automatic and you only verify it:

from openai import OpenAI
client = OpenAI()
resp = client.responses.create(
    model="gpt-5.4", instructions=SYSTEM_PROMPT, tools=tools,
    input=conversation, prompt_cache_key="research-agent-v3",
)
u = resp.usage
cached = u.input_tokens_details.cached_tokens
# GPT-5.4 per 1M: input $2.50, cached read ~$0.25, output $15
cost = ((u.input_tokens - cached) * 2.5 + cached * 0.25
        + u.output_tokens * 15) / 1_000_000

Log cost per step. If the cached figure is zero on step 2, stop and find the invalidator before running anything longer.

• A timestamp, date or "today is" line in the system prompt. It changes every call, so the prefix never matches.
• Changing tool order or the tool set between steps. Tools render first, so a reordered tool invalidates the system prompt and the whole conversation behind it.
• Serialising tool schemas from a dict with unstable key order. Sort keys before you send.
• A request or trace ID inside the system prompt. Put it in metadata or the first user message.
• On Anthropic, editing an earlier message (for example truncating an old tool result) rather than appending. The cache ends at the first changed byte.
• Switching models mid-run. Caches are per model; a Haiku 4.5 worker cannot read a Sonnet 5 cache.
• On Gemini, forgetting to delete an explicit cache. Storage bills per hour whether or not you call it.

Caching does not change which model you should use; it changes how much the expensive one costs. For a well-specified tool loop, Gemini 3.8 Flash at $0.75 / $3.75 or DeepSeek V4 Flash at $0.05 / $0.16 finish the 12-step run for $0.15 and under $0.01 respectively before any caching. See [cheapest LLM for AI agents](/guides/cheapest-llm-for-ai-agents) for the reliability trade-offs.

Pay for Claude Opus 5 ($5 / $25) or GPT-5.5 ($5 / $30) when the loop involves planning, ambiguous instructions or code that must compile first time, because a retry on a cheap model re-sends the whole context and can cost more than getting it right once. With caching, the uncached 12-step Opus run at $0.98 becomes roughly $0.35, which is less than uncached Sonnet 5.

Caching lowers the slope, not the ceiling. Keep four controls on every agent:

• max_tokens per call, sized to the longest legitimate tool call, not the model's maximum
• A hard step limit per run; 12 to 25 steps covers most tasks
• A stop condition the model can hit, such as a finish tool, so it does not loop on "let me check one more thing"
• A per-run dollar budget computed from the usage fields in the snippets above, aborting when the running total passes it

The [token budget guardrails](/guides/ai-agent-token-budget-guardrails) article has a 30-line runner that does the last one. If your agent connects several MCP servers, read [MCP server token cost](/guides/mcp-server-token-cost) next, because a large tool list is the prefix that benefits most from caching and hurts most without it.