Prompt caching

When many requests start with the same text (a long system prompt, a document, tool definitions, the earlier part of a chat), the provider can reuse its work on that prefix. The reused part is billed at the model’s cached input rate, which is far below the normal input rate.

Cached rates

Models not in this table bill repeated input at their normal input rate.

How you are billed

• Cache reads: the cached part of the prompt is billed at the cached rate above. The rest of the prompt is billed at the normal input rate, and output as usual.
• Cache writes (Claude only): writing a prefix to Claude’s cache is billed at 1.25 times the input rate for the default 5-minute cache, and 2 times the input rate for the 1-hour cache ("ttl": "1h"). A write pays for itself after one or two reads.
• Other providers do not charge extra for writes. Caching is automatic there, so you only see the discount.
• A cache entry expires after a few minutes without use (about 5 minutes for Claude’s default). Nothing is billed for keeping it.

Your usage page shows the cost of each request with the cached tokens already priced in.

Automatic caching (non-Claude models)

The other providers in the table cache long prompt prefixes on their own, typically once a prompt passes about 1,024 tokens. You do not change anything in the request. To get more hits:

• Put what stays the same first (instructions, documents, tool definitions, examples) and what changes last (the user’s question).
• Keep that prefix byte-for-byte identical: no timestamps, request ids or shuffled examples at the top.
• Send related requests close together; caches expire after a few idle minutes.
• On OpenAI models, prompt_cache_key (any stable string, for example a user or document id) helps route similar requests to the same cache.

Check usage.prompt_tokens_details.cached_tokens in the response to see how much was a hit:

"usage": {
  "prompt_tokens": 5120,
  "completion_tokens": 180,
  "total_tokens": 5300,
  "prompt_tokens_details": { "cached_tokens": 4864 }
}

Claude: mark what to cache

Claude caches only the prefixes you mark with cache_control, and the marker exists only in Anthropic’s format. Use the Anthropic Messages endpoint and put "cache_control": {"type": "ephemeral"} on the last block of the stable part (a system block, a tool, or a message). Everything up to and including that block is cached.

import anthropic, os

client = anthropic.Anthropic(base_url="https://api.oneaiapikey.com", api_key=os.environ["RELAYKIT_API_KEY"])
handbook = open("handbook.md").read()   # a long document you ask many questions about

def ask(question: str):
    return client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        system=[
            {"type": "text", "text": "Answer questions about the company handbook."},
            {"type": "text", "text": handbook, "cache_control": {"type": "ephemeral"}},  # cache up to here
        ],
        messages=[{"role": "user", "content": question}],
    )

first = ask("How many vacation days do new staff get?")
print(first.usage.cache_creation_input_tokens, "tokens written to the cache")
second = ask("What is the policy on remote work?")
print(second.usage.cache_read_input_tokens, "tokens read from the cache")

• Claude only caches prefixes above a minimum length (around 1,024 tokens on most models; shorter prefixes are processed normally and not cached).
• You can set up to four breakpoints, for example one after the tools and one after the documents.
• usage.cache_creation_input_tokens is what was written (billed at the write rate), usage.cache_read_input_tokens what was read (billed at the cached rate), and usage.input_tokens the rest.

When caching helps

Related

• Cost control: other ways to spend less.
• Models: all rates.