# Prompt caching

> How repeated prompt prefixes are cached and billed at the lower cached rate, how to mark cache breakpoints for Claude, and how to check cache hits in usage.

Source: https://oneaiapikey.com/docs/prompt-caching

When many requests start with the same text (a long system prompt, a document, tool definitions, the earlier part of a chat), the provider can reuse its work on that prefix. The reused part is billed at the model’s **cached input** rate, which is far below the normal input rate.

## Cached rates

| Model id | Input / 1M | Cached input / 1M |
| --- | --- | --- |
| [claude-opus-5-5](https://oneaiapikey.com/models/claude-opus-5-5) | $4.40 | $0.22 |
| [claude-sonnet-5](https://oneaiapikey.com/models/claude-sonnet-5) | $2.20 | $0.22 |
| [claude-haiku-4-5](https://oneaiapikey.com/models/claude-haiku-4-5) | $1.10 | $0.11 |
| [gpt-6.1-sol](https://oneaiapikey.com/models/gpt-6-1-sol) | $2.20 | $0.11 |
| [gpt-5.6-sol](https://oneaiapikey.com/models/gpt-5-6-sol) | $5.50 | $0.55 |
| [gpt-5.6-terra](https://oneaiapikey.com/models/gpt-5-6-terra) | $2.20 | $0.22 |
| [grok-4.6](https://oneaiapikey.com/models/grok-4-6) | $2.20 | $0.55 |
| [claude-sonnet-5-5](https://oneaiapikey.com/models/claude-sonnet-5-5) | $2.20 | $0.22 |
| [grok-4.7](https://oneaiapikey.com/models/grok-4-7) | $2.20 | $0.55 |
| [deepseek-v4-flash](https://oneaiapikey.com/models/deepseek-v4-flash) | $0.24 | $0.0048 |
| [deepseek-v4-pro](https://oneaiapikey.com/models/deepseek-v4-pro) | $1.32 | $0.044 |

Models not in this table bill repeated input at their normal input rate.

## How you are billed

- **Cache reads**: the cached part of the prompt is billed at the cached rate above. The rest of the prompt is billed at the normal input rate, and output as usual.
- **Cache writes (Claude only)**: writing a prefix to Claude’s cache is billed at 1.25 times the input rate for the default 5-minute cache, and 2 times the input rate for the 1-hour cache (`"ttl": "1h"`). A write pays for itself after one or two reads.
- **Other providers** do not charge extra for writes. Caching is automatic there, so you only see the discount.
- A cache entry expires after a few minutes without use (about 5 minutes for Claude’s default). Nothing is billed for keeping it.

Your [usage page](https://oneaiapikey.com/console/usage) shows the cost of each request with the cached tokens already priced in.

## Automatic caching (non-Claude models)

The other providers in the table cache long prompt prefixes on their own, typically once a prompt passes about 1,024 tokens. You do not change anything in the request. To get more hits:

- Put what stays the same first (instructions, documents, tool definitions, examples) and what changes last (the user’s question).
- Keep that prefix byte-for-byte identical: no timestamps, request ids or shuffled examples at the top.
- Send related requests close together; caches expire after a few idle minutes.
- On OpenAI models, `prompt_cache_key` (any stable string, for example a user or document id) helps route similar requests to the same cache.

Check `usage.prompt_tokens_details.cached_tokens` in the response to see how much was a hit:

A cache hit in the OpenAI format:

```json
"usage": {
  "prompt_tokens": 5120,
  "completion_tokens": 180,
  "total_tokens": 5300,
  "prompt_tokens_details": { "cached_tokens": 4864 }
}
```

## Claude: mark what to cache

Claude caches only the prefixes you mark with `cache_control`, and the marker exists only in Anthropic’s format. Use the [Anthropic Messages](https://oneaiapikey.com/docs/anthropic-messages) endpoint and put `"cache_control": {"type": "ephemeral"}` on the last block of the stable part (a system block, a tool, or a message). Everything up to and including that block is cached.

Caching a long document with Claude:

```python
import anthropic, os

client = anthropic.Anthropic(base_url="https://api.oneaiapikey.com", api_key=os.environ["RELAYKIT_API_KEY"])
handbook = open("handbook.md").read()   # a long document you ask many questions about

def ask(question: str):
    return client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        system=[
            {"type": "text", "text": "Answer questions about the company handbook."},
            {"type": "text", "text": handbook, "cache_control": {"type": "ephemeral"}},  # cache up to here
        ],
        messages=[{"role": "user", "content": question}],
    )

first = ask("How many vacation days do new staff get?")
print(first.usage.cache_creation_input_tokens, "tokens written to the cache")
second = ask("What is the policy on remote work?")
print(second.usage.cache_read_input_tokens, "tokens read from the cache")
```

- Claude only caches prefixes above a minimum length (around 1,024 tokens on most models; shorter prefixes are processed normally and not cached).
- You can set up to four breakpoints, for example one after the tools and one after the documents.
- `usage.cache_creation_input_tokens` is what was written (billed at the write rate), `usage.cache_read_input_tokens` what was read (billed at the cached rate), and `usage.input_tokens` the rest.

> **Chat completions does not cache on Claude:** When you call a Claude model through `/v1/chat/completions`, the gateway translates the request to Anthropic’s format and does not carry `cache_control` markers over. Those requests get no cache hits and pay the full input rate for the whole prompt. For long, repeated prompts on Claude, use `/v1/messages`.

## When caching helps

| Workload | Benefit |
| --- | --- |
| Coding agents and long chats (the history is resent every turn) | Large: most of every request is the same prefix. |
| Questions over one long document or codebase | Large, if the document comes first. |
| Many short, unrelated prompts | None: there is nothing to reuse. |
| Batch jobs with a shared instruction block | Moderate, if requests run close together. |

## Related

- [Cost control](https://oneaiapikey.com/docs/cost-control): other ways to spend less.
- [Models](https://oneaiapikey.com/docs/models#current-models): all rates.