# Streaming responses

> Stream replies token by token with server-sent events in both formats, get token usage in the last chunk, and handle timeouts, disconnects and errors mid-stream.

Source: https://oneaiapikey.com/docs/streaming

With `"stream": true` the reply arrives as [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) while the model writes it, instead of as one JSON body at the end. Use it for anything a person watches: the first words appear much sooner, and long answers do not run into idle timeouts.

Streaming costs the same as a normal request. You pay for the same tokens.

## OpenAI format

Set `stream: true` on [chat completions](https://oneaiapikey.com/docs/chat-completions). Add `stream_options: {"include_usage": true}` to get the token counts in one extra chunk at the end.

curl:

```bash
curl -N https://api.oneaiapikey.com/v1/chat/completions \
  -H "Authorization: Bearer $RELAYKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6-terra",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "Write a haiku about latency."}]
  }'
```

Python:

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.oneaiapikey.com/v1", api_key=os.environ["RELAYKIT_API_KEY"])

stream = client.chat.completions.create(
    model="gpt-5.6-terra",
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:  # last chunk, choices is empty
        print("\n", chunk.usage)
```

Node:

```js
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.oneaiapikey.com/v1", apiKey: process.env.RELAYKIT_API_KEY });

const stream = await client.chat.completions.create({
  model: "gpt-5.6-terra",
  messages: [{ role: "user", content: "Write a haiku about latency." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\n", chunk.usage); // last chunk, choices is empty
}
```

On the wire, each event is a `data:` line with one JSON chunk. The text is in `choices[0].delta.content`; the last content chunk carries `finish_reason`; the usage chunk has an empty `choices` array; `data: [DONE]` ends the stream.

text/event-stream:

```text
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"gpt-5.6-terra","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"gpt-5.6-terra","choices":[{"index":0,"delta":{"content":"Packets"}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"gpt-5.6-terra","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"gpt-5.6-terra","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":19,"total_tokens":33}}

data: [DONE]
```

> **Usage only when you ask for it:** Without `stream_options.include_usage`, a streamed response has no usage chunk. The request is still metered and billed in full; you just do not see the counts in the stream. They are on the [usage page](https://oneaiapikey.com/console/usage) either way.

Reasoning models may stream their thinking first, as `delta.reasoning_content`, before the answer arrives in `delta.content`. See [Reasoning models](https://oneaiapikey.com/docs/reasoning).

## Anthropic format

Set `"stream": true` on [`/v1/messages`](https://oneaiapikey.com/docs/anthropic-messages). The events are Anthropic’s own: `message_start`, then `content_block_start`, `content_block_delta` (text in `delta.text`), `content_block_stop` for each block, `message_delta` with `stop_reason` and the output token count, and `message_stop`. The Anthropic SDKs assemble them for you.

curl:

```bash
curl -N https://api.oneaiapikey.com/v1/messages \
  -H "x-api-key: $RELAYKIT_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "max_tokens": 512,
    "stream": true,
    "messages": [{"role": "user", "content": "Write a haiku about latency."}]
  }'
```

Python:

```python
import os
import anthropic

client = anthropic.Anthropic(base_url="https://api.oneaiapikey.com", api_key=os.environ["RELAYKIT_API_KEY"])

with client.messages.stream(
    model="claude-sonnet-5-5",
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()
print("\n", final.usage)
```

Node:

```js
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ baseURL: "https://api.oneaiapikey.com", apiKey: process.env.RELAYKIT_API_KEY });

const stream = client.messages.stream({
  model: "claude-sonnet-5-5",
  max_tokens: 512,
  messages: [{ role: "user", content: "Write a haiku about latency." }],
});
stream.on("text", (text) => process.stdout.write(text));
const final = await stream.finalMessage();
console.log("\n", final.usage);
```

Usage is always included in this format: input tokens in `message_start`, output tokens in `message_delta`.

## Tool calls in a stream

When a model calls a tool while streaming, the arguments arrive in pieces. In the OpenAI format they are in `delta.tool_calls[i].function.arguments`, keyed by `index`; in the Anthropic format they are `input_json_delta` events. Join the pieces and parse the JSON only after the call is complete (`finish_reason: "tool_calls"` or `content_block_stop`). The SDK helpers above do this for you. See [Tool calling](https://oneaiapikey.com/docs/tool-calling).

## Timeouts and disconnects

- **Idle timeout.** The gateway ends a stream if the provider sends nothing for 300 seconds. Very long thinking phases on reasoning models are the usual cause; a lower `reasoning_effort` or a smaller `max_tokens` helps.
- **Client timeouts.** Set your HTTP client’s read timeout above the longest gap you expect between chunks, not above the total response time. A stream can run for many minutes while chunks keep arriving.
- **If you disconnect,** the gateway closes the request to the provider. You are billed for the tokens produced up to that point, not for the full `max_tokens`.
- **Proxies and buffering.** If you relay the stream through your own server, turn off response buffering (for nginx: `proxy_buffering off`) or the text arrives in one lump at the end.

## Errors during a stream

Errors that happen before the first byte (bad key, unknown model, low balance) come back as a normal JSON error with a 4xx or 5xx status; see [Errors](https://oneaiapikey.com/docs/errors). Once the stream has started the status is already `200`, so a later failure shows up as an `error` object inside a `data:` event, or the stream ends without `finish_reason` / `message_stop`. Treat a stream that ends without one of those as incomplete, and retry it if your app needs the full answer.

## Related

- [Chat completions](https://oneaiapikey.com/docs/chat-completions) and [Anthropic Messages](https://oneaiapikey.com/docs/anthropic-messages): the request fields.
- [Rate limits and retries](https://oneaiapikey.com/docs/rate-limits): when to retry, and how.
- [Performance metrics](https://oneaiapikey.com/docs/performance): first-token time and tokens per second.