Streaming

Set "stream": true to receive the response as it is generated. The response is a stream of server-sent events (SSE). Each event is a line starting with data: that holds a JSON chunk. The stream ends with data: [DONE].

curl -N https://api.router.africa/v1/chat/completions \
-H "Authorization: Bearer $ROUTER_AFRICA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "openai/gpt-4o-mini",
  "stream": true,
  "stream_options": { "include_usage": true },
  "messages": [{ "role": "user", "content": "Write a two-line status message for a delayed parcel." }]
}'

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.router.africa/v1", api_key=os.environ["ROUTER_AFRICA_API_KEY"])

stream = client.chat.completions.create(
  model="openai/gpt-4o-mini",
  messages=[{"role": "user", "content": "Write a two-line status message for a delayed parcel."}],
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in stream:
  if chunk.choices:
      print(chunk.choices[0].delta.content or "", end="")
  if chunk.usage:
      print("\ntokens:", chunk.usage.total_tokens)

import OpenAI from "openai";

const client = new OpenAI({
baseURL: "https://api.router.africa/v1",
apiKey: process.env.ROUTER_AFRICA_API_KEY,
});

const stream = await client.chat.completions.create({
model: "openai/gpt-4o-mini",
messages: [{ role: "user", content: "Write a two-line status message for a delayed parcel." }],
stream: true,
stream_options: { include_usage: true },
});

for await (const chunk of stream) {
if (chunk.choices.length) process.stdout.write(chunk.choices[0].delta.content ?? "");
if (chunk.usage) console.log("\ntokens:", chunk.usage.total_tokens);
}

Token usage

Add "stream_options": {"include_usage": true} to get one last chunk before [DONE]. It has an empty choices array and a usage object with the tokens used.

data: {"id":"gen-8f2c","choices":[],"usage":{"prompt_tokens":38,"completion_tokens":21,"total_tokens":59}}
data: [DONE]

Errors during a stream

An error before the first token is a normal HTTP error. See Errors. Once the stream has started, the status is already 200, so an error is sent as a final event with the same body as other errors, and then the stream ends.

data: {"error":{"code":529,"message":"The model provider is overloaded. Retry shortly.","metadata":{"error_type":"provider_overloaded","reason":"provider_overloaded"}}}

If you disconnect

If you close the connection before the stream ends, you are charged for the tokens used up to that point, as reported at that time, and the rest of the hold is released.

Stream with the Anthropic SDK

POST /v1/messages streams in the Anthropic event format. Set the base URL to https://api.router.africa and pass your key as the usual api_key. See Messages for curl, Python and TypeScript examples. Disconnecting is charged the same way as above.

Stream with the Responses API

POST /v1/responses streams Responses events, each with an event: line and a data: line, and no [DONE] line. The response.completed event carries the usage. See Responses.