Streaming

Set stream: true. Both wire formats stream, and both stream the shape your SDK already parses.

1curl -N https://api.aile.sh/v1/chat/completions \
2-H "Authorization: Bearer $AILE_KEY" \
3-H "Content-Type: application/json" \
4-d '{
5  "model": "cc/claude-sonnet-5",
6  "stream": true,
7  "messages": [{ "role": "user", "content": "Count to five." }]
8}'

-N matters for curl: without it curl buffers the output itself and the stream looks broken when it is not.

Response headers on a stream

1content-type: text/event-stream
2cache-control: no-cache, no-transform
3x-accel-buffering: no

x-accel-buffering: no tells intermediate proxies not to accumulate the body before passing it on, and no-transform keeps a compressor from buffering it. Without them a correctly streaming response arrives as one block at the end. Self-hosted (local/) chat streams, image and speech streams, and a held stream (below) send cache-control: no-cache alone.

The x-aile-* decision headers arrive with the response head, before the first token, so you know which sharer is serving before you know what it says — unless the stream is held (below).

What can and cannot stream

The image and speech streams need an API key: a keyless request with stream: true is refused. An image stream sends no partial frames (partial_images is refused) and bills the images it delivered. Their 200 is sent once the provider accepts, so a stream that then delivers nothing is a 200 that bills nothing.

x402 stays buffered because its settlement receipt is single-shot: the payment result is one artifact produced when the call completes, and there is no honest way to emit it incrementally.

The two timeouts

Time to first byte is bounded across the whole request, not per attempt (180 s by default). The first attempt is never shortened (a cold high-reasoning model can legitimately take a long time to produce its first token), so the budget only squeezes later attempts after an earlier sharer has already failed.

Idle timeout takes over once the upstream's headers arrive. It rearms on every chunk, so a long answer that keeps producing tokens is never cut off for being long. Only silence ends it, and on chat only after 10 minutes by default (3 on image and speech streams), so a reasoning model that thinks quietly for minutes is not cut off. Meanwhile you receive keepalive frames (an Anthropic ping event on /v1/messages, a : keepalive comment otherwise), so your own proxies see a live connection.

The practical consequence: failover happens before the stream starts. Once tokens are flowing, you are with that sharer for the rest of the response.

Errors mid-stream

An error before the first byte is an ordinary HTTP error with a JSON body. See Errors.

One exception: a streamed chat or image request with no answer after 25 s is committed to 200 with x-aile-edge-hold and only the x-aile-* headers that could no longer change (not the sharer or price, except on self-hosted capacity — see Headers) and kept alive with a keepalive frame every 15 s. Any failure after that arrives as the terminal frame below, not as an HTTP status. A request paid with x402 is never held.

An error after the stream has started arrives as a terminal frame in the dialect you are reading:

1data: {"error":{"message":"…","type":"api_error","code":null}}
2
3data: [DONE]

Anthropic readers get an error event instead:

1event: error
2data: {"type":"error","error":{"type":"api_error","message":"…"}}

A stream that errors before any text is not charged. One cut short otherwise bills the usage the provider reported (an Anthropic stream's opening count bills its input); with none, it bills an estimate from the text that reached you, capped at the quote. Reasoning, a tool call or audio count as output too: a stream you abandon after only those bills its input, and one that ends cleanly with only those bills the estimate, capped at the quote.

When the provider answers a stream with an error status, the request is not charged, and the PAYMENT-RESPONSE receipt on a key's stream says so (success: false). A Claude answer cut by max_tokens or the model's context window ends with finish_reason: "length" on the OpenAI endpoints.

Counting tokens first

1curl https://api.aile.sh/v1/messages/count_tokens \
2  -H "x-api-key: $AILE_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{ "model": "cc/claude-sonnet-5", "messages": [{"role":"user","content":"…"}] }'

It is advisory only. It probes no sharer, bills nothing, and estimates at roughly four characters per token, so use it as a sanity check rather than for billing arithmetic.