Routing & failover

Price is primary

The sharer pool is sorted by quoted price ascending. Un-quotable listings sort last, never first.

Only among sharers quoting the exact same price does anything else run: a health-weighted draw over

• live cooldown: a veto;
• probe result: did the credential answer recently;
• backoff ladder: how recently it failed;
• capacity headroom: how much of its rate window is left;
• observed time-to-first-byte and success rate.

The last two signals are in-memory averages and are deliberately not persisted. A restart forgets them, and every account then reads neutral, exactly like a freshly linked one. A cold map cannot misroute anything, because it has no opinions to be wrong about.

An untested model is listed only after a sharer's account passes a test of it, but untested capacity still serves when you name it, in price order with everyone else. A passing test only breaks a tie between sharers at the same price.

Two things override that order:

• A silent node. A machine that has not been heard from for 75 seconds (two missed pings) leaves selection, and is routable again on its next message.
• A dead key. An API-key account the provider refused outright, at its last probe or on a real request, sorts after every other candidate and leaves the chat catalogue. It is demoted, never removed, and a successful serve, a passing model test or a re-link restores it (a passing probe too, when only a probe refused it). See Verification.

Listings read the same gate: an account that is cooling down or has its breaker open is not shown as available.

Strictly per provider

There is no cross-provider fallback. Naming a provider means that provider serves you or the request fails:

• unknown prefix → 400, pointing you to GET /v1/models;
• a provider that cannot speak the endpoint you posted to → 404, naming the provider and the path, before anything is held;
• known provider, no sharer → 503, naming that provider;
• only your own capacity (or a related account's) left → 403, after every other sharer was tried. Nobody is paid for serving themselves.

Every request names its provider: a prefix, a bare name plus x-aile-provider, or a bare name on a provider-pinned key. A bare name alone is a 400, with two exceptions: a bare name from Claude Code routes to claude, and one from the Codex CLI routes to codex, so their built-in model pickers work on a base URL and key alone. A request may also name any provider, as auto/<model>: the model is then tried at each provider that lists it, cheapest first. See Model names.

A provider named by header, key pin or route receives the model exactly as written; Aile does not look it up first. A model that provider does not serve is its refusal to give: its 400 is passed through, and a 404 fails over to the next sharer and ends as a 503. Either way you are not charged.

One case is caught before any dial: a pinned provider whose own model ids carry no prefix (claude, codex and most others) sent another provider's route as the prefix, such as codex/gpt-5.6-sol on a key pinned to claude. That id cannot be the pinned provider's, so it is a 400 naming both providers and the fix. An aggregator such as openrouter takes namespaced ids (deepseek/…) as written.

Two independent bounds on your wait

They count different things, and conflating them was a real bug.

Dispatched dials. Aile dials up to four sharers. Pre-dispatch refusals (a cooling account, a sharer with no payout wallet, a saturated slot) are free and bounded separately by a scan cap.

When refusals shared the dial budget, a pool whose first four entries were skipped returned 503 to a caller while a healthy fifth sharer sat untried.

Total time to first byte. One budget covers every attempt of one request.

• The first attempt is never shortened. A cold high-reasoning model can legitimately take a long time to produce its first token.
• The budget therefore only squeezes later attempts, after an earlier sharer has already failed.
• It covers the dial and header phase only. Once the provider's response head arrives, a rearming idle timeout governs instead: 10 minutes of silence, so a reasoning model that says nothing while it thinks is not cut off, and a long answer is not cut off for being long.

When Aile dials a provider itself (nodeless), getting a connection up is bounded separately, at 12 seconds for connect and TLS, so a host that never finishes its handshake cannot eat the first-byte budget.

What fails over. A rate limit, a provider's overload (529), any other server error and a broken TLS path all move on to the next sharer. A request the provider rejects as malformed (a 400) does not: another sharer would refuse it the same way.

A slow stream is held open. When a streaming request has no response head after 25 seconds, Aile commits it to 200 and sends keepalives until the answer exists, so no proxy on the way cuts it. The per-attempt headers were not known when that 200 went out, so such a reply carries x-aile-edge-hold instead of x-aile-attempts, the sharer and the price. It keeps the provider and model, which failover cannot change, on a plain request or a route's last entry; a self-hosted reply, which never fails over, keeps them all. A failure after that point arrives as the SDK's own error frame in the stream rather than as a status code. A request paid with x402 is never held: its receipt rides a response header.

Whose fault is a timeout

A sharer cut short by Aile's own shortened budget is not benched for it. That would let a busy hour of Aile's deadlines bench the marketplace, punishing sharers for a queue they did not create.

The slowness still feeds sharer selection, so a consistently slow sharer loses ties. It does not trip a breaker.

Cooldowns are jittered

When several accounts share one upstream rate limit, a shared 429 would otherwise expire them all at the same instant and send them back in lockstep. A positive-only jitter spreads expiry.

Positive-only matters: a provider's own stated reset time is never jittered, and a retry is never made earlier than the computed backoff. A Retry-After is honoured up to 24 hours, so a malformed header cannot bench an account for years.

A cut stream cools its account. On the Anthropic and Responses wires, a stream that ends cleanly before the provider's own end event (an account at its quota edge often answers that way) cools that account for ten minutes, so the next request goes to another sharer. Billing is unchanged.

A model the key has no quota for. Google's "limit: 0" refusal on an API key fails over and takes that one model off that account, rather than cooling the whole account; its other models keep serving.

Session affinity

1x-aile-session: <id>

Pins a conversation to the account that served it, which matters where a provider caches per account. The verdict comes back as x-aile-session-affinity. The id a client already sends (Claude Code's, the Codex CLI's, OpenCode's) works too; a prompt_cache_key pins each conversation under it rather than every request that shares it, and a multi-turn conversation that sends none is recognised by its opening. The answer a client sends back counts most: its encrypted reasoning or tool call id names the account that produced it, and the next turn goes there, session id or not. A busy subscription account that holds the conversation is waited for up to 10 seconds on a stream before the turn moves. See Choosing a sharer.

Steering it yourself

Every lever, and the CLI flags that mirror them: Choosing a sharer.

The default listing order in aile sharers is the routing order, so the top row is the machine your next request goes to. --sort asks for a reading order instead, and the footer says so rather than letting you mistake one for the other.