Routing

When more than one provider can serve a model, the router picks between them on price, latency and live health. The provider object lets you constrain that choice.

{
  "model": "meta/llama-3.3-70b",
  "messages": [{"role": "user", "content": "Hello"}],
  "provider": {
    "only": ["together", "fireworks"],
    "sort": "price",
    "max_price": {"prompt": 0.9},
    "zdr": true
  }
}

Fields

• order — try these providers first, in this order.
• only — restrict to these providers.
• ignore — exclude these providers.
• sort — price, throughput or latency.
• max_price — a per-SKU ceiling in USD per million tokens.
• quantizations — restrict to given quantizations. bf16 and int4 are not the same model.
• require_parameters — only endpoints supporting every parameter you sent.
• zdr — only endpoints that retain nothing.
• data_collection — allow or deny training on your inputs.

Preferences reorder; constraints reject

sort changes the order of candidates. only, ignore, max_price, zdr and quantizations remove them. If a constraint removes every candidate the request fails with 503 and names the filter that emptied the set — it never quietly falls through to something you excluded.

{
  "error": {
    "code": 503,
    "message": "No endpoints satisfy the request",
    "metadata": {"filters": ["provider.only=[together]", "zdr=true"]}
  }
}

Guardrails outrank the request

Organisation guardrails can restrict providers, models and data regions, and a request cannot widen them — only narrow them further. If your request asks for a provider a guardrail excludes, the guardrail wins and the error says so.