Models
One endpoint for every model, on-premises or hosted.
Every provider you use
On-premises vLLM, Azure, Bedrock, Vertex, OCI, IBM watsonx and regional hosted APIs.
No client changes
OpenAI- and Anthropic-compatible APIs, streaming included.
Routing that thinks
Semantic routing by request, load-aware routing across self-hosted models, and fallback when a backend fails.
Keys per team and app
Virtual API keys, hashed and expiring, bound to teams, models and source networks.
Residency enforced
Cloud backends admitted only in allowlisted regions. In-Kingdom by configuration, not by promise.
Response cache
Exact and semantic caching, with prompt-cache pricing counted correctly in spend.