Skip to main content
Portrix routing lets you define how requests are directed to AI providers, enabling fallback chains, load distribution, and cost optimization — all without changing your core application logic. By default, routing is automatic and implicit, but you can layer in sophisticated strategies as your needs grow.

​
How routing works

By default, Portrix routes each request to the provider implied by the model ID you specify. If you send a request with model: "anthropic/claude-3-5-sonnet", the gateway routes it to Anthropic. No additional configuration is required. Advanced routing goes further. You can specify:
  • Fallback models — an ordered list of alternatives to try if the primary model is unavailable or returns an error.
  • Cost-based routing — automatically select the cheapest model that meets your latency threshold.
  • Latency-based selection — route to whichever provider responds fastest at the time of the request.
All routing behavior is controlled through request headers, so you can apply different strategies per request without any server-side configuration changes.

​
Fallback chains

A fallback chain is an ordered list of models Portrix will try in sequence if the primary model fails. Failures that trigger a fallback include provider outages, rate limit errors (HTTP 429), and server errors (HTTP 5xx). Specify fallbacks using the x-portrix-fallback header as a comma-separated list of model IDs. Portrix tries each model in order and returns the first successful response.
In this example, if openai/gpt-4o is unavailable or over quota, Portrix automatically retries with anthropic/claude-3-5-sonnet, then google/gemini-2.0-flash if Anthropic also fails.
Test your fallback chains in development before deploying to production. You can force a fallback by specifying a deliberately invalid primary model name or by using a model that is known to be rate-limited in your test environment.

​
Load balancing

For high-throughput scenarios, Portrix can distribute requests across multiple providers to avoid hitting any single provider’s rate limits. Configure load balancing from your Portrix dashboard by creating a routing group — a named set of models with associated weights. Once a routing group is configured, reference it by name in the x-portrix-model header. Portrix distributes traffic according to the weights you defined, transparently balancing load across providers.

​
Cost routing

With cost routing, Portrix automatically selects the cheapest model that meets your performance requirements. You set a maximum acceptable latency (in milliseconds), and the gateway chooses the lowest-cost model that can reliably respond within that window. Enable cost routing by setting x-portrix-route: cheapest. Portrix evaluates current pricing and historical latency data to make the selection at request time. This strategy is particularly useful for batch processing workloads where quality differences between models are small but cost differences are significant.

​
Routing headers

Use the following x-portrix-* headers to control routing behavior on individual requests.
Headers take precedence over the model field in the request body when both are present. Use x-portrix-model if you need to programmatically override routing at the request level without changing your payload structure.