> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portix.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure AI Fallbacks and Load Balancing in Portrix

> Set up automatic fallbacks and load balancing in Portrix to keep your AI application resilient when a provider experiences downtime or rate limits.

Portrix lets you define fallback models and distribute load across providers so your application stays available even when individual providers hit rate limits or experience outages. Instead of your users seeing a hard error, Portrix silently retries the request against the next model in your chain — all within the same API call.

## Why fallbacks matter

AI provider reliability is not guaranteed. Providers can:

* **Go down** — planned or unplanned outages affect even major providers
* **Rate limit your account** — high-traffic periods can trigger 429 responses
* **Introduce latency spikes** — cold model starts or traffic surges slow responses significantly

Without fallbacks, any of these events surfaces directly as an error in your application. With fallbacks configured in Portrix, the gateway retries transparently before returning an error to your code.

## Setting up fallbacks

### Using the `x-portrix-fallback` header

Pass a comma-separated list of fallback model IDs in the `x-portrix-fallback` header. Portrix tries the primary `model` first, then each fallback in the order listed.

<CodeGroup>
  ```bash cURL theme={null}
  curl https://api.portrix.ai/v1/chat/completions \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -H "x-portrix-fallback: anthropic/claude-3-5-sonnet, mistral/mistral-large" \
    -d '{
      "model": "openai/gpt-4o",
      "messages": [
        {"role": "user", "content": "Summarise the key principles of REST APIs."}
      ]
    }'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      api_key="YOUR_API_KEY",
      base_url="https://api.portrix.ai/v1",
  )

  response = client.chat.completions.create(
      model="openai/gpt-4o",
      messages=[
          {"role": "user", "content": "Summarise the key principles of REST APIs."}
      ],
      extra_headers={
          "x-portrix-fallback": "anthropic/claude-3-5-sonnet, mistral/mistral-large",
      },
  )

  print(response.choices[0].message.content)
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: "YOUR_API_KEY",
    baseURL: "https://api.portrix.ai/v1",
  });

  const response = await client.chat.completions.create(
    {
      model: "openai/gpt-4o",
      messages: [
        { role: "user", content: "Summarise the key principles of REST APIs." },
      ],
    },
    {
      headers: {
        "x-portrix-fallback": "anthropic/claude-3-5-sonnet, mistral/mistral-large",
      },
    }
  );

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

In the examples above, Portrix will:

1. Attempt `openai/gpt-4o` first
2. If that fails, attempt `anthropic/claude-3-5-sonnet`
3. If that also fails, attempt `mistral/mistral-large`
4. If all three fail, return an error to the caller

## Fallback conditions

Portrix triggers a fallback when the upstream provider returns any of the following:

| Condition       | HTTP Status | Description                                            |
| --------------- | ----------- | ------------------------------------------------------ |
| Provider error  | `5xx`       | Internal server errors, model unavailability           |
| Rate limited    | `429`       | Account or IP-level rate limit exceeded                |
| Request timeout | —           | Provider did not respond within the configured timeout |

Fallbacks are **not** triggered for client errors (`4xx` other than `429`), such as invalid request shapes or authentication failures. Those are returned immediately.

## Load balancing

### Using `x-portrix-route: balanced`

When you want to spread traffic evenly across multiple models — distributing cost or staying within per-provider rate limits — set the routing strategy to `balanced` alongside a fallback list:

```python theme={null}
response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
        {"role": "user", "content": "Generate a product description for a wireless keyboard."}
    ],
    extra_headers={
        "x-portrix-fallback": "anthropic/claude-3-5-sonnet, google/gemini-2.0-flash",
        "x-portrix-route": "balanced",
    },
)
```

With `balanced` routing, Portrix distributes requests round-robin across `openai/gpt-4o`, `anthropic/claude-3-5-sonnet`, and `google/gemini-2.0-flash`. If any node is unhealthy, it is temporarily removed from the rotation and the remaining nodes absorb its share.

Available routing strategies:

| Strategy   | Behaviour                                        |
| ---------- | ------------------------------------------------ |
| `balanced` | Round-robin across all models in the chain       |
| `cheapest` | Always routes to the lowest-cost available model |
| `fastest`  | Routes to the model with lowest current latency  |

## Dashboard configuration

Sending headers on every request works well for dynamic configurations, but for stable production setups you can define **named routes** in the [Portrix dashboard](https://app.portrix.ai):

<Steps>
  <Step title="Open the Routes section">
    Navigate to **Gateway → Routes** in the Portrix dashboard.
  </Step>

  <Step title="Create a new route">
    Click **New Route**, give it a name (e.g. `production-chat`), and add your primary model plus ordered fallbacks.
  </Step>

  <Step title="Set the routing strategy">
    Choose `balanced`, `cheapest`, or `fastest` from the dropdown.
  </Step>

  <Step title="Use the route name in requests">
    Reference the route by name instead of a model ID:

    ```python theme={null}
    response = client.chat.completions.create(
        model="route/production-chat",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    ```
  </Step>
</Steps>

Named routes are updated centrally — changing a fallback chain in the dashboard immediately affects all requests using that route, with no code changes required.

## Testing fallbacks

To verify your fallback chain is working, temporarily swap the primary model for an invalid model ID. Portrix will fail on the primary and cascade to your first fallback:

```python theme={null}
response = client.chat.completions.create(
    model="openai/this-model-does-not-exist",   # forces immediate fallback
    messages=[{"role": "user", "content": "Hello!"}],
    extra_headers={
        "x-portrix-fallback": "anthropic/claude-3-5-sonnet",
    },
)

# Response should come from anthropic/claude-3-5-sonnet
print(response.model)
```

The `model` field in the response reflects whichever model actually served the request, making it easy to confirm which fallback was used.

<Warning>
  Fallback models may have different capabilities, context windows, or output styles. A prompt carefully tuned for `openai/gpt-4o` may produce different results on `mistral/mistral-large`. Test every model in your fallback chain with your actual prompts before enabling fallbacks in production.
</Warning>