Skip to main content
A Config is a JSON object you attach to your gateway requests to control how they are handled. Configs let you define provider routing, fallback chains, load balancing weights, retry policies, caching rules, and guardrails — all in one place.

​
Passing a config

You can attach a config in two ways: via a request header, or inline in the Portkey client.

​
Via the x-portkey-config header

Base64-encode your config object and pass it as the x-portkey-config header on any request. 
import base64, json
from portkey_ai import Portkey

config = {
  "retry": {"attempts": 3},
  "strategy": {"mode": "fallback"},
  "targets": [
    {"provider": "openai", "api_key": "sk-..."},
    {"provider": "anthropic", "api_key": "sk-ant-..."}
  ]
}

client = Portkey(
    base_url="http://localhost:8787/v1",
    default_headers={
        "x-portkey-config": base64.b64encode(
            json.dumps(config).encode()
        ).decode()
    }
)

​
Inline in the Portkey client

Pass the config directly when constructing the client, or use with_options to scope it to specific calls. 
from portkey_ai import Portkey

client = Portkey(
    provider="openai",
    Authorization="sk-...",
    config={
      "retry": {"attempts": 5},
      "output_guardrails": [{
        "default.contains": {"operator": "none", "words": ["Apple"]},
        "deny": True
      }]
    }
)

# Or scope the config to a specific set of calls
scoped = client.with_options(config={"retry": {"attempts": 2}})
scoped.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}]
)

​
Config schema

A config object supports the following top-level fields:
​
strategy
object
Controls the routing strategy. See routing, fallbacks, and load balancing.
  • mode — "single" | "fallback" | "loadbalance" | "conditional"
  • on_status_codes — HTTP status codes that trigger a fallback (fallback mode only)
  • conditions — routing conditions evaluated against request metadata (conditional mode only)
  • default — name of the default target when no condition matches
​
targets
array
List of provider targets to route to. Each target can specify:
  • provider — provider name ("openai", "anthropic", "vertex-ai", etc.)
  • api_key / virtual_key — authentication credentials
  • weight — routing weight for load balancing (0–1)
  • override_params — model parameters to override for this target
  • retry — per-target retry settings
  • cache — per-target cache settings
  • name — logical name used in conditional routing
​
retry
object
Automatic retry configuration. See retries.
  • attempts — maximum retry attempts (up to 5)
  • on_status_codes — status codes that trigger a retry (default: [429, 500, 502, 503, 504])
​
cache
object
Response caching configuration. See caching.
  • mode — "simple" or "semantic"
  • max_age — cache TTL in seconds
​
output_guardrails
array
Guardrail checks applied to LLM responses. Requests that fail a guardrail check with deny: true are automatically retried.
​
input_guardrails
array
Guardrail checks applied to incoming requests before they are forwarded to a provider.
​
custom_host
string
Override the base URL used to reach the provider (useful for local proxies or custom deployments).

​
Example: retries + guardrails

This config retries failed requests up to 5 times and denies any response containing the word “Apple”:
python
config = {
  "retry": {"attempts": 5},
  "output_guardrails": [{
    "default.contains": {"operator": "none", "words": ["Apple"]},
    "deny": True
  }]
}

client = client.with_options(config=config)
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Reply randomly with Apple or Bat"}]
)
# Always returns "Bat" — guardrail retries until a compliant response is received

​
Example: fallback + load balancing combined

{
  "strategy": {
    "mode": "loadbalance"
  },
  "targets": [
    {
      "strategy": {"mode": "fallback"},
      "weight": 0.7,
      "targets": [
        {"provider": "openai", "override_params": {"model": "gpt-4o"}},
        {"provider": "azure-openai", "override_params": {"model": "gpt-4o"}}
      ]
    },
    {
      "provider": "anthropic",
      "weight": 0.3,
      "override_params": {"model": "claude-3-5-sonnet-20241022"}
    }
  ]
}
Configs are composable — you can nest targets and strategies to build complex multi-provider routing trees.

Routing

Route requests to different providers based on metadata conditions.

Fallbacks

Automatically switch to a backup provider on failure.

Load balancing

Distribute traffic across providers and API keys.

Retries

Automatically retry failed requests with exponential backoff.

Build docs developers (and LLMs) love

Get started for freeTalk to us