Configs

A Config is a JSON object you attach to your gateway requests to control how they are handled. Configs let you define provider routing, fallback chains, load balancing weights, retry policies, caching rules, and guardrails — all in one place.

Passing a config

You can attach a config in two ways: via a request header, or inline in the Portkey client.

Via the `x-portkey-config` header

Base64-encode your config object and pass it as the x-portkey-config header on any request.

import base64, json
from portkey_ai import Portkey

config = {
  "retry": {"attempts": 3},
  "strategy": {"mode": "fallback"},
  "targets": [
    {"provider": "openai", "api_key": "sk-..."},
    {"provider": "anthropic", "api_key": "sk-ant-..."}
  ]
}

client = Portkey(
    base_url="http://localhost:8787/v1",
    default_headers={
        "x-portkey-config": base64.b64encode(
            json.dumps(config).encode()
        ).decode()
    }
)

import OpenAI from "openai";

const config = {
  retry: { attempts: 3 },
  strategy: { mode: "fallback" },
  targets: [
    { provider: "openai", api_key: "sk-..." },
    { provider: "anthropic", api_key: "sk-ant-..." }
  ]
};

const client = new OpenAI({
  baseURL: "http://localhost:8787/v1",
  defaultHeaders: {
    "x-portkey-config": Buffer.from(JSON.stringify(config)).toString("base64")
  }
});

curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-config: $(echo '{"retry":{"attempts":3}}' | base64)" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

Inline in the Portkey client

Pass the config directly when constructing the client, or use with_options to scope it to specific calls.

from portkey_ai import Portkey

client = Portkey(
    provider="openai",
    Authorization="sk-...",
    config={
      "retry": {"attempts": 5},
      "output_guardrails": [{
        "default.contains": {"operator": "none", "words": ["Apple"]},
        "deny": True
      }]
    }
)

# Or scope the config to a specific set of calls
scoped = client.with_options(config={"retry": {"attempts": 2}})
scoped.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}]
)

import Portkey from "portkey-ai";

const client = new Portkey({
  provider: "openai",
  Authorization: "sk-...",
  config: {
    retry: { attempts: 5 }
  }
});

Config schema

A config object supports the following top-level fields:

object

Controls the routing strategy. See routing, fallbacks, and load balancing.

mode — "single" | "fallback" | "loadbalance" | "conditional"
on_status_codes — HTTP status codes that trigger a fallback (fallback mode only)
conditions — routing conditions evaluated against request metadata (conditional mode only)
default — name of the default target when no condition matches

array

List of provider targets to route to. Each target can specify:

provider — provider name ("openai", "anthropic", "vertex-ai", etc.)
api_key / virtual_key — authentication credentials
weight — routing weight for load balancing (0–1)
override_params — model parameters to override for this target
retry — per-target retry settings
cache — per-target cache settings
name — logical name used in conditional routing

object

Automatic retry configuration. See retries.

attempts — maximum retry attempts (up to 5)
on_status_codes — status codes that trigger a retry (default: [429, 500, 502, 503, 504])

object

Response caching configuration. See caching.

mode — "simple" or "semantic"
max_age — cache TTL in seconds

array

Guardrail checks applied to LLM responses. Requests that fail a guardrail check with deny: true are automatically retried.

array

Guardrail checks applied to incoming requests before they are forwarded to a provider.

string

Override the base URL used to reach the provider (useful for local proxies or custom deployments).

Example: retries + guardrails

This config retries failed requests up to 5 times and denies any response containing the word “Apple”:

python

config = {
  "retry": {"attempts": 5},
  "output_guardrails": [{
    "default.contains": {"operator": "none", "words": ["Apple"]},
    "deny": True
  }]
}

client = client.with_options(config=config)
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Reply randomly with Apple or Bat"}]
)
# Always returns "Bat" — guardrail retries until a compliant response is received

Example: fallback + load balancing combined

{
  "strategy": {
    "mode": "loadbalance"
  },
  "targets": [
    {
      "strategy": {"mode": "fallback"},
      "weight": 0.7,
      "targets": [
        {"provider": "openai", "override_params": {"model": "gpt-4o"}},
        {"provider": "azure-openai", "override_params": {"model": "gpt-4o"}}
      ]
    },
    {
      "provider": "anthropic",
      "weight": 0.3,
      "override_params": {"model": "claude-3-5-sonnet-20241022"}
    }
  ]
}

Configs are composable — you can nest targets and strategies to build complex multi-provider routing trees.

Routing

Route requests to different providers based on metadata conditions.

Fallbacks

Automatically switch to a backup provider on failure.

Load balancing

Distribute traffic across providers and API keys.

Retries

Automatically retry failed requests with exponential backoff.

Get Started

Deployment

Core Concepts

Guardrails

MCP Gateway

Integrations

Plugin Development

Passing a config

Via the `x-portkey-config` header

Inline in the Portkey client

Config schema

Example: retries + guardrails

Example: fallback + load balancing combined

Routing

Fallbacks

Load balancing

Retries

​Passing a config

​Via the x-portkey-config header

​Inline in the Portkey client

​Config schema

​Example: retries + guardrails

​Example: fallback + load balancing combined

​Related concepts

Routing

Fallbacks

Load balancing

Retries

Passing a config

Via the `x-portkey-config` header

Inline in the Portkey client

Config schema

Example: retries + guardrails

Example: fallback + load balancing combined

Related concepts