Skip to main content
1

Run the gateway

Start the gateway locally with a single command. Node.js is required.
npx @portkey-ai/gateway
The gateway starts two endpoints:
  • API: http://localhost:8787/v1
  • Console: http://localhost:8787/public/
You can also run the gateway with Bun: bunx @portkey-ai/gateway. For Docker, Cloudflare Workers, and other deployment targets, see the deployment overview.
2

Make your first request

Send a chat completion through the gateway. The gateway accepts your provider API key directly via the Authorization header — no Portkey account required.
# pip install -qU portkey-ai

from portkey_ai import Portkey

# OpenAI-compatible client pointed at your local gateway
client = Portkey(
    provider="openai",  # or 'anthropic', 'bedrock', 'groq', etc.
    Authorization="sk-***"  # your provider API key
)

response = client.chat.completions.create(
    messages=[{"role": "user", "content": "What's the weather like?"}],
    model="gpt-4o-mini"
)

print(response.choices[0].message.content)
Swap provider="openai" for any of the 250+ supported providers — the request shape stays the same.
3

Add routing and guardrails

Config objects let you attach routing rules, reliability settings, and guardrails to any client. Pass a config to with_options (Python) or withOptions (JavaScript) to apply it.
python
config = {
  "retry": {"attempts": 5},

  "output_guardrails": [{
    "default.contains": {"operator": "none", "words": ["Apple"]},
    "deny": True
  }]
}

# Attach the config to the client
client = client.with_options(config=config)

client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Reply randomly with Apple or Bat"}]
)

# Always responds with "Bat" — the guardrail blocks replies containing "Apple".
# The retry config retries up to 5 times before giving up.
The config above uses two features:
  • retry — automatically retries failed or blocked requests with exponential backoff, up to the specified number of attempts.
  • output_guardrails — evaluates the LLM response before returning it. Here, any response containing the word “Apple” is denied and the request retries.
Configs also support fallbacks, load balancing, caching, request timeouts, and conditional routing. See configs for the full reference.
4

View logs in the Gateway Console

Open http://localhost:8787/public/ to see all requests logged in real time.The console shows request and response bodies, latency, provider used, model, token counts, and whether any guardrails fired. No configuration needed — logging is on by default.

​
Next steps

Supported providers

Browse all 250+ providers, including OpenAI, Anthropic, Gemini, Bedrock, Ollama, and more.

Routing & configs

Define fallbacks, load balancing, retries, and conditional routing in a single JSON config.

Guardrails

Validate LLM inputs and outputs with 50+ built-in checks or bring your own plugin.

Deployment

Deploy to Docker, Node.js, Cloudflare Workers, AWS EC2, or use the managed Portkey Cloud.

Build docs developers (and LLMs) love

Get started for freeTalk to us