What is the AI Gateway?
The Portkey AI Gateway is a lightweight, open-source middleware layer that sits between your application and any AI provider. It gives you a single, consistent API to call 250+ language, vision, audio, and image models — with production-grade reliability features built in.- Sub-1ms overhead — tiny 122kb footprint, runs on Node.js, Cloudflare Workers, and Docker
- Battle-tested — processing over 10 billion tokens per day in production
- OpenAI-compatible — drop-in replacement for the OpenAI client; works with any SDK
Core capabilities
Quickstart
Get your gateway running and make your first request in under 2 minutes.
Supported providers
Explore all 250+ supported LLM providers and model types.
Routing & configs
Define routing rules, fallbacks, retries, and load balancing in JSON configs.
Guardrails
Validate and filter LLM inputs and outputs with 50+ built-in checks.
MCP Gateway
Manage MCP servers with centralized auth, access control, and observability.
Deployment
Deploy to Docker, Node.js, Cloudflare Workers, AWS, GCP, or Azure.
Key features
Reliable routing
- Fallbacks — automatically switch to another provider or model on failure
- Automatic retries — exponential backoff with up to 5 retry attempts
- Load balancing — distribute traffic across multiple API keys or providers by weight
- Request timeouts — terminate slow requests after a configurable duration
- Conditional routing — route to different targets based on request metadata
Security & accuracy
- Guardrails — 50+ pre-built checks for content, format, PII, and more
- Custom plugins — bring your own guardrail logic via the plugin system
- Partner integrations — Aporia, Pangea, Patronus, Pillar, and more
- JWT validation — verify bearer tokens before forwarding requests
Multi-modal support
- Chat completions, text completions, and embeddings
- Image generation and editing
- Audio speech synthesis, transcription, and translation
- Realtime WebSocket APIs (OpenAI Realtime)
- Batch processing and fine-tuning job management
MCP Gateway
The MCP Gateway provides a centralized control plane for Model Context Protocol servers — with authentication, access control, observability, and identity forwarding out of the box. Works with Claude Desktop, Cursor, VS Code, and any MCP-compatible client.Self-hosted and managed
Self-hosted
Run the gateway on your own infrastructure — Docker, Node.js, Kubernetes, and more.
Portkey Cloud
Use the managed Portkey API — zero infrastructure, free developer tier available.
Integrations
The gateway works natively with popular AI frameworks and SDKs:OpenAI SDK
Drop-in compatible with the official OpenAI Python and Node.js SDKs.
LangChain
Use Portkey as the LLM provider in any LangChain chain or agent.
LlamaIndex
Integrate with LlamaIndex data pipelines and query engines.
Autogen
Power multi-agent conversations with any provider through Autogen.
CrewAI
Route CrewAI agent crews through the gateway for observability.
More integrations
Explore cookbooks for Vercel AI SDK, Phidata, Control Flow, and more.