Skip to main content
The LLM Proxy is Archestra’s security layer that sits between your AI agents, chatbots, and applications and the LLM providers they rely on — OpenAI, Anthropic, Google Gemini, AWS Bedrock, and more. Every request passes through the proxy, where Archestra intercepts and analyzes it, applies security policies, and then forwards it to the upstream provider. Responses travel the same path in reverse, giving you a single control plane for visibility, compliance, and cost management across all LLM traffic.

​
How It Works

Security Policies

Every request is evaluated against your configured security and compliance policies before reaching the provider.

Unified Endpoint

One proxy URL per provider route replaces direct provider endpoints in your client code, requiring minimal changes.

Full Observability

All interactions are logged with token counts, costs, latency, and trace attributes for complete audit trails.

​
Getting Started

1

Create an LLM Proxy

Go to LLM Proxies in the Archestra dashboard and create a new proxy. Give it a descriptive name to identify the project or team it serves.
2

Copy the Provider URL

Click the Connect icon on the proxy, choose the LLM provider your application uses, and copy the provided proxy URL.
3

Update Your Application

Replace the provider’s original endpoint in your application with the Archestra proxy URL. All other request parameters — model name, API key, headers — stay the same.

​
OpenAI-Compatible Model Router

The Model Router exposes a single OpenAI-compatible endpoint that can reach models from multiple configured providers. Use it when your application supports the OpenAI API style but you want to route to Anthropic, Gemini, Groq, Bedrock, or other providers without changing client code. The router accepts OpenAI Responses and Chat Completions requests, resolves provider-qualified model IDs such as openai:gpt-5.4 or anthropic:claude-opus-4-6-20250918 to the backing provider, runs the full LLM Proxy security pipeline, and returns a matching OpenAI-format response.
Generic OpenAI-compatible clients should use a virtual key mapped to the providers they need. Backend services and bots should use an LLM OAuth client access token. See Authentication for setup details.

​
Custom Headers

Archestra supports optional custom headers that enrich logs, metrics, and traces without altering the underlying LLM request. All headers are optional and can be combined freely.

​
Usage Examples

The X-Archestra-Meta header is useful for brevity when you want to set all three identifiers in one header. Leave a segment empty with consecutive slashes if you only need some fields, as shown above (agent ID and session ID, no execution ID).

​
Authentication

The LLM Proxy supports direct provider API keys, virtual API keys, LLM OAuth client access tokens, and JWKS via an external identity provider. See Authentication for full details on each method and when to use them.

​
Supported Providers

Archestra supports OpenAI, Anthropic, Google Gemini and Vertex AI, Azure AI Foundry, AWS Bedrock, Groq, Mistral, xAI, OpenRouter, Cerebras, Cohere, DeepSeek, MiniMax, ZhipuAI, Perplexity, vLLM, and Ollama. See Supported Providers for connection details and setup instructions for each provider.