Documentation Index
Fetch the complete documentation index at: https://mintlify.com/ghuntley/loom/llms.txt
Use this file to discover all available pages before exploring further.
Overview
The LLM proxy endpoints allow clients to make LLM completion requests through the Loom server without exposing API keys. The server handles:
- API key management and rotation
- Request/response logging and auditing
- Rate limiting and retry logic
- Token usage tracking
- Server-to-client queries during streaming
Supported Providers:
- Anthropic (Claude models)
- OpenAI (GPT models)
- Vertex AI (Google Cloud)
- Z.ai (Z.ai models)
All proxy endpoints accept a standard LlmRequest payload:
Model identifier (e.g., "claude-sonnet-4", "gpt-4o")
Array of message objects with role and content
Tool definitions for function calling
Maximum tokens to generate
Sampling temperature (0.0-1.0)
Anthropic Complete
POST /proxy/anthropic/complete
Request Body
See Request Format above.
Response
Assistant message with role: "assistant" and content
Array of tool call objects (if any)
Token usage: {input_tokens: number, output_tokens: number}
Reason for completion: "stop", "length", "tool_use", etc.
Example
curl -X POST https://loom.ghuntley.com/proxy/anthropic/complete \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4",
"messages": [
{"role": "user", "content": "Explain Rust ownership in one sentence."}
],
"max_tokens": 100
}'
{
"message": {
"role": "assistant",
"content": "Rust ownership ensures memory safety by enforcing that each value has a single owner, automatically deallocating memory when the owner goes out of scope."
},
"tool_calls": [],
"usage": {
"input_tokens": 18,
"output_tokens": 31
},
"finish_reason": "stop"
}
Anthropic Stream
POST /proxy/anthropic/stream
Request Body
See Request Format above.
Response
Returns text/event-stream with events tagged as event: llm.
Event Types
Text Delta
{"type": "text_delta", "content": "Rust ownership"}
{
"type": "tool_call_delta",
"call_id": "call_abc123",
"tool_name": "read_file",
"arguments_fragment": "{\"path\":\"/src"
}
Server Query
Server requests information from the client:
{
"type": "server_query",
"id": "Q-abc123",
"kind": {"ReadFile": {"path": "/test.txt"}},
"sent_at": "2026-03-03T12:00:00Z",
"timeout_secs": 30,
"metadata": {}
}
POST /api/sessions/{session_id}/query-response.
Completed
{
"type": "completed",
"response": {
"message": {...},
"tool_calls": [...],
"usage": {...},
"finish_reason": "stop"
}
}
Error
{"type": "error", "message": "Rate limited; retry after 30 seconds"}
Example
curl -N -X POST https://loom.ghuntley.com/proxy/anthropic/stream \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4",
"messages": [
{"role": "user", "content": "Write a haiku about coding."}
]
}'
event: llm
data: {"type":"text_delta","content":"Code"}
event: llm
data: {"type":"text_delta","content":" flows like"}
event: llm
data: {"type":"text_delta","content":" water\\n"}
event: llm
data: {"type":"completed","response":{"message":{...},"tool_calls":[],"usage":{...},"finish_reason":"stop"}}
OpenAI Complete
POST /proxy/openai/complete
Request/Response
Same format as Anthropic endpoints. See Request Format and Anthropic Complete.
Example
curl -X POST https://loom.ghuntley.com/proxy/openai/complete \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}'
OpenAI Stream
POST /proxy/openai/stream
Example
curl -N -X POST https://loom.ghuntley.com/proxy/openai/stream \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "Count to 5."}
]
}'
Vertex Complete
POST /proxy/vertex/complete
Request/Response
Same format as Anthropic endpoints. See Request Format and Anthropic Complete.
Example
curl -X POST https://loom.ghuntley.com/proxy/vertex/complete \
-H "Content-Type: application/json" \
-d '{
"model": "gemini-2.0-flash-exp",
"messages": [
{"role": "user", "content": "Summarize quantum computing."}
]
}'
Vertex Stream
POST /proxy/vertex/stream
Example
curl -N -X POST https://loom.ghuntley.com/proxy/vertex/stream \
-H "Content-Type: application/json" \
-d '{
"model": "gemini-2.0-flash-exp",
"messages": [
{"role": "user", "content": "Explain async/await."}
]
}'
Z.ai Complete
Synchronous Z.ai completion.
Request/Response
Same format as Anthropic endpoints. See Request Format and Anthropic Complete.
Example
curl -X POST https://loom.ghuntley.com/proxy/zai/complete \
-H "Content-Type: application/json" \
-d '{
"model": "z.ai-model",
"messages": [
{"role": "user", "content": "Hello, world!"}
]
}'
Z.ai Stream
Streaming Z.ai completion via SSE. Same event format as Anthropic Stream.
Example
curl -N -X POST https://loom.ghuntley.com/proxy/zai/stream \
-H "Content-Type: application/json" \
-d '{
"model": "z.ai-model",
"messages": [
{"role": "user", "content": "Tell me a joke."}
]
}'
Error Handling
All endpoints return errors in this format:
{
"error": "service_unavailable",
"message": "Anthropic provider is not configured on the server"
}
Common Errors
| Status | Error Code | Description |
|---|
| 503 | service_unavailable | Provider not configured or unavailable |
| 503 | rate_limited | Upstream rate limit hit |
| 504 | timeout | LLM request timed out |
| 500 | upstream_error | Provider returned an error |
Rate Limiting
When rate limited, the response includes retry information:
{
"error": "rate_limited",
"message": "LLM rate limited; retry after 30 seconds"
}
Audit Logging
All LLM requests are logged for audit purposes:
- LlmRequestStarted: Provider, model, message count
- LlmRequestCompleted: Provider, model, tool call count
- LlmRequestFailed: Provider, model, error message
Logs are queryable via the admin audit log endpoints.