Chat completions

POST /v1/chat/completions

Sends a list of messages to a supported chat model and returns a generated response. This endpoint is fully compatible with the OpenAI Chat Completions API, so any OpenAI-compatible SDK works without code changes. Requests are routed through the gateway according to the config or headers you provide. Fallback, retry, load balancing, and caching all apply transparently.

Request headers

string

The provider to route the request to (e.g. openai, anthropic, azure-openai). Required when not using a config.

string

Your provider API key. Alternative to setting the key in a virtual key or config.

string

A JSON config object (or config ID from Portkey Cloud) that defines routing, fallbacks, retries, and more.

string

A virtual key ID from Portkey Cloud that maps to a stored provider credential.

Request body

string

required

The model identifier to use (e.g. gpt-4o, claude-3-5-sonnet-20241022). The gateway forwards this value to the target provider; use the provider’s model name.

object[]

required

The conversation history as an array of message objects. Each object must have a role and content.

Show properties

string

required

The sender role. One of system, user, assistant, tool, or developer.

string | object[]

required

The message content. A plain string for most messages. An array of content blocks (type text, image_url, or input_audio) for multimodal input.

string

An optional name for the participant, used to differentiate multiple participants with the same role.

object[]

Tool call objects generated by a previous assistant turn. Required when role is tool.

string

The ID of the tool call this message is responding to. Required when role is tool.

number

default:"1"

Sampling temperature between 0 and 2. Higher values produce more varied output; lower values produce more focused, deterministic output. Avoid using this alongside top_p.

integer

default:"100"

The maximum number of tokens to generate. The request fails if the prompt plus max_tokens exceeds the model’s context length.

integer

An upper bound on tokens generated for a completion, including visible and reasoning tokens. Preferred over max_tokens for models that support reasoning.

boolean

default:"false"

When true, responses are sent as server-sent events (text/event-stream). Each event is a JSON delta prefixed with data: . The stream ends with data: [DONE].

number

default:"1"

Nucleus sampling threshold between 0 and 1. The model considers only the tokens whose cumulative probability reaches top_p. Avoid using alongside temperature.

number

default:"0"

Number between -2.0 and 2.0. Positive values penalize tokens that have already appeared in the output, reducing repetition.

number

default:"0"

Number between -2.0 and 2.0. Positive values penalize tokens that appear anywhere in the text so far, encouraging the model to introduce new topics.

integer

default:"1"

The number of chat completion choices to generate for each input message.

string | string[]

One or more sequences where the model stops generating further tokens. The stop sequence itself is not included in the output.

object[]

A list of tools the model may call. Each tool describes a function the model can invoke.

Show properties

string

required

Always function.

object

required

Function definition with name, optional description, and parameters as a JSON Schema object.

string | object

default:"auto"

Controls how the model selects tools. One of none, auto, required, or an object {"type": "function", "function": {"name": "..."}}.

object

Specifies the output format. Set type to json_object to enable JSON mode, or json_schema to enforce a specific schema.

Show properties

string

required

One of text, json_object, or json_schema.

object

The schema to enforce when type is json_schema. Include name, optional description, schema (JSON Schema), and strict (boolean).

integer

A seed for deterministic sampling. The same seed and parameters should return the same result, though determinism is not guaranteed.

boolean

default:"false"

Whether to return log probabilities for the output tokens.

integer

The number of most likely tokens (0–20) to return at each position, along with their log probabilities. Requires logprobs to be true.

object

Options that modify streaming behavior.

Show properties

boolean

When true, a final chunk is emitted before [DONE] that includes the usage field.

string

A unique identifier for the end user. Helps with monitoring and abuse detection.

string

Controls the amount of reasoning for models that support extended thinking (e.g. o3, Claude). One of none, minimal, low, medium, or high.

boolean

Whether to store the output of this request for use with model distillation or evals.

object

Key-value pairs attached to the stored output when store is true.

Response

string

A unique identifier for the completion in the form chatcmpl-....

string

Always chat.completion for non-streaming responses, or chat.completion.chunk for streaming deltas.

integer

Unix timestamp of when the completion was created.

string

The model used for the completion.

object[]

An array of completion choices. Contains one item unless n is greater than 1.

Show properties

integer

The index of this choice in the array.

object

The generated message.

Show properties

string

Always assistant.

string | null

The text content of the message. null when the model called a tool.

object[]

Tool calls made by the model, if any.

string

Why the model stopped generating. One of stop, length, tool_calls, or content_filter.

object | null

Log probability information for output tokens. Present when logprobs was true in the request.

object

Token usage for this request.

Show properties

integer

Number of tokens in the input.

integer

Number of tokens in the generated output.

integer

Total tokens consumed (prompt_tokens + completion_tokens).

string

A fingerprint representing the backend configuration used to serve the request.

Code examples

curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the Portkey AI Gateway?"}
    ],
    "temperature": 0.7,
    "max_tokens": 256
  }'

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8787/v1",
    api_key="$OPENAI_API_KEY",
    default_headers={
        "x-portkey-provider": "openai",
    },
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the Portkey AI Gateway?"},
    ],
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8787/v1",
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    "x-portkey-provider": "openai",
  },
});

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the Portkey AI Gateway?" },
  ],
  temperature: 0.7,
  max_tokens: 256,
});

console.log(response.choices[0].message.content);

Overview

Chat

Multimodal

Files & Batches

Other

POST /v1/chat/completions

Request headers

Request body

Response

Code examples

​POST /v1/chat/completions

​Request headers

​Request body

​Response

​Code examples

POST /v1/chat/completions

Request headers

Request body

Response

Code examples