Batches

The Batches API lets you submit a collection of requests to be processed asynchronously. This is useful for large workloads where you want to avoid rate limits and reduce cost. Portkey proxies batch requests to the configured provider.

Batch processing is provider-dependent. Check your provider’s documentation for supported endpoints and file formats.

Create a batch

POST /v1/batches Creates a new batch job from a previously uploaded .jsonl file.

Request

string

required

The provider to route the request to (e.g. openai).

string

required

Your Portkey API key or provider API key.

string

required

The ID of the uploaded file containing the batch requests. Use POST /v1/files to upload this file with purpose=batch.

string

required

The API endpoint to run the batch against. For example, /v1/chat/completions.

string

required

The time window for completion. Currently 24h is the most commonly supported value.

object

Optional key-value metadata to attach to the batch object.

Response

string

Unique identifier for the batch.

string

Always batch.

string

The API endpoint the batch runs against.

string

Current status of the batch. Values: validating, in_progress, completed, failed, cancelled.

string

The ID of the input file.

string

The ID of the output file once the batch has completed.

number

Unix timestamp when the batch was created.

number

Unix timestamp when the batch completed, or null if still in progress.

object

Show properties

number

Total number of requests in the batch.

number

Number of requests completed.

number

Number of requests that failed.

curl https://your-gateway.example.com/v1/batches \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: YOUR_API_KEY" \
  -H "x-portkey-provider: openai" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://your-gateway.example.com/v1",
    api_key="YOUR_API_KEY",
    default_headers={
        "x-portkey-provider": "openai",
    },
)

batch = client.batches.create(
    input_file_id="file-abc123",
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

print(batch.id, batch.status)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://your-gateway.example.com/v1",
  apiKey: "YOUR_API_KEY",
  defaultHeaders: {
    "x-portkey-provider": "openai",
  },
});

const batch = await client.batches.create({
  input_file_id: "file-abc123",
  endpoint: "/v1/chat/completions",
  completion_window: "24h",
});

console.log(batch.id, batch.status);

List batches

GET /v1/batches Returns a list of batch jobs for the configured provider.

Request

string

required

The provider to route the request to.

string

required

Your Portkey API key or provider API key.

number

default:"20"

Maximum number of batches to return.

string

Cursor for pagination. Pass the id of the last batch from the previous page.

Response

string

Always list.

array

Array of batch objects.

boolean

Whether additional batches are available beyond this page.

curl https://your-gateway.example.com/v1/batches \
  -H "x-portkey-api-key: YOUR_API_KEY" \
  -H "x-portkey-provider: openai"

batches = client.batches.list()
for batch in batches.data:
    print(batch.id, batch.status)

const batches = await client.batches.list();
for (const batch of batches.data) {
  console.log(batch.id, batch.status);
}

Retrieve a batch

GET /v1/batches/:id Retrieves a single batch job by its ID. Use this to poll the status of an in-progress batch.

Request

string

required

The ID of the batch to retrieve.

string

required

The provider to route the request to.

string

required

Your Portkey API key or provider API key.

Response

Returns a single batch object. See Create a batch for the full field list.

curl https://your-gateway.example.com/v1/batches/batch-xyz789 \
  -H "x-portkey-api-key: YOUR_API_KEY" \
  -H "x-portkey-provider: openai"

batch = client.batches.retrieve("batch-xyz789")
print(batch.status)

const batch = await client.batches.retrieve("batch-xyz789");
console.log(batch.status);

Get batch output

GET /v1/batches/*/output Retrieves the output file content for a completed batch. The wildcard path segment accepts the batch ID.

The batch must have status: completed before output is available. Retrieve the batch first to check its output_file_id, then use GET /v1/files/:id/content to download the output directly.

Request

string

required

The batch ID. For example, /v1/batches/batch-xyz789/output.

string

required

The provider to route the request to.

string

required

Your Portkey API key or provider API key.

Response

Returns the raw .jsonl output content. Each line corresponds to a response for a single request in the batch.

curl https://your-gateway.example.com/v1/batches/batch-xyz789/output \
  -H "x-portkey-api-key: YOUR_API_KEY" \
  -H "x-portkey-provider: openai" \
  --output batch_results.jsonl

Cancel a batch

POST /v1/batches/:id/cancel Cancels an in-progress batch. Requests that have already been processed are not reversed.

Cancellation is asynchronous. The batch status transitions to cancelling and then cancelled. Already-processed requests are not refunded.

Request

string

required

The ID of the batch to cancel.

string

required

The provider to route the request to.

string

required

Your Portkey API key or provider API key.

Response

Returns the updated batch object with status: cancelling.

curl https://your-gateway.example.com/v1/batches/batch-xyz789/cancel \
  -X POST \
  -H "x-portkey-api-key: YOUR_API_KEY" \
  -H "x-portkey-provider: openai"

batch = client.batches.cancel("batch-xyz789")
print(batch.status)  # cancelling

const batch = await client.batches.cancel("batch-xyz789");
console.log(batch.status); // cancelling

Overview

Chat

Multimodal

Files & Batches

Other

Create a batch

Request

Response

List batches

Request

Response

Retrieve a batch

Request

Response

Get batch output

Request

Response

Cancel a batch

Request

Response

​Create a batch

​Request

​Response

​List batches

​Request

​Response

​Retrieve a batch

​Request

​Response

​Get batch output

​Request

​Response

​Cancel a batch

​Request

​Response

Create a batch

Request

Response

List batches

Request

Response

Retrieve a batch

Request

Response

Get batch output

Request

Response

Cancel a batch

Request

Response