Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/openai/openai-cookbook/llms.txt

Use this file to discover all available pages before exploring further.

OpenAI’s gpt-oss models are open-weight reasoning models that you can download, self-host, and customize without going through the OpenAI API. They bring the same reasoning architecture as OpenAI’s hosted models to your own infrastructure — giving you direct control over latency, cost, data residency, and model behavior. Whether you’re running on a consumer laptop with Ollama or a fleet of H100s with vLLM, gpt-oss is designed to fit into your stack.

​
Available models

Two model sizes are available, both quantized in MXFP4 format by default:

gpt-oss-20b

The smaller model. Requires approximately 16 GB of VRAM (or unified memory on Apple Silicon). Well-suited for high-end consumer GPUs, MacBook Pros with M-series chips, and single-GPU cloud instances.

gpt-oss-120b

The full-sized model. Requires 60 GB or more of VRAM. Designed for multi-GPU workstations, H100-class data center hardware, or large unified-memory Mac Pro configurations.
Both models are reasoning models: they generate an internal chain of thought before producing a final answer. You can control reasoning depth — low, medium, or high — through the system message when running your own inference stack.

​
OpenAI Harmony response format

The gpt-oss models were trained on the OpenAI Harmony response format, a structured prompt interface that defines how conversations, tool calls, and chain-of-thought are encoded as tokens. Harmony is what allows the models to separate internal reasoning from user-facing output, and to perform tool calls mid-reasoning.
If you use a compatible provider — Ollama, LM Studio, vLLM, or Hugging Face Transformers — the chat template handles Harmony automatically. You only need to work with Harmony directly if you are building a custom inference solution.

​
Roles and channels

Every message in a Harmony conversation carries a role and, for assistant messages, a channel:
RolePurpose
systemSpecifies reasoning effort, dates, and available built-in tools
developerProvides the system prompt (instructions) and function tool definitions
userRepresents user input
assistantModel output — either a reasoning step, a tool call, or a final response
toolThe result of a tool call, returned to the model
Assistant messages can appear on three channels:
ChannelPurpose
finalThe user-facing answer — safe to display
analysisInternal chain-of-thought reasoning — do not show to users
commentaryFunction tool call invocations and optional preambles
The analysis channel contains raw chain-of-thought output. The model has not been trained to the same safety standards in its reasoning as in its final output. Always filter analysis messages before showing any model output to end users.

​
The openai-harmony library

OpenAI publishes the openai-harmony library on PyPI to handle Harmony encoding and decoding automatically: 
from openai_harmony import (
    Author,
    Conversation,
    DeveloperContent,
    HarmonyEncodingName,
    Message,
    Role,
    SystemContent,
    ToolDescription,
    load_harmony_encoding,
    ReasoningEffort
)

encoding = load_harmony_encoding(HarmonyEncodingName.HARMONY_GPT_OSS)

system_message = (
    SystemContent.new()
        .with_reasoning_effort(ReasoningEffort.HIGH)
        .with_conversation_start_date("2025-06-28")
)

developer_message = (
    DeveloperContent.new()
        .with_instructions("You are a helpful assistant.")
)

convo = Conversation.from_messages([
    Message.from_role_and_content(Role.SYSTEM, system_message),
    Message.from_role_and_content(Role.DEVELOPER, developer_message),
    Message.from_role_and_content(Role.USER, "What is 2 + 2?"),
])

tokens = encoding.render_conversation_for_completion(convo, Role.ASSISTANT)

​
OpenAI API compatibility

Because popular inference servers implement a Chat Completions-compatible API on top of Harmony, you can point the standard OpenAI Python client at any local provider with minimal changes. Only base_url and api_key need to change: 
from openai import OpenAI

# When running via a compatible provider (e.g., vLLM, Ollama)
client = OpenAI(
    base_url="http://localhost:11434/v1",  # local provider URL
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="gpt-oss-20b",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
This means existing code written for the OpenAI API can be redirected to a local gpt-oss instance by changing two lines. Function calling, streaming, and structured output all work the same way through compatible providers.

​
When to use open models vs. the hosted API

Choosing between gpt-oss and OpenAI’s hosted API depends on your requirements around data, cost, and customization:

Data privacy and compliance

Self-hosting keeps all prompts and completions on infrastructure you control. No data leaves your environment — important for regulated industries or sensitive workloads.

Cost at scale

Once you have hardware, inference is effectively free. High-volume workloads that would generate large API bills can be significantly cheaper to run locally.

Customization and fine-tuning

Open weights mean you can fine-tune the model on your own data, adjust the chat template, or modify model behavior in ways the hosted API does not support.

Offline and air-gapped environments

Run the model without any internet connectivity — useful for edge deployments, on-device applications, or environments without external network access.
The hosted API remains the better choice when you want the latest model updates automatically, don’t have GPU hardware available, or need managed scaling without operational overhead.

​
Hosting options

gpt-oss models can run across a range of environments:

Ollama

One-command install for Mac, Linux, and Windows. Exposes a Chat Completions-compatible API at localhost:11434. Best for local development and consumer hardware.

LM Studio

GUI-based desktop app for Windows, macOS, and Linux. Loads GGUF models via llama.cpp or Apple MLX, and exposes a local API server. Great for non-developers or quick experimentation.

vLLM

High-throughput, production-grade inference engine. Exposes both a Chat Completions API and a Responses API. Designed for dedicated GPU servers (H100 and above).

Hugging Face Transformers

Flexible Python-based inference. Run with a pipeline, low-level generate calls, or use transformers serve for a hosted endpoint. Also the foundation for fine-tuning.
For local development on consumer hardware, start with Ollama — it handles the Harmony chat template automatically and exposes an OpenAI-compatible API with a single command. For production deployments on dedicated GPU hardware, use vLLM for best throughput.

​
Next steps

Run locally with Ollama or LM Studio

Step-by-step setup to run gpt-oss-20b on your own machine with Ollama or LM Studio.

Fine-tune with Hugging Face

Fine-tune gpt-oss on your own data using Hugging Face Transformers and TRL.

Build docs developers (and LLMs) love

Get started for freeTalk to us