Fine-tuning gpt-oss models with Hugging Face

Fine-tuning a gpt-oss model lets you specialize its behavior for a specific task — whether that’s domain-specific reasoning, multilingual output, or a custom response style — without starting from scratch. Because gpt-oss ships as open weights, you have full control over the training process, the data, and the resulting model. This guide walks through fine-tuning gpt-oss-20b using Hugging Face Transformers and the TRL library’s SFTTrainer, covering data preparation, model loading, the training loop, and how to handle gpt-oss-specific behaviors like the Harmony format and chain-of-thought in your training data.

This workflow is designed to run on a single H100 GPU with 80 GB of memory. If you are using a smaller GPU, reduce per_device_train_batch_size and max_seq_length in the training arguments. You can also use LoRA (see below) to dramatically reduce VRAM requirements.

Install dependencies

Set up a fresh Python environment and install the required libraries:

pip install -U transformers accelerate trl torch triton==3.4 kernels datasets peft

trl provides the SFTTrainer for supervised fine-tuning. peft adds LoRA support for memory-efficient training. kernels and triton are needed for MXFP4 inference with Transformers.

Prepare your training data

OpenAI message format

The most convenient format for fine-tuning data is the same message structure used by the OpenAI Chat Completions API. Each training example is a list of {"role": ..., "content": ...} dictionaries:

[
  {
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of Australia?"},
      {"role": "assistant", "content": "The capital of Australia is Canberra."}
    ]
  },
  {
    "messages": [
      {"role": "user", "content": "Summarize the following article in one sentence."},
      {"role": "assistant", "content": "The article describes recent advances in quantum computing."}
    ]
  }
]

Save your data as a JSONL file (one JSON object per line) or load it from a Hugging Face dataset.

Load a dataset

from datasets import load_dataset

# Load from a JSONL file
dataset = load_dataset("json", data_files="train.jsonl", split="train")

# Or load from the Hugging Face Hub
dataset = load_dataset("your-org/your-dataset", split="train")

Converting to Hugging Face format

If your data is already in OpenAI format, SFTTrainer can apply the Harmony chat template automatically using the tokenizer.apply_chat_template method. Verify that your dataset has a messages column containing the list of role/content dictionaries:

def format_example(example):
    # SFTTrainer expects a "messages" key with a list of role/content dicts
    return {"messages": example["messages"]}

dataset = dataset.map(format_example)

Load the model and tokenizer

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "openai/gpt-oss-20b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",       # automatically distributes across available GPUs
    torch_dtype="auto",      # uses MXFP4 by default when supported
)

If you are on hardware that does not support MXFP4 (Hopper architecture or later — H100, GB200, RTX 50xx), set torch_dtype=torch.bfloat16. This increases memory usage to approximately 48 GB for the 20B model.

Adding LoRA for memory-efficient fine-tuning

LoRA (Low-Rank Adaptation) dramatically reduces the number of trainable parameters, making it possible to fine-tune on smaller GPUs. Configure a LoRA adapter using peft:

from peft import LoraConfig, get_peft_model, TaskType

lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,                       # rank — higher means more capacity, more memory
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Outputs something like: trainable params: 41,943,040 || all params: 20,000,000,000

Fine-tune with SFTTrainer

SFTTrainer from TRL handles the training loop, chat template application, and sequence packing automatically. Pass your model, tokenizer, dataset, and training arguments:

from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer, SFTConfig

model = AutoModelForCausalLM.from_pretrained("openai/gpt-oss-20b")
tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    args=SFTConfig(
        output_dir="./finetuned-model",
        num_train_epochs=3,
        per_device_train_batch_size=4,
    )
)

trainer.train()

Full training configuration

For more control over training, provide a complete SFTConfig:

from trl import SFTTrainer, SFTConfig

training_args = SFTConfig(
    output_dir="./finetuned-gpt-oss-20b",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=2,      # effective batch size = 4 * 2 = 8
    learning_rate=2e-5,
    lr_scheduler_type="cosine",
    warmup_ratio=0.05,
    max_seq_length=2048,
    logging_steps=10,
    save_strategy="epoch",
    bf16=True,                          # use bfloat16 training precision
    gradient_checkpointing=True,        # trades compute for lower memory usage
)

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    args=training_args,
    peft_config=lora_config,            # omit this if not using LoRA
)

trainer.train()

SFTTrainer applies tokenizer.apply_chat_template to your messages column automatically, so the Harmony format is handled without manual tokenization.

Save the fine-tuned model

trainer.save_model("./finetuned-gpt-oss-20b")
tokenizer.save_pretrained("./finetuned-gpt-oss-20b")

If you used LoRA, save and optionally merge the adapter weights into the base model:

# Save the LoRA adapter only
model.save_pretrained("./finetuned-gpt-oss-20b-lora")

# Or merge adapter into base weights for a standalone model
merged_model = model.merge_and_unload()
merged_model.save_pretrained("./finetuned-gpt-oss-20b-merged")

OpenAI Harmony format in training data

The Harmony format defines how the gpt-oss models expect their training examples to be encoded. When you use SFTTrainer with the Transformers chat template, Harmony encoding is applied automatically. However, if you are constructing training examples at the token level, be aware of the key rules:

Training targets (the assistant’s response) should end with <|return|> — not <|end|>. The <|return|> token signals end of generation during sampling. When you store a completed response in conversation history for a subsequent turn, replace <|return|> with <|end|>.
The analysis channel (chain-of-thought) should be included in training targets if you want the model to learn reasoning patterns. See the section below on handling chain-of-thought.
The reasoning effort level (low, medium, high) is set in the system message and is part of the training signal.

Handling chain-of-thought in training data

The gpt-oss models output raw chain-of-thought (CoT) to the analysis channel before producing a final answer on the final channel. When preparing training data, you need to decide whether to include CoT in your targets. Include CoT when: you want to teach the model how to reason through problems, you are fine-tuning for a task where reasoning quality matters, or you are doing interpretability research. Exclude CoT when: you only care about the final answer format, your dataset does not contain reasoning traces, or you want to reduce training sequence length.

Do not show analysis channel content to end users. The model’s raw chain-of-thought has not been trained to the same safety standards as its final channel output, and may contain content that is not appropriate for user-facing applications.

If your training data includes CoT, structure the assistant messages to reflect the Harmony channel separation:

# Example training conversation with chain-of-thought
training_example = {
    "messages": [
        {"role": "system", "content": "Reasoning: high"},
        {"role": "user", "content": "What is 17 × 24?"},
        # CoT in the analysis channel (internal reasoning)
        {"role": "assistant", "content": "<|channel|>analysis<|message|>17 × 24. I can compute this as (17 × 20) + (17 × 4) = 340 + 68 = 408.<|end|>"},
        # Final answer in the final channel
        {"role": "assistant", "content": "<|channel|>final<|message|>17 × 24 = 408.<|return|>"}
    ]
}

When CoT is not available in your training data, omit the analysis message and include only the final response.

Verifying your fine-tuned model

After fine-tuning, verify that the model produces well-formed outputs and that the chat template is applied correctly.

Quick inference check

Run a simple generation to check that outputs look correct:

from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="./finetuned-gpt-oss-20b",
    tokenizer=tokenizer,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "What is the capital of France?"}
]

output = pipe(messages, max_new_tokens=256)
print(output[0]["generated_text"][-1]["content"])

Check Harmony channel routing

Verify that the model correctly separates reasoning from final output by inspecting the raw generated tokens. If you see content on the final channel, the Harmony template is working correctly:

# Raw token-level verification using openai-harmony
from openai_harmony import load_harmony_encoding, HarmonyEncodingName, Role

encoding = load_harmony_encoding(HarmonyEncodingName.HARMONY_GPT_OSS)
# Inspect that parsed messages have the expected channels

API-level verification

If you are serving the fine-tuned model via vLLM or Transformers serve, use the verification scripts from the gpt-oss GitHub repository to run tool-calling smoke tests against the Chat Completions and Responses API endpoints:

# Clone the gpt-oss repository
git clone https://github.com/openai/gpt-oss
cd gpt-oss

# Run tool calling verification against your local server
node evals/verify.js --base-url http://localhost:8000/v1 --model openai/gpt-oss-20b

These tests verify that function calling works correctly and that the API response shapes match what OpenAI SDKs expect.

Next steps

Run locally with Ollama or LM Studio

Test your fine-tuned model locally before deploying it to a server.

gpt-oss overview

Review the Harmony format, model variants, and hosting options for gpt-oss models.

Responses API

Agent Patterns

Real-World Use Cases

Open Models

Fine-tuning gpt-oss models with Hugging Face

Install dependencies

Prepare your training data

OpenAI message format

Load a dataset

Converting to Hugging Face format

Load the model and tokenizer

Adding LoRA for memory-efficient fine-tuning

Fine-tune with SFTTrainer

Full training configuration

Save the fine-tuned model

OpenAI Harmony format in training data

Handling chain-of-thought in training data

Verifying your fine-tuned model

Quick inference check

Check Harmony channel routing

API-level verification

Next steps

Run locally with Ollama or LM Studio

gpt-oss overview

Build docs developers (and LLMs) love

Responses API

Agent Patterns

Real-World Use Cases

Open Models

Documentation Index

​Install dependencies

​Prepare your training data

​OpenAI message format

​Load a dataset

​Converting to Hugging Face format

​Load the model and tokenizer

​Adding LoRA for memory-efficient fine-tuning

​Fine-tune with SFTTrainer

​Full training configuration

​Save the fine-tuned model

​OpenAI Harmony format in training data

​Handling chain-of-thought in training data

​Verifying your fine-tuned model

​Quick inference check

​Check Harmony channel routing

​API-level verification

​Next steps

Run locally with Ollama or LM Studio

gpt-oss overview

Build docs developers (and LLMs) love

Install dependencies

Prepare your training data

OpenAI message format

Load a dataset

Converting to Hugging Face format

Load the model and tokenizer

Adding LoRA for memory-efficient fine-tuning

Fine-tune with SFTTrainer

Full training configuration

Save the fine-tuned model

OpenAI Harmony format in training data

Handling chain-of-thought in training data

Verifying your fine-tuned model

Quick inference check

Check Harmony channel routing

API-level verification

Next steps