Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/openai/openai-cookbook/llms.txt

Use this file to discover all available pages before exploring further.

Fine-tuning lets you specialize an OpenAI model for a particular task by continuing its training on your own curated dataset. Rather than relying solely on prompts to steer model behavior, fine-tuning bakes your requirements directly into the model weights — enabling more consistent outputs, shorter prompts, and better performance on domain-specific tasks. This guide covers when to fine-tune, how to prepare your data, and how to run and monitor a fine-tuning job from start to finish.

​
When to fine-tune

Before investing in fine-tuning, it is worth evaluating whether prompt engineering alone can meet your needs. Fine-tuning is most valuable in the following scenarios:

Consistent style or format

When your application requires responses to follow a specific tone, structure, or schema — and prompting alone produces inconsistent results.

Shorter prompts

When you want to reduce token usage and latency by embedding complex instructions into the model itself rather than repeating them in every request.

Domain specialization

When the model needs to perform reliably in a specialized domain — such as legal, medical, or financial text — using terminology and conventions specific to that field.

Improved instruction following

When the base model struggles to follow complex multi-step instructions, fine-tuning on high-quality examples can improve reliability substantially.
Fine-tuning is not a substitute for retrieval. If your use case requires the model to access up-to-date or proprietary knowledge, consider retrieval-augmented generation (RAG) alongside or instead of fine-tuning.
Fine-tuning is not recommended for tasks where the base model has no underlying capability. If the model cannot perform a task at all in zero-shot or few-shot settings, fine-tuning on labeled examples is unlikely to help. Start by establishing a baseline and only reach for fine-tuning once you have confirmed that prompt engineering has hit a ceiling.

​
Fine-tuning methods

OpenAI’s platform supports several fine-tuning methods. The two most commonly used for chat models are supervised fine-tuning (SFT) and direct preference optimization (DPO).
SFT is the standard approach. You provide input-output pairs — a conversation that ends with an ideal assistant response — and the model is trained to reproduce those outputs. This technique works well for:
  • Teaching a specific response format (JSON, Markdown, structured reports)
  • Adjusting tone or persona
  • Distilling a larger model’s expertise into a smaller, faster model
  • Correcting instruction-following failures
SFT adjusts model weights to minimize the difference between predicted and target outputs across all of your training examples.
DPO uses pairwise comparisons — a preferred response and a rejected response for the same prompt — to teach the model which outputs are better. It is a lightweight alternative to reinforcement learning from human feedback (RLHF) that does not require a separate reward model.DPO is best suited for:
  • Aligning outputs with human preferences (tone, politeness, helpfulness)
  • Refining an already-capable model using human-rated feedback
  • Achieving nuanced behavioral alignment that is difficult to express as exact output labels
You will need a dataset of preference pairs where each sample includes a prompt, a chosen response, and a rejected response.

​
Preparing your training data

Fine-tuning for chat models uses the JSONL format. Each line is a separate JSON object representing one training example. A training example is a full conversation including a system message, one or more user turns, and the desired assistant response. 
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"}, {"role": "assistant", "content": "Paris."}]}
Every example must:
  • Include at least one user message and one assistant message
  • End with an assistant turn (the target output the model learns to produce)
  • Use the system, user, and assistant role values
A good starting point is 50–100 high-quality examples. More data generally helps, but quality matters more than quantity. Noisy or inconsistent examples will degrade your fine-tuned model.

​
Validating your data

Before uploading, check your dataset for format errors, token length distribution, and per-message role consistency. The Chat fine-tuning data preparation cookbook provides a validation script that checks error counts, estimates training cost, and surfaces common issues. 
import json
from collections import defaultdict

data_path = "training_data.jsonl"

with open(data_path) as f:
    dataset = [json.loads(line) for line in f]

format_errors = defaultdict(int)

for ex in dataset:
    if not isinstance(ex, dict):
        format_errors["data_type"] += 1
        continue
    messages = ex.get("messages", None)
    if not messages:
        format_errors["missing_messages_list"] += 1
        continue
    for message in messages:
        if "role" not in message or "content" not in message:
            format_errors["message_missing_key"] += 1
        if message.get("role") not in ("system", "user", "assistant"):
            format_errors["unrecognized_role"] += 1

if format_errors:
    print("Found errors:", dict(format_errors))
else:
    print("Dataset looks valid.")

​
Running a fine-tuning job

1

Install the OpenAI SDK

pip install --upgrade openai
Set your API key as an environment variable:
export OPENAI_API_KEY=your_api_key_here
2

Upload your training file

Upload your JSONL file to the Files API with purpose="fine-tune". The returned file ID is used when creating the job.
from openai import OpenAI
client = OpenAI()

# Upload training file
with open("training_data.jsonl", "rb") as f:
    training_file = client.files.create(file=f, purpose="fine-tune")

# Create fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4o-mini"
)

print(job.id)
3

Monitor your job

Jobs can take anywhere from a few minutes to several hours depending on dataset size. Poll the job status or list events to track progress:
# Check job status
job_status = client.fine_tuning.jobs.retrieve(job.id)
print(job_status.status)

# Stream events as training progresses
for event in client.fine_tuning.jobs.list_events(fine_tuning_job_id=job.id, limit=10):
    print(event.message)
The job transitions through validating_files → queued → running → succeeded (or failed).
4

Use your fine-tuned model

Once the job status is succeeded, retrieve the model name from the job object and use it like any other model:
job = client.fine_tuning.jobs.retrieve(job.id)
fine_tuned_model = job.fine_tuned_model

response = client.chat.completions.create(
    model=fine_tuned_model,
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

​
Hyperparameters

OpenAI exposes a small set of hyperparameters that you can tune to influence training. In most cases the defaults work well, but adjusting them can help if your loss curve shows over- or underfitting.
HyperparameterDefaultNotes
n_epochsAutoNumber of passes over the training data. A value of 3–5 is a common starting point.
batch_sizeAutoControls how many examples are processed per gradient update. Larger batches stabilize training but require more memory.
learning_rate_multiplierAutoScales the learning rate. If the model overfits, try reducing this.
job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4o-mini",
    hyperparameters={
        "n_epochs": 4,
        "learning_rate_multiplier": 0.5
    }
)
Higher epoch counts can cause overfitting — especially on small datasets. Monitor your validation loss (if you supply a validation file) and stop early if it starts increasing while training loss continues to drop.

​
Adding a validation file

To catch overfitting, provide a separate held-out validation set. The API computes validation loss at the end of each epoch and surfaces it in the training events. 
with open("validation_data.jsonl", "rb") as f:
    validation_file = client.files.create(file=f, purpose="fine-tune")

job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    validation_file=validation_file.id,
    model="gpt-4o-mini"
)

​
Using DPO

For DPO, each training example must contain a prompt and both a chosen and rejected completion. The format differs slightly from SFT: 
{
  "prompt": [{"role": "user", "content": "Explain quantum entanglement simply."}],
  "chosen": [{"role": "assistant", "content": "Quantum entanglement is when two particles become linked so that measuring one instantly affects the other, no matter the distance."}],
  "rejected": [{"role": "assistant", "content": "It's a complicated quantum mechanics phenomenon involving particle states."}]
}
Specify "dpo" as the method when creating the job: 
job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4o-mini",
    method={
        "type": "dpo",
        "dpo": {
            "hyperparameters": {"beta": 0.1}
        }
    }
)
The beta parameter controls how strongly the model is pulled toward the preferred response. Higher values enforce preferences more aggressively; lower values allow more deviation from the base model.

​
Next steps

Evaluation

Measure your fine-tuned model’s quality using LLM-as-a-judge, rule-based evals, and the Evals API.

Reinforcement fine-tuning

For tasks with verifiable outputs, RFT uses reward signals to push reasoning capabilities beyond what labeled data alone can achieve.

Build docs developers (and LLMs) love

Get started for freeTalk to us