Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/openai/openai-cookbook/llms.txt

Use this file to discover all available pages before exploring further.

When a large language model fails on a difficult task, the problem is often not that the model is incapable — it is that the prompt leaves no room for the model to think. Just as a person cannot multiply two three-digit numbers in their head instantaneously, a model asked to jump straight to an answer on a complex problem will often confabulate rather than reason. The techniques below give the model the time and structure it needs to work things out reliably.

​
Why complex tasks fail

Consider this problem: 
Q: A juggler has 16 balls. Half of the balls are golf balls and half of
the golf balls are blue. How many blue golf balls are there?
A:
A model answering immediately says 8 — wrong. But with a single additional instruction: 
Q: A juggler has 16 balls. Half of the balls are golf balls and half of
the golf balls are blue. How many blue golf balls are there?
A: Let's think step by step.
The model now reasons through the problem and arrives at the correct answer of 4. The capability was there all along; the prompt just didn’t create conditions for the model to use it.
The “Let’s think step by step” technique raised GPT-3’s solve rate on the MultiArith math benchmark from 18% to 79% in published research by Kojima et al. (2022).

​
Core techniques

​
Chain-of-thought prompting

Chain-of-thought (CoT) prompting asks the model to reason through intermediate steps before producing a final answer. You can trigger this in two ways. Zero-shot — just append a reasoning instruction: 
Q: If a store has 50 shirts and sells 60% of them, then receives a new
shipment of 30, how many shirts does it have?

A: Let's think step by step.
Few-shot — provide examples that include the reasoning chain: 
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each
can has 3 balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans × 3 balls = 6 new balls.
   5 + 6 = 11. The answer is 11.

Q: A juggler has 16 balls. Half are golf balls and half of those are blue.
How many blue golf balls are there?
A:
The few-shot approach lets you control the format, length, and style of the reasoning trace, which is useful when the default reasoning doesn’t fit your output requirements.
You can customize the chain-of-thought trigger to match your domain. Instead of the generic “Let’s think step by step”, try something like: “First, list the known quantities. Then, identify the unknown. Finally, solve step by step.”

​
Task decomposition

Breaking a large task into an explicit sequence of smaller subtasks improves reliability in two ways: each step is simpler, and errors don’t compound silently. Without decomposition (the model fails): 
Use the following clues to answer the multiple-choice question.

Clues:
1. Miss Scarlett was the only person in the lounge.
2. The person with the pipe was in the kitchen.
3. Colonel Mustard was the only person in the observatory.
4. Professor Plum was not in the library nor the billiard room.
5. The person with the candlestick was in the observatory.

Question: Was Colonel Mustard in the observatory with the candlestick?
(a) Yes  (b) No  (c) Unknown

Solution:
The model skips over clues 3 and 5 and answers (c) — wrong. With decomposition (the model succeeds): 
Use the following clues to answer the multiple-choice question by following
this procedure:
(1) Go through the clues one by one and note which are relevant.
(2) Combine the relevant clues to reason out the answer.
(3) Map the answer to (a), (b), or (c).

[same clues and question]

Solution:
(1) First, go through the clues one by one...
By making each step explicit, the model correctly identifies clues 3 and 5 as relevant, combines them, and arrives at (a) Yes. Decomposition also keeps the model on task. When asked to “summarize the text in its original language,” the model may drift to English. A decomposed version — “First, identify the language of the text. Second, summarize using that language.” — reliably preserves the original language.

​
Self-consistency

For tasks with a discrete answer, generate multiple independent answers at non-zero temperature and take a majority vote. Different reasoning paths often converge on the correct answer even when individual paths are flawed. 
import openai

client = openai.OpenAI()

def self_consistent_answer(prompt: str, n: int = 5) -> str:
    """Sample n answers and return the most common one."""
    responses = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        n=n,
        temperature=0.7,
    )
    answers = [choice.message.content.strip() for choice in responses.choices]
    return max(set(answers), key=answers.count)

result = self_consistent_answer(
    "Q: A store has 50 shirts. 60% are sold. 30 new arrive. How many remain?\n"
    "A: Let's think step by step."
)
print(result)
Self-consistency multiplies your API cost by the number of samples. Use it selectively on high-stakes tasks where accuracy justifies the expense. It also only helps when there are multiple valid reasoning paths — it won’t improve open-ended tasks like creative writing.

​
Selection-inference prompting

For multi-step reasoning, alternate between two specialized prompts:
  1. Selection prompt — identify which facts from the context are relevant to the current step.
  2. Inference prompt — draw a single conclusion from the selected facts.
Iterate this loop until the model has enough information to produce a final answer. This makes the reasoning process transparent and reduces the chance that the model skips necessary steps. This approach is particularly valuable for long reasoning chains (more than three or four steps) where a single chain-of-thought prompt may lose track of earlier inferences.

​
Least-to-most prompting

When the overall problem is long but individual examples in your prompt are short, chain-of-thought may not generalize. Least-to-most prompting addresses this by eliciting a decomposition from the model itself: 
To solve {question}, we need to first solve:
The model identifies a simpler sub-problem. You solve it (or let the model solve it), append the solution, then repeat until the original question is answered. This approach has shown gains from 16% to nearly 100% accuracy on benchmarks requiring long compositional reasoning.

​
Structured output with a reasoning preamble

Combine a structured prompt format with an explicit reasoning step for tasks that require both correct logic and a clean final answer: 
Using the IRS guidance below, answer whether I qualify for the EV tax
credit. Use this format for each criterion:
- {Criterion}: Let's think step by step. {explanation} {yes / no / N/A}

After checking each criterion, conclude with:
"Because of {reasons}, the answer is likely {yes or no}."

IRS guidance:
"""
[guidance text]
"""

Question: Can I claim a federal tax credit for my Toyota Prius Prime
bought in 2021?

Solution:
(1) For each criterion...
This forces the model to evaluate every criterion explicitly before committing to a conclusion, which dramatically reduces the chance of skipping a disqualifying condition.

​
Advanced: verifiers and generators

A powerful pattern for tasks with measurable correct answers is to train or prompt a verifier — a second model or prompt that judges whether a candidate answer is correct — and use it to select among multiple candidates:
1

Generate candidates

Sample several answers from the main model at non-zero temperature.
2

Score candidates

Pass each candidate to a verifier model or use a scoring heuristic (e.g., does the answer parse correctly? Does it pass unit tests?).
3

Select the best

Return the candidate with the highest verifier score.
This pattern is especially effective for code generation, where execution provides a natural verifier, and for math, where a symbolic checker can validate results.

​
Which technique should you use?

Chain of thought

Best for arithmetic, symbolic reasoning, multi-step logic, and strategy problems. Add “Let’s think step by step” to any prompt where the model is rushing to an answer.

Task decomposition

Best when the task involves multiple distinct sub-steps or when the model is losing track of requirements in a single long prompt.

Self-consistency

Best for high-stakes tasks with discrete answers where you can afford multiple API calls. Overkill for simple or open-ended tasks.

Least-to-most

Best when you’re working with long compositional problems where the model needs to generalize from short examples to long chains of reasoning.

​
Guiding principles

All of the techniques above share a common insight: reliability comes from giving the model more time and structure to reason, not from expecting it to answer difficult questions in a single inference step. Build your prompts to:
  • Break complex tasks into smaller, more reliable sub-operations.
  • Give the model space to work out intermediate steps before committing to an answer.
  • Generate multiple outputs when possible and use a discriminator to pick the best.
  • Reduce hallucination by constraining what the model can say (explicit formats, structured criteria, sentence labels).

Build docs developers (and LLMs) love

Get started for freeTalk to us