File search with the OpenAI Responses API

Building a retrieval system from scratch means parsing documents, chunking text, generating embeddings, managing a vector database, writing retrieval logic, and wiring it all into your LLM calls. The file search tool in the Responses API handles all of that for you. Upload your files, create a vector store, and pass the store ID in your API call — OpenAI takes care of chunking, embedding, retrieval scoring, and synthesis. The result is grounded, document-cited answers in a single request.

How it works

When you attach the file_search tool to a Responses API call, the model:

Decides whether the query requires document retrieval.
Runs a hybrid search (keyword + semantic) against your vector store.
Retrieves the most relevant chunks, ranked by a relevancy score.
Synthesizes a response grounded in the retrieved content, with inline file citations.

This replaces the traditional multi-step RAG pipeline with a single, managed API surface.

Quickstart

Upload a file

Upload your document using the Files API with purpose="assistants". The file is stored on OpenAI’s servers and made available for indexing.

from openai import OpenAI

client = OpenAI()

with open("knowledge_base.pdf", "rb") as f:
    file = client.files.create(file=f, purpose="assistants")

print(file.id)  # file-abc123

Create a vector store

Create a vector store and associate your uploaded file with it. OpenAI reads the file, splits it into chunks, generates embeddings, and indexes them automatically.

vector_store = client.vector_stores.create(
    name="My Knowledge Base",
    file_ids=[file.id]
)

print(vector_store.id)  # vs_abc123

Indexing is asynchronous. Check vector_store.file_counts.completed to confirm the file has finished processing before querying.

Query using file search

Pass the vector store ID in the file_search tool definition. The model retrieves relevant context and generates a grounded answer.

response = client.responses.create(
    model="gpt-4o",
    input="What does the document say about pricing?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id]
    }]
)
print(response.output_text)

Uploading multiple files in parallel

For larger knowledge bases, upload files concurrently using ThreadPoolExecutor to reduce wall-clock time.

import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from openai import OpenAI

client = OpenAI()

def upload_single_file(file_path: str, vector_store_id: str) -> dict:
    file_name = os.path.basename(file_path)
    try:
        file_response = client.files.create(
            file=open(file_path, "rb"),
            purpose="assistants"
        )
        client.vector_stores.files.create(
            vector_store_id=vector_store_id,
            file_id=file_response.id
        )
        return {"file": file_name, "status": "success"}
    except Exception as e:
        return {"file": file_name, "status": "failed", "error": str(e)}


# Create the vector store first
vector_store = client.vector_stores.create(name="Docs Knowledge Base")

pdf_files = ["doc1.pdf", "doc2.pdf", "doc3.pdf"]

with ThreadPoolExecutor(max_workers=10) as executor:
    futures = {
        executor.submit(upload_single_file, path, vector_store.id): path
        for path in pdf_files
    }
    for future in as_completed(futures):
        result = future.result()
        print(result["file"], "->", result["status"])

Standalone vector search

You can query the vector store directly — without an LLM call — using the vector search API. This is useful for inspecting retrieval quality or building custom ranking logic.

search_results = client.vector_stores.search(
    vector_store_id=vector_store.id,
    query="What's the refund policy?"
)

for result in search_results.data:
    print(f"{result.filename}  score={result.score:.4f}")
    print(result.content[0].text[:200])
    print()

Scores are calculated using hybrid search — a combination of BM25 keyword matching and semantic similarity — and range from 0 to 1. Higher scores indicate greater relevance.

Inspecting retrieved chunks

To see exactly which chunks were retrieved and used to generate an answer, include output[*].file_search_call.search_results in the include parameter.

response = client.responses.create(
    model="gpt-4o",
    input="Summarize the key findings from the Q4 report.",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id]
    }],
    include=["output[*].file_search_call.search_results"]
)

# The file search call is output[0]; the text response is output[1]
file_search_call = response.output[0]
for chunk in file_search_call.search_results:
    print(f"From: {chunk.filename}  score={chunk.score:.4f}")

# Extract cited filenames from annotations
annotations = response.output[1].content[0].annotations
cited_files = {a.filename for a in annotations}
print("Files cited in response:", cited_files)

print(response.output[1].content[0].text)

The annotations list on the text output item maps citation markers in the response text to the specific file chunks that were used. This gives you full traceability from answer back to source.

Multi-turn file search conversations

Because the Responses API is stateful, you can run multi-turn conversations that reference documents without re-specifying the vector store on every turn.

# First turn
response = client.responses.create(
    model="gpt-4o",
    input="What are the main topics covered in the handbook?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id]
    }]
)
print(response.output_text)

# Follow-up — context is preserved automatically
follow_up = client.responses.create(
    model="gpt-4o",
    input="Which of those topics covers remote work policies?",
    previous_response_id=response.id,
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id]
    }]
)
print(follow_up.output_text)

Supported file types

Documents

PDF, DOCX, TXT, MD, HTML, RTF, TeX

Data

CSV, TSV, JSON, JSONL, XML

Code

Python, JavaScript, TypeScript, Java, C, C++, Go, Ruby, PHP, Shell, and more

Individual file size is limited to 512 MB. A single vector store can hold up to 10,000 files. For files larger than this limit, consider splitting them before uploading.

Chunking and retrieval scoring

OpenAI automatically splits your documents into overlapping chunks of approximately 800 tokens each. You do not need to choose a chunking strategy — the default works well for most document types. During retrieval, chunks are ranked using a hybrid scoring model:

Semantic search — embedding-based cosine similarity captures meaning and paraphrase
Keyword search (BM25) — term frequency matching captures exact phrases and named entities
Hybrid reranking — scores from both signals are fused and reranked before the top results are passed to the model

In practice, this means a query like “What’s Deep Research?” returns highly relevant chunks even when the document uses phrasing like “our deep research capability” rather than the exact query string.

Metadata filtering

You can attach metadata to files at upload time and filter on it during search. This is useful for multi-tenant applications or document collections with logical partitions (by department, date range, or document type).

# Attach metadata when creating the vector store file association
client.vector_stores.files.create(
    vector_store_id=vector_store.id,
    file_id=file.id,
    attributes={"department": "engineering", "year": 2024}
)

# Filter at query time
response = client.responses.create(
    model="gpt-4o",
    input="What were the engineering team's goals?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
        "filters": {
            "type": "eq",
            "key": "department",
            "value": "engineering"
        }
    }]
)

Metadata filtering narrows the retrieval pool before scoring. If your filter is too restrictive, the model may not find relevant chunks and will say so in its response rather than hallucinating.

Next steps

Responses API intro

Learn the fundamentals of the stateful Responses API, including multi-turn conversations and built-in tools.

Structured Outputs

Guarantee that model responses match a JSON schema you define, with full Pydantic support.

Responses API

Agent Patterns

Real-World Use Cases

Open Models

File search with the OpenAI Responses API

How it works

Quickstart

Uploading multiple files in parallel

Standalone vector search

Inspecting retrieved chunks

Multi-turn file search conversations

Supported file types

Documents

Data

Code

Chunking and retrieval scoring

Metadata filtering

Next steps

Responses API intro

Structured Outputs

Build docs developers (and LLMs) love

Responses API

Agent Patterns

Real-World Use Cases

Open Models

Documentation Index

​How it works

​Quickstart

​Uploading multiple files in parallel

​Standalone vector search

​Inspecting retrieved chunks

​Multi-turn file search conversations

​Supported file types

Documents

Data

Code

​Chunking and retrieval scoring

​Metadata filtering

​Next steps

Responses API intro

Structured Outputs

Build docs developers (and LLMs) love

How it works

Quickstart

Uploading multiple files in parallel

Standalone vector search

Inspecting retrieved chunks

Multi-turn file search conversations

Supported file types

Chunking and retrieval scoring

Metadata filtering

Next steps