Documentation Index
Fetch the complete documentation index at: https://mintlify.com/avnlp/dspy-opt/llms.txt
Use this file to discover all available pages before exploring further.
The HotpotQA pipeline is built for questions that cannot be answered from a single passage — each question requires chaining evidence across two or more documents. The pipeline uses the distractor subset of HotpotQA, which interleaves the relevant supporting paragraphs with distractor passages, making both retrieval precision and multi-hop reasoning critical. Documents are indexed into a HotpotQA Weaviate collection with title and category metadata to guide the hybrid search filter step.
Dataset
| Property | Value |
|---|
| HuggingFace ID | hotpotqa/hotpot_qa |
| Subset | distractor |
| Split | train (optimization) / test (evaluation) |
| Weaviate collection | HotpotQA |
| Complexity type | Multi-hop reasoning |
Pipeline Class
The HotpotQARAG class is defined in hotpotqa_rag_module.py and subclasses dspy.Module. It composes the five shared utility stages — query rewriting, sub-query generation, metadata extraction, hybrid retrieval, and chain-of-thought answer generation — into a single forward() call.
from dspy_opt.hotpotqa.hotpotqa_rag_module import HotpotQARAG
class HotpotQARAG(dspy.Module):
def __init__(
self,
query_rewriter: QueryRewriter,
sub_query_generator: SubQueryGenerator,
metadata_extractor: MetadataExtractor,
metadata_schema: Dict[str, Any],
weaviate_retriever: WeaviateRetriever,
embedding_model: SentenceTransformer,
top_k: int = 3,
): ...
def forward(self, question: str) -> dspy.Prediction:
"""Execute the complete RAG pipeline."""
forward() Return Fields
forward() returns a dspy.Prediction containing the following fields:
| Field | Description |
|---|
question | The original input question (passed through unchanged) |
rewritten_query | Search-optimized version of the question produced by QueryRewriter |
sub_queries | List of decomposed sub-queries from SubQueryGenerator |
retrieved_context | Deduplicated list of passages returned by WeaviateRetriever |
answer | Concise answer generated by dspy.ChainOfThought |
reasoning | Explanation of how the answer was derived across multiple hops |
MetadataExtractor parses each query against the following JSON schema to produce Weaviate filter values:
| Field | Type | Description |
|---|
title | string | The main title or name of the subject |
category | string | Primary category or type of content |
metadata_schema:
properties:
title:
type: "string"
description: "The main title or name of the subject"
category:
type: "string"
description: "Primary category or type of content"
For multi-hop questions, the SubQueryGenerator is especially impactful: it decomposes the original question into independent sub-queries, each of which can retrieve a different supporting document. The MetadataExtractor then filters each sub-query retrieval independently by title and category.
Models
| Role | Model |
|---|
| Answer LLM | groq/qwen3-32b |
| Extractor LLM | groq/llama-3.3-70b-versatile |
| Embedding | Qwen/Qwen3-Embedding-0.6B |
| Evaluator LLM | groq/qwen3-32b |
Scripts
| Script | Description |
|---|
hotpotqa_indexing.py | Load dataset from HuggingFace, extract metadata, embed, and store in Weaviate |
hotpotqa_rag_module.py | Pipeline class definition — imported by optimizer and evaluation scripts |
hotpotqa_rag_mipro.py | Run MIPROv2 optimization |
hotpotqa_rag_copro.py | Run COPRO optimization |
hotpotqa_rag_bootstrap_few_shot.py | Run BootstrapFewShot optimization |
hotpotqa_rag_simba.py | Run SIMBA optimization |
hotpotqa_rag_gepa.py | Run GEPA optimization |
hotpotqa_rag_evaluation.py | Evaluate the optimized pipeline with DeepEval metrics |
Configuration Files
| File | Description |
|---|
hotpotqa_indexing_config.yml | Indexing parameters: embedding model, metadata schema, collection name |
hotpotqa_rag_mipro_config.yml | MIPROv2 parameters: max_bootstrapped_demos, max_labeled_demos, auto |
hotpotqa_rag_copro_config.yml | COPRO parameters: breadth, depth, init_temperature |
hotpotqa_rag_bootstrap_few_shot_config.yml | BootstrapFewShot parameters: max_bootstrapped_demos, max_rounds |
hotpotqa_rag_simba_config.yml | SIMBA parameters: bsize, num_candidates, max_steps, max_demos |
hotpotqa_rag_gepa_config.yml | GEPA parameters: max_full_evals, reflection_minibatch_size, candidate_selection_strategy |
hotpotqa_rag_evaluation_config.yml | Evaluation settings and DeepEval metric thresholds |
MIPROv2 Configuration
answer_llm:
model: "groq/qwen3-32b"
api_key_env: "GROQ_API_KEY"
extractor_llm:
model: "groq/llama-3.3-70b-versatile"
api_key_env: "GROQ_API_KEY"
embedding:
model: "Qwen/Qwen3-Embedding-0.6B"
tokenizer_kwargs:
padding_side: "left"
weaviate:
url_env: "WEAVIATE_URL"
api_key_env: "WEAVIATE_API_KEY"
collection_name: "HotpotQA"
top_k: 5
dataset:
name: "hotpotqa/hotpot_qa"
subset_name: "distractor"
split: "train"
optimizer:
max_bootstrapped_demos: 3
max_labeled_demos: 16
auto: "medium"
Running the Pipeline
All scripts must be run from the hotpotqa/ directory so that relative config file paths resolve correctly.
# Index documents into Weaviate
cd src/dspy_opt/hotpotqa
python hotpotqa_indexing.py
# Run MIPROv2 optimization
cd src/dspy_opt/hotpotqa
python hotpotqa_rag_mipro.py
# Run SIMBA optimization
cd src/dspy_opt/hotpotqa
python hotpotqa_rag_simba.py
# Run GEPA optimization
cd src/dspy_opt/hotpotqa
python hotpotqa_rag_gepa.py
# Evaluate the optimized pipeline
cd src/dspy_opt/hotpotqa
python hotpotqa_rag_evaluation.py
Programmatic Usage
import dspy
from sentence_transformers import SentenceTransformer
from dspy_opt.hotpotqa.hotpotqa_rag_module import HotpotQARAG
from dspy_opt.utils.metadata_extractor import MetadataExtractor
from dspy_opt.utils.query_rewriter import QueryRewriter
from dspy_opt.utils.sub_query_generator import SubQueryGenerator
from dspy_opt.utils.weaviate_retriever import WeaviateRetriever
# Configure LLMs
answer_lm = dspy.LM("groq/qwen3-32b", api_key="your-groq-api-key")
extractor_lm = dspy.LM("groq/llama-3.3-70b-versatile", api_key="your-groq-api-key")
dspy.configure(lm=answer_lm)
# Initialize components
query_rewriter = QueryRewriter()
sub_query_generator = SubQueryGenerator()
metadata_extractor = MetadataExtractor(extractor_llm=extractor_lm)
embedding_model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")
retriever = WeaviateRetriever(
weaviate_url="your-weaviate-url",
weaviate_api_key="your-weaviate-api-key",
collection_name="HotpotQA",
top_k=5,
)
metadata_schema = {
"properties": {
"title": {"type": "string", "description": "The main title or name of the subject"},
"category": {"type": "string", "description": "Primary category or type of content"},
}
}
# Build and run the pipeline
pipeline = HotpotQARAG(
query_rewriter=query_rewriter,
sub_query_generator=sub_query_generator,
metadata_extractor=metadata_extractor,
metadata_schema=metadata_schema,
weaviate_retriever=retriever,
embedding_model=embedding_model,
top_k=5,
)
result = pipeline("What nationality was the director of the film that won the 1972 Academy Award for Best Picture?")
print(result.answer)
print(result.reasoning)