The Core Problem: Similarity ≠ Relevance
Traditional vector-based RAG systems rely on semantic similarity to retrieve relevant document chunks. However, similarity is not the same as relevance.Vector-Based RAG Limitations
Opaque Retrieval
Vector similarity is a “black box” - it’s unclear why certain chunks were retrieved
No Context Understanding
Embeddings don’t understand document structure or relationships between sections
Surface-Level Matching
Similar words don’t guarantee relevant content for complex queries
Lost Hierarchy
Chunking destroys the document’s natural organization and context
Example: A query about “risk mitigation strategies” might retrieve chunks containing similar words like “risk” and “strategy”, but miss the actual section discussing mitigation approaches because it uses different terminology.
Reasoning-Based Retrieval
PageIndex uses LLMs to reason about which document sections are relevant to a query, simulating how human experts navigate and extract knowledge from complex documents.Two-Step Process
- Tree Structure Generation: Create a hierarchical index of the document
- Reasoning-Based Tree Search: Use LLM reasoning to navigate the tree and find relevant sections
The LLM evaluates each node based on its title, summary, and context to determine relevance - not just keyword matching.
How It Works: Tree Search
PageIndex performs retrieval through agentic tree search, where the LLM acts as an intelligent agent navigating the document structure:Step 1: Understanding the Query
The LLM first analyzes the user’s query to understand what information is being requested:Step 2: Navigating the Tree
The LLM examines node titles and summaries to decide which branches to explore:Step 3: Intelligent Pruning
The LLM can reason about which sections to skip:Comparison: Vector RAG vs Reasoning RAG
| Aspect | Vector-Based RAG | PageIndex Reasoning RAG |
|---|---|---|
| Retrieval Method | Embedding similarity | LLM reasoning over structure |
| Document Processing | Chunking (arbitrary boundaries) | Tree structure (natural sections) |
| Context Preservation | Lost in chunks | Hierarchical relationships maintained |
| Explainability | Opaque similarity scores | Traceable reasoning path |
| Query Complexity | Best for simple queries | Handles multi-step reasoning |
| Accuracy | Depends on embeddings | 98.7% on FinanceBench |
| Traceability | Approximate chunk locations | Exact page and section references |
State-of-the-Art Results
PageIndex-powered systems achieve superior performance on professional document analysis:FinanceBench Achievement
98.7% accuracy on the FinanceBench benchmark using reasoning-based RAG with PageIndex tree structures.This significantly outperforms traditional vector-based RAG solutions in financial document analysis.
Key Advantages
1. True Relevance
The LLM can determine if a section is truly relevant to the query, not just similar:2. Multi-Step Reasoning
PageIndex can handle queries requiring multiple reasoning steps:3. Domain Expertise
The LLM can apply domain knowledge during retrieval:This level of understanding is impossible with vector similarity alone, which operates purely on mathematical distance between embeddings.
Human-Like Document Navigation
PageIndex simulates how experts work with documents:- Scan the table of contents to identify relevant sections
- Use context and structure to navigate to the right location
- Apply domain knowledge to interpret section relevance
- Follow logical paths through hierarchical information
When to Use Reasoning-Based RAG
Reasoning-based RAG with PageIndex is especially powerful for:- Professional documents requiring domain expertise (financial, legal, technical)
- Long documents exceeding LLM context windows
- Complex queries requiring multi-step reasoning
- Scenarios requiring explainability and traceability
- Hierarchically structured documents (reports, manuals, textbooks)
Next Steps
Vectorless Approach
Learn why PageIndex doesn’t need vector databases
Try Vectorless RAG
Hands-on tutorial with working code examples