Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Jatin-Mehra119/PDF-Insight-Beta/llms.txt

Use this file to discover all available pages before exploring further.

PDF Insight Pro’s RAG pipeline operates in two distinct phases: document ingestion — run once at upload time — and query processing — run on every chat turn. What makes the pipeline truly agentic is the second phase: instead of doing a single retrieval pass and feeding the result blindly to the LLM, the system wraps the LLM in a LangChain AgentExecutor that is given explicit tools it can call repeatedly. The agent decides on its own whether to search the vector database, invoke the Tavily web-search API, or combine both — before composing a final answer. This tool-calling loop runs for up to Config.AGENT_MAX_ITERATIONS (2) iterations, after which a three-tier fallback strategy guarantees a response is always returned.

​
Document Ingestion Phase

1

PDF loading with PyMuPDFLoader

process_pdf_file(file_path) in utils/text_processing.py uses LangChain’s PyMuPDFLoader to load the uploaded PDF. The loader returns a list of Document objects, one per page, each carrying the page text and a metadata dict (page number, source path, etc.).
from langchain_community.document_loaders import PyMuPDFLoader

def process_pdf_file(file_path: str) -> List[Document]:
    loader = PyMuPDFLoader(file_path)
    documents = loader.load()
    return documents
2

Paragraph-aware chunking with chunk_text()

chunk_text(documents, max_length=Config.DEFAULT_CHUNK_SIZE) walks every Document, splits the page text on double newlines to produce paragraphs, and assembles paragraphs into chunks whose token estimate stays at or below max_length (1000). Paragraphs shorter than Config.MIN_PARAGRAPH_LENGTH (10 characters) are skipped. A chunk is only committed to the output list when its stripped length exceeds Config.MIN_CHUNK_LENGTH (20 characters).
def chunk_text(documents: List[Document], max_length: int = None) -> List[Dict[str, Any]]:
    if max_length is None:
        max_length = Config.DEFAULT_CHUNK_SIZE  # 1000

    chunks = []
    for doc in documents:
        paragraphs = doc.page_content.split("\n\n")
        current_chunk = ""
        for paragraph in paragraphs:
            if len(paragraph.strip()) < Config.MIN_PARAGRAPH_LENGTH:  # 10
                continue
            if estimate_tokens(current_chunk + paragraph) <= max_length // 4:
                current_chunk += paragraph + "\n\n"
            else:
                if current_chunk.strip() and len(current_chunk.strip()) > Config.MIN_CHUNK_LENGTH:  # 20
                    chunks.append({"text": current_chunk.strip(), "metadata": doc.metadata})
                current_chunk = paragraph + "\n\n"
        if current_chunk.strip() and len(current_chunk.strip()) > Config.MIN_CHUNK_LENGTH:
            chunks.append({"text": current_chunk.strip(), "metadata": doc.metadata})
    return chunks
Token estimation uses the approximation len(text) // 4 — fast and dependency-free, traded off for precision.
3

Embedding generation with SentenceTransformer

create_embeddings(chunks, model) in utils/text_processing.py uses a SentenceTransformer loaded with Config.EMBEDDING_MODEL (BAAI/bge-large-en-v1.5) to encode all chunk texts in a single batched call. The function returns a (numpy_array, chunks) tuple.
def create_embeddings(
    chunks: List[Dict[str, Any]],
    model: SentenceTransformer
) -> Tuple[np.ndarray, List[Dict[str, Any]]]:
    texts = [chunk["text"] for chunk in chunks]
    embeddings = model.encode(texts, show_progress_bar=True, convert_to_tensor=True)
    return embeddings.cpu().numpy(), chunks
4

FAISS HNSW index construction

build_faiss_index(embeddings) in utils/faiss_utils.py creates a faiss.IndexHNSWFlat index from the embedding array. HNSW (Hierarchical Navigable Small World) provides approximate-nearest-neighbour search in sub-linear time.
def build_faiss_index(embeddings: np.ndarray) -> faiss.IndexHNSWFlat:
    dim = embeddings.shape[1]
    index = faiss.IndexHNSWFlat(dim, Config.FAISS_NEIGHBORS)   # 32 neighbours
    index.hnsw.efConstruction = Config.FAISS_EF_CONSTRUCTION    # 200
    index.hnsw.efSearch = Config.FAISS_EF_SEARCH                # 50
    index.add(embeddings)
    return index
ParameterValueEffect
FAISS_NEIGHBORS32Graph connectivity; higher = better recall, slower build
efConstruction200Build-time beam width; higher = more accurate graph
efSearch50Query-time beam width; higher = better recall, slower search

​
Query Processing Phase

1

Top-k chunk retrieval via retrieve_similar_chunks()

At query time the chat handler calls retrieve_similar_chunks(query, index, chunks, model, k=Config.INITIAL_CONTEXT_CHUNKS) for an initial 5-chunk pass. The function encodes the query string with the same SentenceTransformer, runs index.search(), and returns a list of (text, distance, metadata) tuples for every valid index position.
def retrieve_similar_chunks(query, index, chunks_with_metadata, embedding_model, k=None, ...):
    query_embedding = embedding_model.encode([query], convert_to_tensor=True).cpu().numpy()
    distances, indices = index.search(query_embedding, k)

    valid_results = []
    for idx_pos, chunk_idx in enumerate(indices[0]):
        if 0 <= chunk_idx < len(chunks_with_metadata):
            chunk_text = chunks_with_metadata[chunk_idx]["text"][:max_chunk_length]
            if chunk_text.strip():
                result = (chunk_text, distances[0][idx_pos], chunks_with_metadata[chunk_idx]["metadata"])
                if validate_chunk_data(result):
                    valid_results.append(result)
    return valid_results
2

Similarity filtering with filter_relevant_chunks()

filter_relevant_chunks(chunks_data, threshold=Config.SIMILARITY_THRESHOLD) removes any tuple whose distance value is ≥ 1.5. Because HNSW distances are L2-style (lower = more similar), chunks that exceed this threshold are considered too dissimilar to be useful.
def filter_relevant_chunks(chunks_data, threshold=None):
    if threshold is None:
        threshold = Config.SIMILARITY_THRESHOLD  # 1.5
    return [chunk for chunk in chunks_data if len(chunk) >= 3 and chunk[1] < threshold]
3

Context assembly with prepare_context_from_chunks()

prepare_context_from_chunks(context_chunks, max_tokens=Config.MAX_CONTEXT_TOKENS) sorts the filtered chunks by distance (ascending), then concatenates chunk text until the running token count reaches MAX_CONTEXT_TOKENS (7000). The resulting string is passed as the {context} variable in the agent prompt.
def prepare_context_from_chunks(context_chunks, max_tokens=None):
    if max_tokens is None:
        max_tokens = Config.MAX_CONTEXT_TOKENS  # 7000
    sorted_chunks = sorted(context_chunks, key=lambda x: x[1])
    relevant_chunks = filter_relevant_chunks(sorted_chunks)
    context = ""
    total_tokens = 0
    for chunk, _, _ in relevant_chunks:
        chunk_tokens = estimate_tokens(chunk)
        if total_tokens + chunk_tokens <= max_tokens:
            context += chunk + "\n\n"
            total_tokens += chunk_tokens
        else:
            break
    return context.strip() or "No initial context provided from preliminary search."
4

LangChain agent creation

RAGService.execute_agent() in services/rag_service.py calls create_tool_calling_agent(llm, tools, prompt) and wraps it in an AgentExecutor. The executor is configured with handle_parsing_errors=True, max_iterations=Config.AGENT_MAX_ITERATIONS (2), and early_stopping_method="generate".
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    memory=memory,
    verbose=Config.AGENT_VERBOSE,
    handle_parsing_errors=True,
    max_iterations=Config.AGENT_MAX_ITERATIONS,  # 2
    return_intermediate_steps=False,
    early_stopping_method="generate"
)
response_payload = agent_executor.invoke({"input": query, "context": context})
5

Tool execution: vector_database_search and tavily_search_results_json

The agent always has access to vector_database_search, a @tool-decorated closure over the session’s FAISS index, chunks, and embedding model. When use_search=True is set in the ChatRequest, the Tavily tool (tavily_search_results_json) is appended to the tool list.
# RAGService.create_agent_tools()
tools = []

vector_tool = create_vector_search_tool(
    faiss_index=faiss_index,
    document_chunks_with_metadata=document_chunks,
    embedding_model=embedding_model,
    max_chunk_length=Config.DEFAULT_CHUNK_SIZE,  # 1000
    k=10
)
tools.append(vector_tool)

if use_web_search and self.tavily_tool:
    tools.append(self.tavily_tool)
The vector_database_search tool internally calls retrieve_similar_chunks() followed by filter_relevant_chunks() and returns a formatted string. Queries shorter than 3 characters are rejected with an explanatory message.
6

Response validation and output extraction

After agent_executor.invoke() returns, execute_agent() checks that response_payload["output"] is non-empty and longer than 10 characters, and that it does not match a list of known incomplete-response prefixes (e.g. "I need to", "Let me"). If validation fails, an exception is raised to trigger the fallback chain.

​
Fallback Strategy

The system uses a three-tier fallback to ensure a response is always returned, even when the agent errors or produces an unusable output.
1

Tier 1 — Agent execution (primary)

execute_agent() runs the full AgentExecutor loop with tool-calling and conversation memory. This is the preferred path.
2

Tier 2 — fallback_response() (direct tool use + LLM)

If the agent raises an exception, generate_response() calls fallback_response(). This method:
  1. Invokes vector_database_search.run(query) directly (bypassing the agent loop).
  2. Optionally invokes tavily_search_results_json.run(query) if use_tavily=True.
  3. Appends tool results to the original context string as "Additional Information".
  4. Calls llm.invoke() directly with a ChatPromptTemplate that instructs the LLM to use the enriched context.
3

Tier 3 — Simple LLM prompt (final safety net)

If fallback_response() also raises, generate_response() catches the error and returns a static apology string: "I'm sorry, I encountered an error processing your request. Please try again." In practice, fallback_response() itself contains a nested try/except that falls back to an even simpler prompt before giving up.

​
Agentic Prompt

The system prompt is built dynamically by RAGService.create_agent_prompt() based on which tools are available. The tool_instructions string is interpolated at prompt-creation time, not at inference time. 
def create_agent_prompt(self, has_document_search: bool, has_web_search: bool) -> ChatPromptTemplate:
    tool_instructions = ""
    if has_document_search:
        tool_instructions += "Use vector_database_search to find information in the uploaded document. "
    if has_web_search:
        tool_instructions += "Use tavily_search_results_json for web searches when document search is insufficient. "
    if not tool_instructions:
        tool_instructions = "Answer based on the provided context only. "

    return ChatPromptTemplate.from_messages([
        ("system", f"""You are a helpful AI assistant that answers questions about documents.

Context: {{context}}

Tools available: {tool_instructions}

Instructions:
- Use the provided context first
- If context is insufficient, use available tools to search for more information
- Provide clear, helpful answers
- If you cannot find an answer, say so clearly"""),
        ("human", "{input}"),
        MessagesPlaceholder(variable_name="chat_history"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ])
The chat_history placeholder is populated from ConversationBufferMemory that the chat handler pre-fills from the session’s persisted chat_history list before each AgentExecutor.invoke() call. This gives the agent full multi-turn context without loading the entire history into the system prompt.

​
RAG Evaluation Metrics

The pipeline was evaluated against the neural-bridge dataset using semantic similarity (cosine) and ROUGE-L F1 as the primary metrics. Results are documented in test_RAG.ipynb and summarised below.

​
Key Metrics

MetricValue
Semantic Similarity (Mean)0.852
ROUGE-L F1 Score (Mean)0.395
Semantic Similarity (Max)1.000
ROUGE-L F1 Score (Max)1.000
Semantic Similarity (Min)0.592
ROUGE-L F1 Score (Min)0.099
Standard Deviation (Similarity)0.089
Standard Deviation (ROUGE-L F1)0.217

​
Quantile Distribution

PercentileSemantic SimilarityROUGE-L F1 Score
25th0.79460.2516
50th0.87320.3256
75th0.91810.4951

​
Evaluation Status

StatusCountPercentage
PASS6485.3%
FAIL1114.7%
A response is classified as PASS when it clears the minimum thresholds for both semantic similarity and ROUGE-L F1 simultaneously. The 14.7% failure rate is concentrated in edge cases where the queried fact does not appear anywhere in the uploaded document and web search was disabled.

Build docs developers (and LLMs) love

Get started for freeTalk to us