Skip to main content
DocMind processes natural language queries through a multi-stage workflow involving decomposition, retrieval, generation, and evaluation.

​
Quick Start

The simplest way to run queries is using the run_docmind function: 
import asyncio
from starter import run_docmind

async def main():
    response = await run_docmind("What are the penalties for late payment?")
    print(response)

asyncio.run(main())

​
Understanding the Workflow

DocMind uses a LangGraph workflow with 5 nodes:
1

Decompose

The query is decomposed into structured components:
components.py:46-63
async def decompose(self, query: str) -> Dict:
    intent = self._extract_intent(query)
    entities = self._extract_entities(query)
    constraints = self._extract_constraints(query)
    temporals = self._extract_temporals(query)
    
    return {
        "intent": intent,
        "entities": entities,
        "constraints": constraints,
        "temporals": temporals
    }
Example Output:
{
  "intent": "penalty",
  "entities": ["penalties", "late", "payment"],
  "constraints": {},
  "temporals": []
}
2

Retrieve

Sections are retrieved using strategic, intent-based scoring:
components.py:114-144
async def retrieve(self, query: str, decomposition: Dict) -> List[Dict]:
    all_sections = await self.doc_store.full_text_search(query)
    
    scored_sections = []
    for section in all_sections:
        score = self._score_section(section, query, decomposition)
        section_copy = section.copy()
        section_copy["_relevance_score"] = score
        scored_sections.append(section_copy)
    
    scored_sections.sort(key=lambda x: x["_relevance_score"], reverse=True)
    relevant = self._filter_irrelevant(scored_sections, threshold=2.0)
    
    return relevant[:5]
Returns 3-5 most relevant sections with page numbers.
3

Generate

A response is generated from retrieved sections:
components.py:312-329
def generate(self, sections: List[Dict]) -> str:
    response_parts = []
    
    for section in sections:
        title = section.get("title", "Unknown")
        page = section.get("page_num", "?")
        content = section.get("content", "")
        
        sentences = re.split(r'(?<=[.!?])\s+', content)
        key_info = sentences[0] if sentences else content[:200]
        
        response_parts.append(f"{key_info} (See {title}, page {page})")
    
    return " ".join(response_parts)
4

Judge

The LLM Judge evaluates the response for hallucinations:
components.py:244-308
async def evaluate(self, response: str, context: List[Dict]) -> Dict:
    claims = self._extract_claims(response)
    
    supported_count = 0
    contradicted_count = 0
    unsupported_count = 0
    
    for claim in claims:
        source_quote = self._find_supporting_quote(claim["text"], context)
        is_contradicted = self._check_contradiction(claim["text"], context)
        
        # ... claim evaluation logic ...
    
    confidence_score = 1.0
    confidence_score -= (contradicted_count / total_claims) * 0.8
    confidence_score -= (unsupported_count / total_claims) * 0.3
    
    return {
        "confidence_score": round(confidence_score, 2),
        "is_hallucinated": contradicted_count > 0 or confidence_score < 0.5,
        "should_return": not is_hallucinated
    }
5

Output

Final output is returned if judge approves:
nodes.py:39-45
async def output_node(state: DocMindState) -> DocMindState:
    if state["judge_verdict"] and state["judge_verdict"].get("should_return", False):
        state["final_output"] = state["generated_response"]
    else:
        state["final_output"] = "Unable to provide a confident response. Please rephrase your query."
    return state

​
Running Multiple Queries

The starter.py script demonstrates running multiple queries:
starter.py:32-46
if __name__ == "__main__":
    async def main():
        queries = [
            "What are the penalties for late payment?",
            "What are the indemnification obligations?",
            "What happens if IP is infringed?"
        ]
        
        for i, query in enumerate(queries, 1):
            print(f"\n{'='*80}")
            print(f"Query {i}: {query}")
            print('='*80)
            response = await run_docmind(query)
            print(f"\nResponse: {response}\n")
    
    asyncio.run(main())

​
Example Output

================================================================================
Query 1: What are the penalties for late payment?
================================================================================

Response: If payment is not received within thirty (30) days, Client shall be 
assessed a late fee of 1.5% per month (18% annually) on the outstanding balance. 
(See Late Payment Penalties, page 8)

​
State Tracking

DocMind tracks workflow state through the DocMindState type:
state_types.py:3-12
class DocMindState(TypedDict):
    query: str
    decomposition: Optional[Dict]
    retrieved_sections: List[Dict]
    generated_response: Optional[str]
    judge_verdict: Optional[Dict]
    final_output: Optional[str]
    retry_count: int  # track number of retries
    node_history: List[str]  # track nodes visited

​
Accessing State Information

from workflow import build_graph_workflow
from state_types import DocMindState

graph = build_graph_workflow()
initial_state: DocMindState = {
    "query": "What are the payment terms?",
    "decomposition": None,
    "retrieved_sections": [],
    "generated_response": None,
    "judge_verdict": None,
    "final_output": None,
    "retry_count": 0,
    "node_history": []
}

final_state = await graph.ainvoke(initial_state)

# Access results
print(f"Intent: {final_state['decomposition']['intent']}")
print(f"Sections retrieved: {len(final_state['retrieved_sections'])}")
print(f"Confidence: {final_state['judge_verdict']['confidence_score']}")
print(f"Nodes visited: {final_state['node_history']}")

​
Retry Logic

DocMind automatically retries if the judge detects hallucinations:
nodes.py:47-55
def should_retry(state: DocMindState) -> str:
    verdict = state.get("judge_verdict", {})
    retry_count = state.get("retry_count", 0)
    
    # retry if hallucinated and haven't exceeded max retries (2 attempts max)
    if verdict.get("is_hallucinated", False) and retry_count < 2:
        log_retry_attempt(retry_count + 1, 2)
        return "retry"
    return "output"
The workflow supports up to 2 retry attempts. After that, it returns: “Unable to provide a confident response. Please rephrase your query.”

​
Supported Query Types

DocMind recognizes these intent patterns:
Pattern: late fee, overdue, penaltiesExample: “What are the penalties for late payment?”Intent: penaltyTarget Sections: Late Payment Penalties, Payment Terms
Pattern: pay, invoice, paymentExample: “When is payment due?”Intent: payment_termsTarget Sections: Payment Terms, Late Payment Penalties
Pattern: owned by, license, IP, infringeExample: “What happens if IP is infringed?”Intent: intellectual_propertyTarget Sections: Intellectual Property Rights
Pattern: indemnify, indemnification, third-party claimsExample: “What are the indemnification obligations?”Intent: indemnificationTarget Sections: Indemnification
Pattern: terminate, termination, written noticeExample: “How can the contract be terminated?”Intent: terminationTarget Sections: Termination for Convenience
Pattern: confidential, proprietary informationExample: “What are the confidentiality obligations?”Intent: confidentialityTarget Sections: Confidentiality
Pattern: shall provide, servicesExample: “What services are provided?”Intent: scope_of_servicesTarget Sections: Scope of Services

​
Next Steps

Testing

Run the comprehensive test suite

Customization

Customize components and extend the system

Build docs developers (and LLMs) love

Get started for freeTalk to us