Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/EllisYuan/ChatAgents/llms.txt

Use this file to discover all available pages before exploring further.

ChatAgents is structured as two cooperating services — a Streamlit frontend and a FastAPI backend — running from the same Docker image. This split keeps the chat UI decoupled from agent execution: the frontend owns rendering and user interaction, while the backend owns all LLM calls, tool invocations, and conversation state. Streaming NDJSON events bridge them in real time, so users see tokens and tool-call cards appear as the agent works rather than waiting for a complete response.

​
Service Overview

Streamlit Frontend

Port 8501 — Renders the chat interface, sidebar API-key management, and tool-call visualization cards. Communicates with the backend exclusively via HTTP streaming (POST /stream_agent). Maintains lightweight session state in Streamlit’s own session store.

FastAPI Backend

Port 8080 — Owns the LangGraph WebAgent, the /stream_agent streaming endpoint, and all session CRUD routes (/api/sessions). Validates API keys, builds the ReAct graph per request, and persists conversation state through a shared MemorySaver checkpointer.

​
Request Lifecycle

1

User types a message

The user submits a query in the Streamlit chat input. The frontend packages the message together with session metadata and sends it to the backend.
2

Streamlit POSTs to /stream_agent

The frontend sends an HTTP POST to /stream_agent with a JSON body matching the AgentRequest model:
class AgentRequest(BaseModel):
    input: str        # user message
    thread_id: str    # unique session identifier
    agent_type: str   # "fast" or "deep"
    llm_provider: str # e.g. "claude"
    llm_model: str    # e.g. "sonnet"
API keys are passed as request headers (X-Tavily-Key, X-Claude-Key, etc.) or read from environment variables as a fallback.
3

FastAPI validates API keys

The backend calls check_api_key() on the Tavily key (format and authorization check) and verifies that the requested LLM provider key is present. If validation fails, a 401 or 400 HTTPException is raised before any LLM cost is incurred.
4

WebAgent.build_graph() creates the ReAct agent

Based on agent_type, the backend selects the correct prompt (get_simple_prompt() for Fast, get_reasoning_prompt() for Deep) and calls WebAgent.build_graph() with the main LLM, a fast Claude Haiku summarizer LLM, the Tavily API key, and the mode string — build_graph() constructs the three Tavily tools internally:
agent_runnable = app.state.agent.build_graph(
    api_key=tavily_api_key,
    llm=main_llm,
    prompt=prompt,
    summary_llm=summary_llm,
    user_message=body.input,
    mode=body.agent_type,   # "fast" or "deep"
)
Internally, build_graph() configures TavilySearch, TavilyExtract, and TavilyCrawl with mode-appropriate parameters, then calls create_react_agent() with the shared MemorySaver checkpointer.
5

astream_events() runs the agent and yields NDJSON

The backend drives the agent asynchronously using LangGraph’s astream_events() API. Each LangGraph event is mapped to one of four NDJSON event types and written to the streaming response:
async for event in agent_runnable.astream_events(
    input={"messages": [HumanMessage(content=body.input)]},
    config={"configurable": {"thread_id": body.thread_id}},
    version="v2",
):
    # emit chatbot / tool_start / tool_end / error events
6

Streamlit renders tokens and tool cards in real time

The frontend reads the chunked response line-by-line. chatbot events are appended to the chat bubble as tokens arrive. tool_start and tool_end events render collapsible tool-call cards showing the tool name, type, input parameters, and source links returned by the tool.
7

Session saved to JSON after stream ends

Once the stream is complete (inside the generator’s finally block), the backend calls session_manager.save_session(), which writes the full exchange — user message, assistant response, and any tool calls — to data/sessions/<thread_id>.json. If no session file exists yet, the first user message is used to auto-generate a title.

​
Conversation Memory

Multi-turn context is maintained by a LangGraph MemorySaver checkpointer. The checkpointer is created once at application startup and stored on app.state: 
@asynccontextmanager
async def lifespan(app: FastAPI):
    checkpointer = MemorySaver()
    agent = WebAgent(checkpointer=checkpointer)
    app.state.agent = agent
    yield
Every call to build_graph() passes self.checkpointer to create_react_agent(). LangGraph then uses the thread_id from the request config to read and write checkpoint snapshots — so the agent picks up the full message history from prior turns automatically, without the frontend needing to replay messages.
The agent graph is re-built on every request (new LLM instance, new tool instances), but the single MemorySaver instance lives for the entire application lifetime. This means conversation context survives across requests, while each request gets fresh LLM and tool configuration (model choice, API key, mode parameters).

​
NDJSON Event Reference

Every line the backend streams is a JSON object terminated by \n. The Streamlit frontend dispatches on the type field:
Event typeEmitted byPayload fields
chatboton_chat_model_stream handlertype, content — one text chunk from the LLM’s final answer
tool_starton_tool_start handlertype, tool_name, tool_type (search/extract/crawl), operation_index, content (serialized input params)
tool_endon_tool_end handlertype, tool_name, tool_type, operation_index, content (serialized tool output, including source URLs and favicons)
errorException handler in generatortype, content — human-readable error message

​
Tech Stack

LayerTechnologyPurpose
FrontendStreamlit 1.32+Python-native chat UI, sidebar controls, real-time streaming render
BackendFastAPI 0.109+Async HTTP server, streaming endpoint, session CRUD API
Agent orchestrationLangGraphReAct agent graph, MemorySaver checkpointer, astream_events streaming
Primary LLMClaude (Haiku / Sonnet / Opus)Agent reasoning; OpenAI and Groq interfaces also supported
Web toolsTavily (langchain-tavily)Search, extract, and crawl the live web
Session storageJSON files (data/sessions/)Lightweight persistence; no database dependency
ContainerizationDocker + Docker ComposeSingle-image, two-service deployment
Configurationpython-dotenv.env-based API key management

Build docs developers (and LLMs) love

Get started for freeTalk to us