POST /stream_agent — LangGraph Streaming Agent API

The /stream_agent endpoint is the heart of ChatAgents. It accepts a user message along with session and model configuration, runs the LangGraph ReAct agent with Tavily web-search tooling, and streams the result back as a sequence of newline-delimited JSON (NDJSON) events. Clients receive LLM tokens as they are generated, plus lifecycle events for every tool call the agent makes — giving the UI everything it needs to render a live, step-by-step response. Once the stream completes, the full conversation turn is automatically persisted to disk under the supplied thread_id.

Endpoint

POST /stream_agent

Request Headers

X-Tavily-Key

string

required

Your Tavily API key. Required for all requests — the agent cannot perform web searches without it. Falls back to the TAVILY_API_KEY environment variable if the header is absent.

X-Claude-Key

string

Your Anthropic API key. Required when llm_provider is "claude". Falls back to ANTHROPIC_API_KEY.

X-OpenAI-Key

string

Your OpenAI API key. Required when llm_provider is "openai". Falls back to OPENAI_API_KEY.

X-Groq-Key

string

Your Groq API key. Required when llm_provider is "groq". Falls back to GROQ_API_KEY.

Content-Type

string

required

Must be application/json.

Request Body

input

string

required

The user’s message or question. This is passed directly to the LangGraph ReAct agent as a HumanMessage.

thread_id

string

required

A UUID that identifies the conversation session. LangGraph uses this value as the thread_id for its MemorySaver checkpointer, enabling multi-turn context across requests. After the stream ends, the session is saved to data/sessions/<thread_id>.json.

agent_type

string

required

The reasoning mode for this request. Accepted values:

"fast" — uses Tavily basic search depth, fetches up to 3 results, excludes images. Optimised for speed and lower cost.
"deep" — uses Tavily advanced search depth, fetches up to 5 results, includes images. More thorough but takes longer and costs more.

llm_provider

string

default:"claude"

The language model provider to use for agent reasoning. Accepted values: "claude", "openai", "groq".

llm_model

string

default:"sonnet"

The model name within the chosen provider. Accepted values by provider:

Provider	Accepted values
`claude`	`"haiku"`, `"sonnet"`, `"opus"`
`openai`	`"gpt-5.1"`, `"gpt-5-mini"`, `"gpt-5-nano"`, `"gpt-5"`, `"gpt-4.1-nano"`
`groq`	Pass the Groq model identifier directly

Response

Content-Type: application/json The response body is an NDJSON stream — one JSON object per line, delivered as the agent produces output. Parse each line independently as it arrives. Do not attempt to parse the entire body as a single JSON document.

Response Event Types

type

string

required

Discriminator field. One of "chatbot", "tool_start", "tool_end", or "error".

`chatbot` Event

Carries a text chunk from the language model. Concatenate all chatbot chunks in order to reconstruct the full assistant reply.

type

"chatbot"

Always "chatbot".

content

string

A token or short sequence of tokens from the LLM output.

`tool_start` Event

Emitted when the agent begins invoking a Tavily tool. The content field contains the serialised tool input so the UI can show the query before results arrive.

type

"tool_start"

Always "tool_start".

tool_name

string

The LangChain tool class name, e.g. "TavilySearch", "TavilyExtract", "TavilyCrawl".

tool_type

string

Simplified category derived from the tool name: "search", "extract", or "crawl".

operation_index

integer

Zero-based counter that increments with each completed tool call in the request. Pair this with operation_index in the corresponding tool_end event to correlate start and end.

content

object | string

The tool’s input arguments, serialised to strings for safe JSON transport.

`tool_end` Event

Emitted when a Tavily tool returns its result.

type

"tool_end"

Always "tool_end".

tool_name

string

The LangChain tool class name.

tool_type

string

"search", "extract", or "crawl".

operation_index

integer

Matches the operation_index in the preceding tool_start event. The counter increments after tool_end is emitted.

content

string | object | array

The tool’s output, serialised for safe JSON transport. For TavilyExtract and TavilyCrawl, this is a summarised version of the raw content.

`error` Event

Emitted if an unrecoverable error occurs during streaming. The stream ends immediately after this event.

type

"error"

Always "error".

content

string

A human-readable description of what went wrong.

Error Responses

Status	Condition
`400 Bad Request`	`X-Tavily-Key` header is missing and `TAVILY_API_KEY` env var is not set; or `agent_type` is not `"fast"` or `"deep"`
`401 Unauthorized`	The supplied Tavily API key failed validation
`500 Internal Server Error`	LLM instantiation failed (bad model name, wrong API key) or the LangGraph agent graph could not be built

Examples

Fast Mode — Claude Sonnet

curl -X POST http://localhost:8080/stream_agent \
  -H "Content-Type: application/json" \
  -H "X-Tavily-Key: tvly-your-key" \
  -H "X-Claude-Key: sk-ant-api-your-key" \
  -d '{
    "input": "What are the latest developments in LangGraph?",
    "thread_id": "550e8400-e29b-41d4-a716-446655440000",
    "agent_type": "fast",
    "llm_provider": "claude",
    "llm_model": "sonnet"
  }'

Streaming Response

Each line below is a separate NDJSON object delivered as the agent runs:

{"type": "tool_start", "tool_name": "TavilySearch", "tool_type": "search", "operation_index": 0, "content": {"query": "LangGraph latest developments 2024"}}
{"type": "tool_end", "tool_name": "TavilySearch", "tool_type": "search", "operation_index": 0, "content": "LangGraph 0.2 introduced the new streaming API and improved checkpointing..."}
{"type": "chatbot", "content": "Here are the latest developments in LangGraph:\n\n"}
{"type": "chatbot", "content": "**LangGraph 0.2** brought several major improvements, including"}
{"type": "chatbot", "content": " a redesigned streaming API that delivers fine-grained token-level events,"}
{"type": "chatbot", "content": " enhanced `MemorySaver` checkpointing for multi-turn conversations,"}
{"type": "chatbot", "content": " and first-class support for multi-agent workflows with shared state."}

Deep Thinking Mode — Claude Opus

curl -X POST http://localhost:8080/stream_agent \
  -H "Content-Type: application/json" \
  -H "X-Tavily-Key: tvly-your-key" \
  -H "X-Claude-Key: sk-ant-api-your-key" \
  -d '{
    "input": "Analyze the differences between LangChain and LangGraph",
    "thread_id": "550e8400-e29b-41d4-a716-446655440000",
    "agent_type": "deep",
    "llm_provider": "claude",
    "llm_model": "opus"
  }'

Python Client

import httpx
import json

with httpx.stream(
    "POST",
    "http://localhost:8080/stream_agent",
    headers={
        "X-Tavily-Key": "tvly-your-key",
        "X-Claude-Key": "sk-ant-api-your-key",
    },
    json={
        "input": "What is LangGraph?",
        "thread_id": "my-session-id",
        "agent_type": "fast",
        "llm_provider": "claude",
        "llm_model": "sonnet",
    },
    timeout=120.0,
) as response:
    for line in response.iter_lines():
        if line:
            event = json.loads(line)
            if event["type"] == "chatbot":
                print(event["content"], end="", flush=True)
            elif event["type"] == "tool_start":
                print(f"\n[{event['tool_type']} started: {event['tool_name']}]")
            elif event["type"] == "tool_end":
                print(f"[{event['tool_type']} finished]\n")
            elif event["type"] == "error":
                print(f"\nError: {event['content']}")

After the stream ends, the full conversation turn — user message and assistant reply — is automatically saved to data/sessions/<thread_id>.json. If no session file exists for the given thread_id, a new one is created with a title auto-generated from the first message.

Set a generous timeout (120 seconds or more) on your HTTP client. Deep Thinking Mode uses Tavily’s advanced search depth and may invoke multiple tools, which can push total latency well above 30 seconds for complex queries.

Backend API

POST /stream_agent — LangGraph Streaming Agent API

Endpoint

Request Headers

Request Body

Response

Response Event Types

`chatbot` Event

`tool_start` Event

`tool_end` Event

`error` Event

Error Responses

Examples

Fast Mode — Claude Sonnet

Streaming Response

Deep Thinking Mode — Claude Opus

Python Client

Build docs developers (and LLMs) love

Backend API

Documentation Index

​Endpoint

​Request Headers

​Request Body

​Response

​Response Event Types

​chatbot Event

​tool_start Event

​tool_end Event

​error Event

​Error Responses

​Examples

​Fast Mode — Claude Sonnet

​Streaming Response

​Deep Thinking Mode — Claude Opus

​Python Client

Build docs developers (and LLMs) love

Endpoint

Request Headers

Request Body

Response

Response Event Types

`chatbot` Event

`tool_start` Event

`tool_end` Event

`error` Event

Error Responses

Examples

Fast Mode — Claude Sonnet

Streaming Response

Deep Thinking Mode — Claude Opus

Python Client