Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/EllisYuan/ChatAgents/llms.txt

Use this file to discover all available pages before exploring further.

ChatAgents delegates all language-model initialization to the LLMConfig class in backend/llm_config.py. The class provides a dedicated factory method for each provider (create_claude, create_openai, create_groq) as well as a unified create_llm dispatcher. Provider identity is referenced via the LLMProvider constants rather than raw strings, keeping all provider names in one place and avoiding typos across the codebase.

​
Provider Constants

class LLMProvider:
    CLAUDE = "claude"
    OPENAI = "openai"
    GROQ   = "groq"
Use these constants whenever you pass a provider argument — for example LLMProvider.CLAUDE instead of the literal "claude".

​
Supported Providers

ProviderConstantAPI Key Env VarNotes
Anthropic ClaudeLLMProvider.CLAUDEANTHROPIC_API_KEYDefault provider
OpenAILLMProvider.OPENAIOPENAI_API_KEYOptional
GroqLLMProvider.GROQGROQ_API_KEYReserved for future use

​
Claude

Claude is the default provider and is the most thoroughly tested option in ChatAgents. Three model tiers are available, mapped from short aliases to full model identifiers:
AliasModel IDBest For
haikuclaude-haiku-4-5-20251001High-volume, cost-sensitive, or testing scenarios
sonnetclaude-sonnet-4-5-20250929Balanced performance — recommended default
opusclaude-opus-4-1-202508059Highest quality, most capable reasoning

​
LLMConfig.create_claude()

@staticmethod
def create_claude(
    model: str = "sonnet",
    api_key: Optional[str] = None,
    temperature: float = 0.7,
    max_tokens: int = 4096,
    streaming: bool = True,
) -> BaseChatModel:
Parameters
  • model — Short alias: "haiku", "sonnet", or "opus". Falls back to "sonnet" if an unrecognised alias is provided.
  • api_key — Anthropic API key. If None, the value of ANTHROPIC_API_KEY is read from the environment.
  • temperature — Sampling temperature in the range 0–1. Lower values produce more deterministic output; higher values increase creativity.
  • max_tokens — Maximum number of tokens in the generated response.
  • streaming — When True, the model is configured with {"tags": ["streaming"]} and responses are streamed token-by-token to the frontend.
Example 
from backend.llm_config import LLMConfig, LLMProvider

llm = LLMConfig.create_claude(
    model="sonnet",
    api_key="sk-ant-api-...",
    temperature=0.7,
    max_tokens=4096,
    streaming=True,
)
Use model="haiku" for high-volume or testing scenarios. Haiku is significantly cheaper per token than Sonnet or Opus and is fast enough for most informational queries.

​
OpenAI

OpenAI models are available when OPENAI_API_KEY is set. The default temperature for OpenAI is 1 (matching OpenAI’s own API default), which differs from the Claude default of 0.7.
Model KeyModel ID
gpt-5.1gpt-5.1
gpt-5-minigpt-5-mini
gpt-5-nanogpt-5-nano
gpt-5gpt-5
gpt-4.1-nanogpt-4.1-nano

​
LLMConfig.create_openai()

@staticmethod
def create_openai(
    model: str = "gpt-5.1-mini",
    api_key: Optional[str] = None,
    temperature: float = 1,
    max_tokens: int = 4096,
    streaming: bool = True,
) -> BaseChatModel:
Example 
llm = LLMConfig.create_openai(
    model="gpt-5-mini",
    api_key="sk-proj-...",
    temperature=1,
    max_tokens=4096,
    streaming=True,
)
The temperature default for OpenAI is 1, not 0.7. This matches OpenAI’s API default and generally produces natural, varied responses for chat use cases. Lower it toward 0 if you need more deterministic or factual answers.
The create_llm() dispatcher passes "gpt-4o" as the default model when no model is specified for the OpenAI provider (model=model or "gpt-4o"). "gpt-4o" is not present in OPENAI_MODELS — if the key is not found, create_openai falls back to OPENAI_MODELS["gpt-5.1"] (i.e. "gpt-5.1"). To avoid this silent fallback, always pass an explicit model key from OPENAI_MODELS when calling create_llm or create_openai directly.

​
Groq

Groq support is implemented in LLMConfig but is reserved for future use and not yet exposed in the Streamlit UI. The integration is present so Groq can be wired in without structural changes once the UI surface is ready.

​
LLMConfig.create_groq()

@staticmethod
def create_groq(
    model: str = "llama-3.3-70b",
    api_key: Optional[str] = None,
    temperature: float = 0.7,
    max_tokens: int = 4096,
    streaming: bool = True,
) -> BaseChatModel:
Groq model aliases (llama-3.3-70b, mixtral-8x7b, kimi-k2) are defined in the source but commented out in GROQ_MODELS pending UI support. Set GROQ_API_KEY in your .env to have the key available when Groq is enabled.

​
Unified Interface

For code that needs to work with any provider interchangeably, create_llm dispatches to the correct factory method based on the provider string: 
@staticmethod
def create_llm(
    provider: str = LLMProvider.CLAUDE,
    model: Optional[str] = None,
    api_key: Optional[str] = None,
    max_tokens: int = 4096,
    streaming: bool = True,
) -> BaseChatModel:
Note that create_llm does not accept a temperature argument — each provider uses its own default. Use the provider-specific factory method directly when you need temperature control. Example 
llm = LLMConfig.create_llm(
    provider=LLMProvider.OPENAI,
    model="gpt-5-mini",
    api_key="sk-proj-...",
    max_tokens=4096,
    streaming=True,
)
If provider is not one of the three supported values, create_llm raises a ValueError.

​
Selecting a Model at Request Time

The LLM is chosen per-request, not globally. The AgentRequest body sent to the /stream_agent endpoint carries both the provider and model: 
class AgentRequest(BaseModel):
    input: str          # User's message
    thread_id: str      # Conversation session ID
    agent_type: str     # "fast" or "deep"
    llm_provider: str = LLMProvider.CLAUDE  # e.g. "claude", "openai", "groq"
    llm_model: str = "sonnet"               # e.g. "sonnet", "gpt-5-mini"
The backend then instantiates the correct model on the fly: 
if body.llm_provider == LLMProvider.CLAUDE:
    llm = LLMConfig.create_claude(
        model=body.llm_model,
        api_key=claude_api_key,
        streaming=True,
    )
elif body.llm_provider == LLMProvider.OPENAI:
    llm = LLMConfig.create_openai(
        model=body.llm_model,
        api_key=openai_api_key,
        streaming=True,
    )
This design lets the Streamlit sidebar switch providers and models without restarting the backend.

​
Summary LLM

Regardless of which main model is selected, a Claude Haiku instance is always instantiated as summary_llm. It is passed to create_output_summarizer() inside WebAgent.build_graph() and used exclusively to compress raw TavilyExtract and TavilyCrawl output before the main LLM sees it. Using Haiku for summarization is a deliberate cost-optimization: Haiku’s pricing is a fraction of Sonnet or Opus, and summarizing tool output does not require the reasoning depth of the main model. 
# summary_llm is always Haiku, independent of the main model choice
summary_llm = LLMConfig.create_claude(
    model="haiku",
    api_key=claude_api_key,
    streaming=False,
)
If you are running ChatAgents at high volume or just exploring the system, set the main model to haiku as well. The full pipeline — main LLM + summary LLM — will then run entirely on Haiku, keeping costs minimal while still producing useful responses.

Build docs developers (and LLMs) love

Get started for freeTalk to us