Skip to main content
The orchestrator is the entry point for every text-based user request. It is an Agent instance powered by GPT-4o that holds the full registry of specialized agents as handoffs, decides which of them to invoke, and synthesizes their outputs into a final response.

​
Agent definition

ORCHESTRATOR_AGENT = Agent(
    name="ODAI",
    model="gpt-4o",
    instructions=(
        RECOMMENDED_PROMPT_PREFIX +
        (
            "You are a helpful AI assistant that can help with a wide range of tasks. "
            "You can hand off to available agents and make tool calls. "
            "If someone tries to access Google Docs, GMAil, or Google Calendar, you can tell them to type in "
            "\"Connect my Google account\" or \"Connect to Google\" to connect their Google account."
            "If someone asks to connect to their Google account, hand off to the Google Connections agent."
            "If someone asks to connect to their bank or credit card account, hand off to the Plaid Connector agent."
            "If someone asks for their bank or credit card account balance and they have no accounts connected, hand off to the Plaid Connector agent."
        )
    ),
    handoffs=[
        YELP_AGENT,
        COINMARKETCAP_AGENT,
        GMAIL_AGENT,
        FLIGHTAWARE_AGENT,
        PLAID_AGENT,
        FINNHUB_AGENT,
        GOOGLE_CALENDAR_AGENT,
        GOOGLE_DOCS_AGENT,
        # ... 25 agents total
    ],
    model_settings=ModelSettings(include_usage=True)
)
The Orchestrator class in connectors/orchestrator.py wraps this definition and is instantiated per WebSocket connection, allowing future dynamic agent registration (e.g. injecting user-specific integrations via build_dynamic_agents).

​
The O.D.A.R. loop

Every request the orchestrator processes follows a four-phase loop:
1

Observe

Parse the incoming user message together with any prior conversation context and tool results already in the thread.
2

Decide

Determine whether the request can be answered from existing context or requires delegating to one or more agents. Apply the H.A.N.D.O.F.F. criteria (see below) to select the best agent(s).
3

Act

Execute the chosen tool calls. Where multiple agents are needed and their calls are independent, they are parallelized.
4

Respond

Summarize the results in a concise, user-facing reply. Mention which services were consulted (“I checked your calendar…”) without exposing raw API output.

​
The H.A.N.D.O.F.F. decision framework

When choosing which agent to invoke, the orchestrator evaluates seven criteria:
Does the agent explicitly solve this task? Only agents whose declared capabilities match the user’s intent are considered.
Does the agent have the required data or API permissions for this user? For example, the Gmail agent is only invoked when the user has connected their Google account.
Is a tool call actually necessary, or can the orchestrator answer from known context or cached information? Unnecessary calls are avoided.
Prefer fewer or cheaper API calls when they produce equivalent quality. The orchestrator minimizes the number of agent invocations per request.
Will the agent return the format and information the request requires? Agents are assessed for fit before being selected.
If the first-choice agent is likely to fail (e.g. due to missing credentials), an alternate agent is chosen proactively.
When a task spans multiple domains, the orchestrator coordinates several agents in parallel and merges their outputs into a single coherent response.

​
Tool call progress messages

Every tool function exposed by a specialized agent is registered in the TOOL_CALLS dictionary. When the orchestrator invokes a tool, this dict is consulted to surface a user-friendly progress string over the WebSocket connection in real time. 
TOOL_CALLS = {
    # Yelp
    "search_businesses_at_yelp": "Searching Yelp...",
    "get_business_reviews_at_yelp": "Getting Reviews...",

    # Gmail
    "fetch_google_email_inbox": "Fetching Inbox...",
    "search_google_mail": "Searching Emails...",
    "send_google_email": "Sending Email...",
    "reply_to_google_email": "Replying to Email...",

    # Finnhub
    "get_stock_price_at_finnhub": "Getting Stock Price...",
    "get_annual_financials_at_finnhub": "Getting Annual Stock Financials...",

    # FlightAware
    "get_current_flight_status": "Getting Flight Status...",

    # AccuWeather
    "get_current_weather_by_latitude_longitude": "Getting Current Weather...",
    "get_daily_forecast_weather_by_latitude_longitude": "Getting Daily Forecast...",

    # ... 100+ entries total
}
Tool entries with a None value are internal bookkeeping calls (e.g. store_location) that do not produce a visible progress message.

​
Parallel agent execution

When the O.D.A.R. loop reaches the Act phase and determines that multiple independent tool calls are required, the orchestrator issues them concurrently rather than sequentially. The response synthesis step waits for all results before generating the final reply.
Tool calls that depend on each other’s output — for example, fetching a flight status and then looking up the arrival airport weather — are still issued sequentially, with each result fed into the next call’s context.

​
Error handling

If a tool call fails or returns malformed output, the orchestrator retries once with corrected arguments. If the retry also fails, it degrades gracefully by acknowledging the failure to the user and offering alternative approaches. Raw error messages, stack traces, and internal identifiers are never surfaced in the user-facing response.

Build docs developers (and LLMs) love

Get started for freeTalk to us