Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/BerriAI/litellm/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Function calling (also called tool calling) allows LLMs to interact with external functions and APIs. LiteLLM standardizes function calling across 100+ providers using the OpenAI format.

​
Basic Usage

from litellm import completion

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name, e.g. San Francisco"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"]
                }
            },
            "required": ["location"]
        }
    }
}]

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools
)

# Check if function was called
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

​
Function Calling Parameters

​
tools
List[Dict]
List of tool definitions. Each tool must have:
  • type: Always "function"
  • function: Object with name, description, and parameters
​
tool_choice
Union[str, Dict]
Controls which tools the model can call:
  • "auto" (default): Model decides whether to call a function
  • "none": Model will not call any functions
  • {"type": "function", "function": {"name": "function_name"}}: Force specific function
  • "required": Model must call at least one function
​
parallel_tool_calls
bool
Whether to allow multiple function calls in a single response. Default: True

​
Tool Definition Format

tool = {
    "type": "function",
    "function": {
        "name": "function_name",           # Required: Function identifier
        "description": "What it does",     # Required: Help model understand usage
        "parameters": {                     # Required: JSON Schema
            "type": "object",
            "properties": {
                "param1": {
                    "type": "string",
                    "description": "Parameter description"
                },
                "param2": {
                    "type": "number",
                    "description": "Another parameter"
                }
            },
            "required": ["param1"]           # Optional: Required parameters
        }
    }
}

​
Response Format

When a function is called, the response contains tool calls: 
class ChatCompletionMessageToolCall:
    id: str                          # Unique call ID
    type: str                        # Always "function"
    function: Function

class Function:
    name: str                        # Function name
    arguments: str                   # JSON string of arguments

​
Examples

​
Single Function Call

from litellm import completion
import json

# Define function
def get_weather(location: str, unit: str = "fahrenheit") -> dict:
    # Implementation
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "forecast": "Sunny"
    }

# Define tool
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

# Call LLM
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools
)

# Process function call
message = response.choices[0].message
if message.tool_calls:
    tool_call = message.tool_calls[0]
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)
    
    # Execute function
    if function_name == "get_weather":
        result = get_weather(**function_args)
        print(f"Weather: {result}")

​
Complete Conversation Flow

from litellm import completion
import json

def get_weather(location: str, unit: str = "fahrenheit") -> dict:
    return {"temperature": 72, "unit": unit, "forecast": "Sunny"}

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

messages = [{"role": "user", "content": "What's the weather in Paris?"}]

# First call - LLM decides to call function
response = completion(model="gpt-4", messages=messages, tools=tools)
message = response.choices[0].message
messages.append(message)

# Execute function
if message.tool_calls:
    for tool_call in message.tool_calls:
        function_args = json.loads(tool_call.function.arguments)
        result = get_weather(**function_args)
        
        # Add function result to messages
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result)
        })

# Second call - LLM uses function result
final_response = completion(model="gpt-4", messages=messages, tools=tools)
print(final_response.choices[0].message.content)
# "The weather in Paris is 72°F and sunny."

​
Multiple Functions

from litellm import completion
import json

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_time",
            "description": "Get current time for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_restaurants",
            "description": "Search for restaurants",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "cuisine": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

response = completion(
    model="gpt-4",
    messages=[{
        "role": "user",
        "content": "What's the weather in Tokyo and find me Italian restaurants there?"
    }],
    tools=tools
)

# Model might call multiple functions
for tool_call in response.choices[0].message.tool_calls:
    print(f"Calling: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

​
Forcing Function Call

from litellm import completion

tools = [{
    "type": "function",
    "function": {
        "name": "calculate",
        "description": "Perform calculation",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {"type": "string"}
            },
            "required": ["expression"]
        }
    }
}]

# Force the model to call the calculate function
response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is 25 * 4?"}],
    tools=tools,
    tool_choice={
        "type": "function",
        "function": {"name": "calculate"}
    }
)

print(response.choices[0].message.tool_calls[0].function.arguments)
# {"expression": "25 * 4"}

​
Parallel Function Calls

from litellm import completion
import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"]
        }
    }
}]

response = completion(
    model="gpt-4",
    messages=[{
        "role": "user",
        "content": "What's the weather in London, Paris, and Tokyo?"
    }],
    tools=tools,
    parallel_tool_calls=True  # Allow multiple calls
)

# Model calls function 3 times in parallel
for tool_call in response.choices[0].message.tool_calls:
    args = json.loads(tool_call.function.arguments)
    print(f"Getting weather for {args['location']}")

​
Function Calling with Streaming

from litellm import completion
import json

tools = [{
    "type": "function",
    "function": {
        "name": "search",
        "description": "Search the web",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string"}
            },
            "required": ["query"]
        }
    }
}]

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Search for Python tutorials"}],
    tools=tools,
    stream=True
)

function_name = ""
function_args = ""

for chunk in response:
    delta = chunk.choices[0].delta
    
    if delta.tool_calls:
        for tool_call in delta.tool_calls:
            if tool_call.function.name:
                function_name += tool_call.function.name
            if tool_call.function.arguments:
                function_args += tool_call.function.arguments
    
    if chunk.choices[0].finish_reason == "tool_calls":
        print(f"Function: {function_name}")
        print(f"Arguments: {json.loads(function_args)}")

​
Provider-Specific Examples

from litellm import completion

tools = [{"type": "function", "function": {...}}]

response = completion(
    model="gpt-4",
    messages=[...],
    tools=tools
)

​
Advanced Usage

​
Function with Complex Parameters

tools = [{
    "type": "function",
    "function": {
        "name": "book_hotel",
        "description": "Book a hotel room",
        "parameters": {
            "type": "object",
            "properties": {
                "hotel_name": {"type": "string"},
                "check_in": {
                    "type": "string",
                    "description": "Check-in date (YYYY-MM-DD)"
                },
                "check_out": {
                    "type": "string",
                    "description": "Check-out date (YYYY-MM-DD)"
                },
                "guests": {
                    "type": "object",
                    "properties": {
                        "adults": {"type": "integer"},
                        "children": {"type": "integer"}
                    },
                    "required": ["adults"]
                },
                "preferences": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Room preferences"
                }
            },
            "required": ["hotel_name", "check_in", "check_out"]
        }
    }
}]

​
Error Handling

from litellm import completion
from litellm.exceptions import BadRequestError
import json

try:
    response = completion(
        model="gpt-4",
        messages=[{"role": "user", "content": "Get weather"}],
        tools=tools
    )
    
    if response.choices[0].message.tool_calls:
        for tool_call in response.choices[0].message.tool_calls:
            try:
                args = json.loads(tool_call.function.arguments)
                # Execute function
            except json.JSONDecodeError:
                print("Invalid function arguments")
            except Exception as e:
                print(f"Function execution failed: {e}")
                
except BadRequestError as e:
    print(f"Invalid request: {e}")

​
Function Calling with Pydantic

from litellm import completion
from pydantic import BaseModel, Field
import json

class WeatherParams(BaseModel):
    location: str = Field(description="City name")
    unit: str = Field(description="Temperature unit", default="fahrenheit")

# Convert Pydantic model to tool schema
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": WeatherParams.model_json_schema()
    }
}]

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Weather in NYC?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    # Validate with Pydantic
    params = WeatherParams.model_validate_json(tool_call.function.arguments)
    print(f"Location: {params.location}, Unit: {params.unit}")

​
Best Practices

  1. Clear descriptions: Write detailed function and parameter descriptions
  2. Use enums: Constrain parameter values when possible
  3. Handle errors: Always validate function arguments before execution
  4. Complete the loop: Send function results back to the model
  5. Consider parallel calls: Enable for independent operations
  6. Set tool_choice wisely: Use “auto” for flexibility, force calls when needed

​
Troubleshooting

​
Function Not Called

  • Ensure the function description clearly states when to use it
  • Try setting tool_choice="required" to force a function call
  • Check if the model supports function calling

​
Invalid Arguments

  • Validate the JSON schema in your tool definition
  • Add detailed descriptions for each parameter
  • Use json.loads() with error handling

​
Multiple Unwanted Calls

  • Set parallel_tool_calls=False to limit to one call
  • Be specific in function descriptions about when to use each tool

Build docs developers (and LLMs) love

Get started for freeTalk to us