Creating chatbots with ChatInterface

Chatbots are a popular application of large language models (LLMs). Using Gradio, you can easily build a chat application and share it with your users, or try it yourself using an intuitive UI. This guide uses gr.ChatInterface(), which is a high-level abstraction that allows you to create your chatbot UI fast, often with a few lines of Python. It can be easily adapted to support multimodal chatbots, or chatbots that require further customization.

Prerequisites

Make sure you are using the latest version of Gradio:

pip install --upgrade gradio

OpenAI-API compatible endpoints

If you have a chat server serving an OpenAI-API compatible endpoint (such as Ollama), you can spin up a ChatInterface in a single line of Python. First, also run pip install openai. Then, with your own URL, model, and optional token:

import gradio as gr

gr.load_chat("http://localhost:11434/v1/", model="llama3.2", token="***").launch()

Read about gr.load_chat in the docs. If you have your own model, keep reading to see how to create an application around any chat model in Python!

Defining a chat function

To create a chat application with gr.ChatInterface(), the first thing you should do is define your chat function. In the simplest case, your chat function should accept two arguments: message and history.

message: a str representing the user’s most recent message
history: a list of openai-style dictionaries with role and content keys, representing the previous conversation history

The history would look like this:

[
    {"role": "user", "content": [{"type": "text", "text": "What is the capital of France?"}]},
    {"role": "assistant", "content": [{"type": "text", "text": "Paris"}]}
]

while the next message would be:

"And what is its largest city?"

Your chat function simply needs to return a str value, which is the chatbot’s response based on the chat history and most recent message, for example:

Paris is also the largest city.

Example: a random response chatbot

Let’s write a chat function that responds Yes or No randomly:

import random

def random_response(message, history):
    return random.choice(["Yes", "No"])

Now, we can plug this into gr.ChatInterface() and call the .launch() method to create the web interface:

import gradio as gr

gr.ChatInterface(
    fn=random_response, 
).launch()

That’s it! Here’s our running demo:

Example: incorporating user input and history

Of course, the previous example was very simplistic—it didn’t take user input or the previous history into account! Here’s another simple example showing how to incorporate both:

import gradio as gr

def alternatingly_agree(message, history):
    if len([h for h in history if h['role'] == "assistant"]) % 2 == 0:
        return f"Yes, I do think that: {message}"
    else:
        return "I don't think so"

gr.ChatInterface(
    fn=alternatingly_agree, 
).launch()

Streaming chatbots

In your chat function, you can use yield to generate a sequence of partial responses, each replacing the previous ones. This way, you’ll end up with a streaming chatbot:

import time
import gradio as gr

def slow_echo(message, history):
    for i in range(len(message)):
        time.sleep(0.3)
        yield "You typed: " + message[: i+1]

gr.ChatInterface(
    fn=slow_echo, 
).launch()

While the response is streaming, the “Submit” button turns into a “Stop” button that can be used to stop the generator function.

Even though you are yielding the latest message at each iteration, Gradio only sends the “diff” of each message from the server to the frontend, which reduces latency and data consumption over your network.

Customizing the chat UI

If you’re familiar with Gradio’s gr.Interface class, the gr.ChatInterface includes many of the same arguments that you can use to customize the look and feel of your Chatbot. For example, you can:

Add a title and description above your chatbot using title and description arguments
Add a theme or custom css using theme and css arguments respectively in the launch() method
Add examples and even enable cache_examples, which make your Chatbot easier for users to try it out
Customize the chatbot (e.g. to change the height or add a placeholder) or textbox (e.g. to add a max number of characters or add a placeholder)

Adding examples

You can add preset examples to your gr.ChatInterface with the examples parameter, which takes a list of string examples. Any examples will appear as “buttons” within the Chatbot before any messages are sent. If you’d like to include images or other files as part of your examples, you can do so by using this dictionary format for each example instead of a string:

{"text": "What's in this image?", "files": ["cheetah.jpg"]}

Each file will be a separate message that is added to your Chatbot history. You can change the displayed text for each example by using the example_labels argument. You can add icons to each example as well using the example_icons argument. Both of these arguments take a list of strings, which should be the same length as the examples list. If you’d like to cache the examples so that they are pre-computed and the results appear instantly, set cache_examples=True.

Customizing the chatbot or textbox component

If you want to customize the gr.Chatbot or gr.Textbox that compose the ChatInterface, then you can pass in your own chatbot or textbox components. Here’s an example:

import gradio as gr

def yes_man(message, history):
    if message.endswith("?"):
        return "Yes"
    else:
        return "Ask me anything!"

gr.ChatInterface(
    yes_man,
    chatbot=gr.Chatbot(height=300),
    textbox=gr.Textbox(placeholder="Ask me a yes or no question", container=False, scale=7),
    title="Yes Man",
    description="Ask Yes Man any question",
    examples=["Hello", "Am I cool?", "Are tomatoes vegetables?"],
    cache_examples=True,
).launch(theme="ocean")

Here’s another example that adds a “placeholder” for your chat interface, which appears before the user has started chatting. The placeholder argument of gr.Chatbot accepts Markdown or HTML:

gr.ChatInterface(
    yes_man,
    chatbot=gr.Chatbot(placeholder="<strong>Your Personal Yes-Man</strong><br>Ask Me Anything"),
    ...
)

The placeholder appears vertically and horizontally centered in the chatbot.

Multimodal chat interface

You may want to add multimodal capabilities to your chat interface. For example, you may want users to be able to upload images or files to your chatbot and ask questions about them. You can make your chatbot “multimodal” by passing in a single parameter (multimodal=True) to the gr.ChatInterface class. When multimodal=True, the signature of your chat function changes slightly: the first parameter of your function should accept a dictionary consisting of the submitted text and uploaded files that looks like this:

{
    "text": "user input", 
    "files": [
        "updated_file_1_path.ext",
        "updated_file_2_path.ext", 
        ...
    ]
}

The second parameter of your chat function, history, will be in the same openai-style dictionary format as before. However, if the history contains uploaded files, the content key will be a dictionary with a “type” key whose value is “file” and the file will be represented as a dictionary. The return type of your chat function does not change when setting multimodal=True (i.e. in the simplest case, you should still return a string value). If you are customizing a multimodal chat interface, you should pass in an instance of gr.MultimodalTextbox to the textbox parameter. You can customize the MultimodalTextbox further by passing in the sources parameter, which is a list of sources to enable. Here’s an example:

import gradio as gr

def count_images(message, history):
    num_images = len(message["files"])
    total_images = 0
    for message in history:
        for content in message["content"]:
            if content["type"] == "file":
                total_images += 1
    return f"You just uploaded {num_images} images, total uploaded: {total_images+num_images}"

demo = gr.ChatInterface(
    fn=count_images, 
    examples=[
        {"text": "No files", "files": []}
    ], 
    multimodal=True,
    textbox=gr.MultimodalTextbox(file_count="multiple", file_types=["image"], sources=["upload", "microphone"])
)

demo.launch()

Additional inputs

You may want to add additional inputs to your chat function and expose them to your users through the chat UI. For example, you could add a textbox for a system prompt, or a slider that sets the number of tokens in the chatbot’s response. The gr.ChatInterface class supports an additional_inputs parameter which can be used to add additional input components. The additional_inputs parameters accepts a component or a list of components. You can pass the component instances directly, or use their string shortcuts (e.g. "textbox" instead of gr.Textbox()). If you pass in component instances, and they have not already been rendered, then the components will appear underneath the chatbot within a gr.Accordion(). Here’s a complete example:

import gradio as gr
import time

def echo(message, history, system_prompt, tokens):
    response = f"System prompt: {system_prompt}\n Message: {message}."
    for i in range(min(len(response), int(tokens))):
        time.sleep(0.05)
        yield response[: i+1]

demo = gr.ChatInterface(
    echo,
    additional_inputs=[
        gr.Textbox("You are helpful AI.", label="System Prompt"),
        gr.Slider(10, 100),
    ]
)

demo.launch()

If the components you pass into the additional_inputs have already been rendered in a parent gr.Blocks(), then they will not be re-rendered in the accordion. This provides flexibility in deciding where to lay out the input components.

Examples with additional inputs

You can also add example values for your additional inputs. Pass in a list of lists to the examples parameter, where each inner list represents one sample, and each inner list should be 1 + len(additional_inputs) long. The first element in the inner list should be the example value for the chat message, and each subsequent element should be an example value for one of the additional inputs, in order. When additional inputs are provided, examples are rendered in a table underneath the chat interface.

Additional outputs

In the same way that you can accept additional inputs into your chat function, you can also return additional outputs. Simply pass in a list of components to the additional_outputs parameter in gr.ChatInterface and return additional values for each component from your chat function.

Unlike the case of additional inputs, the components passed in additional_outputs must be already defined in your gr.Blocks context—they are not rendered automatically. If you need to render them after your gr.ChatInterface, you can set render=False when they are first defined and then .render() them in the appropriate section of your gr.Blocks().

Next steps

Now that you’ve learned about the gr.ChatInterface class and how it can be used to create chatbot UIs quickly, we recommend:

Check out examples of using gr.ChatInterface with popular LLM libraries in the ChatInterface examples guide
Learn how to build custom chatbots from scratch using the low-level Blocks API in the Custom chatbots with Blocks guide
Explore how to display intermediate thoughts and tool usage in the Building agents with tool usage guide

Get Started

Core Concepts

Building Interfaces

Building with Blocks

Chatbots

Advanced Features

Custom Components

Clients & Deployment

Creating chatbots with ChatInterface

Prerequisites

OpenAI-API compatible endpoints

Defining a chat function

Example: a random response chatbot

Example: incorporating user input and history

Streaming chatbots

Customizing the chat UI

Adding examples

Customizing the chatbot or textbox component

Multimodal chat interface

Additional inputs

Examples with additional inputs

Additional outputs

Next steps

​Prerequisites

​OpenAI-API compatible endpoints

​Defining a chat function

​Example: a random response chatbot

​Example: incorporating user input and history

​Streaming chatbots

​Customizing the chat UI

​Adding examples

​Customizing the chatbot or textbox component

​Multimodal chat interface

​Additional inputs

​Examples with additional inputs

​Additional outputs

​Next steps

Prerequisites

OpenAI-API compatible endpoints

Defining a chat function

Example: a random response chatbot

Example: incorporating user input and history

Streaming chatbots

Customizing the chat UI

Adding examples

Customizing the chatbot or textbox component

Multimodal chat interface

Additional inputs

Examples with additional inputs

Additional outputs

Next steps