AI assistant

Overview

Duet includes an AI assistant that acts as an additional pair programmer in your session. It’s powered by Meta’s Llama 3 8B model running on Cloudflare’s edge network, providing fast responses without leaving your terminal.

Activating the AI

Press Ctrl+G while in a room to open the AI input prompt:

Open AI input

Press Ctrl+G in the terminal. You’ll see:

Ask the AI... |

Type your question

Ask anything about your code, request help with commands, or get debugging suggestions:

How do I list all files modified in the last hour?

Submit with Enter

The AI will respond in the sidebar. Press Esc to cancel without sending.

The AI assistant requires a Cloudflare Worker URL to be configured when starting the Duet server:

duet --worker https://duet-cf-worker.your-subdomain.workers.dev

AI architecture

The AI runs as a Cloudflare Durable Object, maintaining conversation state per room:

export class DuetAgent extends Agent<Env, DuetAgentState> {
  override initialState: DuetAgentState = { messages: [] };

  private async runAI(messages: AIMessage[]): Promise<string> {
    const result = await this.env.AI.run("@cf/meta/llama-3-8b-instruct", {
      messages,
    });
    return result.response?.trim() || "";
  }
}

Model
Context window
System prompt

Llama 3 8B Instruct

8 billion parameter model
Optimized for instruction following
Runs on Cloudflare’s global network
Typical response time: 1-3 seconds

Duet keeps the last 10 messages in context:

...this.state.messages.slice(-10).map<AIMessage>((m) => ({
  role: m.role === "agent" ? "assistant" : "user",
  content: m.text,
}))

This balances memory usage with conversation coherence.

{
  role: "system",
  content:
    "You are Duet, a concise pair-programming assistant. " +
    "You can run commands in a sandbox using <run>command</run> tags. " +
    "When asked to perform an action, briefly explain what you will do " +
    "and wrap the exact shell command(s) in <run> tags. " +
    "Do NOT include predicted output in your response - just provide " +
    "the explanation and command."
}

The AI chat appears in a collapsible sidebar on the right side of your terminal:

┌─────────────────┬──────────────────────┬────────────────────┐
│   Users         │   Terminal           │   AI Assistant     │
│                 │                      │                    │
│   alice (host)  │   $ ls -la           │   You: How do I... │
│   bob           │   total 48           │                    │
│                 │   drwxr-xr-x ...     │   AI: You can use..│
└─────────────────┴──────────────────────┴────────────────────┘

Keyboard shortcuts

Key	Action
`Ctrl+A`	Toggle AI sidebar visibility
`Ctrl+J`	Scroll chat down (3 lines)
`Ctrl+K`	Scroll chat up (3 lines)
`Ctrl+G`	Open AI input prompt

The AI sidebar only appears when your terminal is at least 120 columns wide and 24 rows tall. On smaller windows, the feature is automatically disabled.

Command execution

The AI can execute commands in a Cloudflare Sandbox by wrapping them in <run> tags:

private async executeCommands(text: string, roomId: string): Promise<string> {
  const matches = Array.from(text.matchAll(/<run>([\s\S]*?)<\/run>/g));
  let result = text;

  for (const match of matches) {
    const cmd = match[1]?.trim();
    if (!cmd) continue;

    try {
      const sandbox = getSandbox(this.env.Sandbox, `sandbox-${roomId}`);
      const { stderr, stdout } = await sandbox.exec(cmd);

      const summary = stdout.slice(0, 500) || stderr.slice(0, 500) || "[no output]";
      result += `\n\nOutput (${cmd}):\n${summary}`;
    } catch (e) {
      result += `\n\nError (${cmd}):\n${e.message}`;
    }
  }
  return result.replace(/<run>[\s\S]*?<\/run>/g, "").trim();
}

Example conversation

How do I find all Python files?

Commands run in an isolated Cloudflare Sandbox, not your shared terminal workspace. The sandbox is ephemeral and destroyed when the room ends.

Conversation persistence

AI messages are synchronized across all participants in real time:

case AIResponseMsg:
    if m.currentRoom != nil {
        m.currentRoom.SetAIMessages(msg.Messages)
        // Notify other clients to sync their viewport
        m.currentRoom.BroadcastEvent(room.RoomEvent{
            Type: "ai_sync",
        }, m.clientID)
    }
    m.syncAIViewportContent()
    m.scrollToLastPrompt()

When someone asks the AI a question:

The response is stored in the room’s shared state
An ai_sync event is broadcast to all participants
Everyone’s AI sidebar updates to show the new messages

Late joiners can see the full AI conversation history when they enter the room. The last 50 messages are kept in memory.

Message format

Each message in the conversation includes:

type AIMessage struct {
    Role   string `json:"role"`    // "user" or "agent"
    UserID string `json:"user_id"` // Username of the person who asked
    Text   string `json:"text"`    // Message content
    Ts     int64  `json:"ts"`      // Unix timestamp
}

This allows the UI to display who asked each question:

You: How do I list files?

AI: Use the ls command...

alice: What about sorting by date?

AI: Add the -t flag...

API endpoints

The Cloudflare Worker exposes these AI endpoints:

POST /api/rooms/:roomId/message

endpoint

Send a message to the AIRequest body:

{
  "text": "How do I list files?",
  "userId": "alice"
}

Response:

{
  "reply": "Use the ls command to list files...",
  "messages": [
    {"role": "user", "userId": "alice", "text": "How do I...", "ts": 1234567890},
    {"role": "agent", "text": "Use the ls command...", "ts": 1234567891}
  ]
}

DELETE /api/rooms/:roomId

endpoint

Clean up AI state and sandbox for a roomCalled automatically when the last participant leaves.Response:

{
  "cleaned": true,
  "roomId": "a3f8e9d2-..."
}

Client implementation

The Go client communicates with the Worker:

func (c *Client) SendMessage(ctx context.Context, roomID, text, userID string) (*MessageResponse, error) {
    url := fmt.Sprintf("%s/api/rooms/%s/message", c.baseURL, roomID)

    body := MessageRequest{
        Text:   text,
        UserID: userID,
    }

    jsonBody, err := json.Marshal(body)
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(jsonBody))
    req.Header.Set("Content-Type", "application/json")

    resp, err := c.http.Do(req)
    // ... handle response
}

Requests timeout after 30 seconds to prevent hanging if the Worker is slow.

Error handling

No Worker URL configured

If you press Ctrl+G without a Worker URL:

AI not configured (no worker URL)

Start the server with the --worker flag.

Request timeout

If the AI doesn’t respond within 30 seconds:

Error: context deadline exceeded

Try again or check your Worker status.

Validation error

Empty messages are rejected:

{
  "error": "invalid request",
  "details": {"text": ["Text cannot be empty"]}
}

Best practices

Be specific

Instead of “help with this code”, ask “how do I parse JSON in Go?”

One question at a time

The AI works best with focused questions. Break complex tasks into steps.

Include context

Mention what you’re trying to do: “I’m debugging a Python script that…”

Review commands

Always verify AI-suggested commands before running them in your terminal.

Limitations

The AI doesn’t have access to your terminal history or current directory
It can’t see files in your workspace (only in its own sandbox)
It doesn’t remember conversations across different rooms
The 10-message context window means very long discussions may lose coherence

Next steps

Sandbox execution

Learn how the AI’s command execution sandbox works

Deploy a Worker

Set up your own Cloudflare Worker for AI features

Get Started

Core Features

Guides

Deployment

AI assistant

Overview

Activating the AI

AI architecture

AI sidebar

Keyboard shortcuts

Command execution

Example conversation

Conversation persistence

Message format

API endpoints

Client implementation

Error handling

Best practices

Be specific

One question at a time

Include context

Review commands

Limitations

Next steps

Sandbox execution

Deploy a Worker

Build docs developers (and LLMs) love

Get Started

Core Features

Guides

Deployment

Documentation Index

​Overview

​Activating the AI

​AI architecture

​AI sidebar

​Keyboard shortcuts

​Command execution

​Example conversation

​Conversation persistence

​Message format

​API endpoints

​Client implementation

​Error handling

​Best practices

Be specific

One question at a time

Include context

Review commands

​Limitations

​Next steps

Sandbox execution

Deploy a Worker

Build docs developers (and LLMs) love

Overview

Activating the AI

AI architecture

AI sidebar

Keyboard shortcuts

Command execution

Example conversation

Conversation persistence

Message format

API endpoints

Client implementation

Error handling

Best practices

Limitations

Next steps