Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/jaypopat/cf_ai_duet/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Duet uses Cloudflare’s AI service with the Llama 3 8B Instruct model to provide intelligent pair programming assistance. The LLM generates responses based on conversation history and can output commands for sandbox execution.

​
Model Configuration

The worker uses Cloudflare AI binding configured in wrangler.toml: 
[ai]
binding = "AI"
Source: ~/workspace/source/cf-worker/wrangler.toml:12-13 The model is accessed via the @cf/meta/llama-3-8b-instruct identifier: 
private async runAI(messages: AIMessage[]): Promise<string> {
  const result = await this.env.AI.run("@cf/meta/llama-3-8b-instruct", {
    messages,
  });
  return result.response?.trim() || "";
}
Source: ~/workspace/source/cf-worker/index.ts:122-127

​
Message Format

The AI expects messages in a specific format: 
interface AIMessage {
  role: "system" | "user" | "assistant";
  content: string;
}
Source: ~/workspace/source/cf-worker/index.ts:84-87

​
System Prompt

Duet’s AI behavior is defined by a system prompt that instructs it to be concise and use special tags for command execution: 
const aiMessages: AIMessage[] = [
  {
    role: "system",
    content:
      "You are Duet, a concise pair-programming assistant. " +
      "You can run commands in a sandbox using <run>command</run> tags. " +
      "When asked to perform an action, briefly explain what you will do and wrap the exact shell command(s) in <run> tags. " +
      "Do NOT include predicted output in your response - just provide the explanation and command.",
  },
  // ...
];
Source: ~/workspace/source/cf-worker/index.ts:154-162 This prompt establishes:
  • Identity: Duet is a pair-programming assistant
  • Tone: Concise and action-oriented
  • Command syntax: Use <run>command</run> tags
  • Output handling: Don’t predict output, let sandbox provide it

​
Conversation Context

The AI receives context from recent conversation history: 
const aiMessages: AIMessage[] = [
  {
    role: "system",
    content: "You are Duet, a concise pair-programming assistant..."
  },
  ...this.state.messages.slice(-10).map<AIMessage>((m) => ({
    role: m.role === "agent" ? "assistant" : "user",
    content: m.text,
  })),
  { role: "user", content: userMsg.text },
];
Source: ~/workspace/source/cf-worker/index.ts:154-168 The agent:
  1. Includes the system prompt
  2. Adds the last 10 messages from conversation history
  3. Appends the current user message
  4. Converts message roles (“agent” → “assistant”, “user” → “user”)

​
Command Execution Flow

The AI can trigger sandbox commands using special tags:

​
1. AI Response with Commands

The AI wraps commands in <run> tags: 
I'll check the directory contents.
<run>ls -la</run>

​
2. Command Extraction

The agent extracts commands using regex: 
private async executeCommands(text: string, roomId: string): Promise<string> {
  const matches = Array.from(text.matchAll(/<run>([\s\S]*?)<\/run>/g));
  let result = text;

  for (const match of matches) {
    const cmd = match[1]?.trim();
    if (!cmd) {
      continue;
    }
    // ...
  }
}
Source: ~/workspace/source/cf-worker/index.ts:185-193

​
3. Sandbox Execution

Each extracted command is executed in the room’s sandbox: 
try {
  const sandbox = getSandbox(this.env.Sandbox, `sandbox-${roomId}`);
  const { stderr, stdout } = await sandbox.exec(cmd);

  const summary =
    stdout.slice(0, 500) || stderr.slice(0, 500) || "[no output]";
  result += `\n\nOutput (${cmd}):\n${summary}`;
} catch (e) {
  const msg = e instanceof Error ? e.message : String(e);
  result += `\n\nError (${cmd}):\n${msg}`;
}
Source: ~/workspace/source/cf-worker/index.ts:195-205

​
4. Output Appending

Command outputs are appended to the AI’s response:
  • First 500 characters of stdout or stderr
  • Error messages if execution fails
  • “[no output]” if command produces nothing

​
5. Tag Removal

The <run> tags are stripped from the final response: 
return result.replace(/<run>[\s\S]*?<\/run>/g, "").trim() || "";
Source: ~/workspace/source/cf-worker/index.ts:207

​
Complete Message Flow

private async handleMessage(roomId: string, rawBody: unknown): Promise<Response> {
  // 1. Validate request
  const parseResult = MessageRequestSchema.safeParse(rawBody);
  
  // 2. Create user message
  const userMsg: DuetMessage = {
    role: "user",
    userId: data.userId?.trim(),
    text: data.text.trim(),
    ts: Date.now(),
  };

  // 3. Build AI context
  const aiMessages: AIMessage[] = [
    { role: "system", content: "..." },
    ...this.state.messages.slice(-10).map(...),
    { role: "user", content: userMsg.text },
  ];

  // 4. Get AI response
  const text = await this.runAI(aiMessages);
  
  // 5. Execute any commands in response
  const textWithOutputs = await this.executeCommands(text, roomId);

  // 6. Create agent message
  const agentMsg: DuetMessage = {
    role: "agent",
    text: textWithOutputs,
    ts: Date.now(),
  };

  // 7. Update state
  const nextMessages = [...this.state.messages, userMsg, agentMsg].slice(-50);
  this.setState({ messages: nextMessages });

  // 8. Return response
  return Response.json({ reply: agentMsg.text, messages: nextMessages });
}
Source: ~/workspace/source/cf-worker/index.ts:129-183

​
API Request/Response

​
Request Format

{
  "text": "list files in the current directory",
  "userId": "user-123"
}
Validated by: 
const MessageRequestSchema = z.object({
  text: z.string().min(1, "Text cannot be empty"),
  userId: z.string().optional(),
});
Source: ~/workspace/source/cf-worker/index.ts:9-12

​
Response Format

{
  "reply": "I'll check the directory contents.\n\nOutput (ls -la):\ntotal 48\ndrwxr-xr-x  12 user  staff   384 Mar  1 10:00 .",
  "messages": [
    {
      "role": "user",
      "userId": "user-123",
      "text": "list files in the current directory",
      "ts": 1709294400000
    },
    {
      "role": "agent",
      "text": "I'll check the directory contents.\n\nOutput (ls -la):\n...",
      "ts": 1709294401000
    }
  ]
}
Source: ~/workspace/source/internal/ai/client.go:42-47

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us