Cua-Bench provides theDocumentation Index
Fetch the complete documentation index at: https://mintlify.com/trycua/cua/llms.txt
Use this file to discover all available pages before exploring further.
cb command-line tool for defining, running, and analyzing computer-use benchmark evaluations. Install it from the libs/cua-bench environment, then run cb --help or cb <command> --help to see options for your installed version.
Task and dataset commands
These commands let you inspect, list, scaffold, and build benchmark tasks and dataset artifacts.| Command | Purpose |
|---|---|
cb task info <path> | Inspect a task’s provider, variants, and lifecycle functions. |
cb task list [path] | List available task directories under an optional path. |
cb task create | Scaffold a new task interactively. |
cb task generate | Generate a task scaffold from a natural-language prompt. |
cb dataset list | List all known datasets. |
cb dataset build <path> | Build a dataset artifact from a task directory. |
cb task info
Print a task’s metadata: provider, available variants, and which lifecycle hooks are defined.
Path to the task directory or task definition file.
cb task list
List available task directories.
Optional root path to search. Defaults to the current working directory.
cb task create
Scaffold a new task interactively. The CLI prompts for a name, provider, and optional variants, then writes the task files.
cb task generate
Generate a task scaffold from a prompt string. Useful for quickly bootstrapping new evaluations.
cb dataset list
List all datasets known to the current Cua-Bench environment.
cb dataset build
Build a dataset artifact from a task directory.
Path to the dataset definition or task directory.
Run commands
These commands manage the full benchmark run lifecycle — from launching a single interactive task to running an entire dataset in parallel.| Command | Purpose |
|---|---|
cb interact <task-path> | Run one task interactively with the VM display visible. |
cb run task <task-path> | Run a single task, optionally with an agent or oracle. |
cb run dataset <dataset> | Run all tasks in a dataset, optionally in parallel. |
cb run list | List all runs and their current status. |
cb run info <run-id> | Show detailed information for a specific run. |
cb run watch <run-id> | Stream a run’s progress in the terminal. |
cb run logs <run-id> | Print logs for a run or session. |
cb run stop <run-id> | Stop a running benchmark. |
cb run task
Run a single benchmark task with an optional agent or oracle.
Usage: cb run task <task-path> [options]
Path to the task definition file or directory.
Agent to use for the run. Built-in values include
cua-agent. Pass a custom image name to use a containerized agent.Model identifier to pass to the agent (e.g.
gpt-4o, claude-3-5-sonnet).Oracle evaluator to use. Overrides agent-based evaluation.
Maximum number of agent steps before the task is marked failed.
Environment image to use for the task sandbox. Overrides the task’s default image.
Execution platform. See
cb platform for available options.Directory to write trajectory files and run artifacts. Defaults to a timestamped directory in the current working directory.
cb run dataset
Run all tasks in a dataset, with optional parallelism.
Usage: cb run dataset <dataset> [options]
Dataset name or path to the dataset definition. Use
cb dataset list to find available datasets.Agent to use for all tasks in the dataset. Built-in:
cua-agent. Custom: any agent image name.Model identifier to pass to the agent.
Oracle evaluator for all tasks.
Maximum agent steps per task.
Number of tasks to run concurrently. Higher values require more system resources. Defaults to
1 (sequential).Override the environment image for all tasks.
Execution platform.
Directory for trajectory files and run artifacts.
cb interact
Run a single task interactively with the VM display visible. Useful for debugging or manually completing a task.
Usage: cb interact <task-path> [options]
Path to the task to run.
--image, --platform, and --output-dir options as cb run task.
cb run list
List all benchmark runs and their status.
cb run info
Show detailed information about a specific run.
Run ID from
cb run list.cb run watch
Stream a run’s progress live in the terminal.
Run ID to follow.
cb run logs
Print logs for a run or a specific session within the run.
Run ID to fetch logs for.
cb run stop
Stop an active benchmark run.
Run ID to stop.
Environment and artifact commands
These command groups manage the infrastructure that Cua-Bench tasks run on.cb image
Manage sandbox environment images used by tasks.
| Subcommand | Purpose |
|---|---|
cb image list | List all available environment images. |
cb image create <name> | Build a new environment image. |
cb image delete <name> | Delete an environment image. |
cb image clone <src> <dst> | Clone an image to a new name. |
cb image shell <name> | Open an interactive shell in an image. |
cb image create
Build a new base environment image for use in benchmark tasks.
Name for the new image.
cb image list
List all environment images available to Cua-Bench, with their names and statuses.
cb platform
Inspect available execution platforms.
| Subcommand | Purpose |
|---|---|
cb platform list | List all configured platforms. |
cb platform info <name> | Show details for a specific platform. |
cb agent
Manage agent images that evaluate tasks.
| Subcommand | Purpose |
|---|---|
cb agent init | Initialize a new agent project. |
cb agent build | Build the agent container image. |
cb agent push | Push the agent image to a registry. |
cb trace
View traces and agent trajectories from completed runs.
| Subcommand | Purpose |
|---|---|
cb trace view <run-id> | Open a run’s trace in the terminal viewer. |
cb trace open <run-id> | Open agent trajectory files from a completed run. |