Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/trycua/cua/llms.txt

Use this file to discover all available pages before exploring further.

Cua Bench is an MIT-licensed framework for computer-use benchmarks and reinforcement-learning environments. It packages the task definition, starting state, agent interface, and evaluator needed to run a repeatable experiment — across Linux, Windows, Android, browser, or simulated desktops. A computer-use score is meaningful only when the starting state and success condition are reproducible; that is the problem Cua Bench solves.

​
Why tasks are environments

A benchmark task is not just a prompt. A Cua Bench task defines:
  1. The prompt — what the agent is asked to do
  2. The starting state — what the computer looks like before the agent acts
  3. Setup — code that prepares the environment for each run
  4. Variations — different inputs or conditions that test the same objective
  5. Oracle solution — a reference implementation that correctly solves the task
  6. Evaluator — task-specific checks that score the final state
The evaluator is the single source of truth. It makes the score reproducible regardless of who — or what model — attempts the task.

​
How the pieces fit

Task definition
  └── Prompt + variations
  └── Setup function
  └── Oracle (reference solution)
  └── Evaluator (scoring function)
        |
        v
     Provider
  (browser / simulated / container / VM / hosted computer)
        |
        v
  Agent adapter
  (translates model output into computer actions)
        |
        v
     Runner
  (executes tasks, records traces, supports parallel workers)
Each layer is independent:
  • Tasks define the prompt, setup, variations, and evaluator.
  • Providers supply the environment: a browser, a simulated surface, a Docker container, a VM, or a hosted computer.
  • Agent adapters connect a model or agent implementation to the environment interface.
  • The runner executes one task or a full dataset, records trajectories, and supports parallel workers.
This separation lets the same task definition run against different agents and infrastructure. Evaluation logic stays independent from the model being tested.

​
Task lifecycle

A single Cua Bench task execution follows this lifecycle:
1

Setup

The setup function runs and prepares the environment. This may install applications, seed files, configure system state, or start services. Each variation can have its own setup parameters.
2

Observation

The agent adapter receives the initial observation — a screenshot, an accessibility tree, or both — along with the task prompt.
3

Act

The agent takes actions against the environment (clicks, typing, shell commands) until it decides the task is complete or a step budget is exhausted.
4

Evaluate

The evaluator inspects the final state and returns a reward score between 0.0 and 1.0. A score of 1.0 means the task was completed correctly.
5

Trace export

The complete trajectory — observations, actions, rewards, and timing — is recorded. Trajectories can be inspected, scored, or exported as training data for reinforcement learning.

​
Supported benchmarks

Cua Bench ships adapters for several established computer-use benchmarks:

OSWorld

A large-scale benchmark with tasks across Windows, macOS, and Ubuntu desktop environments. Covers web browsers, office apps, file management, and coding tools.

ScreenSpot

A grounding benchmark that tests whether a model can locate UI elements from natural language descriptions. Covers mobile, desktop, and web screenshots.

Windows Arena

A benchmark for Windows desktop automation. Tasks require multi-step reasoning across Windows applications with verifiable outcomes.

Custom tasks

Define your own tasks with any environment, prompt, and evaluator. Publish them to the task registry for others to use.

​
The reinforcement-learning environment interface

Cua Bench exposes an RL environment interface compatible with standard training loops. Each step returns:
  • observation — screenshot bytes, accessibility tree, or both
  • reward — a float from the task’s evaluator
  • done — whether the episode has ended
  • info — metadata including timing, step count, and evaluator details
This interface lets you plug Cua Bench tasks directly into RL training frameworks. Trajectories collected during evaluation can be filtered, re-scored, and used as offline training data.

​
Running a dataset with an agent

# Install Cua Bench
uv tool install 'cua-bench[browser]'

# Create a base image for container tasks
cb image create linux-docker

# Run a benchmark dataset with the built-in cua-agent, 4 tasks in parallel
cb run dataset datasets/cua-bench-basic --agent cua-agent --max-parallel 4
The runner produces a results directory with one trajectory file per task execution.

​
Relationship to the rest of Cua

Cua Bench supplies the task and evaluation layer. Cua Sandbox components supply machine environments — containers, VMs, and hosted computers — while agent adapters translate model output into computer actions. 
Cua Bench  →  task definition, evaluation, trajectory export
Cua Sandbox →  isolated computer environments (Docker, QEMU, Lume, cloud)
Cua Driver  →  UI automation on real or virtualized machines
Agent       →  model + harness that drives the action loop
The resulting trajectory can be inspected, scored, or used as training data for improving computer-use models.

​
Further reading

First Bench task tutorial

Scaffold a task, run the oracle, and evaluate your own attempt — no Docker needed.

Task registry

Browse published tasks and the applications they cover.

Run a task

Run, inspect, and validate existing tasks from the registry.

Task definition reference

Complete specification for task setup, variations, oracle, and evaluator functions.

Build docs developers (and LLMs) love

Get started for freeTalk to us