Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/trycua/cua/llms.txt

Use this file to discover all available pages before exploring further.

Every Sandbox instance exposes a set of typed interface objects that control the running VM or container. This page documents every method on every interface with exact signatures, parameter types, and return types.
AttributeClassPurpose
sb.shellShellRun shell commands and inspect output
sb.mouseMousePointer control — clicks, drags, scrolling
sb.keyboardKeyboardKeyboard input — typing, key combos, press/release
sb.screenScreenScreenshots and display dimensions
sb.clipboardClipboardClipboard read and write
sb.tunnelTunnelTCP port forwarding from sandbox to host
sb.terminalTerminalPTY (pseudo-terminal) session management
sb.windowWindowActive window inspection
sb.mobileMobileAndroid touch gestures and hardware keys

​
Shell

sb.shell runs commands inside the sandbox and returns structured output.

​
sb.shell.run

async def run(command: str, timeout: int = 30) -> CommandResult
​
command
str
required
Shell command to execute inside the sandbox.
​
timeout
int
default:"30"
Maximum number of seconds to wait for the command to finish before raising a timeout error.
Returns: CommandResult 
result = await sb.shell.run('echo hello')
print(result.stdout)    # 'hello\n'
print(result.success)   # True

result = await sb.shell.run('ls /tmp', timeout=10)

​
Mouse

sb.mouse controls the pointer. All coordinates are in screen pixels with the origin (0, 0) at the top-left corner.

​
sb.mouse.click

async def click(x: int, y: int, button: str = 'left') -> None
​
x
int
required
Horizontal screen coordinate in pixels.
​
y
int
required
Vertical screen coordinate in pixels.
​
button
str
default:"'left'"
Mouse button to click: 'left', 'right', or 'middle'.

​
sb.mouse.right_click

async def right_click(x: int, y: int) -> None
Convenience wrapper for a right-button click at (x, y).
​
x
int
required
Horizontal coordinate.
​
y
int
required
Vertical coordinate.

​
sb.mouse.double_click

async def double_click(x: int, y: int) -> None
​
x
int
required
Horizontal coordinate.
​
y
int
required
Vertical coordinate.

​
sb.mouse.move

async def move(x: int, y: int) -> None
Move the pointer to (x, y) without clicking.
​
x
int
required
Target horizontal coordinate.
​
y
int
required
Target vertical coordinate.

​
sb.mouse.scroll

async def scroll(x: int, y: int, scroll_x: int = 0, scroll_y: int = 3) -> None
Scroll at position (x, y). Positive scroll_y scrolls down; negative scrolls up.
​
x
int
required
Horizontal coordinate of the scroll target.
​
y
int
required
Vertical coordinate of the scroll target.
​
scroll_x
int
default:"0"
Horizontal scroll amount.
​
scroll_y
int
default:"3"
Vertical scroll amount. Positive = down, negative = up.

​
sb.mouse.mouse_down

async def mouse_down(x: int, y: int, button: str = 'left') -> None
Press and hold a mouse button at (x, y) without releasing.
​
x
int
required
Horizontal coordinate.
​
y
int
required
Vertical coordinate.
​
button
str
default:"'left'"
Mouse button: 'left', 'right', or 'middle'.

​
sb.mouse.mouse_up

async def mouse_up(x: int, y: int, button: str = 'left') -> None
Release a held mouse button at (x, y).
​
x
int
required
Horizontal coordinate.
​
y
int
required
Vertical coordinate.
​
button
str
default:"'left'"
Mouse button to release.

​
sb.mouse.drag

async def drag(start_x: int, start_y: int, end_x: int, end_y: int, button: str = 'left') -> None
Press, drag, and release the pointer from one point to another.
​
start_x
int
required
Starting horizontal coordinate.
​
start_y
int
required
Starting vertical coordinate.
​
end_x
int
required
Ending horizontal coordinate.
​
end_y
int
required
Ending vertical coordinate.
​
button
str
default:"'left'"
Mouse button to use for the drag.

​
Keyboard

sb.keyboard sends keyboard input to the sandbox.

​
sb.keyboard.type

async def type(text: str) -> None
Type a string of text character by character.
​
text
str
required
Text to type into the focused element.

​
sb.keyboard.keypress

async def keypress(keys: Union[List[str], str]) -> None
Press a single key or a key combination simultaneously.
​
keys
List[str] | str
required
A single key name (e.g. 'enter') or a list of keys to press together (e.g. ['ctrl', 'c']). Key names follow X11 keysym conventions.
await sb.keyboard.keypress('enter')
await sb.keyboard.keypress(['ctrl', 'c'])
await sb.keyboard.keypress(['cmd', 'shift', '4'])

​
sb.keyboard.key_down

async def key_down(key: str) -> None
Press and hold a key without releasing it.
​
key
str
required
Key name to hold down.

​
sb.keyboard.key_up

async def key_up(key: str) -> None
Release a previously held key.
​
key
str
required
Key name to release.

​
Screen

sb.screen captures screenshots and reports display dimensions.

​
sb.screen.screenshot

async def screenshot(format: str = 'png', quality: int = 95) -> bytes
Capture a screenshot of the sandbox display.
​
format
str
default:"'png'"
Image format: 'png' (lossless) or 'jpeg' (lossy, roughly 5–10× smaller).
​
quality
int
default:"95"
JPEG quality from 1 to 95. Ignored when format='png'.
Returns: bytes — raw image data.

​
sb.screen.screenshot_base64

async def screenshot_base64(format: str = 'png', quality: int = 95) -> str
Same as screenshot() but returns the image as a base64-encoded string — convenient for embedding in JSON payloads or LLM vision inputs.
​
format
str
default:"'png'"
Image format: 'png' or 'jpeg'.
​
quality
int
default:"95"
JPEG quality 1–95.
Returns: str — base64-encoded image.

​
sb.screen.size

async def size() -> Tuple[int, int]
Returns the display dimensions as a (width, height) tuple in pixels. 
width, height = await sb.screen.size()
print(f'Display: {width}×{height}')

​
Clipboard

sb.clipboard reads from and writes to the sandbox clipboard.

​
sb.clipboard.get

async def get() -> str
Returns: str — the current clipboard text content.

​
sb.clipboard.set

async def set(text: str) -> None
​
text
str
required
Text to place on the clipboard.

​
Tunnel

sb.tunnel forwards sandbox TCP ports (or Android abstract sockets) to the host machine.

​
sb.tunnel.forward

def forward(*ports) -> _TunnelContext
Forward one or more sandbox ports to the host. Each positional argument is an int (TCP port) or str (Android abstract socket name, e.g. 'chrome_devtools_remote'). Use as an async context manager to auto-close when done, or await directly to keep the tunnel alive until you call .close().
​
*ports
int | str
required
One or more port numbers or Android abstract socket names to forward.
Returns:
  • A single TunnelInfo when one port is given.
  • A dict[sandbox_port, TunnelInfo] when multiple ports are given.
# Single port — context manager
async with sb.tunnel.forward(8080) as tunnel:
    print(tunnel.url)   # e.g. http://localhost:54321

# Multiple ports
tunnels = await sb.tunnel.forward(8080, 9229)
print(tunnels[8080].url)
print(tunnels[9229].url)

​
TunnelInfo.close

async def close() -> None
Close this tunnel. A no-op if already closed or if the tunnel was opened via a context manager.

​
Terminal

sb.terminal manages PTY (pseudo-terminal) sessions inside the sandbox, enabling interactive shell access.

​
sb.terminal.create

async def create(command: Optional[str] = None, cols: int = 80, rows: int = 24) -> dict
Create a new PTY session. If command is omitted, the default login shell is used.
​
command
str | None
default:"None"
Command to run in the PTY. Defaults to the user’s login shell.
​
cols
int
default:"80"
Terminal width in columns.
​
rows
int
default:"24"
Terminal height in rows.
Returns: dict with fields pid (int), cols (int), rows (int).

​
sb.terminal.send_input

async def send_input(pid: int, data: str) -> None
​
pid
int
required
PTY session PID returned by create().
​
data
str
required
Input data to send to the terminal.

​
sb.terminal.info

async def info(pid: int) -> Optional[dict]
Return session information for a PTY, or None if the session no longer exists.
​
pid
int
required
PTY session PID.

​
sb.terminal.close

async def close(pid: int) -> bool
Kill a PTY session.
​
pid
int
required
PTY session PID to terminate.
Returns: bool — True on success.

​
Window

sb.window provides information about the active window.

​
sb.window.get_active_title

async def get_active_title() -> str
Returns: str — the title of the currently focused window. 
title = await sb.window.get_active_title()
print(title)  # e.g. 'Visual Studio Code'

​
Mobile

sb.mobile provides touch and hardware-key control for Android sandboxes. All coordinates are in screen pixels. Single-touch methods use adb shell input; multi-touch gestures use MT Protocol B via adb root + sendevent.
sb.mobile is only available on Android sandboxes created with Image.android().

​
Touch gestures

​
.tap(x, y)
(int, int)
Single tap at (x, y).
​
.long_press(x, y, duration_ms=1000)
Long-press at (x, y) for duration_ms milliseconds.
​
.double_tap(x, y, delay=0.1)
Two taps at (x, y) separated by delay seconds.
​
.type_text(text)
str
Type text using adb shell input text.
​
.swipe(x1, y1, x2, y2, duration_ms=300)
Swipe from (x1, y1) to (x2, y2) over duration_ms milliseconds.
​
.scroll_up(x, y, distance=600, duration_ms=400)
Scroll up from (x, y).
​
.scroll_down(x, y, distance=600, duration_ms=400)
Scroll down from (x, y).
​
.scroll_left(x, y, distance=400, duration_ms=300)
Scroll left from (x, y).
​
.scroll_right(x, y, distance=400, duration_ms=300)
Scroll right from (x, y).
​
.fling(x1, y1, x2, y2)
Fast fling gesture from (x1, y1) to (x2, y2).

​
Multi-touch

​
.gesture(*finger_paths, duration_ms=400, steps=0)
N-finger gesture using MT Protocol B. Each positional argument is an (x, y) tuple; pairs of consecutive tuples define one finger’s start and end point. Requires an even number of tuples ≥ 4 (at least two fingers × two points each). steps=0 auto-calculates steps as max(5, duration_ms // 20).
​
.pinch_in(cx, cy, spread=300, duration_ms=400)
Two-finger pinch-in (zoom out) centered at (cx, cy). spread is the initial finger separation in pixels.
​
.pinch_out(cx, cy, spread=300, duration_ms=400)
Two-finger pinch-out (zoom in) centered at (cx, cy).

​
Hardware keys

await sb.mobile.home()           # Home button
await sb.mobile.back()           # Back button
await sb.mobile.recents()        # Recents / app switcher
await sb.mobile.power()          # Power button
await sb.mobile.volume_up()      # Volume up
await sb.mobile.volume_down()    # Volume down
await sb.mobile.enter()          # Enter key
await sb.mobile.backspace()      # Backspace key
await sb.mobile.wake()           # Wake the screen
await sb.mobile.notifications()  # Pull down notification shade
await sb.mobile.close_notifications()  # Dismiss notification shade
You can also send any Android keycode directly: 
await sb.mobile.key(keycode=3)   # KEYCODE_HOME
​
.key(keycode)
int
Send an arbitrary Android keycode integer via adb shell input keyevent.

Build docs developers (and LLMs) love

Get started for freeTalk to us