Action functions simulate user interactions in Chrome via the DevTools Protocol. They handle cursor movement, clicks, keyboard input, and element discovery.
Navigation
navigate()
Navigate to a URL and wait for page load.
await navigate(client, url)
Chrome DevTools Protocol client
import { navigate } from '@webreel/core';
await navigate(client, 'https://example.com');
pause()
Wait for a specified duration.
Duration to wait in milliseconds
import { pause } from '@webreel/core';
await pause(2000); // Wait 2 seconds
Element Discovery
waitForSelector()
Wait for an element to appear in the DOM.
await waitForSelector(client, selector, timeoutMs?)
Chrome DevTools Protocol client
Maximum time to wait in milliseconds
await waitForSelector(client, '.submit-button');
waitForText()
Wait for text content to appear.
await waitForText(client, text, within?, timeoutMs?)
Chrome DevTools Protocol client
Optional CSS selector to search within
Maximum time to wait in milliseconds
await waitForText(client, 'Welcome back');
await waitForText(client, 'Submit', '.form-actions');
findElementByText()
Find an element’s bounding box by text content.
const box = await findElementByText(client, text, within?)
Chrome DevTools Protocol client
Text content to search for
Optional CSS selector to search within
Element bounds, or null if not foundinterface BoundingBox {
x: number;
y: number;
width: number;
height: number;
}
const button = await findElementByText(client, 'Sign In');
if (button) {
await clickAt(ctx, client, button.x + button.width / 2, button.y + button.height / 2);
}
findElementBySelector()
Find an element’s bounding box by CSS selector.
const box = await findElementBySelector(client, selector, within?)
Chrome DevTools Protocol client
CSS selector to search for
Optional CSS selector to search within
Element bounds, or null if not found
const input = await findElementBySelector(client, '#email-input');
Cursor & Mouse
moveCursorTo()
Animate cursor movement to a position.
await moveCursorTo(ctx, client, x, y)
Recording context for cursor tracking
Chrome DevTools Protocol client
Target horizontal position in pixels
Target vertical position in pixels
await moveCursorTo(ctx, client, 500, 300);
clickAt()
Click at specific coordinates.
await clickAt(ctx, client, x, y, modifiers?)
Recording context for cursor tracking
Chrome DevTools Protocol client
Horizontal click position in pixels
Vertical click position in pixels
Optional modifier keys: 'cmd', 'ctrl', 'shift', 'alt', 'mod'
- Moves cursor to position
- Shows modifier key HUD if specified
- Dispatches synthetic click events with correct modifier flags
- Triggers browser focus behavior
- Records event for sound effects
// Simple click
await clickAt(ctx, client, 400, 200);
// Click with modifiers
await clickAt(ctx, client, 400, 200, ['cmd']); // Cmd+Click
await clickAt(ctx, client, 400, 200, ['mod']); // Cmd on Mac, Ctrl on Windows/Linux
dragFromTo()
Drag an element from one position to another.
await dragFromTo(ctx, client, fromBox, toBox)
Recording context for cursor tracking
Chrome DevTools Protocol client
const item = await findElementBySelector(client, '.draggable-item');
const dropzone = await findElementBySelector(client, '.drop-zone');
if (item && dropzone) {
await dragFromTo(ctx, client, item, dropzone);
}
Keyboard
pressKey()
Press a keyboard key or shortcut.
await pressKey(ctx, client, key, label?)
Recording context for event tracking
Chrome DevTools Protocol client
Key combination (e.g., 'Enter', 'cmd+s', 'ctrl+shift+p')
Optional custom label for HUD display
Enter, Tab, EscapeBackspace, DeleteArrowUp, ArrowDown, ArrowLeft, ArrowRight- Letter keys:
a-z
Modifiers:
cmd or meta - Command key (Mac) or Windows keyctrl - Control keyshift - Shift keyalt - Alt/Option keymod - Automatically resolves to cmd on Mac, ctrl elsewhere
// Simple keys
await pressKey(ctx, client, 'Enter');
await pressKey(ctx, client, 'Escape');
// Shortcuts
await pressKey(ctx, client, 'cmd+s');
await pressKey(ctx, client, 'mod+shift+p'); // Platform-aware
await pressKey(ctx, client, 'ctrl+a');
// Custom label
await pressKey(ctx, client, 'cmd+k', 'Search');
typeText()
Type text character by character.
await typeText(ctx, client, text, delayMs?)
Recording context for event tracking
Chrome DevTools Protocol client
Base delay between keystrokes in milliseconds (randomized for realism)
await typeText(ctx, client, '[email protected]');
await typeText(ctx, client, 'Fast typing', 60);
modKey()
Get the platform-specific modifier key name.
Returns 'cmd' on macOS, 'ctrl' elsewhere
import { modKey } from '@webreel/core';
const modifier = modKey();
await pressKey(ctx, client, `${modifier}+s`); // Cmd+S on Mac, Ctrl+S elsewhere
Utilities
captureScreenshot()
Capture a screenshot of the current page.
await captureScreenshot(client, outputPath)
Chrome DevTools Protocol client
Path where PNG screenshot will be saved
await captureScreenshot(client, './screenshot.png');
Complete Example
import {
launchChrome,
connectCDP,
Recorder,
RecordingContext,
navigate,
waitForSelector,
findElementByText,
clickAt,
typeText,
pressKey
} from '@webreel/core';
const chrome = await launchChrome();
const client = await connectCDP(chrome.wsUrl);
const ctx = new RecordingContext();
const recorder = new Recorder(1920, 1080, { fps: 60 });
await recorder.start(client, './demo.mp4', ctx);
// Navigate and wait
await navigate(client, 'https://example.com');
await waitForSelector(client, '.login-form');
// Find and click
const signInButton = await findElementByText(client, 'Sign In');
if (signInButton) {
await clickAt(ctx, client,
signInButton.x + signInButton.width / 2,
signInButton.y + signInButton.height / 2
);
}
// Type credentials
await typeText(ctx, client, '[email protected]');
await pressKey(ctx, client, 'Tab');
await typeText(ctx, client, 'password123');
await pressKey(ctx, client, 'Enter');
await recorder.stop();
await client.close();
await chrome.process.kill();