Skip to main content

​
Overview

Skyvern provides a unified interface that combines traditional Playwright browser automation with AI-powered task execution. You can seamlessly mix traditional CSS selector-based interactions with natural language AI commands in the same session.

​
The Hybrid Approach

Skyvern pages extend standard Playwright pages with additional AI capabilities. You get:
  1. Traditional Playwright methods - All standard Playwright page methods (click, fill, goto, etc.)
  2. AI-augmented methods - Traditional methods enhanced with optional AI-powered element location
  3. Pure AI methods - New AI-powered commands for complex tasks

​
AI-Augmented Playwright Actions

All standard Playwright actions support an optional prompt parameter for AI-powered element location.

​
Click

Three interaction modes: 
# 1. Traditional Playwright - CSS/XPath selectors
await page.click("#submit-button")

# 2. AI-powered - natural language
await page.click(prompt="Click the green Submit button")

# 3. AI fallback - tries selector first, falls back to AI if it fails
await page.click("#submit-btn", prompt="Click the Submit button")
Additional Options: 
# With timeout
await page.click(prompt="Click login", timeout=5000)

# With other Playwright click options
await page.click(prompt="Click checkbox", force=True, noWaitAfter=True)

​
Fill

Fill input fields using selectors or natural language: 
# Traditional selector
await page.fill("#email-input", "[email protected]")

# AI-powered
await page.fill(prompt="Fill email address", value="[email protected]")

# AI can also extract value from prompt
await page.fill(prompt="Fill '[email protected]' in the email field")

# Fallback mode
await page.fill("#email", "[email protected]", prompt="Fill email field")

​
Select Option

Select dropdown options using selectors or natural language: 
# Traditional selector
await page.select_option("#country", "us")

# AI-powered
await page.select_option(prompt="Select 'United States' from country dropdown")

# With value specified
await page.select_option(prompt="Select country", value="us")

# Fallback mode
await page.select_option("#country", "us", prompt="Select United States")

​
When to Use Each Approach

​
Use Traditional Selectors When:

  • You know the exact CSS selector or XPath
  • The page structure is stable and well-documented
  • Performance is critical (selectors are faster)
  • You’re automating your own website

​
Use AI-Powered Element Location When:

  • Selectors are fragile or frequently change
  • You need to work across multiple websites with different structures
  • Element location is ambiguous (e.g., “the second blue button”)
  • You’re automating third-party websites

​
Use Fallback Mode When:

  • You have a selector but want a safety net
  • You’re migrating from traditional automation to AI
  • You want optimal performance with AI resilience

​
Complete Example: Mixing Traditional and AI

import asyncio
from skyvern import Skyvern
from skyvern.schemas.run_blocks import CredentialType

async def main():
    skyvern = Skyvern(api_key="your-api-key")
    browser = await skyvern.launch_cloud_browser()
    page = await browser.get_working_page()
    
    # Traditional Playwright navigation
    await page.goto("https://example.com")
    
    # Traditional selector-based click
    await page.click("#login-button")
    
    # Wait for navigation using Playwright
    await page.wait_for_selector("#email")
    
    # AI-powered login (handles complex login flows)
    await page.agent.login(
        credential_type=CredentialType.skyvern,
        credential_id="cred_123",
    )
    
    # Traditional navigation and interaction
    await page.click("#invoices-tab")
    await page.fill("#search-input", "Invoice-2024-001")
    await page.click("button[type='submit']")
    
    # AI-augmented click with fallback
    await page.click(
        "#download-btn",
        prompt="Click the download button for the invoice"
    )
    
    # AI-powered data extraction
    invoice_data = await page.extract(
        prompt="Extract invoice details",
        schema={
            "invoice_number": "string",
            "amount": "number",
            "date": "string"
        }
    )
    
    print(f"Invoice: {invoice_data}")
    
    # Traditional screenshot
    await page.screenshot(path="invoice.png", full_page=True)
    
    await browser.close()
    await skyvern.aclose()

asyncio.run(main())

​
Best Practices

​
1. Start with Traditional Selectors

Use traditional selectors for critical paths where you know the structure: 
# Fast and reliable for known structure
await page.click("#submit-order")

​
2. Add AI Fallbacks for Resilience

Add prompt fallbacks for elements that might change: 
# Tries selector first, AI if it fails
await page.click("#submit-btn", prompt="Click the Submit Order button")

​
3. Use AI for Complex Element Location

Let AI handle ambiguous or complex element selection: 
# AI excels at context-dependent selection
await page.click(prompt="Click the blue 'Add to Cart' button next to the iPhone 15")

​
4. Combine Traditional and AI Strategically

# Traditional for structure, AI for content
await page.goto("https://example.com/products")
await page.wait_for_selector(".product-grid")  # Traditional wait
await page.click(prompt="Click the product with '4.5 stars'")  # AI selection

​
5. Use AI for Multi-Step Workflows

For complex multi-step tasks, use page.agent.run_task(): 
# Let AI handle the entire checkout flow
await page.agent.run_task(
    "Complete checkout with shipping to 123 Main St, NYC, pay with card ending in 1234"
)

​
Performance Considerations

MethodSpeedResilienceBest For
Traditional selectorsFastestLow (breaks on changes)Stable, known structures
AI-augmented (fallback)Fast with safety netHighProduction automation
Pure AISlowerHighestDynamic content, third-party sites
AI tasksVariableHighestComplex multi-step workflows

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us