Automation Skills

Browser automation, screenshot-based interaction, and request routing. These skills give CLI-JAW the ability to control Chrome via CDP, click UI elements from screenshots, and route web requests programmatically.

browser

Chrome browser control via the Chrome DevTools Protocol (CDP). Opens pages, navigates, takes reference snapshots, clicks elements, types text, extracts content, and captures screenshots. Requires the cli-jaw browser server to be running.

Capabilities

Example Usage

# Open a page and take a screenshot
/browser open https://example.com
/browser screenshot

# Click a button by selector
/browser click "#submit-btn"

# Type into an input field
/browser type "#search-input" "CLI-JAW documentation"

# Extract text content from an element
/browser extract ".main-content" text

# Execute JavaScript in the page
/browser eval "document.title"

CDP Connection

The browser skill connects to Chrome through the cli-jaw browser server, which manages the CDP WebSocket connection. The server must be started before using browser commands.

# Start the browser server (runs on a dedicated port)
jaw browser serve

# The skill auto-connects when invoked
# Connection is reused across commands in the same session

vision-click

Screenshot-based UI interaction using vision models. Instead of relying on DOM selectors, this skill takes a screenshot, identifies UI elements visually, and clicks at the correct coordinates. Works with any application visible on screen -- not limited to browsers.

How It Works

1. Capture -- Takes a screenshot of the current screen or a specified region
2. Analyze -- Sends the screenshot to a vision model to identify the target element
3. Locate -- The model returns pixel coordinates of the element to click
4. Act -- Performs the click (or other action) at the identified coordinates

Example Usage

# Click a button described by its visual label
/vision-click "the blue Submit button"

# Click on a specific icon
/vision-click "the gear icon in the top-right corner"

# Right-click on a file in Finder
/vision-click --right "the README.md file"

# Describe what is currently visible
/vision-click --describe

When to Use vision-click vs browser

web-routing

Programmatic HTTP request routing and interception. Configures request handlers that can redirect, rewrite, block, or transform web traffic. Useful for local development proxying, API mocking, and traffic shaping.

Routing Rules

Example Usage

# Redirect API calls to local dev server
/web-routing add /api/* -> http://localhost:8080/api/*

# Mock a specific endpoint with a JSON response
/web-routing mock GET /api/users '[{"id":1,"name":"Test"}]'

# Block tracking requests
/web-routing block *.analytics.com

# Add auth header to all outgoing API requests
/web-routing transform /api/* --header "Authorization: Bearer $TOKEN"

# List active routing rules
/web-routing list

# Remove a routing rule
/web-routing remove /api/*

Pattern Matching

Route patterns support glob syntax and can match on path, host, method, and headers.

# Glob patterns
/api/*                    # matches /api/users, /api/posts/1, etc.
/api/users/:id            # named parameter capture
*.example.com             # subdomain wildcard

# Method-specific routes
GET /api/users            # only match GET requests
POST /api/users           # only match POST requests

# Header-based matching
/api/* [Content-Type: application/json]   # match by header

Skill Comparison

Choose the right automation skill based on your task.

Common Patterns

Scrape and Extract

Combine the browser skill with content extraction to pull structured data from web pages.

# Open the target page
/browser open https://news.example.com

# Extract all article titles
/browser eval "Array.from(document.querySelectorAll('h2.title')).map(el => el.textContent)"

# Take a screenshot for reference
/browser screenshot

Form Automation

Fill and submit forms by chaining type and click commands.

# Navigate to login page
/browser open https://app.example.com/login

# Fill in credentials
/browser type "#email" "user@example.com"
/browser type "#password" "secretpassword"

# Submit the form
/browser click "#login-btn"

# Wait for navigation and verify
/browser wait "nav.dashboard"
/browser screenshot

Desktop App + Browser Combo

Use vision-click for desktop app elements and browser for web content in a hybrid workflow.

# Click "Open in Browser" in a desktop app
/vision-click "the Open in Browser button"

# Switch to browser skill for DOM-level control
/browser wait ".main-content"
/browser extract ".main-content" text