Concept

The runtime reuses headed Chrome and keeps each CLI command short-lived.

Commands connect to a CDP port, inspect or mutate the current target, then exit. Browser state and Web-AI session metadata live under BROWSER_AGENT_HOME.

Operating model

agbrowse start --headed launches or reuses Chrome.
Snapshots produce refs; actions consume refs or explicit selectors.
Tab lifecycle cleanup keeps provider pools bounded.
Headless browser state is rejected for Web-AI commands that need visible UI.

Adaptive-fetch escalation ladder

The agbrowse fetch command reads a single candidate URL through a 6-rung adaptive escalation ladder. Each rung produces a content score; the ladder stops when the score is strong enough or all rungs are exhausted.

Public endpoints + direct HTTP using known API resolvers (GitHub, Reddit, HN, Wikipedia, npm, PyPI, arXiv, Stack Exchange, and others) plus direct fetch with browser-grade identity headers.
TLS/JA3 impersonation via curl-impersonate. Triggered on 403, 429, or challenge responses before launching a browser. This lets the runtime bypass fingerprint-based blocks without a full browser render.
Third-party readers such as Jina Reader (opt-in with --allow-third-party-reader).
Isolated browser render using Camoufox or headless Chrome, with automatic network API discovery to capture XHR/fetch payloads that contain the target content.
User session (opt-in with --browser-session user): reuses the authenticated browser profile for paywalled or login-gated content.
Human-in-the-loop (opt-in with --browser-session interactive): manual challenge resolution in the headed browser window.

WAF profile detection

During HTTP-phase escalation, the runtime inspects response headers, status codes, and challenge page signatures to identify the upstream WAF product. Recognized profiles include Cloudflare (managed rules + Turnstile), Akamai Bot Manager, AWS WAF, Imperva/Incapsula, DataDome, and PerimeterX. Detection results feed into the escalation decision: a Cloudflare Turnstile challenge, for example, skips straight to the browser rung rather than retrying with different headers.

TLS/JA3 impersonation

When a direct HTTP fetch returns 403, 429, or a recognized challenge page, the runtime attempts a TLS-impersonation retry using curl-impersonate before escalating to a full browser. This matches the JA3 fingerprint of a real Chrome browser at the TLS layer, which is enough to pass many fingerprint-based WAF blocks without the cost of launching Chrome.

Search command

agbrowse search is a standalone deep search pipeline that any CLI agent can use. It rewrites the user query into focused sub-queries, discovers candidate URLs, runs each through the adaptive-fetch ladder for original-page verification, scores evidence, and returns structured results. Use --verify <url> to check a single URL, --stdin-results to enrich externally-sourced search results, or --deep --vendor grok to escalate to web-ai when evidence is insufficient.

Diagnostics

agbrowse status --json
agbrowse tabs --json
agbrowse console --limit 50
agbrowse network --duration 1000

Web-AI sessions Config