Guide

Run provider web UI tasks with explicit prompts and recoverable sessions.

Web-AI commands operate through the visible browser UI. They are not provider API clients and do not use API keys.

Render before send

agbrowse web-ai render --vendor chatgpt --prompt "Explain the plan" --json

Send and poll

SID=$(agbrowse web-ai send --vendor chatgpt --inline-only --prompt "Analyze this" --json | jq -r .sessionId)
agbrowse web-ai poll --vendor chatgpt --session "$SID"

One-shot query

agbrowse web-ai query --vendor grok --inline-only --prompt "Find risks in this text"

Capability registry

The declarative capability registry provides per-vendor capability lookup with fail-closed gating. Each capability entry declares its status, owner PRD, browser gate, fail-closed stage, family, and frontend observation preset. When a command targets an unsupported capability, the registry throws capability.unsupported before any browser mutation occurs.

Use agbrowse web-ai status --vendor chatgpt --json to inspect which capabilities are currently available for a vendor.

Annotated screenshots

Annotated (set-of-mark) screenshots capture the page and overlay bounding-box highlights on selected ref elements. The result is a typed AnnotatedScreenshotResult with image hash, dimensions, highlight count, and timestamp. This is used by the doctor diagnostics pipeline and visual grounding workflows.

# Annotated screenshots are used internally by doctor and diagnostics.
agbrowse web-ai doctor --vendor chatgpt --json

Interstitial detector

The unified interstitial detector classifies blocking pages before send or poll proceeds. It recognizes five kinds:

The classifier is pure (all page signals passed in) so it is fully unit-testable. When an interstitial is detected, the typed result includes the kind, evidence text, URL, and retry hint.

Diagnostics stages

Failure diagnostics now use a richer stage taxonomy. Each error envelope carries a typed stage field that classifies where in the pipeline the failure occurred (e.g., visual-fallback, composer-resolve, model-select, poll-timeout). Stage-typed envelopes make it possible to route failures to the correct recovery path without string matching on error messages.

Live status report

The typed standalone LiveStatusReport struct provides a vendor-agnostic health check. It evaluates whether the active tab is on the correct provider URL, whether the user is signed in, and whether the composer is visible, then returns a structured { vendor, status, runtimeEnabled, notes, sources } object.

agbrowse web-ai status --vendor chatgpt --json

Freshness gate

The docs-first freshness gate enforces that a capability cannot be trusted until official-doc retrieval evidence is recorded. A valid freshness record must include: retrieval date, vendor docs searched, official sources used, visible updated dates, feature changes since prior PRD, contradictions or unstable limits, UI-authoritative plan-limit assertion, implementation impact, and tests updated because of docs. Missing fields or unasserted UI authority cause the gate to reject.

Copy-markdown fallback

When inline text extraction fails, --allow-copy-markdown-fallback asks the runtime to locate and click the provider Copy/Markdown button as a lenient fallback. The 203.9 update adds a lenient button-matching heuristic and a real click path so the fallback works even when the button label or position shifts between provider UI revisions.

agbrowse web-ai query \
  --vendor chatgpt \
  --inline-only \
  --allow-copy-markdown-fallback \
  --prompt "Summarize this document."

Polling timeouts

Timeout resolution is explicit timeout → stored session deadline remainder → tier default → vendor fallback.

Unknown tiers fall back to ChatGPT 1200, Gemini 1200, or Grok 600 seconds. The user-reported roughly 40-minute ChatGPT Pro budget was not present in the inspected DOM and is not the agbrowse timeout.

ChatGPT Work

agbrowse web-ai work send --prompt "Analyze this repository" --power 4

The same Work task can be submitted via MCP with web_ai_work_send. Chat commands and web_ai_submit_prompt reject Work; they do not accept surface=work.

ChatGPT model contract

Legacy effort is remapped: light|standard → medium, extended → high (one stderr warning), heavy → xhigh. Legacy Pro effort resolves to flat Pro and emits one no-selection stderr warning. Extended labels belong to UI before 2026-07-10 only.

Generated images

ChatGPT generated-image output is beta and opt-in via --output-image:

agbrowse web-ai query \
  --vendor chatgpt \
  --inline-only \
  --output-image ./out.png \
  --prompt "Create an image of a small robot holding a banana."

When ChatGPT returns multiple images, agbrowse writes sibling files as out.png, out-2.png, out-3.png. Image output is fail-closed: if no generated image can be detected or saved, the command returns provider.image-output instead of silently succeeding.

Batch follow-ups

ChatGPT batch follow-ups send multiple sequential prompts in one command:

agbrowse web-ai query \
  --vendor chatgpt \
  --inline-only \
  --prompt "Analyze this design." \
  --follow-up "Summarize the risks." \
  --follow-up "List the next three actions."

For a later follow-up in the same saved conversation, use query --session <id> --prompt <text>. --follow-up is ChatGPT-only and cannot be combined with --research deep.

Deep Research (experimental)

--research deep activates ChatGPT Deep Research mode as an experimental beta:

agbrowse web-ai query \
  --vendor chatgpt \
  --inline-only \
  --research deep \
  --timeout 1800 \
  --prompt "Research the current official status and cite sources."

Deep Research saves a report artifact when available, records researchMode: "deep" in the session, skips auto archive, and auto-confirms the post-submit Deep Research plan card. Account blocks or missing provider UI surfaces are reported explicitly; do not treat this as a ready cross-provider capability.

ChatGPT Project Sources

Project Sources are append-only and require an explicit ChatGPT project URL:

agbrowse web-ai project-sources list \
  --chatgpt-url https://chatgpt.com/g/project_123 --json

agbrowse web-ai project-sources add \
  --chatgpt-url https://chatgpt.com/g/project_123 \
  --file ./docs/context.md \
  --dry-run summary

--dry-run validates the project URL and local files without browser mutation. Live add waits for upload evidence before reporting uploaded: true. Delete, replace, and clear operations are intentionally unsupported.

Provider boundaries

ChatGPT, Gemini, and Grok each have separate model aliases and UI contracts. Check agbrowse web-ai --help before scripting.