Commands API

Autopilot Commands

`act`

Executes a complex instruction on the current page. Instruction could be relatively general or extremely specific. The AI agent will autonomously plan and perform a sequence of commands (like clicking, typing, scrolling or waiting) until the goal is achieved or the maximum number of steps is reached. Optionally, it can also extract data once the instruction is completed.

Payload:

json
{
  "commands": [
    {
      "command": "act",
      "params": {
        "instruction": "Open a page with the best rated laptop on the page",
        "max_steps": 5,
        "model_size": "medium"
      }
    }
  ]
}

Payload with enabled data extraction mode (useful when you want to extract data scattered across multiple pages / sites or you want let agent find the data by itself - agent will collect data from different places and assemble unified output JSON):

json
{
  "command": "act",
  "params": {
    "instruction": "Open the first 3 laptop listings for the following query and extract data (from each item page): 'cool laptop'",
    "max_steps": 15,
    "model_size": "medium",
    "extract_data": true,
    "data_instruction": "Extract basic info of the laptop",
    "json_schema": [
      {
        "title": "<short laptop listing title>",
        "price": "<listed price with currency without any special symbols>",
        "price_currency": "<ISO currency code of the price>",
        "specs": "<short summary of main specs such as CPU, RAM, storage, GPU>",
        "summary": "<brief description of the laptop and its standout features>",
        "product_url": "<direct valid URL to the laptop product page>"
      }
    ]
  }
}

Parameters:
- instruction (string, required): The instruction describing the goal you want the AI to achieve.
- max_steps (integer, required): The maximum number of interaction steps the agent is allowed to take to complete the instruction. Minimum 1, maximum 200.
- model_size (string, optional): The size of the AI model to use for reasoning. Larger models are more capable but cost more. Allowed values: "small", "medium", "large". Default "medium".
- extract_data (boolean, optional): If true, the agent will be able to extract structured data on the fly. Default false. Consider using this option as a Deep research when agent navigates to multiple pages looking for the data you want. If you need to extract data from a static page, we suggest you to use extract_data command.
- data_instruction (string, optional): The instruction describing the exact data you want to extract (used when extract_data is true).
- json_schema (object, array or string, optional): A JSON schema defining the precise structure of the data you want returned. Useful for ensuring the output matches your programmatic expectations (used when extract_data is true).
- viewport_min (integer, optional): Restrict the agent's initial attention to start from a specific 0-based viewport index.
- viewport_max (integer, optional): Restrict the agent's attention to end at a specific 0-based viewport index.

Response Example (without data extraction):

json
{
    "command": "act",
    "offloaded_data_url": null,
    "metadata": {
        "ocr": {
            "images_processed": 5,
            "images_failed": 0,
            "chars_extracted": 6000
        },
        "tokens": {
            "completion": 75,
            "total": 2500,
            "prompt": 2425
        },
        "actions_executed": 3,
        "pages_extracted": 0,
        "steps_executed": 4
    },
    "step": 0,
    "success": true,
    "data": {
        "objective_accomplished": true,
        "action_steps": [
            {
                "stop": 0,
                "name": "type_at",
                "comment": "Typing query into the search bar",
                "url": "https://example-store.com/",
                "element": {
                    "bbox": [100, 150, 130, 450],
                    "point": [115, 300]
                },
                "instruction": "Type 'laptop' into the search input",
                "text": "laptop",
                "hint": "Typing search query",
                "before": "https://example-cloud-storage.com/screenshots/before_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_1.png"
            },
            {
                "stop": 0,
                "name": "press",
                "comment": "Pressing enter to submit the search",
                "url": "https://example-store.com/",
                "instruction": "Press Enter",
                "keys": ["enter"],
                "hint": "Submitting search",
                "before": "https://example-cloud-storage.com/screenshots/after_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_2.png"
            },
            {
                "stop": 1,
                "name": "click",
                "comment": "Clicking on the first laptop in the search results",
                "url": "https://example-store.com/search?q=laptop",
                "element": {
                    "bbox": [300, 200, 400, 500],
                    "point": [350, 350]
                },
                "instruction": "Click on the first product listing",
                "mouse_button": "left",
                "clicks": 1,
                "label": "SuperBook Pro 15",
                "hint": "Clicking first result",
                "before": "https://example-cloud-storage.com/screenshots/after_2.png",
                "after": "https://example-cloud-storage.com/screenshots/after_3.png"
            }
        ],
        "stop_type": "objective_accomplished"
    }
}

Response Example (with data extraction enabled):

json
{
    "command": "act",
    "offloaded_data_url": null,
    "metadata": {
        "ocr": {
            "images_processed": 2,
            "images_failed": 0,
            "chars_extracted": 12000
        },
        "tokens": {
            "completion": 125,
            "total": 4500,
            "prompt": 4375
        },
        "actions_executed": 3,
        "pages_extracted": 1,
        "steps_executed": 4
    },
    "step": 0,
    "success": true,
    "data": {
        "objective_accomplished": true,
        "action_steps": [
            {
                "stop": 0,
                "name": "type_at",
                "comment": "Typing query into the search bar",
                "url": "https://example-store.com/",
                "element": {
                    "bbox": [100, 150, 130, 450],
                    "point": [115, 300]
                },
                "instruction": "Type 'laptop' into the search input",
                "text": "laptop",
                "hint": "Typing search query",
                "before": "https://example-cloud-storage.com/screenshots/before_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_1.png"
            },
            {
                "stop": 0,
                "name": "press",
                "comment": "Pressing enter to submit the search",
                "url": "https://example-store.com/",
                "instruction": "Press Enter",
                "keys": ["enter"],
                "hint": "Submitting search",
                "before": "https://example-cloud-storage.com/screenshots/after_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_2.png"
            },
            {
                "stop": 0,
                "name": "click",
                "comment": "Clicking on the first laptop in the search results",
                "url": "https://example-store.com/search?q=laptop",
                "element": {
                    "bbox": [300, 200, 400, 500],
                    "point": [350, 350]
                },
                "instruction": "Click on the first product listing",
                "mouse_button": "left",
                "clicks": 1,
                "label": "SuperBook Pro 15",
                "hint": "Clicking first result",
                "before": "https://example-cloud-storage.com/screenshots/after_2.png",
                "after": "https://example-cloud-storage.com/screenshots/after_3.png"
            },
            {
                "name": "extract_data",
                "after": "https://example-cloud-storage.com/screenshots/after_4.png",
                "comment": "Extracting laptop info from the current page.",
                "stop": 0,
                "before": "https://example-cloud-storage.com/screenshots/after_3.png",
                "hint": "",
                "url": "https://example-store.com/product/superbook-pro-15"
            },
            {
                "name": "stop",
                "after": "https://example-cloud-storage.com/screenshots/after_5.png",
                "comment": "Laptop info extracted from the dedicated page.",
                "stop": 1,
                "before": "https://example-cloud-storage.com/screenshots/after_4.png",
                "hint": "Stopping execution. Comment: Laptop info extracted from the dedicated page.",
                "url": "https://example-store.com/product/superbook-pro-15"
            }
        ],
        "extracted_json": [
            {
                "title": "SuperBook Pro 15",
                "price": "1299.00",
                "price_currency": "USD",
                "specs": "Intel Core i7, 16GB RAM, 512GB SSD",
                "summary": "A powerful laptop for professionals.",
                "product_url": "https://example-store.com/product/superbook-pro-15"
            }
        ],
        "screenshots": [
            "https://example-cloud-storage.com/screenshots/example_1.png",
            "https://example-cloud-storage.com/screenshots/example_2.png"
        ],
        "text": "Raw text extracted from the page...",
        "stop_type": "objective_accomplished"
    }
}

(Note: The exact contents of data depend on whether extract_data is true and what the agent encountered during execution).

`extract`

Extracts structured data directly from the current static page layout using AI. You define what you want to extract (via instructions and a JSON schema) and the AI analyzes the DOM and visual data on the page to extract JSON for you. It is highly resilient to website structural changes. When both data_instruction and json_schema are not provided, AI will design schema layout on its own.

By default, the AI uses a combination of HTML parsing and visual screenshots to gather data. You can tune extraction parameters to lower your AI resource costs. For instance, by disabling vision or limiting how much of the page the AI analyzes.

Parameters:
- data_instruction (string, optional): Plain-text instruction describing exactly what you want the AI to find and extract.
- json_schema (object, array or string, optional): A schema indicating the expected JSON structure. Schema is not strictly typed - might be represented as { field_name: description, ... }. Wrap object into an array [{...}] if you expect to extract a list of items from the page. Example: [{"title": "<title of the product>", "price": "<price of the prodcut without any special symbols (float)>"}].
- model_size (string, optional): AI model size. Larger models are smarter but more expensive. Allowed: "small", "medium", "large". Default "small".
- viewport_min (integer, optional): Start index (0-based) for the viewports the AI should analyze.
- viewport_max (integer, optional): Maximum viewport index the AI should look at. Useful for making extraction cheaper by ignoring irrelevant footer content on long pages.
- vision (boolean, optional): If false, the AI will skip stage with recognizing page text from screenshots using OCR and rely purely on screenshots. Turning vision off reduces prompt (input) tokens usage approximately by 20-30%. Default true.
- feed_urls (boolean, optional): If true, the AI receives exact DOM hyperlinks. Set to false to save token costs if you don't need to extract URLs. Default true.
- disable_ocr (boolean, optional): Disables Optical Character Recognition during visual analysis. Default false.
- paginate (boolean, optional): Set to true to let the AI automatically find the "Next" button, navigate and merge data across multiple pages. Default false.
- pages_to_paginate (integer, optional): Max pages to navigate through. Minimum 1, max 100. Default 1.
- pagination_hint (string, optional): Custom text hints for the AI to find the pagination button (useful when navigation element on the page is not obvious / hard to find).

Example 1: Plain Extraction

Standard extraction from a static page using data_instruction and json_schema.

Payload:

json
{
  "commands": [
    {
      "command": "extract",
      "params": {
        "data_instruction": "Extract all product names and their prices.",
        "json_schema": [
          {
            "name": "<product name>",
            "price": "<price as string>"
          }
        ],
        "model_size": "medium"
      }
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "screenshots": [
      "https://cloud-storage-url.com/screenshot1.jpg"
    ],
    "extracted_json": [
      {
        "name": "SuperBook Pro 15",
        "price": "$1299.00"
      },
      {
        "name": "MegaPad Air",
        "price": "$599.00"
      }
    ],
    "text": "Raw text extracted from the page..."
  }
}

Example 2: Cheap Extraction (Custom Params)

Optimize for cost by disabling visual analysis (AI will take raw text recognized by OCR as inputs only), ignoring page links and only looking at the top few viewports.

Payload:

json
{
  "commands": [
    {
      "command": "extract",
      "params": {
        "data_instruction": "Extract the main article title and author.",
        "json_schema": {
          "title": "<article title>",
          "author": "<author name>"
        },
        "model_size": "small",
        "vision": false,
        "viewport_max": 3,
        "feed_urls": false
      }
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "screenshots": [
      "https://cloud-storage-url.com/screenshot_page1.jpg",
      "https://cloud-storage-url.com/screenshot_page2.jpg",
      "https://cloud-storage-url.com/screenshot_page3.jpg"
    ],
    "extracted_json": {
      "title": "The Future of AI",
      "author": "John Smith"
    },
    "text": "Raw text extracted from the page..."
  }
}

Example 3: Extraction with Pagination

Automatically navigate through a list and compile the data into a single array.

Payload:

json
{
  "commands": [
    {
      "command": "extract",
      "params": {
        "data_instruction": "Extract job postings from the list.",
        "json_schema": [
          {
            "job_title": "<title>",
            "company": "<company name>"
          }
        ],
        "model_size": "medium",
        "paginate": true,
        "pages_to_paginate": 3
      }
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "screenshots": [
      "https://cloud-storage-url.com/screenshot_page1.jpg",
      "https://cloud-storage-url.com/screenshot_page2.jpg",
      "https://cloud-storage-url.com/screenshot_page3.jpg"
    ],
    "extracted_json": [
      { "job_title": "Software Engineer", "company": "Tech Corp" },
      { "job_title": "Data Analyst", "company": "Data Inc" },
      { "job_title": "Product Manager", "company": "Innovate LLC" },
      { "job_title": "DevOps Engineer", "company": "Cloud Co" }
    ],
    "text": "Raw text extracted from the page..."
  }
}

Manual Commands

`open_url`

Navigates the browser to the specified URL and automatically waits for the page to finish loading.

Payload:

json
{
  "commands": [
    {
      "command": "open_url",
      "params": {
        "url": "https://example.com"
      }
    }
  ]
}

Parameters:
- url (string, required): Destination URL to open. Accepts URLs, IPv4 addresses, IPv6 addresses and about:blank page.

Response:

json
{
  "success": true,
  "data": {
    "url": "https://example.com"
  }
}

`get_url`

Returns the current URL natively using the browser's navigation history.

Payload:

json
{
  "commands": [
    {
      "command": "get_url"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "url": "https://example.com/current-page"
  }
}

`refresh`

Reloads the current web page and waits for it to load completely. Returns the URL of the page post-refresh.

Payload:

json
{
  "commands": [
    {
      "command": "refresh"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "url": "https://example.com/current-page"
  }
}

`back`

Navigates back to the previous web page in the browser's history natively and waits for it to load. Returns the updated URL.

Payload:

json
{
  "commands": [
    {
      "command": "back"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "url": "https://example.com/previous-page"
  }
}

`forward`

Navigates forward to the next web page in the browser's history natively and waits for it to load. Returns the updated URL.

Payload:

json
{
  "commands": [
    {
      "command": "forward"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "url": "https://example.com/next-page"
  }
}

`sleep`

Pauses the browser execution for a specific amount of time. Includes a slight random fluctuation by default to make sleeping less predictable.

Payload:

json
{
  "commands": [
    {
      "command": "sleep",
      "params": {
        "wait_time": 2.5
      }
    }
  ]
}

Parameters:
- wait_time (float, required): Minimum time to sleep in seconds.
- fluctuation (float, optional): Maximum random deviation added to the base wait_time. Default 0.05.
Response: {"success": true}

`get_pdf`

Exports the current page to a PDF document and uploads it to the cloud.

Payload:

json
{
  "commands": [
    {
      "command": "get_pdf",
      "params": {
        "landscape": false,
        "print_background": true
      }
    }
  ]
}

Parameters:
- landscape (boolean, optional): Paper orientation. Defaults to false.
- print_background (boolean, optional): Print background graphics. Defaults to true.
- scale (float, optional): Scale of the webpage rendering. Defaults to 1.0.
- paper_width (float, optional): Paper width in inches. Defaults to 8.5.
- paper_height (float, optional): Paper height in inches. Defaults to 11.0.
- margin_top (float, optional): Top margin in inches. Defaults to 0.4.
- margin_bottom (float, optional): Bottom margin in inches. Defaults to 0.4.
- margin_left (float, optional): Left margin in inches. Defaults to 0.4.
- margin_right (float, optional): Right margin in inches. Defaults to 0.4.

Response:

json
{
  "success": true,
  "data": {
    "pdf_url": "https://cloud-storage-url.com/document.pdf"
  }
}

`get_screenshot`

Captures a screenshot of the current page. The response behavior depends on the requested screenshot mode. Captured images are automatically uploaded to the cloud.

Payload:

json
{
  "commands": [
    {
      "command": "get_screenshot",
      "params": {
        "mode": "viewport"
      }
    }
  ]
}

Parameters:
- mode (string, optional): Method that'll be used to capture a screenshot or screenshots.
  - "viewport" (Default): Captures only the currently visible part of the page (viewport).
  - "full_page": Captures full-length image of the page.
  - "viewports": Captures viewport images of the page.
- viewport_min (integer, optional): If mode is "full_page" or "viewports", sets the starting index (0-based) for the captured viewports. Useful for skipping the top of the page.
- viewport_max (integer, optional): Sets the maximum amount of viewports to capture. Max limit is 50.

Response (viewport / full_page modes):

json
{
  "success": true,
  "data": {
    "image_url": "https://cloud-storage-url.com/image.jpg"
  }
}

Response (viewports mode):

json
{
  "success": true,
  "data": {
    "image_urls": [
      "https://cloud-storage-url.com/chunk_1.jpg",
      "https://cloud-storage-url.com/chunk_2.jpg"
    ]
  }
}

`get_html`

Retrieves the entire outer HTML of the current page. If the HTML response is extremely large and exceeds internal limits, it is offloaded to the cloud and you'll receive an offloaded_data_url instead of inline JSON data in the response payload.

Payload:

json
{
  "commands": [
    {
      "command": "get_html"
    }
  ]
}

Response (Standard):

json
{
  "success": true,
  "data": {
    "html": "<html>...</html>"
  },
  "offloaded_data_url": null
}

Response (Offloaded payload):

json
{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/your-large-html-file"
}

`get_snapshot`

Captures detailed representation of the DOM tree (similar to Chrome DevTools snapshot), mapped with exact x/y coordinates, dimensions, visibilities, text contents and specific computed styles of each element. Snapshot payloads are very large so they are always offloaded to the cloud.

Payload:

json
{
  "commands": [
    {
      "command": "get_snapshot",
      "params": {
        "computed_styles": ["color", "font-weight"]
      }
    }
  ]
}

Parameters:
- computed_styles (array of strings, optional): A list of CSS property names to explicitly extract for every node (e.g., ["background-color", "display"]).

Response:

json
{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/snapshot.json"
}

`locate`

Locates an element on the page based on the specified strategy. Automatically penetrates iframes and shadow DOMs. Assigns an element_id to the element for easier future interaction.

Payload:

json
{
  "commands": [
    {
      "command": "locate",
      "params": {
        "strategy": "css",
        "value": "button.submit"
      }
    }
  ]
}

Parameters:
- strategy (string, required): Strategy to locate the element (css, xpath, role, text, label, placeholder, alt_text, title, test_id).
- value (string, required): The core search string. For css/xpath, this is the selector. For built-in locators like role, this is the ARIA role. For text, it is the inner text.
- name (string, optional): Used when strategy="role". Acts as an accessible name filter.
- exact (boolean, optional): Used for text or role strategies. Requires an exact whole-string match. Defaults to false.
- index (integer, optional): Indicates which matching element to return if there are multiple matches. Defaults to 0.
- state (string, optional): Wait for the element to reach a specific DOM state (visible, attached, enabled, hidden). Defaults to visible.
- timeout_ms (integer, optional): Maximum time in milliseconds to wait for the element state. Defaults to 15000.
- parent_id (string, optional): A previously located element_id to scope this search inside of.

`inject_cookies`

Injects the provided cookies into the running session. Automatic page refresh will apply them.

Parameters:
- cookies (array of objects, required): An array of cookie objects to inject into the new browser session. Each object must contain name, value and domain.

Payload:

json
{
  "commands": [
    {
      "command": "inject_cookies",
      "params": {
        "cookies": [
          {
            "name": "session_id",
            "value": "123456789",
            "domain": ".example.com",
            "path": "/"
          }
        ]
      }
    }
  ]
}

Response:
```
json
{
  "success": true
}
```

`dump_cookies`

Retrieves a list of all browser cookies for the current URL.

Payload:

json
{
  "commands": [
    {
      "command": "dump_cookies"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "cookies": [
      {
        "name": "session_id",
        "value": "xyz123",
        "domain": "example.com",
        "path": "/",
        "expires": 1700000000,
        "size": 16,
        "httpOnly": true,
        "secure": true,
        "session": false
      }
    ]
  },
  "offloaded_data_url": null
}

`dump_console_logs`

Retrieves a list of all browser console logs for the current session.

Payload:

json
{
  "commands": [
    {
      "command": "dump_console_logs"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "console_logs": [
      {
        "type": "log",
        "timestamp": 123456789.0,
        "message": "Hello world"
      }
    ]
  },
  "offloaded_data_url": null
}

`dump_local_storage`

Retrieves a dictionary representing the local storage data for the currently opened origin.

Payload:

json
{
  "commands": [
    {
      "command": "dump_local_storage"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "local_storage": {
      "theme": "dark",
      "user_preferences": "{\"notifications\":true}"
    }
  },
  "offloaded_data_url": null
}

`dump_har_logs`

Retrieves the HAR (HTTP Archive) logs for the current session. These logs capture detailed information about all network requests and responses routed through the proxy, including headers, timings and statuses.

Since HAR logs can become exceptionally large, if they exceed internal payload limits, they will be offloaded to the cloud and you will receive an offloaded_data_url pointing to the JSON file instead of inline data.

Payload:

json
{
  "commands": [
    {
      "command": "dump_har_logs"
    }
  ]
}

Response (Standard Sample):

json
{
  "success": true,
  "data": {
    "har_logs": {
      "log": {
        "version": "1.2",
        "creator": {
          "name": "BroTrafficRouter",
          "version": "1.0"
        },
        "entries": [
          {
            "startedDateTime": "2023-10-15T12:00:00.000Z",
            "time": 150,
            "proxied": true,
            "request": {
              "method": "GET",
              "url": "https://example.com/api/data",
              "httpVersion": "HTTP/2.0",
              "cookies": [],
              "headers": [
                {
                  "name": "Accept",
                  "value": "application/json"
                }
              ],
              "queryString": [],
              "headersSize": 150,
              "bodySize": 0
            },
            "response": {
              "status": 200,
              "statusText": "OK",
              "httpVersion": "HTTP/2.0",
              "cookies": [],
              "headers": [
                {
                  "name": "Content-Type",
                  "value": "application/json"
                }
              ],
              "content": {
                "size": 1024,
                "compression": 0,
                "mimeType": "application/json",
                "text": "{\"key\": \"value\"}"
              },
              "redirectURL": "",
              "headersSize": 160,
              "bodySize": 1024
            },
            "cache": {},
            "timings": {
              "send": 2,
              "receive": 48,
              "wait": 100,
              "connect": -1,
              "ssl": -1
            },
            "serverIPAddress": "93.185.218.44"
          }
        ]
      }
    }
  },
  "offloaded_data_url": null
}

Response (Offloaded):

json
{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/your-large-har-file.json"
}

`parse_text`

Crawls the current page's DOM (including shadow DOM and iframes) to discover and extract all visible text elements. If the extracted object is too large, the data will be offloaded to the cloud.

Payload:

json
{
  "command": "parse_text",
  "params": {
    "format": "json"
  }
}

Parameters:
- format (string, optional): The output format for the parsed text. Options are "plain", "md" or "json". Defaults to "plain".
- computed_styles (array of strings, optional): A list of CSS property names to extract for every node.

Response (Standard for format="json"):

json
{
  "success": true,
  "data": [
    {
      "id": 0,
      "tag": "p",
      "text": "Hello World",
      "viewport": 0,
      "coords": {
        "x": 100,
        "y": 200,
        "width": 300,
        "height": 20
      }
    }
  ],
  "offloaded_data_url": null
}

Response (Standard for format="plain" or format="md"):

json
{
  "success": true,
  "data": "Hello World\n...",
  "offloaded_data_url": null
}

`parse_element_text`

Crawls the specified element's DOM (including shadow DOM and iframes) to discover and extract all visible text elements inside it. Either element_id or locator must be provided. If the extracted object is too large, the data will be offloaded to the cloud.

Parameters:
- element_id (string, optional): Use an existing element_id from a previous locate call. (Exactly one of element_id or locator is required).
- locator (object, optional): Pass a full locator object containing strategy, value, etc. (Exactly one of element_id or locator is required).

Payload:

json
{
  "command": "parse_element_text",
  "params": {
    "element_id": "...",
    "format": "json"
  }
}

Parameters:
- element_id (string) OR locator (object) (Exactly one is required): Use an existing element_id from a previous locate call or pass a full locator object containing strategy, value, etc.
- format (string, optional): The output format for the parsed text. Options are "plain", "md" or "json". Defaults to "plain".
- computed_styles (array of strings, optional): A list of CSS property names to extract for every node.

Response (Standard for format="json"):

json
{
  "success": true,
  "data": [
    {
      "id": 0,
      "tag": "span",
      "text": "Extracted Element Text",
      "viewport": 0,
      "coords": {
        "x": 100,
        "y": 200,
        "width": 150,
        "height": 20
      }
    }
  ],
  "offloaded_data_url": null
}

Response (Standard for format="plain" or format="md"):

json
{
  "success": true,
  "data": "Extracted Element Text\n...",
  "offloaded_data_url": null
}

`parse_urls`

Crawls the current page's DOM to discover and extract all interactive links, routing objects and standard anchor tags. If the extracted object is too large, the data will be offloaded to the cloud.

Payload:
```
json
{
  "command": "parse_urls"
}
```
Parameters:
- computed_styles (array of strings, optional): A list of CSS property names to extract for every link node.

Response (Standard):

json
{
  "success": true,
  "data": {
    "links": [
      {
        "is_base64": false,
        "url": "https://example.com/widget/sample-script.min.js",
        "id": 0,
        "count": 1,
        "tag": "script",
        "viewport": 0,
        "text": ""
      }
    ],
    "images": [
      {
        "is_base64": false,
        "url": "https://www.example.com/content/uploads/image.png",
        "id": 383,
        "count": 1,
        "tag": "img",
        "viewport": 0,
        "text": ""
      }
    ]
  },
  "offloaded_data_url": null
}

`run_js`

Executes raw JavaScript code in the context of the active page. Use this for highly custom operations or retrieving local storage values.

Payload:

json
{
  "commands": [
    {
      "command": "run_js",
      "params": {
        "js_code": "window.location.href",
        "out_type": "str"
      }
    }
  ]
}

Parameters:
- js_code (string, required): The JavaScript code to execute. Must be written as a block that resolves or returns a value (e.g., return window.location.href;).
- out_type (string, optional): Validates and casts the JavaScript result into a specific type.
  - Allowed values: "list", "dict", "bool", "float", "int", "str".

Response:

json
{
  "success": true,
  "data": {
    "result": "https://news.ycombinator.com/"
  },
  "offloaded_data_url": null
}

`copy`

Copies the currently highlighted text to system clipboard using a standard OS keyboard shortcut (Ctrl+C). Returns the copied text. Like get_html, if the copied text is extremely large, it will be automatically offloaded to the cloud.

Payload:

json
{
  "commands": [
    {
      "command": "copy"
    }
  ]
}

Response:

json
{
  "success": true,
  "data": {
    "copied_text": "Copied text..."
  },
  "offloaded_data_url": null
}

`paste`

Pastes the contents currently stored in the system clipboard into the active page element using a standard OS keyboard shortcut (Ctrl+V).

Payload:

json
{
  "commands": [
    {
      "command": "paste"
    }
  ]
}

Response:
```
json
{
  "success": true
}
```

HTML Locator targets

`click`

Clicks on a targeted element. Identifies the target via element_id or locator.

Payload:

json
{
  "commands": [
    {
      "command": "click",
      "params": {
        "locator": {
          "strategy": "css",
          "value": "button.submit"
        }
      }
    }
  ]
}

Parameters:
- element_id (string) OR locator (object) (Exactly one is required): Use an existing element_id from a previous locate call or pass a full locator object containing strategy, value, etc.
- mouse_button (string, optional): "left", "right", "middle". Default "left".
- clicks (integer, optional): Number of clicks. Default 1.
Response:
```
json
{
  "success": true
}
```

`type`

Types text into a targeted element. Identifies the target via element_id or locator.

Payload:

json
{
  "commands": [
    {
      "command": "type",
      "params": {
        "locator": {
          "strategy": "css",
          "value": "input[name='search']"
        },
        "text": "Hello World"
      }
    }
  ]
}

Parameters:
- element_id (string) OR locator (object) (Exactly one is required).
- text (string, required): The string to type.
- clear_first (boolean, optional): Clear the input before typing. Default true.
Response:
```
json
{
  "success": true
}
```

`select`

Selects an option from a <select> dropdown. Identifies the target via element_id or locator.

Payload:

json
{
  "commands": [
    {
      "command": "select",
      "params": {
        "element_id": "id_x123",
        "select_value": "option1"
      }
    }
  ]
}

Parameters:
- element_id (string) OR locator (object) (Exactly one is required).
- select_value (string, optional): Matches the <option value="..."> attribute.
- label (string, optional): Matches the visible text of the option. (Must provide either select_value or label).
Response:
```
json
{
  "success": true
}
```

Mouse and viewport controls

`hover_at`

Moves the mouse cursor to the specified coordinates.

Payload:

json
{
  "commands": [
    {
      "command": "hover_at",
      "params": {
        "x": 250.0,
        "y": 500.0
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate. Minimum 0, maximum is the window height (1024).
- scope (string, optional): The reference frame for the coordinates.
  - "viewport" (Default): Coordinates are relative to the webpage viewport.
  - "browser": Coordinates are relative to the browser window.
  - "os": Coordinates are relative to the operating system's screen.
Response:
```
json
{
  "success": true
}
```

`click_at`

Moves the cursor to the target coordinates and clicks the specified mouse button.

Payload:

json
{
  "commands": [
    {
      "command": "click_at",
      "params": {
        "x": 250.0,
        "y": 500.0,
        "clicks": 2
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate. Minimum 0, maximum is the window height (1024).
- mouse_button (string, optional): The mouse button to press. Allowed values: "left", "right", "middle". Default "left".
- clicks (integer, optional): The number of times to click on the mouse_button (e.g., 2 for a double-click). Minimum 1. Default 1.
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

`drag`

Performs a drag-and-drop interaction by moving the mouse from a start coordinate to a destination coordinate while holding down a specified mouse button.

Payload:

json
{
  "commands": [
    {
      "command": "drag",
      "params": {
        "start_x": 100.0,
        "start_y": 100.0,
        "dest_x": 300.0,
        "dest_y": 300.0
      }
    }
  ]
}

Parameters:
- start_x (float, required): The starting X coordinate. Minimum 0, maximum is the window width (1280).
- start_y (float, required): The starting Y coordinate. Minimum 0, maximum is the window height (1024).
- dest_x (float, required): The destination X coordinate. Minimum 0, maximum is the window width (1280).
- dest_y (float, required): The destination Y coordinate. Minimum 0, maximum is the window height (1024).
- mouse_button (string, optional): The mouse button to hold down while dragging. Allowed values: "left", "right", "middle". Default "left".
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

`scroll`

Scrolls the mouse wheel by a specified amount at the given coordinates.

Payload:

json
{
  "commands": [
    {
      "command": "scroll",
      "params": {
        "clicks": -5,
        "x": 400,
        "y": 500
      }
    }
  ]
}

Parameters:
- clicks (integer, required): The amount of scroll wheel clicks to apply (negative - scroll down, positive - scroll up).
- x (float, required): The X coordinate to place the cursor before scrolling. Minimum 0, maximum is the window width (1280).
- y (float, required): The Y coordinate to place the cursor before scrolling. Minimum 0, maximum is the window height (1024).
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
- axis (string, optional): The direction to scroll. Allowed values: "x" (horizontal), "y" (vertical). Default "y".
Response:
```
json
{
  "success": true
}
```

`scroll_to_viewport`

Automatically scrolls the page until a specific viewport chunk comes into view. This is useful when paginating through large pages so you won't need to call scroll command multiple times.

Payload:

json
{
  "commands": [
    {
      "command": "scroll_to_viewport",
      "params": {
        "viewport_idx": 5
      }
    }
  ]
}

Parameters:
- viewport_idx (integer, required): The target 0-based viewport index to scroll to. Minimum 0.
Response:
```
json
{
  "success": true
}
```

`click_and_hold`

Moves the cursor to the target coordinates, presses down the specified mouse button, holds it for a specific duration and then releases it.

Payload:

json
{
  "commands": [
    {
      "command": "click_and_hold",
      "params": {
        "x": 100.0,
        "y": 200.0,
        "hold_duration": 2.5
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate. Minimum 0, maximum is the window height (1024).
- hold_duration (float, required): The duration in seconds to hold the mouse button. Minimum 0.0, maximum 60.0.
- mouse_button (string, optional): The mouse button to hold down. Allowed values: "left", "right", "middle". Default "left".
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

Keyboard controls

`type_at`

Clicks at the specified coordinates and inputs the given text into the focused element.

Payload:

json
{
  "commands": [
    {
      "command": "type_at",
      "params": {
        "x": 200.0,
        "y": 150.0,
        "text": "Hello world"
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate to click before typing. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate to click before typing. Minimum 0, maximum is the window height (1024).
- text (string, required): The text to type.
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

`press`

Presses a single keyboard key or a combination of keys (hotkey).

Payload:

json
{
  "commands": [
    {
      "command": "press",
      "params": {
        "keys": ["ctrl", "a"]
      }
    }
  ]
}

Parameters:
- keys (string or array of strings, required): A single key (e.g., "enter") or an array of keys to trigger a shortcut (e.g., ["ctrl", "c"]).
Response:
```
json
{
  "success": true
}
```

`hold_a_key_and_click`

Holds down specified keys (like "shift" or "control"), clicks at the target coordinates and then releases the keys. Useful for selecting multiple items or opening links in a new tab.

Payload:

json
{
  "commands": [
    {
      "command": "hold_a_key_and_click",
      "params": {
        "x": 300.0,
        "y": 400.0,
        "keys": "shift"
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate. Minimum 0, maximum is the window height (1024).
- keys (string or array of strings, required): The key or sequence of keys to hold down during the click.
- mouse_button (string, optional): The mouse button to press. Allowed values: "left", "right", "middle". Default "left".
- clicks (integer, optional): The number of times to click. Minimum 1. Default 1.
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

`hold_a_key_and_drag`

Holds down specified keys, drags the mouse from a start coordinate to a destination coordinate and then releases the keys.

Payload:

json
{
  "commands": [
    {
      "command": "hold_a_key_and_drag",
      "params": {
        "start_x": 100.0,
        "start_y": 100.0,
        "dest_x": 200.0,
        "dest_y": 200.0,
        "keys": "shift"
      }
    }
  ]
}

Parameters:
- start_x (float, required): The starting X coordinate. Minimum 0, maximum is the window width (1280).
- start_y (float, required): The starting Y coordinate. Minimum 0, maximum is the window height (1024).
- dest_x (float, required): The destination X coordinate. Minimum 0, maximum is the window width (1280).
- dest_y (float, required): The destination Y coordinate. Minimum 0, maximum is the window height (1024).
- keys (string or array of strings, required): The key or sequence of keys to hold down during the drag.
- mouse_button (string, optional): The mouse button to hold down while dragging. Allowed values: "left", "right", "middle". Default "left".
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

`clear_textbox`

Clicks on a textbox at the specified coordinates, selects all existing text and deletes it.

Payload:

json
{
  "commands": [
    {
      "command": "clear_textbox",
      "params": {
        "x": 150.0,
        "y": 250.0
      }
    }
  ]
}

Parameters:
- x (float, required): The target X coordinate. Minimum 0, maximum is the window width (1280).
- y (float, required): The target Y coordinate. Minimum 0, maximum is the window height (1024).
- scope (string, optional): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".
Response:
```
json
{
  "success": true
}
```

Commands API

Autopilot Commands

act

extract

Example 1: Plain Extraction

Example 2: Cheap Extraction (Custom Params)

Example 3: Extraction with Pagination

Manual Commands

Navigation and page state

open_url

get_url

refresh

back

forward

sleep

get_pdf

get_screenshot

get_html

get_snapshot

locate

inject_cookies

dump_cookies

dump_console_logs

dump_local_storage

dump_har_logs

parse_text

parse_element_text

parse_urls

run_js

copy

paste

HTML Locator targets

click

type

select

Mouse and viewport controls

hover_at

click_at

drag

scroll

scroll_to_viewport

click_and_hold

Keyboard controls

type_at

press

hold_a_key_and_click

hold_a_key_and_drag

clear_textbox

`act`

`extract`

`open_url`

`get_url`

`refresh`

`back`

`forward`

`sleep`

`get_pdf`

`get_screenshot`

`get_html`

`get_snapshot`

`locate`

`inject_cookies`

`dump_cookies`

`dump_console_logs`

`dump_local_storage`

`dump_har_logs`

`parse_text`

`parse_element_text`

`parse_urls`

`run_js`

`copy`

`paste`

`click`

`type`

`select`

`hover_at`

`click_at`

`drag`

`scroll`

`scroll_to_viewport`

`click_and_hold`

`type_at`

`press`

`hold_a_key_and_click`

`hold_a_key_and_drag`

`clear_textbox`