Commands API

Autopilot Commands

act

Executes a complex instruction on the current page. Instruction could be relatively general or extremely specific. The AI agent will autonomously plan and perform a sequence of commands (like clicking, typing, scrolling or waiting) until the goal is achieved or the maximum number of steps is reached. Optionally, it can also extract data once the instruction is completed.

Parameters:

• instruction (str): The instruction describing the goal you want the AI to achieve.

• max_steps (int): The maximum number of interaction steps the agent is allowed to take to complete the instruction. Minimum 1, maximum 200.

• model_size (str): The size of the AI model to use for reasoning. Larger models are more capable but cost more. Allowed values: "small", "medium", "large". Default "medium".

• extract_data (bool): If true, the agent will be able to extract structured data on the fly. Default false. Consider using this option as a Deep research when agent navigates to multiple pages looking for the data you want. If you need to extract data from a static page, we suggest you to use extract_data command.

• data_instruction (str): The instruction describing the exact data you want to extract (used when extract_data is true).

• json_schema (Union[dict, list, str]): A JSON schema defining the precise structure of the data you want returned. Useful for ensuring the output matches your programmatic expectations (used when extract_data is true).

• viewport_min (int): Restrict the agent's initial attention to start from a specific 0-based viewport index.

• viewport_max (int): Restrict the agent's attention to end at a specific 0-based viewport index.

• Example Usage:

python
session.execute(
    commands.act(
        instruction="Open a page with the best rated laptop on the page",
        max_steps=5,
        model_size="medium"
    )
)

• Example Usage with enabled data extraction mode (useful when you want to extract data scattered across multiple pages / sites or you want let agent find the data by itself - agent will collect data from different places and assemble unified output JSON):

python
session.execute(
    commands.act(
        instruction="Open the first 3 laptop listings for the following query and extract data (from each item page): 'cool laptop'",
        max_steps=15,
        model_size="medium",
        extract_data=True,
        data_instruction="Extract basic info of the laptop",
        json_schema=[
            {
                "title": "<short laptop listing title>",
                "price": "<listed price with currency without any special symbols>",
                "price_currency": "<ISO currency code of the price>",
                "specs": "<short summary of main specs such as CPU, RAM, storage, GPU>",
                "summary": "<brief description of the laptop and its standout features>",
                "product_url": "<direct valid URL to the laptop product page>"
            }
        ]
    )
)

• Returns (Command Result) Example (without data extraction):

json

{
    "command": "act",
    "offloaded_data_url": null,
    "metadata": {
        "ocr": {
            "images_processed": 5,
            "images_failed": 0,
            "chars_extracted": 6000
        },
        "tokens": {
            "completion": 75,
            "total": 2500,
            "prompt": 2425
        },
        "actions_executed": 3,
        "pages_extracted": 0,
        "steps_executed": 4
    },
    "step": 0,
    "success": true,
    "data": {
        "objective_accomplished": true,
        "action_steps": [
            {
                "stop": 0,
                "name": "type_at",
                "comment": "Typing query into the search bar",
                "url": "https://example-store.com/",
                "element": {
                    "bbox": [100, 150, 130, 450],
                    "point": [115, 300]
                },
                "instruction": "Type 'laptop' into the search input",
                "text": "laptop",
                "hint": "Typing search query",
                "before": "https://example-cloud-storage.com/screenshots/before_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_1.png"
            },
            {
                "stop": 0,
                "name": "press",
                "comment": "Pressing enter to submit the search",
                "url": "https://example-store.com/",
                "instruction": "Press Enter",
                "keys": ["enter"],
                "hint": "Submitting search",
                "before": "https://example-cloud-storage.com/screenshots/after_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_2.png"
            },
            {
                "stop": 1,
                "name": "click",
                "comment": "Clicking on the first laptop in the search results",
                "url": "https://example-store.com/search?q=laptop",
                "element": {
                    "bbox": [300, 200, 400, 500],
                    "point": [350, 350]
                },
                "instruction": "Click on the first product listing",
                "mouse_button": "left",
                "clicks": 1,
                "label": "SuperBook Pro 15",
                "hint": "Clicking first result",
                "before": "https://example-cloud-storage.com/screenshots/after_2.png",
                "after": "https://example-cloud-storage.com/screenshots/after_3.png"
            }
        ],
        "stop_type": "objective_accomplished"
    }
}

• Returns (Command Result) Example (with data extraction enabled):

json

{
    "command": "act",
    "offloaded_data_url": null,
    "metadata": {
        "ocr": {
            "images_processed": 2,
            "images_failed": 0,
            "chars_extracted": 12000
        },
        "tokens": {
            "completion": 125,
            "total": 4500,
            "prompt": 4375
        },
        "actions_executed": 3,
        "pages_extracted": 1,
        "steps_executed": 4
    },
    "step": 0,
    "success": true,
    "data": {
        "objective_accomplished": true,
        "action_steps": [
            {
                "stop": 0,
                "name": "type_at",
                "comment": "Typing query into the search bar",
                "url": "https://example-store.com/",
                "element": {
                    "bbox": [100, 150, 130, 450],
                    "point": [115, 300]
                },
                "instruction": "Type 'laptop' into the search input",
                "text": "laptop",
                "hint": "Typing search query",
                "before": "https://example-cloud-storage.com/screenshots/before_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_1.png"
            },
            {
                "stop": 0,
                "name": "press",
                "comment": "Pressing enter to submit the search",
                "url": "https://example-store.com/",
                "instruction": "Press Enter",
                "keys": ["enter"],
                "hint": "Submitting search",
                "before": "https://example-cloud-storage.com/screenshots/after_1.png",
                "after": "https://example-cloud-storage.com/screenshots/after_2.png"
            },
            {
                "stop": 0,
                "name": "click",
                "comment": "Clicking on the first laptop in the search results",
                "url": "https://example-store.com/search?q=laptop",
                "element": {
                    "bbox": [300, 200, 400, 500],
                    "point": [350, 350]
                },
                "instruction": "Click on the first product listing",
                "mouse_button": "left",
                "clicks": 1,
                "label": "SuperBook Pro 15",
                "hint": "Clicking first result",
                "before": "https://example-cloud-storage.com/screenshots/after_2.png",
                "after": "https://example-cloud-storage.com/screenshots/after_3.png"
            },
            {
                "name": "extract_data",
                "after": "https://example-cloud-storage.com/screenshots/after_4.png",
                "comment": "Extracting laptop info from the current page.",
                "stop": 0,
                "before": "https://example-cloud-storage.com/screenshots/after_3.png",
                "hint": "",
                "url": "https://example-store.com/product/superbook-pro-15"
            },
            {
                "name": "stop",
                "after": "https://example-cloud-storage.com/screenshots/after_5.png",
                "comment": "Laptop info extracted from the dedicated page.",
                "stop": 1,
                "before": "https://example-cloud-storage.com/screenshots/after_4.png",
                "hint": "Stopping execution. Comment: Laptop info extracted from the dedicated page.",
                "url": "https://example-store.com/product/superbook-pro-15"
            }
        ],
        "extracted_json": [
            {
                "title": "SuperBook Pro 15",
                "price": "1299.00",
                "price_currency": "USD",
                "specs": "Intel Core i7, 16GB RAM, 512GB SSD",
                "summary": "A powerful laptop for professionals.",
                "product_url": "https://example-store.com/product/superbook-pro-15"
            }
        ],
        "screenshots": [
            "https://example-cloud-storage.com/screenshots/example_1.png",
            "https://example-cloud-storage.com/screenshots/example_2.png"
        ],
        "text": "Raw text extracted from the page...",
        "stop_type": "objective_accomplished"
    }
}

(Note: The exact contents of data depend on whether extract_data is true and what the agent encountered during execution).

extract

Extracts structured data directly from the current static page layout using AI. You define what you want to extract (via instructions and a JSON schema) and the AI analyzes the DOM and visual data on the page to extract JSON for you. It is highly resilient to website structural changes. When both data_instruction and json_schema are not provided, AI will design schema layout on its own.

Parameters:

• data_instruction (str): Plain-text instruction describing exactly what you want the AI to find and extract.

• json_schema (Union[dict, list, str]): A schema indicating the expected JSON structure. Schema is not strictly typed - might be represented as { field_name: description, ... }. Wrap object into an array [{...}] if you expect to extract a list of items from the page. Example: [{"title": "<title of the product>", "price": "<price of the prodcut without any special symbols (float)>"}].

• model_size (str): AI model size. Larger models are smarter but more expensive. Allowed: "small", "medium", "large". Default "small".

• viewport_min (int): Start index (0-based) for the viewports the AI should analyze.

• viewport_max (int): Maximum viewport index the AI should look at. Useful for making extraction cheaper by ignoring irrelevant footer content on long pages.

• vision (bool): If false, the AI will skip stage with recognizing page text from screenshots using OCR and rely purely on screenshots. Turning vision off reduces prompt (input) tokens usage approximately by 20-30%. Default true.

• feed_urls (bool): If true, the AI receives exact DOM hyperlinks. Set to false to save token costs if you don't need to extract URLs. Default true.

• disable_ocr (bool): Disables Optical Character Recognition during visual analysis. Default false.

• paginate (bool): Set to true to let the AI automatically find the "Next" button, navigate and merge data across multiple pages. Default false.

• pages_to_paginate (int): Max pages to navigate through. Minimum 1, max 100. Default 1.

• pagination_hint (str): Custom text hints for the AI to find the pagination button (useful when navigation element on the page is not obvious / hard to find).

• Example Usage:

python
session.execute(
    commands.extract(
        data_instruction="Extract all product names and their prices.",
        json_schema=[
            {
                "name": "<product name>",
                "price": "<price as string>"
            }
        ],
        model_size="medium"
    )
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "screenshots": [
      "https://cloud-storage-url.com/screenshot1.jpg"
    ],
    "extracted_json": [
      {
        "name": "SuperBook Pro 15",
        "price": "$1299.00"
      },
      {
        "name": "MegaPad Air",
        "price": "$599.00"
      }
    ],
    "text": "Raw text extracted from the page..."
  }
}

Manual Commands

Navigation and page state

open_url

Navigates the browser to the specified URL and automatically waits for the page to finish loading.

Parameters:

• url (str): Destination URL to open. Accepts URLs, IPv4 addresses, IPv6 addresses and about:blank page.

• Example Usage:

python
session.execute(
    commands.open_url(url="https://example.com")
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "url": "https://example.com"
  }
}

get_url

Returns the current URL natively using the browser's navigation history.

Parameters: None

• Example Usage:

python
session.execute(
    commands.get_url()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "url": "https://example.com/current-page"
  }
}

refresh

Reloads the current web page and waits for it to load completely. Returns the URL of the page post-refresh.

Parameters: None

• Example Usage:

python
session.execute(
    commands.refresh()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "url": "https://example.com/current-page"
  }
}

back

Navigates back to the previous web page in the browser's history natively and waits for it to load. Returns the updated URL.

Parameters: None

• Example Usage:

python
session.execute(
    commands.back()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "url": "https://example.com/previous-page"
  }
}

forward

Navigates forward to the next web page in the browser's history natively and waits for it to load. Returns the updated URL.

Parameters: None

• Example Usage:

python
session.execute(
    commands.forward()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "url": "https://example.com/next-page"
  }
}

sleep

Pauses the browser execution for a specific amount of time. Includes a slight random fluctuation by default to make sleeping less predictable.

Parameters:

• wait_time (float): Minimum time to sleep in seconds.

• fluctuation (float): Maximum random deviation added to the base wait_time. Default 0.05.

• Example Usage:

python
session.execute(
    commands.sleep(wait_time=2.5)
)

• Returns (Command Result):

json

{
  "success": true
}

get_pdf

Exports the current page to a PDF document and uploads it to the cloud.

Parameters:

• landscape (bool): Paper orientation. Defaults to false.

• print_background (bool): Print background graphics. Defaults to true.

• scale (float): Scale of the webpage rendering. Defaults to 1.0.

• paper_width (float): Paper width in inches. Defaults to 8.5.

• paper_height (float): Paper height in inches. Defaults to 11.0.

• margin_top (float): Top margin in inches. Defaults to 0.4.

• margin_bottom (float): Bottom margin in inches. Defaults to 0.4.

• margin_left (float): Left margin in inches. Defaults to 0.4.

• margin_right (float): Right margin in inches. Defaults to 0.4.

• Example Usage:

python
session.execute(
    commands.get_pdf(
        landscape=False,
        print_background=True
    )
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "pdf_url": "https://cloud-storage-url.com/document.pdf"
  }
}

get_screenshot

Captures a screenshot of the current page. The response behavior depends on the requested screenshot mode. Captured images are automatically uploaded to the cloud.

Parameters:

• mode (str): Method that'll be used to capture a screenshot or screenshots. - "viewport" (Default): Captures only the currently visible part of the page (viewport). - "full_page": Captures full-length image of the page. - "viewports": Captures viewport images of the page.

• viewport_min (int): If mode is "full_page" or "viewports", sets the starting index (0-based) for the captured viewports. Useful for skipping the top of the page.

• viewport_max (int): Sets the maximum amount of viewports to capture. Max limit is 50.

• Example Usage:

python
session.execute(
    commands.get_screenshot(mode="viewport")
)

• Returns (Command Result) (viewport / full_page modes):

json

{
  "success": true,
  "data": {
    "image_url": "https://cloud-storage-url.com/image.jpg"
  }
}

• Returns (Command Result) (viewports mode):

json

{
  "success": true,
  "data": {
    "image_urls": [
      "https://cloud-storage-url.com/chunk_1.jpg",
      "https://cloud-storage-url.com/chunk_2.jpg"
    ]
  }
}

get_html

Retrieves the entire outer HTML of the current page. If the HTML response is extremely large and exceeds internal limits, it is offloaded to the cloud and you'll receive an offloaded_data_url instead of inline JSON data in the response payload.

Parameters: None

• Example Usage:

python
session.execute(
    commands.get_html()
)

• Returns (Command Result) (Standard):

json

{
  "success": true,
  "data": {
    "html": "<html>...</html>"
  },
  "offloaded_data_url": null
}

• Returns (Command Result) (Offloaded payload):

json

{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/your-large-html-file"
}

get_snapshot

Captures detailed representation of the DOM tree (similar to Chrome DevTools snapshot), mapped with exact x/y coordinates, dimensions, visibilities, text contents and specific computed styles of each element. Snapshot payloads are very large so they are always offloaded to the cloud.

Parameters:

• computed_styles (List[str]): A list of CSS property names to explicitly extract for every node (e.g., ["background-color", "display"]).

• Example Usage:

python
session.execute(
    commands.get_snapshot(
        computed_styles=[
            "color",
            "font-weight"
        ]
    )
)

• Returns (Command Result):

json

{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/snapshot.json"
}

locate

Locates an element on the page based on the specified strategy. Automatically penetrates iframes and shadow DOMs. Assigns an element_id to the element for easier future interaction.

Parameters:

• strategy (str): Strategy to locate the element (css, xpath, role, text, label, placeholder, alt_text, title, test_id).

• value (str): The core search string. For css/xpath, this is the selector. For built-in locators like role, this is the ARIA role. For text, it is the inner text.

• name (str): Used when strategy="role". Acts as an accessible name filter.

• exact (bool): Used for text or role strategies. Requires an exact whole-string match. Defaults to false.

• index (int): Indicates which matching element to return if there are multiple matches. Defaults to 0.

• state (str): Wait for the element to reach a specific DOM state (visible, attached, enabled, hidden). Defaults to visible.

• timeout_ms (int): Maximum time in milliseconds to wait for the element state. Defaults to 15000.

• parent_id (str): A previously located element_id to scope this search inside of.

• Example Usage:

python
session.execute(
    commands.locate(
        strategy="css",
        value="button.submit"
    )
)

inject_cookies

Injects the provided cookies into the running session. Automatic page refresh will apply them.

Parameters:

• cookies (List[dict]): An array of cookie objects to inject into the new browser session. Each object must contain name, value and domain.

  • Payload:

    json
    {
      "commands": [
        {
          "command": "inject_cookies",
          "params": {
            "cookies": [
              {
                "name": "session_id",
                "value": "123456789",
                "domain": ".example.com",
                "path": "/"
              }
            ]
          }
        }
      ]
    }

• Example Usage:

python
session.execute(
    commands.inject_cookies(
        cookies=[
            {
                "name": "session_id",
                "value": "123456789",
                "domain": ".example.com",
                "path": "/"
            }
        ]
    )
)

• Returns (Command Result):

json

{
  "success": true
}

dump_cookies

Retrieves a list of all browser cookies for the current URL.

Parameters: None

• Example Usage:

python
session.execute(
    commands.dump_cookies()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "cookies": [
      {
        "name": "session_id",
        "value": "xyz123",
        "domain": "example.com",
        "path": "/",
        "expires": 1700000000,
        "size": 16,
        "httpOnly": true,
        "secure": true,
        "session": false
      }
    ]
  },
  "offloaded_data_url": null
}

dump_console_logs

Retrieves a list of all browser console logs for the current session.

Parameters: None

• Example Usage:

python
session.execute(
    commands.dump_console_logs()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "console_logs": [
      {
        "type": "log",
        "timestamp": 123456789.0,
        "message": "Hello world"
      }
    ]
  },
  "offloaded_data_url": null
}

dump_local_storage

Retrieves a dictionary representing the local storage data for the currently opened origin.

Parameters: None

• Example Usage:

python
session.execute(
    commands.dump_local_storage()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "local_storage": {
      "theme": "dark",
      "user_preferences": "{"notifications":true}"
    }
  },
  "offloaded_data_url": null
}

dump_har_logs

Retrieves the HAR (HTTP Archive) logs for the current session. These logs capture detailed information about all network requests and responses routed through the proxy, including headers, timings and statuses.

Parameters: None

• Example Usage:

python
session.execute(
    commands.dump_har_logs()
)

• Returns (Command Result) (Standard Sample):

json

{
  "success": true,
  "data": {
    "har_logs": {
      "log": {
        "version": "1.2",
        "creator": {
          "name": "BroTrafficRouter",
          "version": "1.0"
        },
        "entries": [
          {
            "startedDateTime": "2023-10-15T12:00:00.000Z",
            "time": 150,
            "proxied": true,
            "request": {
              "method": "GET",
              "url": "https://example.com/api/data",
              "httpVersion": "HTTP/2.0",
              "cookies": [],
              "headers": [
                {
                  "name": "Accept",
                  "value": "application/json"
                }
              ],
              "queryString": [],
              "headersSize": 150,
              "bodySize": 0
            },
            "response": {
              "status": 200,
              "statusText": "OK",
              "httpVersion": "HTTP/2.0",
              "cookies": [],
              "headers": [
                {
                  "name": "Content-Type",
                  "value": "application/json"
                }
              ],
              "content": {
                "size": 1024,
                "compression": 0,
                "mimeType": "application/json",
                "text": "{"key": "value"}"
              },
              "redirectURL": "",
              "headersSize": 160,
              "bodySize": 1024
            },
            "cache": {},
            "timings": {
              "send": 2,
              "receive": 48,
              "wait": 100,
              "connect": -1,
              "ssl": -1
            },
            "serverIPAddress": "93.185.218.44"
          }
        ]
      }
    }
  },
  "offloaded_data_url": null
}

• Returns (Command Result) (Offloaded):

json

{
  "success": true,
  "data": null,
  "offloaded_data_url": "https://cloud-storage-url.com/your-large-har-file.json"
}

parse_text

Crawls the current page's DOM (including shadow DOM and iframes) to discover and extract all visible text elements. If the extracted object is too large, the data will be offloaded to the cloud.

Parameters:

• format (str): The output format for the parsed text. Options are "plain", "md" or "json". Defaults to "plain".
• computed_styles (List[str]): A list of CSS property names to extract for every node.

CommandPayload: - Payload:

json

{
  "command": "parse_text",
  "params": {
    "format": "json"
  }
}

        - **Response (Standard for `format="json"`):**

json

{
  "success": true,
  "data": [
    {
      "id": 0,
      "tag": "p",
      "text": "Hello World",
      "viewport": 0,
      "coords": {
        "x": 100,
        "y": 200,
        "width": 300,
        "height": 20
      }
    }
  ],
  "offloaded_data_url": null
}

        - **Response (Standard for `format="plain"` or `format="md"`):**

json

{
  "success": true,
  "data": "Hello World

...", "offloaded_data_url": null }

parse_element_text

Crawls the specified element's DOM (including shadow DOM and iframes) to discover and extract all visible text elements inside it. Either element_id or locator must be provided. If the extracted object is too large, the data will be offloaded to the cloud.

Parameters:

• element_id (str): Use an existing element_id from a previous locate call. (Exactly one of element_id or locator is required).
• locator (dict): Pass a full locator object containing strategy, value, etc. (Exactly one of element_id or locator is required). - Payload: json { "command": "parse_element_text", "params": { "element_id": "...", "format": "json" } } - Parameters: - element_id (string) OR locator (object) (Exactly one is required): Use an existing element_id from a previous locate call or pass a full locator object containing strategy, value, etc.
• format (str): The output format for the parsed text. Options are "plain", "md" or "json". Defaults to "plain".
• computed_styles (List[str]): A list of CSS property names to extract for every node.

CommandPayload: - Payload:

json

{
  "command": "parse_element_text",
  "params": {
    "element_id": "...",
    "format": "json"
  }
}

        - **Response (Standard for `format="json"`):**

json

{
  "success": true,
  "data": [
    {
      "id": 0,
      "tag": "span",
      "text": "Extracted Element Text",
      "viewport": 0,
      "coords": {
        "x": 100,
        "y": 200,
        "width": 150,
        "height": 20
      }
    }
  ],
  "offloaded_data_url": null
}

        - **Response (Standard for `format="plain"` or `format="md"`):**

json

{
  "success": true,
  "data": "Extracted Element Text

...", "offloaded_data_url": null }

parse_urls

Crawls the current page's DOM to discover and extract all interactive links, routing objects and standard anchor tags. If the extracted object is too large, the data will be offloaded to the cloud.

Parameters:

• computed_styles (List[str]): A list of CSS property names to extract for every link node.

• Example Usage:

python
session.execute(
    commands.parse_urls()
)

• Returns (Command Result) (Standard):

json

{
  "success": true,
  "data": {
    "links": [
      {
        "is_base64": false,
        "url": "https://example.com/widget/sample-script.min.js",
        "id": 0,
        "count": 1,
        "tag": "script",
        "viewport": 0,
        "text": ""
      }
    ],
    "images": [
      {
        "is_base64": false,
        "url": "https://www.example.com/content/uploads/image.png",
        "id": 383,
        "count": 1,
        "tag": "img",
        "viewport": 0,
        "text": ""
      }
    ]
  },
  "offloaded_data_url": null
}

run_js

Executes raw JavaScript code in the context of the active page. Use this for highly custom operations or retrieving local storage values.

Parameters:

• js_code (str): The JavaScript code to execute. Must be written as a block that resolves or returns a value (e.g., return window.location.href;).

• out_type (str): Validates and casts the JavaScript result into a specific type. - Allowed values: "list", "dict", "bool", "float", "int", "str".

• Example Usage:

python
session.execute(
    commands.run_js(
        js_code="window.location.href",
        out_type="str"
    )
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "result": "https://news.ycombinator.com/"
  },
  "offloaded_data_url": null
}

copy

Copies the currently highlighted text to system clipboard using a standard OS keyboard shortcut (Ctrl+C). Returns the copied text. Like get_html, if the copied text is extremely large, it will be automatically offloaded to the cloud.

Parameters: None

• Example Usage:

python
session.execute(
    commands.copy()
)

• Returns (Command Result):

json

{
  "success": true,
  "data": {
    "copied_text": "Copied text..."
  },
  "offloaded_data_url": null
}

paste

Pastes the contents currently stored in the system clipboard into the active page element using a standard OS keyboard shortcut (Ctrl+V).

Parameters: None

• Example Usage:

python
session.execute(
    commands.paste()
)

• Returns (Command Result):

json

{
  "success": true
}

HTML Locator targets

click

Clicks on a targeted element. Identifies the target via element_id or locator.

Parameters:

• mouse_button (str): "left", "right", "middle". Default "left".

• clicks (int): Number of clicks. Default 1.

• element_id (str): An ID returned from a previous locate command.

• locator (dict): An object containing locator parameters.

• Example Usage:

python
session.execute(
    commands.click(
        locator={
            "strategy": "css",
            "value": "button.submit"
        }
    )
)

• Returns (Command Result):

json

{
  "success": true
}

type

Types text into a targeted element. Identifies the target via element_id or locator.

Parameters:

• text (str): The string to type.

• clear_first (bool): Clear the input before typing. Default true.

• element_id (str): An ID returned from a previous locate command.

• locator (dict): An object containing locator parameters.

• Example Usage:

python
session.execute(
    commands.type(
        locator={
            "strategy": "css",
            "value": "input[name='search']"
        },
        text="Hello World"
    )
)

• Returns (Command Result):

json

{
  "success": true
}

select

Selects an option from a <select> dropdown. Identifies the target via element_id or locator.

Parameters:

• select_value (str): Matches the <option value="..."> attribute.

• label (str): Matches the visible text of the option. (Must provide either select_value or label).

• element_id (str): An ID returned from a previous locate command.

• locator (dict): An object containing locator parameters.

• Example Usage:

python
session.execute(
    commands.select(
        element_id="id_x123",
        select_value="option1"
    )
)

• Returns (Command Result):

json

{
  "success": true
}

Mouse and viewport controls

hover_at

Moves the mouse cursor to the specified coordinates.

Parameters:

• x (float): The target X coordinate. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate. Minimum 0, maximum is the window height (1024).

• scope (str): The reference frame for the coordinates. - "viewport" (Default): Coordinates are relative to the webpage viewport. - "browser": Coordinates are relative to the browser window. - "os": Coordinates are relative to the operating system's screen.

• Example Usage:

python
session.execute(
    commands.hover_at(
        x=250.0,
        y=500.0
    )
)

• Returns (Command Result):

json

{
  "success": true
}

click_at

Moves the cursor to the target coordinates and clicks the specified mouse button.

Parameters:

• x (float): The target X coordinate. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate. Minimum 0, maximum is the window height (1024).

• mouse_button (str): The mouse button to press. Allowed values: "left", "right", "middle". Default "left".

• clicks (int): The number of times to click on the mouse_button (e.g., 2 for a double-click). Minimum 1. Default 1.

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.click_at(
        x=250.0,
        y=500.0,
        clicks=2
    )
)

• Returns (Command Result):

json

{
  "success": true
}

drag

Performs a drag-and-drop interaction by moving the mouse from a start coordinate to a destination coordinate while holding down a specified mouse button.

Parameters:

• start_x (float): The starting X coordinate. Minimum 0, maximum is the window width (1280).

• start_y (float): The starting Y coordinate. Minimum 0, maximum is the window height (1024).

• dest_x (float): The destination X coordinate. Minimum 0, maximum is the window width (1280).

• dest_y (float): The destination Y coordinate. Minimum 0, maximum is the window height (1024).

• mouse_button (str): The mouse button to hold down while dragging. Allowed values: "left", "right", "middle". Default "left".

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.drag(
        start_x=100.0,
        start_y=100.0,
        dest_x=300.0,
        dest_y=300.0
    )
)

• Returns (Command Result):

json

{
  "success": true
}

scroll

Scrolls the mouse wheel by a specified amount at the given coordinates.

Parameters:

• clicks (int): The amount of scroll wheel clicks to apply (negative - scroll down, positive - scroll up).

• x (float): The X coordinate to place the cursor before scrolling. Minimum 0, maximum is the window width (1280).

• y (float): The Y coordinate to place the cursor before scrolling. Minimum 0, maximum is the window height (1024).

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• axis (str): The direction to scroll. Allowed values: "x" (horizontal), "y" (vertical). Default "y".

• Example Usage:

python
session.execute(
    commands.scroll(
        clicks=-5,
        x=400,
        y=500
    )
)

• Returns (Command Result):

json

{
  "success": true
}

scroll_to_viewport

Automatically scrolls the page until a specific viewport chunk comes into view. This is useful when paginating through large pages so you won't need to call scroll command multiple times.

Parameters:

• viewport_idx (int): The target 0-based viewport index to scroll to. Minimum 0.

• Example Usage:

python
session.execute(
    commands.scroll_to_viewport(viewport_idx=5)
)

• Returns (Command Result):

json

{
  "success": true
}

click_and_hold

Moves the cursor to the target coordinates, presses down the specified mouse button, holds it for a specific duration and then releases it.

Parameters:

• x (float): The target X coordinate. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate. Minimum 0, maximum is the window height (1024).

• hold_duration (float): The duration in seconds to hold the mouse button. Minimum 0.0, maximum 60.0.

• mouse_button (str): The mouse button to hold down. Allowed values: "left", "right", "middle". Default "left".

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.click_and_hold(
        x=100.0,
        y=200.0,
        hold_duration=2.5
    )
)

• Returns (Command Result):

json

{
  "success": true
}

Keyboard controls

type_at

Clicks at the specified coordinates and inputs the given text into the focused element.

Parameters:

• x (float): The target X coordinate to click before typing. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate to click before typing. Minimum 0, maximum is the window height (1024).

• text (str): The text to type.

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.type_at(
        x=200.0,
        y=150.0,
        text="Hello world"
    )
)

• Returns (Command Result):

json

{
  "success": true
}

press

Presses a single keyboard key or a combination of keys (hotkey).

Parameters:

• keys (Union[List[str], str]): A single key (e.g., "enter") or an array of keys to trigger a shortcut (e.g., ["ctrl", "c"]).

• Example Usage:

python
session.execute(
    commands.press(
        keys=[
            "ctrl",
            "a"
        ]
    )
)

• Returns (Command Result):

json

{
  "success": true
}

hold_a_key_and_click

Holds down specified keys (like "shift" or "control"), clicks at the target coordinates and then releases the keys. Useful for selecting multiple items or opening links in a new tab.

Parameters:

• x (float): The target X coordinate. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate. Minimum 0, maximum is the window height (1024).

• keys (Union[List[str], str]): The key or sequence of keys to hold down during the click.

• mouse_button (str): The mouse button to press. Allowed values: "left", "right", "middle". Default "left".

• clicks (int): The number of times to click. Minimum 1. Default 1.

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.hold_a_key_and_click(
        x=300.0,
        y=400.0,
        keys="shift"
    )
)

• Returns (Command Result):

json

{
  "success": true
}

hold_a_key_and_drag

Holds down specified keys, drags the mouse from a start coordinate to a destination coordinate and then releases the keys.

Parameters:

• start_x (float): The starting X coordinate. Minimum 0, maximum is the window width (1280).

• start_y (float): The starting Y coordinate. Minimum 0, maximum is the window height (1024).

• dest_x (float): The destination X coordinate. Minimum 0, maximum is the window width (1280).

• dest_y (float): The destination Y coordinate. Minimum 0, maximum is the window height (1024).

• keys (Union[List[str], str]): The key or sequence of keys to hold down during the drag.

• mouse_button (str): The mouse button to hold down while dragging. Allowed values: "left", "right", "middle". Default "left".

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.hold_a_key_and_drag(
        start_x=100.0,
        start_y=100.0,
        dest_x=200.0,
        dest_y=200.0,
        keys="shift"
    )
)

• Returns (Command Result):

json

{
  "success": true
}

clear_textbox

Clicks on a textbox at the specified coordinates, selects all existing text and deletes it.

Parameters:

• x (float): The target X coordinate. Minimum 0, maximum is the window width (1280).

• y (float): The target Y coordinate. Minimum 0, maximum is the window height (1024).

• scope (str): The reference frame for the coordinates. Allowed values: "viewport", "browser", "os". Default "viewport".

• Example Usage:

python
session.execute(
    commands.clear_textbox(
        x=150.0,
        y=250.0
    )
)

• Returns (Command Result):

json

{
  "success": true
}

---