Skip to main content

Request Configuration

This page documents configuration options that are shared across multiple REST API endpoints. These options allow you to control how the browser navigates to pages, what content to block, how to wait for page elements, and how to handle errors.

Universal Configuration Options

The examples on this page use the /content API for demonstration purposes, but all configuration options documented here work across all REST API endpoints including /pdf, /screenshot, /scrape, /download, /unblock, /export, and more.

Complete API Reference

For an exhaustive list of all available configuration options and parameters for each endpoint, please refer to our OpenAPI specification. The OpenAPI docs provide comprehensive details on all request and response schemas.

Waiting for Things

Browserless offers 4 different ways to wait for preconditions to be met on the page before returning the response. These are events, functions, selectors and timeouts.

waitForEvent

Waits for an event to happen on the page before continuing.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"waitForEvent": {
"event": "fullscreenchange",
"timeout": 5000
}
}'

waitForFunction

Waits for the provided function to return before continuing. The function can be any valid JavaScript function including async functions.

Example

JS function:

async () => {
const res = await fetch("https://jsonplaceholder.typicode.com/todos/1");
const json = await res.json();

document.querySelector("h1").innerText = json.title;
};
curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"waitForFunction": {
"fn": "async()=>{let t=await fetch('https://jsonplaceholder.typicode.com/todos/1'),e=await t.json();document.querySelector('h1').innerText=e.title}",
"timeout": 5000
}
}'

waitForSelector

Waits for a selector to appear on the page. If at the moment of calling this API, the selector already exists, the method will return immediately. If the selector doesn't appear after the timeout milliseconds of waiting, the API will return a non-200 response code with an error message.

The object can have any of these values:

  • selector: String, required — A valid CSS selector.
  • hidden: Boolean, optional — Wait for the selected element to not be found in the DOM or to be hidden, i.e. have display: none or visibility: hidden CSS properties.
  • timeout: Number, optional — Maximum number of milliseconds to wait for the selector before failing.
  • visible: Boolean, optional — Wait for the selected element to be present in DOM and to be visible, i.e. to not have display: none or visibility: hidden CSS properties.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"waitForSelector": {
"selector": "h1",
"timeout": 5000
}
}'

waitForTimeout

Waits for a specified timeout before continuing.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"waitForTimeout": 10000
}'

To customize the navigation behavior when loading a page, such as specifying when to consider the page fully loaded (e.g., waiting for network activity to settle), you can use the gotoOptions parameter. The objects mirror Puppeteer's GoToOptions interface.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"gotoOptions": { "waitUntil": "networkidle2" }
}'

Rejecting Undesired Requests

You can use rejectResourceTypes and rejectRequestPattern to block undesired content, resources and requests.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://browserless.io/",
"rejectResourceTypes": ["image"],
"rejectRequestPattern": ["/^.*\\.(css)"]
}'

Continue on Error

You can use bestAttempt to make Browserless attempt to proceed when async events fail or timeout. This includes things like the goto or waitForSelector properties in the JSON payload.

Example

curl -X POST \
https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com/",
"bestAttempt": true,
"waitForSelector": { "selector": "table", "timeout": 500 }
}'

Response codes from sites

When you make a REST API call to Browserless, you may receive an HTTP 200 status code indicating that the connection between Browserless and your client was successful. However, the underlying site that Browserless visits may return a different status response code.

Browserless propagates the actual site's response code through response headers. If a site returns something like "Error 403 - Forbidden", you'll find that 403 status code in the response headers under X-Response-Code.

Response headers provided:

  • X-Response-Code: The HTTP status code from the target site
  • X-Response-Status: The status text from the target site
  • X-Response-URL: The final URL after any redirects
  • X-Response-IP: The IP address of the target site
  • X-Response-Port: The port used to connect to the target site

This means a successful Browserless API call (HTTP 200) can still indicate that the target site returned an error (e.g., 403, 404, 500) - check the X-Response-Code header to determine the actual site response.