/unblock API

info

Currently, Browserless V2 is available in production via two domains: production-sfo.browserless.io and production-lon.browserless.io

The /unblock API is designed to bypass bot detection mechanisms, allowing you to get a bot-protected website's content, cookies, and even a screenshot of the bypassed result. It's available exclusively in cloud-hosted or Enterprise plans and only supports connections via the Chrome DevTools Protocol.

This API is particularly useful for developers who need to automate web interactions on sites that employ sophisticated bot detection and blocking techniques.

Basic Usage

JSON Payload

This is a JSON object containing the URL of the site you wish to unblock, along with the parameters you want in the response. Ideally you always want to set the ttl the browserWSEndpoint and cookies.

{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "content": false,
  "screenshot": false,
  "ttl": 30000
}

cURL Request

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=GOES-HERE' \
  --header 'content-type: application/json' \
  --data '{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "content": false,
  "screenshot": false,
  "ttl": 30000
}'

Which will return a JSON response like this:

{
  "browserWSEndpoint": "wss://production-sfo.browserless.io/p/53616c7465645f5f0d2e4012516859fdda7cc1ae0b16c6c5ec739d5d9f19a3d3c9b49c8a814b0fd1beae934b2e8050a0/devtools/browser/102ea3e9-74d7-42c9-a856-1bf254649b9a",
  "content": null,
  "cookies": [
    {
      name: "session_id",
      value: "XYZ123",
      domain: "example.com",
      path: "/",
      secure: true,
      httpOnly: true,
    },
  ],
  "screenshot": null,
  "ttl": 30000
}

After receiving the response with the browserWSEndpoint and cookies, you can use Puppeteer, Playwright or another CDP library to connect to the browser instance and inject the cookies to continue your scraping process:

Puppeteer Connection

import puppeteer from "puppeteer";
import unblock from "./utils";

// Example response from the API
const { browserWSEndpoint, cookies } = unblock("https://example.com");

const browser = await puppeteer.connect({ browserWSEndpoint });
const page = await browser.newPage();

// Inject cookies into the page
await page.setCookie(...response.cookies);
await page.goto("https://www.targetsite.com");
await page.screenshot({ path: "screenshot.png" });

await browser.close();