/unblock API

The /unblock API is designed to bypass bot detection mechanisms such as Datadome and other passive CAPTCHAs. It offers two main ways to access protected pages:

Direct data extraction: Grab the HTML content, screenshots, or cookies from protected pages with a simple REST call.
WebSocket endpoint generation: Get a WebSocket URL to connect your existing Puppeteer/Playwright code for custom automations.

BrowserQL Recommended

For generic and scale-intensive use-cases involving advanced bot detection, we recommend using BrowserQL instead. Its advanced stealth capabilities include human-like interactions, built-in CAPTCHA solving (Cloudflare, reCAPTCHA, and a variety of others), fingerprint masking, and GraphQL-based automation that outperforms traditional tools.

For Cloudflare turnstiles, use BrowserQL's verify mutation. For reCAPTCHA, use the solve mutation. See the BrowserQL CAPTCHA Solving Guide for details.

The /unblock API works best when combined with our Browserless Residential Proxy Service.

This API is particularly useful for developers who need to automate web interactions on sites that employ sophisticated bot detection and blocking techniques. It offers four different ways to wait for preconditions to be met before returning a response.

You can view the complete Unblock API OpenAPI specification for all properties and documentation.

Examples

The Unblock API is optimized to return only the data you request. Set fields to true for the data you need, and false for everything else to reduce execution time and resource usage.

Get Content
Get Cookies
Get Screenshot
Get WebSocket

Retrieve the HTML content of a page after bypassing bot detection:

{
    "url": "https://www.example.com/",
    "content": true,
    "cookies": false,
    "screenshot": false,
    "browserWSEndpoint": false,
    "ttl": 0
}

Extract cookies from a protected site for session management:

{
    "url": "https://www.example.com/",
    "content": false,
    "cookies": true,
    "screenshot": false,
    "browserWSEndpoint": false,
    "ttl": 0
}

Capture a screenshot from bot-protected pages:

{
    "url": "https://www.example.com/",
    "content": false,
    "cookies": false,
    "screenshot": true,
    "browserWSEndpoint": false,
    "ttl": 0
}

Generate a WebSocket endpoint for custom Puppeteer/Playwright automation:

{
    "url": "https://www.example.com/",
    "content": false,
    "cookies": false,
    "screenshot": false,
    "browserWSEndpoint": true,
    "ttl": 30000
}

Retrieving HTML

If you'd like to retrieve the HTML of a page for scraping, Set content: true in the JSON payload. Here's a complete example with residential proxy enabled:

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com/",
  "content": true
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com/",
  content: true
};

const fetchContent = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const content = await response.json();
  console.log(content);
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com/",
    "content": True
}

response = requests.post(url, headers=headers, json=data)

print(response.json())

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchUnblockedContent {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com/",
            "content": true
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

        string jsonData = @"
        {
            ""url"": ""https://example.com/"",
            ""content"": true
        }";

        using var client = new HttpClient();
        var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

        try
        {
            var response = await client.PostAsync(url, content);
            response.EnsureSuccessStatusCode();

            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
        }
    }
}

Response:

{
  "content": "<!DOCTYPE html><html>...</html>",
  "cookies": [],
  "screenshot": null,
  "browserWSEndpoint": null
}

Process the returned HTML with scraping libraries like Scrapy or Beautiful Soup.

Getting a WebSocket Endpoint

The /unblock API can bypass bot detection, then return a WebSocket endpoint and cookies for you to connect with your own Puppeteer/Playwright code.

JSON Request Body:

Set browserWSEndpoint: true, cookies: true, and specify a ttl (time-to-live in milliseconds) to keep the browser session alive for your automation:

{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "ttl": 30000
}

We recommend using a residential proxy for better bot detection bypass:

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "ttl": 30000
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com",
  browserWSEndpoint: true,
  cookies: true,
  ttl: 30000
};

const fetchBrowserWSEndpoint = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const result = await response.json();
  console.log(result);
};

fetchBrowserWSEndpoint();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com",
    "browserWSEndpoint": True,
    "cookies": True,
    "ttl": 30000
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

print(result)

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchBrowserWSEndpoint {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com",
            "browserWSEndpoint": true,
            "cookies": true,
            "ttl": 30000
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

  using System;
  using System.Net.Http;
  using System.Text;
  using System.Threading.Tasks;

  class Program
  {
      static async Task Main(string[] args)
      {
          string TOKEN = "YOUR_API_TOKEN_HERE";
          string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

          string jsonData = @"
          {
              ""url"": ""https://example.com"",
              ""browserWSEndpoint"": true,
              ""cookies"": true,
              ""ttl"": 30000
          }";

          using var client = new HttpClient();
          var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

          try
          {
              var response = await client.PostAsync(url, content);
              response.EnsureSuccessStatusCode();

              var result = await response.Content.ReadAsStringAsync();
              Console.WriteLine("Response: " + result);
          }
          catch (Exception ex)
          {
              Console.WriteLine($"Error: {ex.Message}");
          }
      }
  }

Response:

{
  "browserWSEndpoint": "wss://production-sfo.browserless.io/p/53616c74...646b292c/devtools/browser/102ea3e9-74d7-42c9-a856-1bf254649b9a",
  "cookies": [
    {
      "name": "session_id",
      "value": "XYZ123",
      "domain": "example.com",
      "path": "/",
      "secure": true,
      "httpOnly": true
    }
  ],
  "content": null,
  "screenshot": null
}

Connecting to the Browser

After receiving the response with the browserWSEndpoint and cookies, you can use Puppeteer, Playwright, or another CDP library to connect to the browser instance and inject the cookies to continue your automation:

Puppeteer
Playwright

import puppeteer from "puppeteer-core";

const TOKEN = "YOUR_API_TOKEN_HERE";

const unblock = async (url) => {
  const response = await fetch(
    `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`,
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        url: url,
        browserWSEndpoint: true,
        cookies: true,
        ttl: 30000
      })
    }
  );
  return await response.json();
};

// Get the WebSocket endpoint after bot detection bypass
const { browserWSEndpoint, cookies } = await unblock("https://example.com/");

// Connect to the browser
const browser = await puppeteer.connect({
  browserWSEndpoint: `${browserWSEndpoint}?token=${TOKEN}`
});
const page = (await browser.pages())[0];

// Inject cookies if needed
// await page.setCookie(...cookies);

await page.screenshot({ path: "screenshot.png" });
await browser.close();

Javascript
Python
Java
C#

import playwright from "playwright-core";

const TOKEN = "YOUR_API_TOKEN_HERE";

const unblock = async (url) => {
  const response = await fetch(
    `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`,
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        url: url,
        browserWSEndpoint: true,
        cookies: true,
        ttl: 30000
      })
    }
  );
  return await response.json();
};

// Get the WebSocket endpoint after bot detection bypass
const { browserWSEndpoint, cookies } = await unblock("https://example.com/");
  
// Connect to the browser
const browser = await playwright.chromium.connect(`${browserWSEndpoint}?token=${TOKEN}`);
const context = await browser.newContext();
const page = await context.newPage();

// Inject cookies if needed
// await context.addCookies(cookies);

await page.screenshot({ path: "screenshot.png" });
await browser.close();

from playwright.sync_api import sync_playwright
import requests

TOKEN = "YOUR_API_TOKEN_HERE"

def unblock(url):
    response = requests.post(
        f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential",
        headers={"Content-Type": "application/json"},
        json={
            "url": url,
            "browserWSEndpoint": True,
            "cookies": True,
            "ttl": 30000
        }
    )
    return response.json()

with sync_playwright() as p:
    # Get the WebSocket endpoint after bot detection bypass
    response = unblock("https://example.com/")
    browserWSEndpoint = response["browserWSEndpoint"]
    cookies = response.get("cookies", [])

    # Connect to the browser
    browser = p.chromium.connect(f"{browserWSEndpoint}?token={TOKEN}")
    context = browser.new_context()
    page = context.new_page()

    # Inject cookies if needed
    # context.add_cookies(cookies)

    page.screenshot(path="screenshot.png")
    browser.close()

import com.microsoft.playwright.*;
import org.json.JSONObject;
import java.net.http.*;
import java.net.URI;
import java.nio.file.Paths;

public class PlaywrightUnblock {
    private static final String TOKEN = "YOUR_API_TOKEN_HERE";

    public static JSONObject unblock(String url) throws Exception {
        HttpClient client = HttpClient.newHttpClient();
        String requestBody = new JSONObject()
                .put("url", url)
                .put("browserWSEndpoint", true)
                .put("cookies", true)
                .put("ttl", 30000)
                .toString();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(new URI("https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential"))
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(requestBody))
                .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        return new JSONObject(response.body());
    }

    public static void main(String[] args) throws Exception {
        // Get the WebSocket endpoint after bot detection bypass
        JSONObject response = unblock("https://example.com/");
        String browserWSEndpoint = response.getString("browserWSEndpoint");

        try (Playwright playwright = Playwright.create()) {
            Browser browser = playwright.chromium().connect(browserWSEndpoint + "?token=" + TOKEN);
            BrowserContext context = browser.newContext();
            Page page = context.newPage();

            // Inject cookies if needed
            // context.addCookies(cookiesArray);

            page.screenshot(new Page.ScreenshotOptions().setPath(Paths.get("screenshot.png")));
            browser.close();
        }
    }
}

  using System;
  using System.Net.Http;
  using System.Text;
  using System.Text.Json;
  using System.Threading.Tasks;
  using Microsoft.Playwright;

  class Program
  {
      private const string TOKEN = "YOUR_API_TOKEN_HERE";

      static async Task<JsonElement> UnblockAsync(string url)
      {
          using var client = new HttpClient();
          var opts = new
          {
              url = url,
              browserWSEndpoint = true,
              cookies = true,
              ttl = 30000
          };

          var response = await client.PostAsync(
              $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential",
              new StringContent(JsonSerializer.Serialize(opts), Encoding.UTF8, "application/json")
          );

          response.EnsureSuccessStatusCode();
          return JsonSerializer.Deserialize<JsonElement>(await response.Content.ReadAsStringAsync());
      }

      static async Task Main(string[] args)
      {
          try
          {
              // Get the WebSocket endpoint after bot detection bypass
              var response = await UnblockAsync("https://example.com/");
              string browserWSEndpoint = response.GetProperty("browserWSEndpoint").GetString();

              var playwright = await Playwright.CreateAsync();
              var browser = await playwright.Chromium.ConnectOverCDPAsync($"{browserWSEndpoint}?token={TOKEN}");
              var context = await browser.NewContextAsync();
              var page = await context.NewPageAsync();

              // Inject cookies if needed
              // await context.AddCookiesAsync(cookiesArray);

              await page.ScreenshotAsync(new PageScreenshotOptions { Path = "screenshot.png" });
              await browser.CloseAsync();
          }
          catch (Exception ex)
          {
              Console.WriteLine($"Error: {ex.Message}");
          }
      }
  }

Additional Configuration Options

The /unblock API supports additional configuration options that are shared across multiple REST API endpoints. See the Request Configuration page for detailed documentation on:

Waiting for Things: Wait for events, functions, selectors, or timeouts before returning the response
Navigation Options: Customize navigation behavior using gotoOptions
Rejecting Undesired Requests: Block specific resources using rejectResourceTypes and rejectRequestPattern
Continue on Error: Use bestAttempt to proceed when async events fail or timeout

Examples​

Retrieving HTML​

Getting a WebSocket Endpoint​

Connecting to the Browser​

Additional Configuration Options​

Examples

Retrieving HTML

Getting a WebSocket Endpoint

Connecting to the Browser

Additional Configuration Options