/unblock API

The /unblock API bypasses captchas/bot detection and either returns content/cookies/screenshot and/or a browserWSEndpoint for custom automation. It offers two main ways to access protected pages:

Direct data extraction: Grab the HTML content, screenshots, or cookies from protected pages with a simple REST call.
WebSocket endpoint generation: Get a WebSocket URL to connect your existing Puppeteer/Playwright code for custom automations.

The /unblock API works best when combined with our Browserless Residential Proxy Service. This API is particularly useful for developers who need to automate web interactions on sites that employ sophisticated bot detection and blocking techniques.

You can view the complete Unblock API OpenAPI specification for all properties and documentation.

Quick Start

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com/",
  "content": true
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com/",
  content: true
};

const fetchContent = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const content = await response.json();
  console.log(content);
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com/",
    "content": True
}

response = requests.post(url, headers=headers, json=data)

print(response.json())

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchUnblockedContent {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com/",
            "content": true
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

        string jsonData = @"
        {
            ""url"": ""https://example.com/"",
            ""content"": true
        }";

        using var client = new HttpClient();
        var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

        try
        {
            var response = await client.PostAsync(url, content);
            response.EnsureSuccessStatusCode();

            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
        }
    }
}

Response

{
  "content": "<!DOCTYPE html><html>...</html>",
  "cookies": [],
  "screenshot": null,
  "browserWSEndpoint": null
}

Additional Examples and Configuration

BrowserQL Recommended

For advanced bot detection and CAPTCHA solving, we recommend using BrowserQL with the solve mutation. It automatically detects and solves CAPTCHAs (reCAPTCHA, Cloudflare, and others) with built-in stealth capabilities.

The /unblock API works best when combined with our Browserless Residential Proxy Service.

Data Extraction Modes

The Unblock API is optimized to return only the data you request. Set fields to true for the data you need, and false for everything else to reduce execution time and resource usage.

Get Content:

If you'd like to retrieve the HTML of a page for scraping, set content: true in the JSON payload. Here's a complete example with residential proxy enabled:

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com/",
  "content": true
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com/",
  content: true
};

const fetchContent = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const content = await response.json();
  console.log(content);
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com/",
    "content": True
}

response = requests.post(url, headers=headers, json=data)

print(response.json())

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchUnblockedContent {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com/",
            "content": true
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

        string jsonData = @"
        {
            ""url"": ""https://example.com/"",
            ""content"": true
        }";

        using var client = new HttpClient();
        var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

        try
        {
            var response = await client.PostAsync(url, content);
            response.EnsureSuccessStatusCode();

            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
        }
    }
}

Response:

{
  "content": "<!DOCTYPE html><html>...</html>",
  "cookies": [],
  "screenshot": null,
  "browserWSEndpoint": null
}

Get Cookies:

{
  "url": "https://www.example.com/",
  "content": false,
  "cookies": true,
  "screenshot": false,
  "browserWSEndpoint": false,
  "ttl": 0
}

Get Screenshot:

{
  "url": "https://www.example.com/",
  "content": false,
  "cookies": false,
  "screenshot": true,
  "browserWSEndpoint": false,
  "ttl": 0
}

Process the returned HTML with scraping libraries like Scrapy or Beautiful Soup.

Getting a WebSocket Endpoint

The /unblock API can bypass bot detection, then return a WebSocket endpoint and cookies for you to connect with your own Puppeteer/Playwright code.

JSON Request Body:

Set browserWSEndpoint: true, cookies: true, and specify a ttl (time-to-live in milliseconds) to keep the browser session alive for your automation:

{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "ttl": 30000
}

We recommend using a residential proxy for better bot detection bypass:

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com",
  "browserWSEndpoint": true,
  "cookies": true,
  "ttl": 30000
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com",
  browserWSEndpoint: true,
  cookies: true,
  ttl: 30000
};

const fetchBrowserWSEndpoint = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const result = await response.json();
  console.log(result);
};

fetchBrowserWSEndpoint();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com",
    "browserWSEndpoint": True,
    "cookies": True,
    "ttl": 30000
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

print(result)

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchBrowserWSEndpoint {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com",
            "browserWSEndpoint": true,
            "cookies": true,
            "ttl": 30000
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

  using System;
  using System.Net.Http;
  using System.Text;
  using System.Threading.Tasks;

  class Program
  {
      static async Task Main(string[] args)
      {
          string TOKEN = "YOUR_API_TOKEN_HERE";
          string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

          string jsonData = @"
          {
              ""url"": ""https://example.com"",
              ""browserWSEndpoint"": true,
              ""cookies"": true,
              ""ttl"": 30000
          }";

          using var client = new HttpClient();
          var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

          try
          {
              var response = await client.PostAsync(url, content);
              response.EnsureSuccessStatusCode();

              var result = await response.Content.ReadAsStringAsync();
              Console.WriteLine("Response: " + result);
          }
          catch (Exception ex)
          {
              Console.WriteLine($"Error: {ex.Message}");
          }
      }
  }

Response:

{
  "browserWSEndpoint": "wss://production-sfo.browserless.io/p/53616c74...646b292c/devtools/browser/102ea3e9-74d7-42c9-a856-1bf254649b9a",
  "cookies": [
    {
      "name": "session_id",
      "value": "XYZ123",
      "domain": "example.com",
      "path": "/",
      "secure": true,
      "httpOnly": true
    }
  ],
  "content": null,
  "screenshot": null
}

Connecting to the Browser

After receiving the response with the browserWSEndpoint and cookies, you can use Puppeteer, Playwright, or another CDP library to connect to the browser instance and inject the cookies to continue your automation:

Puppeteer
Playwright

import puppeteer from "puppeteer-core";

const TOKEN = "YOUR_API_TOKEN_HERE";

const unblock = async (url) => {
  const response = await fetch(
    `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`,
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        url: url,
        browserWSEndpoint: true,
        cookies: true,
        ttl: 30000
      })
    }
  );
  return await response.json();
};

// Get the WebSocket endpoint after bot detection bypass
const { browserWSEndpoint, cookies } = await unblock("https://example.com/");

// Connect to the browser
const browser = await puppeteer.connect({
  browserWSEndpoint: `${browserWSEndpoint}?token=${TOKEN}`
});
const page = (await browser.pages())[0];

// Inject cookies if needed
// await page.setCookie(...cookies);

await page.screenshot({ path: "screenshot.png" });
await browser.close();

Javascript
Python
Java
C#

import playwright from "playwright-core";

const TOKEN = "YOUR_API_TOKEN_HERE";

const unblock = async (url) => {
  const response = await fetch(
    `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`,
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        url: url,
        browserWSEndpoint: true,
        cookies: true,
        ttl: 30000
      })
    }
  );
  return await response.json();
};

// Get the WebSocket endpoint after bot detection bypass
const { browserWSEndpoint, cookies } = await unblock("https://example.com/");
  
// Connect to the browser
const browser = await playwright.chromium.connect(`${browserWSEndpoint}?token=${TOKEN}`);
const context = await browser.newContext();
const page = await context.newPage();

// Inject cookies if needed
// await context.addCookies(cookies);

await page.screenshot({ path: "screenshot.png" });
await browser.close();

from playwright.sync_api import sync_playwright
import requests

TOKEN = "YOUR_API_TOKEN_HERE"

def unblock(url):
    response = requests.post(
        f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential",
        headers={"Content-Type": "application/json"},
        json={
            "url": url,
            "browserWSEndpoint": True,
            "cookies": True,
            "ttl": 30000
        }
    )
    return response.json()

with sync_playwright() as p:
    # Get the WebSocket endpoint after bot detection bypass
    response = unblock("https://example.com/")
    browserWSEndpoint = response["browserWSEndpoint"]
    cookies = response.get("cookies", [])

    # Connect to the browser
    browser = p.chromium.connect(f"{browserWSEndpoint}?token={TOKEN}")
    context = browser.new_context()
    page = context.new_page()

    # Inject cookies if needed
    # context.add_cookies(cookies)

    page.screenshot(path="screenshot.png")
    browser.close()

import com.microsoft.playwright.*;
import org.json.JSONObject;
import java.net.http.*;
import java.net.URI;
import java.nio.file.Paths;

public class PlaywrightUnblock {
    private static final String TOKEN = "YOUR_API_TOKEN_HERE";

    public static JSONObject unblock(String url) throws Exception {
        HttpClient client = HttpClient.newHttpClient();
        String requestBody = new JSONObject()
                .put("url", url)
                .put("browserWSEndpoint", true)
                .put("cookies", true)
                .put("ttl", 30000)
                .toString();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(new URI("https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential"))
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(requestBody))
                .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        return new JSONObject(response.body());
    }

    public static void main(String[] args) throws Exception {
        // Get the WebSocket endpoint after bot detection bypass
        JSONObject response = unblock("https://example.com/");
        String browserWSEndpoint = response.getString("browserWSEndpoint");

        try (Playwright playwright = Playwright.create()) {
            Browser browser = playwright.chromium().connect(browserWSEndpoint + "?token=" + TOKEN);
            BrowserContext context = browser.newContext();
            Page page = context.newPage();

            // Inject cookies if needed
            // context.addCookies(cookiesArray);

            page.screenshot(new Page.ScreenshotOptions().setPath(Paths.get("screenshot.png")));
            browser.close();
        }
    }
}

  using System;
  using System.Net.Http;
  using System.Text;
  using System.Text.Json;
  using System.Threading.Tasks;
  using Microsoft.Playwright;

  class Program
  {
      private const string TOKEN = "YOUR_API_TOKEN_HERE";

      static async Task<JsonElement> UnblockAsync(string url)
      {
          using var client = new HttpClient();
          var opts = new
          {
              url = url,
              browserWSEndpoint = true,
              cookies = true,
              ttl = 30000
          };

          var response = await client.PostAsync(
              $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential",
              new StringContent(JsonSerializer.Serialize(opts), Encoding.UTF8, "application/json")
          );

          response.EnsureSuccessStatusCode();
          return JsonSerializer.Deserialize<JsonElement>(await response.Content.ReadAsStringAsync());
      }

      static async Task Main(string[] args)
      {
          try
          {
              // Get the WebSocket endpoint after bot detection bypass
              var response = await UnblockAsync("https://example.com/");
              string browserWSEndpoint = response.GetProperty("browserWSEndpoint").GetString();

              var playwright = await Playwright.CreateAsync();
              var browser = await playwright.Chromium.ConnectOverCDPAsync($"{browserWSEndpoint}?token={TOKEN}");
              var context = await browser.NewContextAsync();
              var page = await context.NewPageAsync();

              // Inject cookies if needed
              // await context.AddCookiesAsync(cookiesArray);

              await page.ScreenshotAsync(new PageScreenshotOptions { Path = "screenshot.png" });
              await browser.CloseAsync();
          }
          catch (Exception ex)
          {
              Console.WriteLine($"Error: {ex.Message}");
          }
      }
  }

Additional Configuration Options

The /unblock API supports additional configuration options that are shared across multiple REST API endpoints. See the Request Configuration page for detailed documentation on:

Waiting for Things: Wait for events, functions, selectors, or timeouts before returning the response
Navigation Options: Customize navigation behavior using gotoOptions
Rejecting Undesired Requests: Block specific resources using rejectResourceTypes and rejectRequestPattern
Continue on Error: Use bestAttempt to proceed when async events fail or timeout

Quick Start​

Additional Examples and Configuration​

Data Extraction Modes​

Getting a WebSocket Endpoint​

Connecting to the Browser​

Additional Configuration Options​