Version: v2

Quick Start

Welcome to the quick start! Below are some starting quick recipes for the most common and useful use cases.

Quick Recipes

Using our BrowserQL
Bypass bot detectors
Connect with Puppeteer or Playwright
Generate screenshots and PDFs

Advanced Features

Using BrowserQL

BrowserQL is our easy-to-use query-language that comes bundled with its own editor. In order to get started with the editing experience, sign-up for any account and you can download the editor from the Account portal.

To get started you'll need two key pieces of information: the URL for the area you're nearest to and your API key. We have the following locations:

https://production-sfo.browserless.io/ (Based in San Francisco, USA).
https://production-lon.browserless.io/ (Based in London, England).
https://production-ams.browserless.io/ (Based in Amsterdam, Netherlands).

Picking a close location is key to the fastest experience! Once you have these, then you can get started with a simple query like this:

mutation ScrapeHN {
  goto(url: "https://news.ycombinator.com", waitUntil: firstMeaningfulPaint) {
    status
    time
  }

  text {
    text
  }
}

If you want to see more BrowserQL options and recipes, check out our section on BrowserQL.

Bypass active and passive bot-protection

We refer to bot detection as either active or passive:

Passive detectors will check a visitor’s fingerprints and let you through if they think you’re human, such as on most ecommerce sites.
Active detectors will challenge every visitor with a captcha, such as during a sign up or form submission.

We have different solutions for these two detector types, which are /unblock and automated captcha solving.

Avoid passive detectors with BrowserQL

Our BrowserQL can avoid many bot detectors, especially when used with our residential proxy. It can be used in many ways:

Grab the HTML, text or screenshot of a page or specific elements
Create an unlocked endpoint to automate further with Puppeteer or Playwright

Grab rendered HTML with a cURL request

If you simply want the HTML for scraping, you can simply query for the HTML inside the query itself. BrowserQL will get past the detectors, render any dynamic content in our browsers, then return the HTML.

Here's an example with proxies enabled and set to USA. Enter your API token into the query parameters, and replace example.com with your desired site.

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/chromium/bql?token=YOUR_API_TOKEN_HERE&proxy=residential&proxySticky=true&proxyCountry=us' \
  --header 'Content-Type: application/json' \
  --data '{
  "query": "mutation GetContent { goto(url: \"https://example.com\", waitUntil: firstMeaningfulPaint) { status time } html { html } }"
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/chromium/bql?token=${TOKEN}&proxy=residential&proxySticky=true&proxyCountry=us`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  query: 'mutation GetContent { goto(url: "https://example.com", waitUntil: firstMeaningfulPaint) { status time } html { html } }'
};

const fetchContent = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const result = await response.json();
  console.log("Response:", result);
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/chromium/bql?token={TOKEN}&proxy=residential&proxySticky=true&proxyCountry=us"
headers = {
    "Content-Type": "application/json"
}

data = {
    "query": 'mutation GetContent { goto(url: "https://example.com", waitUntil: firstMeaningfulPaint) { status time } html { html } }'
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

print("Response:", result)

import java.net.http.*;
import java.net.URI;
import com.google.gson.*;

public class BrowserlessExample {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/chromium/bql?token=" + TOKEN + "&proxy=residential&proxySticky=true&proxyCountry=us";
        
        HttpClient client = HttpClient.newHttpClient();

        String query = """
            mutation GetContent {
                goto(url: "https://example.com", waitUntil: firstMeaningfulPaint) {
                    status
                    time
                }
                html {
                    html
                }
            }
        """;

        String jsonData = new Gson().toJson(Map.of("query", query));

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;

class Program {
    static async Task Main(string[] args) {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/chromium/bql?token={TOKEN}&proxy=residential&proxySticky=true&proxyCountry=us";

        var query = new {
            query = @"
                mutation GetContent {
                    goto(url: ""https://example.com"", waitUntil: firstMeaningfulPaint) {
                        status
                        time
                    }
                    html {
                        html
                    }
                }
            "
        };

        var jsonContent = new StringContent(JsonSerializer.Serialize(query), Encoding.UTF8, "application/json");

        using var httpClient = new HttpClient();
        try {
            var response = await httpClient.PostAsync(url, jsonContent);
            response.EnsureSuccessStatusCode();
            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        } catch (Exception e) {
            Console.WriteLine("Error: " + e.Message);
        }
    }
}

Which will result in a response containing the unblocked page's HTML:

{
  "data": {
    "goto": {
      "status": 200,
      "time": 957
    },
    "html": {
      "html": "<!DOCTYPE html>...</html>"
    }
  }
}

You can then process this HTML with libraries such as Scrapy or Beautiful Soup.

Unblock and connect with Puppeteer or Playwright

The API can access a site and get approval from the bot detectors, then return return a WebSocket endpoint (browserWSEndpoint) for you to re-connect to for further automation.

First, have your query visit the site (or even fill in form elements if you wish). Then ask for the reconnection endpoints back to connect your library. Here, we use a timeout of 30,000 milliseconds, but you're free to increase or decrease this depending on your use-case. The full query by itself will look like:

mutation Reconnect($url: String!) {
  goto(url: $url, waitUntil: networkIdle) {
    status
  }
  reconnect(timeout: 30000) {
    browserWSEndpoint
  }
}

Here's the full example. Be sure to insert your token and the website you wish to visit as well:

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/chrome/bql?token=YOUR_API_TOKEN_HERE' \
  --header 'Content-Type: application/json' \
  --data '{
  "query": "mutation Reconnect($url: String!) { goto(url: $url, waitUntil: networkIdle) { status } reconnect(timeout: 30000) { browserWSEndpoint } }",
  "variables": { "url": "https://example.com/" }
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/chrome/bql?token=${TOKEN}`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  query: `mutation Reconnect($url: String!) { 
    goto(url: $url, waitUntil: networkIdle) { 
      status 
    } 
    reconnect(timeout: 30000) { 
      browserWSEndpoint 
    } 
  }`,
  variables: { url: "https://example.com/" }
};

const reconnectBrowser = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const result = await response.json();
  console.log("Response:", result);
};

reconnectBrowser();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/chrome/bql?token={TOKEN}"
headers = {
    "Content-Type": "application/json"
}

data = {
    "query": """mutation Reconnect($url: String!) { 
        goto(url: $url, waitUntil: networkIdle) { 
            status 
        } 
        reconnect(timeout: 30000) { 
            browserWSEndpoint 
        } 
    }""",
    "variables": { "url": "https://example.com/" }
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

print("Response:", result)

import java.net.http.*;
import java.net.URI;
import com.google.gson.*;

public class BrowserlessReconnect {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/chrome/bql?token=" + TOKEN;

        HttpClient client = HttpClient.newHttpClient();

        String query = """
            mutation Reconnect($url: String!) {
                goto(url: $url, waitUntil: networkIdle) {
                    status
                }
                reconnect(timeout: 30000) {
                    browserWSEndpoint
                }
            }
        """;

        String jsonData = new Gson().toJson(Map.of("query", query, "variables", Map.of("url", "https://example.com/")));

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;

class Program {
    static async Task Main(string[] args) {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/chrome/bql?token={TOKEN}";

        var query = new {
            query = @"
                mutation Reconnect($url: String!) {
                    goto(url: $url, waitUntil: networkIdle) {
                        status
                    }
                    reconnect(timeout: 30000) {
                        browserWSEndpoint
                    }
                }
            ",
            variables = new { url = "https://example.com/" }
        };

        var jsonContent = new StringContent(JsonSerializer.Serialize(query), Encoding.UTF8, "application/json");

        using var httpClient = new HttpClient();
        try {
            var response = await httpClient.PostAsync(url, jsonContent);
            response.EnsureSuccessStatusCode();
            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        } catch (Exception e) {
            Console.WriteLine("Error: " + e.Message);
        }
    }
}

Which will return a JSON response like this:

{
  "data": {
    "goto": {
      "status": 200
    },
    "reconnect": {
      "browserWSEndpoint": "wss://production-sfo.browserless.io/e/53616c7465645f5f91..."
    }
  }
}

After receiving the response with the browserWSEndpoint and cookies, you can use Puppeteer, Playwright or another CDP library to connect to the browser and continue your scraping process:

Puppeteer
Playwright

import puppeteer from "puppeteer-core";

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = "https://browserless.io/"

const unblock = async (url) => {
  const opts = {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      "query": "mutation Reconnect($url: String!) { goto(url: $url, waitUntil: networkIdle) { status } reconnect(timeout: 30000) { browserWSEndpoint } }",
      "variables": { url }
    }),
  };

  const response = await fetch(
    `https://production-sfo.browserless.io/chromium/bql?token=${TOKEN}`,
    opts,
  );

  return await response.json();
};

// Reconnect
const { data } = await unblock(url);
const browser = await puppeteer.connect({
  browserWSEndpoint: data.reconnect.browserWSEndpoint + `?token=${TOKEN}`,
});
const pages = await browser.pages();
const page = pages.find((p) => p.url.includes(url));

await page.screenshot({ path: `screenshot-${Date.now()}.png` });
await browser.close();

Javascript
Python
Java
C#

import playwright from "playwright-core";

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = "https://browserless.io/"

const unblock = async (url) => {
  const opts = {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      "query": "mutation Reconnect($url: String!) { goto(url: $url, waitUntil: networkIdle) { status } reconnect(timeout: 30000) { browserWSEndpoint } }",
      "variables": { url }
    }),
  };

  const response = await fetch(
    `https://production-sfo.browserless.io/chromium/bql?token=${TOKEN}`,
    opts,
  );

  return await response.json();
};

// Reconnect
const { data } = await unblock(url);
  
const browser = await playwright.chromium.connect(`${data.reconnect.browserWSEndpoint}?token=${TOKEN}`);
const context = await browser.newContext();

const page = await context.newPage();

await page.screenshot({ path: `screenshot-${Date.now()}.png` });
await browser.close();

import asyncio
import requests
from playwright.async_api import async_playwright

TOKEN = "YOUR_API_TOKEN_HERE"
URL = "https://browserless.io/"

async def unblock(url):
    opts = {
        "query": """
            mutation Reconnect($url: String!) {
                goto(url: $url, waitUntil: networkIdle) { status }
                reconnect(timeout: 30000) { browserWSEndpoint }
            }
        """,
        "variables": {"url": url},
    }

    response = requests.post(
        f"https://production-sfo.browserless.io/chromium/bql?token={TOKEN}",
        json=opts,
        headers={"Content-Type": "application/json"},
    )

    response.raise_for_status()
    return response.json()

async def main():
    data = unblock(URL)["data"]

    async with async_playwright() as p:
        browser = await p.chromium.connect(data["reconnect"]["browserWSEndpoint"] + f"?token={TOKEN}")
        context = await browser.new_context()
        page = await context.new_page()

        await page.screenshot(path=f"screenshot-{int(asyncio.time())}.png")
        await browser.close()

asyncio.run(main())

import com.google.gson.*;
import com.microsoft.playwright.*;
import java.net.http.*;
import java.net.URI;
import java.util.Map;

public class PlaywrightExample {
    private static final String TOKEN = "YOUR_API_TOKEN_HERE";
    private static final String URL = "https://browserless.io/";

    private static String getBrowserWSEndpoint() throws Exception {
        HttpClient client = HttpClient.newHttpClient();

        String query = """
            mutation Reconnect($url: String!) {
                goto(url: $url, waitUntil: networkIdle) { status }
                reconnect(timeout: 30000) { browserWSEndpoint }
            }
        """;

        Map<String, Object> payload = Map.of(
            "query", query,
            "variables", Map.of("url", URL)
        );

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://production-sfo.browserless.io/chromium/bql?token=" + TOKEN))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(new Gson().toJson(payload)))
            .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        if (response.statusCode() != 200) {
            throw new Exception("Failed to fetch endpoint: " + response.body());
        }

        JsonObject data = JsonParser.parseString(response.body()).getAsJsonObject().getAsJsonObject("data");
        return data.getAsJsonObject("reconnect").get("browserWSEndpoint").getAsString();
    }

    public static void main(String[] args) {
        try (Playwright playwright = Playwright.create()) {
            String wsEndpoint = getBrowserWSEndpoint() + "?token=" + TOKEN;
            Browser browser = playwright.chromium().connect(wsEndpoint);

            BrowserContext context = browser.newContext();
            Page page = context.newPage();

            page.screenshot(new Page.ScreenshotOptions().setPath("screenshot-" + System.currentTimeMillis() + ".png"));

            browser.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using Microsoft.Playwright;

class Program
{
    private const string TOKEN = "YOUR_API_TOKEN_HERE";
    private const string URL = "https://browserless.io/";

    static async Task<string> GetBrowserWSEndpoint()
    {
        using var httpClient = new HttpClient();
        var payload = new
        {
            query = @"
                mutation Reconnect($url: String!) {
                    goto(url: $url, waitUntil: networkIdle) { status }
                    reconnect(timeout: 30000) { browserWSEndpoint }
                }
            ",
            variables = new { url = URL }
        };

        var content = new StringContent(JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");
        var response = await httpClient.PostAsync($"https://production-sfo.browserless.io/chromium/bql?token={TOKEN}", content);
        response.EnsureSuccessStatusCode();

        var responseBody = await response.Content.ReadAsStringAsync();
        var json = JsonSerializer.Deserialize<JsonElement>(responseBody);

        return json.GetProperty("data").GetProperty("reconnect").GetProperty("browserWSEndpoint").GetString();
    }

    static async Task Main(string[] args)
    {
        var wsEndpoint = await GetBrowserWSEndpoint() + $"?token={TOKEN}";

        var playwright = await Playwright.CreateAsync();
        var browser = await playwright.Chromium.ConnectAsync(wsEndpoint);

        var context = await browser.NewContextAsync();
        var page = await context.NewPageAsync();

        await page.ScreenshotAsync(new PageScreenshotOptions { Path = $"screenshot-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}.png" });

        await browser.CloseAsync();
    }
}

Solve captchas from active detectors

Browserless directly interacts with the Chrome Devtools Protocol for features such as solving captchas. These CDP-based features require a Chrome session, so let's start one and look for a captcha.

Here's how the full script would look like in Puppeteer:

Puppeteer
Playwright

import puppeteer from "puppeteer-core";

const TOKEN = "YOUR_API_TOKEN_HERE";
const browserWSEndpoint = `wss://production-sfo.browserless.io/chromium?token=${TOKEN}&timeout=300000`;

try {
  const browser = await puppeteer.connect({ browserWSEndpoint });

  const page = await browser.newPage();
  const cdp = await page.createCDPSession();

  await page.goto("https://www.google.com/recaptcha/api2/demo", {
    waitUntil: "networkidle0",
  });

  await new Promise((resolve) => cdp.on("Browserless.captchaFound", resolve));
  console.log("Captcha found!");

  const { solved, error } = await cdp.send("Browserless.solveCaptcha");
  console.log({ solved, error });

  // Continue...
  await page.click("#recaptcha-demo-submit");
  await browser.close();
} catch (e) {
  console.error("There was a big error :(", e);
  process.exit(1);
}

Javascript
Python
Java
C#

import playwright from "playwright-core";

const waitForCaptcha = (cdpSession) => {
  return new Promise((resolve) =>
    cdpSession.on("Browserless.captchaFound", resolve)
  );
};
const pwEndpoint = `wss://production-sfo.browserless.io/chromium?token=YOUR_API_TOKEN_HERE`;

try {
  const browser = await playwright.chromium.connectOverCDP(pwEndpoint);
  // 👇 Queue we're re-using the existing context and page
  const context = browser.contexts()[0];
  const page = context.pages()[0];

  await page.goto("https://www.google.com/recaptcha/api2/demo", {
    waitUntil: "networkidle0",
  });

  const cdp = await page.context().newCDPSession(page);
  await waitForCaptcha(cdp);
  console.log("Captcha found!");

  const { solved, error } = await cdp.send("Browserless.solveCaptcha");

  console.log({ solved, error });

  // Continue...
  await page.click("#recaptcha-demo-submit");
  await browser.close();
} catch (e) {
  console.error("There was a big error :(", e);
  process.exit(1);
}

import asyncio
from playwright.async_api import async_playwright

async def wait_for_captcha(cdp_session):
    # Wait for the "Browserless.captchaFound" event
    future = asyncio.Future()
    
    def handle_captcha_found(event):
        print("Captcha found!")
        future.set_result(event)
    
    cdp_session.on("Browserless.captchaFound", handle_captcha_found)
    return await future

async def main():
    pw_endpoint = "wss://production-sfo.browserless.io/chromium?token=YOUR_API_TOKEN_HERE"
    async with async_playwright() as p:
        try:
            # Connect to the browser
            browser = await p.chromium.connect_over_cdp(pw_endpoint)

            # Use the first context and page
            context = browser.contexts[0]
            page = context.pages[0]

            # Navigate to the captcha demo page
            await page.goto("https://www.google.com/recaptcha/api2/demo", wait_until="networkidle")

            # Create a CDP session
            cdp = await page.context.new_cdp_session(page)

            # Wait for captcha to be found
            await wait_for_captcha(cdp)

            # Solve the captcha
            result = await cdp.send("Browserless.solveCaptcha")
            solved, error = result.get("solved"), result.get("error")
            print({"solved": solved, "error": error})

            # Continue after solving captcha
            await page.click("#recaptcha-demo-submit")
            await browser.close()
        except Exception as e:
            print("There was a big error :(", e)

asyncio.run(main())

import com.microsoft.playwright.*;
import com.microsoft.playwright.options.*;

import java.util.*;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;

public class PlaywrightCaptchaExample {
    public static void main(String[] args) {
        String pwEndpoint = "wss://production-sfo.browserless.io/chromium?token=YOUR_API_TOKEN_HERE";

        try (Playwright playwright = Playwright.create()) {
            Browser browser = playwright.chromium().connectOverCDP(pwEndpoint);

            // Reuse the existing context and page
            BrowserContext context = browser.contexts().get(0);
            Page page = context.pages().get(0);

            // Navigate to the captcha demo page
            page.navigate("https://www.google.com/recaptcha/api2/demo", new Page.NavigateOptions()
                .setWaitUntil(LoadState.NETWORKIDLE));

            // Create a CDP session
            CDPSession cdp = page.context().newCDPSession(page);

            // Wait for captcha to be found
            CompletableFuture<Void> captchaFound = new CompletableFuture<>();
            cdp.addListener("Browserless.captchaFound", event -> {
                System.out.println("Captcha found!");
                captchaFound.complete(null);
            });

            captchaFound.get(); // Wait for the event

            // Solve the captcha
            Map<String, Object> result = cdp.send("Browserless.solveCaptcha");
            System.out.println("Result: " + result);

            // Continue after solving captcha
            page.click("#recaptcha-demo-submit");
            browser.close();
        } catch (ExecutionException | InterruptedException e) {
            System.err.println("There was a big error :(" + e.getMessage());
            e.printStackTrace();
        }
    }
}

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Threading.Tasks;
using Microsoft.Playwright;

class Program
{
    static async Task Main(string[] args)
    {
        var pwEndpoint = "wss://production-sfo.browserless.io/chromium?token=YOUR_API_TOKEN_HERE";

        try
        {
            var playwright = await Playwright.CreateAsync();
            var browser = await playwright.Chromium.ConnectOverCDPAsync(pwEndpoint);

            // Use the first context and page
            var context = browser.Contexts[0];
            var page = context.Pages[0];

            // Navigate to the captcha demo page
            await page.GotoAsync("https://www.google.com/recaptcha/api2/demo", new PageGotoOptions
            {
                WaitUntil = WaitUntilState.NetworkIdle
            });

            // Create a CDP session
            var cdp = await page.Context.NewCDPSessionAsync(page);

            // Wait for captcha to be found
            var captchaFound = new TaskCompletionSource<bool>();

            await cdp.SendAsync("Network.enable");

            Console.WriteLine("Searching for Captcha...");
            while (true)
            {
                var frames = page.Frames;
                var captchaFrame = frames.FirstOrDefault(f => f.Url.Contains("recaptcha"));
                if (captchaFrame != null)
                {
                    Console.WriteLine("Captcha Found!");
                    captchaFound.TrySetResult(true);
                    break;
                }
                
                await Task.Delay(1000);
            }

            await captchaFound.Task;

            // Solve the captcha
            var result = await cdp.SendAsync("Browserless.solveCaptcha", new Dictionary<string, object>());
            if (result is JsonElement element)
            {
                var solved = element.GetProperty("solved").GetBoolean();
                var error = element.GetProperty("error").GetString();
                Console.WriteLine($"Solved: {solved}, Error: {error}");
            }
            else
            {
                Console.WriteLine("Response format is invalid.");
            }


            // Continue after solving captcha
            await page.ClickAsync("#recaptcha-demo-submit");
            await browser.CloseAsync();
        }
        catch (Exception e)
        {
            Console.WriteLine("There was a big error :(");
            Console.WriteLine(e.Message);
        }
    }
}

For more information, you read more about it here or refer to our API reference.

Connect using Puppeteer or Playwright

Whether you're looking to get started, or already have an established codebase, browserless aims to make the transition as easy as possible. Puppeteer and Playwright are exceedingly convenient for this use case, since you can start using Browserless by changing just one line.

Via `Puppeteer.connect()`

To go from running Chrome locally to using Browserless, simply use our endpoint. You can also our use proxy with a couple of query parameters.

import puppeteer from "puppeteer-core";

// Connecting to Chrome locally
const browser = await puppeteer.launch();

// Connecting to Browserless and using a proxy
const browser = await puppeteer.connect({
  browserWSEndpoint: `https://production-sfo.browserless.io/?token=${TOKEN}&proxy=residential`,
});

After that your code should remain exactly the same. You can use launch flags, proxies or other details to customize the behavior.

Via `Playwright.BrowserType.connect()`

We support all Playwright protocols, and, just like with Puppeteer, you can easily switch to Browserless. The standard connect method uses playwright's built-in browser-server to handle the connection.

To connect to Browserless using Chrome, WebKit or Firefox, just make sure that the connection string matches the browser:

Javascript
Python
Java
C#

import playwright from "playwright";

// Connecting to Firefox locally
const browser = await playwright.firefox.launch();

// Connecting to Firefox via Browserless and using a proxy
const browser = await playwright.firefox.connect(`https://production-sfo.browserless.io/firefox/playwright?token=${TOKEN}&proxy=residential`);

from playwright.sync_api import sync_playwright

# Connecting to Firefox locally
with sync_playwright() as p:
    browser = p.firefox.launch()

# Connecting to Firefox via Browserless with a proxy
with sync_playwright() as p:
    browser = p.firefox.connect_over_cdp(f"https://production-sfo.browserless.io/firefox/playwright?token={TOKEN}&proxy=residential")

package org.example;

import com.microsoft.playwright.*;
import java.nio.file.Paths;

public class Main {
  public static void main(String[] args) {
    // Connecting to Firefox locally
    Browser browserLocal = playwright.firefox().launch();

    // Connecting to Firefox via Browserless and using a proxy
    String wsEndpoint = String.format(
      "wss://production-sfo.browserless.io/firefox/playwright?token=%s&proxy=residential",
      TOKEN
    );
    BrowserType.ConnectOptions connectOptions = new BrowserType.ConnectOptions();
    connectOptions.setWsEndpoint(wsEndpoint);
  }
}

using System;
using System.Threading.Tasks;
using Microsoft.Playwright;

namespace PlaywrightExample
{
    class Program
    {
        public static async Task Main(string[] args)
        {
            // Connecting to Firefox locally
            using var playwright = await Playwright.CreateAsync();
            var browserLocal = await playwright.Firefox.LaunchAsync();

            // Connecting to Firefox via Browserless and using a proxy
            using var playwright = await Playwright.CreateAsync();
            string wsEndpoint = $"wss://production-sfo.browserless.io/firefox/playwright?token={TOKEN}&proxy=residential";
            var browserRemote = await playwright.Firefox.ConnectAsync(wsEndpoint);
        }
    }
}

As with Puppeteer, you can use launch flags, proxies or other details to customize the behavior. Check out the Playwright docs for instructions for each supported language.

You can read more about our integrations with Puppeteer, Playwright, Scrapy and other libraries.

Generate screenshots and PDFs

All of our HTTP APIs can be used with a JSON or cURL request, which come with various options and properties. For instance, this is how you generate a PDF.

JSON body

options are the options available via puppeteer's Page.pdf() method. Refer to our Open API documentation for details.

// JSON body
{
  "url": "https://example.com/",
  "options": {
    "displayHeaderFooter": true,
    "printBackground": false,
    "format": "A0"
    // Queue the lack of a `path` parameter
  }
}

curl -X POST \
  https://production-sfo.browserless.io/pdf?token=YOUR_API_TOKEN_HERE \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "https://example.com/",
  "options": {
    "displayHeaderFooter": true,
    "printBackground": false,
    "format": "A0"
  }
}'

import { writeFile } from 'fs/promises';

const TOKEN = 'YOUR_API_TOKEN_HERE';
const url = `https://production-sfo.browserless.io/pdf?token=${TOKEN}`;
const headers = {
  'Cache-Control': 'no-cache',
  'Content-Type': 'application/json'
};
const data = {
  url: "https://example.com/",
  options: {
    displayHeaderFooter: true,
    printBackground: false,
    format: "A0"
  }
};

const response = await fetch(url, {
  method: 'POST',
  headers: headers,
  body: JSON.stringify(data)
});

const buffer = await response.arrayBuffer();
await writeFile("output.pdf", Buffer.from(buffer));
console.log("PDF saved as output.pdf");

import requests

TOKEN = 'YOUR_API_TOKEN_HERE'
url = f"https://production-sfo.browserless.io/pdf?token={TOKEN}"
headers = {
    "Cache-Control": "no-cache",
    "Content-Type": "application/json"
}
data = {
    "url": "https://example.com/",
    "options": {
        "displayHeaderFooter": True,
        "printBackground": False,
        "format": "A0"
    }
}

response = requests.post(url, headers=headers, json=data)

with open("output.pdf", "wb") as f:
    f.write(response.content)
print("PDF saved as output.pdf")

import java.net.http.*;
import java.net.URI;
import com.google.gson.*;

public class BrowserlessPDF {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/pdf?token=" + TOKEN;

        HttpClient client = HttpClient.newHttpClient();

        String jsonData = new Gson().toJson(Map.of(
            "url", "https://example.com/",
            "options", Map.of(
                "displayHeaderFooter", true,
                "printBackground", false,
                "format", "A0"
            )
        ));

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .header("Cache-Control", "no-cache")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System.Text;
using System.Text.Json;

class Program
{
    static async Task Main(string[] args)
    {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/pdf?token={TOKEN}";

        var payload = new
        {
            url = "https://example.com/",
            options = new
            {
                displayHeaderFooter = true,
                printBackground = false,
                format = "A0"
            }
        };

        var jsonContent = new StringContent(JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");

        using var httpClient = new HttpClient();
        try
        {
            var request = new HttpRequestMessage(HttpMethod.Post, url)
            {
                Content = jsonContent
            };

            request.Headers.Add("Cache-Control", "no-cache");

            var response = await httpClient.SendAsync(request);
            response.EnsureSuccessStatusCode();
            
            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: " + e.Message);
        }
    }
}

Browserless offers ways to load additional stylesheets and script tags to the page as well. This gives you full control and allows you to override page elements to suit your needs.

For more information you can check out /pdf API, our /screenshot API or our Lighthouse Tests API. Or if you need to avoid bot detection, check out BrowserQL.

Quick Recipes​

Advanced Features​

Using BrowserQL​

Bypass active and passive bot-protection​

Avoid passive detectors with BrowserQL​

Grab rendered HTML with a cURL request​

Unblock and connect with Puppeteer or Playwright​

Solve captchas from active detectors​

Connect using Puppeteer or Playwright​

Via Puppeteer.connect()​

Via Playwright.BrowserType.connect()​

Generate screenshots and PDFs​