/content API

The /content API returns the fully rendered HTML of a page as text/html after running it in a real browser, including the <head> section and all JavaScript-generated content. This API is useful for capturing the full contents of a page that has a lot of JavaScript or other interactivity.

You can check the full Open API schema here.

Quick Start

cURL
Javascript
Python
Java
C#

curl -X POST \
  "https://production-sfo.browserless.io/content?token=YOUR_API_TOKEN_HERE" \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "https://example.com/"
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/content?token=${TOKEN}`;
const headers = {
    "Cache-Control": "no-cache",
    "Content-Type": "application/json"
};
const data = {
    url: "https://example.com/"
};

const fetchContent = async () => {
    const response = await fetch(url, {
        method: 'POST',
        headers: headers,
        body: JSON.stringify(data)
    });

    const content = await response.text();
    console.log(content); 
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/content?token={TOKEN}"
headers = {
    "Cache-Control": "no-cache",
    "Content-Type": "application/json"
}
data = {
    "url": "https://example.com/"
}

response = requests.post(url, headers=headers, json=data)
print(response.text) 

import java.io.*;
import java.net.http.*;
import java.net.URI;

public class FetchContent {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/content?token=" + TOKEN;

        HttpClient client = HttpClient.newHttpClient();

        String jsonData = """
            {
                "url": "https://example.com/"
            }
        """;

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .header("Cache-Control", "no-cache")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

  using System.Text;
  using System.Text.Json;

  class Program
  {
      static async Task Main(string[] args)
      {
          string TOKEN = "YOUR_API_TOKEN_HERE";
          string url = $"https://production-sfo.browserless.io/content?token={TOKEN}";

          var payload = new
          {
              url = "https://example.com/"
          };

          var jsonContent = new StringContent(JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");

          using var httpClient = new HttpClient();
          try
          {
              var request = new HttpRequestMessage(HttpMethod.Post, url)
              {
                  Content = jsonContent
              };

              request.Headers.Add("Cache-Control", "no-cache");

              var response = await httpClient.SendAsync(request);
              response.EnsureSuccessStatusCode();
              var result = await response.Content.ReadAsStringAsync();
              Console.WriteLine("Response: " + result);
          }
          catch (Exception e)
          {
              Console.WriteLine("Error: " + e.Message);
          }
      }
  }

Response

The API returns the fully rendered HTML content as text/html:

<!DOCTYPE html>
<html lang="en">
<head>
  <title>Example Domain</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <style>
    body{
      background:#eee;
      width:60vw;
      margin:15vh auto;
      font-family:system-ui,sans-serif
    }
    h1{font-size:1.5em}
    div{opacity:0.8}
    a:link,a:visited{color:#348}
  </style>
</head>
<body>
  <div>
    <h1>Example Domain</h1>
    <p>This domain is for use in documentation examples without needing permission. Avoid use in operations.</p>
    <p><a href="https://iana.org/domains/example">Learn more</a></p>
  </div>
</body>
</html>

Additional Configuration Options

The /content API supports additional configuration options that are shared across multiple REST API endpoints. See the Request Configuration page for detailed documentation on:

Waiting for Things: Wait for events, functions, selectors, or timeouts before returning the response
Navigation Options: Customize navigation behavior using gotoOptions
Rejecting Undesired Requests: Block specific resources using rejectResourceTypes and rejectRequestPattern
Continue on Error: Use bestAttempt to proceed when async events fail or timeout

Bot Detection Troubleshooting

If you're experiencing issues with the /content API returning no data or partial data, this is typically due to bot detection mechanisms employed by the target website. Websites may use various techniques to detect and block automated browsers, which can result in empty responses or incomplete content.

Recognising failures from Anti Bot Mechanisms

Empty HTML response or minimal content
Partial page content missing key elements
Different content compared to what you see in a regular browser
Blocked requests or access denied messages

Alternative: Unblock API

When encountering bot detection issues, we recommend using the /unblock API as an alternative to the /content API. The /unblock endpoint is specifically designed to bypass bot detection mechanisms and can return HTML content directly in the response.

cURL
Javascript
Python
Java
C#

curl --request POST \
  --url 'https://production-sfo.browserless.io/unblock?token=YOUR_API_TOKEN_HERE&proxy=residential' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com/",
  "content": true
}'

const TOKEN = "YOUR_API_TOKEN_HERE";
const url = `https://production-sfo.browserless.io/unblock?token=${TOKEN}&proxy=residential`;
const headers = {
  "Content-Type": "application/json"
};

const data = {
  url: "https://example.com/",
  content: true
};

const fetchContent = async () => {
  const response = await fetch(url, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  const content = await response.json();
  console.log(content);
};

fetchContent();

import requests

TOKEN = "YOUR_API_TOKEN_HERE"
url = f"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential"
headers = {
    "Content-Type": "application/json"
}

data = {
    "url": "https://example.com/",
    "content": True
}

response = requests.post(url, headers=headers, json=data)

print(response.json())

import java.io.*;
import java.net.URI;
import java.net.http.*;

public class FetchUnblockedContent {
    public static void main(String[] args) {
        String TOKEN = "YOUR_API_TOKEN_HERE";
        String url = "https://production-sfo.browserless.io/unblock?token=" + TOKEN + "&proxy=residential";

        String jsonData = """
        {
            "url": "https://example.com/",
            "content": true
        }
        """;

        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .POST(HttpRequest.BodyPublishers.ofString(jsonData))
            .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            System.out.println("Response: " + response.body());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        string TOKEN = "YOUR_API_TOKEN_HERE";
        string url = $"https://production-sfo.browserless.io/unblock?token={TOKEN}&proxy=residential";

        string jsonData = @"
        {
            ""url"": ""https://example.com/"",
            ""content"": true
        }";

        using var client = new HttpClient();
        var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

        try
        {
            var response = await client.PostAsync(url, content);
            response.EnsureSuccessStatusCode();

            var result = await response.Content.ReadAsStringAsync();
            Console.WriteLine("Response: " + result);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
        }
    }
}

The /unblock API is suitable for more adavanced bot detection bypass use-cases similar to:

Specialized unblocking: Designed specifically to bypass bot detection mechanisms like Datadome and passive CAPTCHAs
Direct content return: Returns HTML content directly in the response when content: true is set
Enhanced success rate: Works best when combined with residential proxies (&proxy=residential)
Simple integration: Provides the same content extraction functionality as the /content API

For more information about the /unblock API and its capabilities, see the /unblock API documentation.

BrowserQL

We recommend using BrowserQL, Browserless' first-class browser automation API, to capture content from any website to be used in complex browser automation tasks.

Quick Start​

Additional Configuration Options​

Bot Detection Troubleshooting​

Recognising failures from Anti Bot Mechanisms​

Alternative: Unblock API​

Quick Start

Additional Configuration Options

Bot Detection Troubleshooting

Recognising failures from Anti Bot Mechanisms

Alternative: Unblock API