Proxies
As of this writing, browserless now comes packaged with a built-in residential proxy for paid cloud-unit accounts. You can sign-up for one here.
Please NOTE that our residential proxy does not currently work with Selenium-based libraries.
In order to effectively use this proxy, you'll need to adjust your code or API calls to let browserless proxy the request for you. For both library connect and REST API calls, the process is the same!
Using puppeteer with our proxy
The following uses our built-in residential proxy, targeting a node in the US:
const puppeteer = require('puppeteer-core');
// Simply add proxy=residential and (optionally) a country
const browserWSEndpoint =
'wss://chrome.browserless.io?token=YOUR-API-KEY&proxy=residential&proxyCountry=us';
(async function job(url = 'https://ipinfo.io/') {
let browser;
try {
browser = await puppeteer.connect({ browserWSEndpoint });
const page = await browser.newPage();
await page.setViewport({ width: 1920, height: 1080 });
await page.goto(url);
await page.screenshot({ path: 'temp.png' });
} catch (e) {
console.log('Error during script:', e.message);
} finally {
browser && browser.close();
}
})();
By default, all requests will go through a random node in the proxy pool. This may not be desireable and can cause other issues. In order to keep your session "sticky" (use the same IP node), add a proxySticky parameter:
'wss://chrome.browserless.io?token=YOUR-API-KEY&proxy=residential&proxyCountry=us&proxySticky';
Using playwright with our proxy
Playwright functions similarly to puppeteer when using the connectOverCDP method. Simply set the variables you want for the proxy, and connect. That's it!
const pw = require('playwright-core');
// Simply add proxy=residential and (optionally) a country
const browserWSEndpoint =
'wss://chrome.browserless.io?token=YOUR-API-KEY&proxy=residential&proxyCountry=us';
(async function job(url = 'https://ipinfo.io/') {
let browser;
try {
browser = await pw.chromium.connectOverCDP({ browserWSEndpoint });
const page = await browser.newPage();
await page.setViewport({ width: 1920, height: 1080 });
await page.goto(url);
await page.screenshot({ path: 'temp.png' });
} catch (e) {
console.log('Error during script:', e.message);
} finally {
browser && browser.close();
}
})();
By default, all requests will go through a random node in the proxy pool. This may not be desireable and can cause other issues. In order to keep your session "sticky" (use the same IP node), add a proxySticky parameter:
'wss://chrome.browserless.io?token=YOUR-API-KEY&proxy=residential&proxyCountry=us&proxySticky';
REST APIs
All of our REST-APIs function the same for proxying: simply add the proxy parameters you care about to the requests' query-string parameters, and you're all done. No need for credentials or otherwise.
Here's an example of collecting content from a United States Proxy:
curl --request POST \
--url 'https://chrome.browserless.io/content?token=YOUR-API-KEY&proxyCountry=us&proxy=residential' \
--header 'Content-Type: application/json' \
--data '{
"url": "https://ipinfo.io/"
}'
And here's a PDF generated document with it as well:
curl --request POST \
--url 'https://chrome.browserless.io/screenshot?token=YOUR-API-KEY&proxyCountry=us&proxy=residential' \
--header 'Content-Type: application/json' \
--data '{
"url": "https://ipinfo.io/"
}'
By default, all requests will go through a random node in the proxy pool. This may not be desireable and can cause other issues. In order to keep your session "sticky" (use the same IP node), add a proxySticky parameter:
'http://chrome.browserless.io/content?token=YOUR-API-KEY&proxy=residential&proxyCountry=us&proxySticky';
All proxy options
Today, we support only residential proxying in our paid cloud-unit plans. In the near future we'll expand this to Dedicated accounts as well as other proxy types. Below is a list of all query parameters and their options:
&proxy=residential: Indicates which proxy type to use. Must be set, and as of today, the only option is residential.&proxyCountry=US: An ISO 3166-1 two-digit country code. Defaults tousor United States.&proxySticky: When present, force all of chrome's network traffic to go through the same IP.
Be sure to refer to this document as we add more options in the future!
Cost
Never worry about topping up or buying more time. When you have a paid cloud-unit plan with us, you're simply charged 6 units per MB of traffic. For instance, a site that is 1MB would equate to using 6 units of cloud-unit plans, or roughly $0.01. Since our technology doesn't interrupt Chrome's requests, you can also instrument chrome (via puppeteer or our REST APIs) to reject requests that are unnecessary, like images and videos. See our other documentation pages for how to do that.
The other cost to consider is time. Using a residential proxy forces Chrome to issue requests through a residential (or consumer computer) proxy, meaning requests might be slower and thus take longer to resolve. You'll likely want to increase your timeout setting 5 - 10x what you might consider normal to allow traffic to flow properly.
Higher tier plans also come with the benefit of less cost per unit, meaning the more units you sign-up for the cheaper the proxying becomes. Our account page shows the amount of units used for proxying to give you an idea of how much is being spent on proxy-related costs.
FAQ
How come I'm getting a ERR_TUNNEL_CONNECTION_FAILED?
There's two reasons this might happen. The first, and most rare, is a bad node or IP that's no longer working. Simply retry your requests and things should work properly.
The second is that some sites are blocked either by legal action. Unfortunately few providers can work around issues like these as they can be legally held responsible. Simply try your request without the proxy, or find a provider you'd like to use as we also allow 3rd party proxies as well.
I'm using Selenium/Webdriver but it's not working.
As of this time we currently don't have support for proxying in Selenium or other webdriver-based integrations (Capybara and more). We apologize for the inconvenience.
I'm using a lot of units on proxying, why is that?
When you launch Chrome with a proxy, it'll use that proxy for every request it makes. Puppeteer and other libraries come with handy network-request rejection APIs that can tell chrome not to make those requests if it's for an image, css, or even JavaScript. To do that is pretty simple:
// Abort image requests as they're big and not necessary most of time.
await page.on('request', (req) => {
if (!req.isInterceptResolutionHandled() && req.resourceType() === 'image') {
return req.abort();
}
});
For most REST-based APIs you can do similarly by adding a payload property:
{
"url": "https://example.com",
"rejectResourceTypes": ["image"]
}
The site is really slow with the proxy, why?
When using residential proxies, Chrome makes many network "hops" in order to route traffic through that IP address. Depending on where you're running your code, and what country you target for proxying, this can mean multiple trips around the globe multiple times for every network request.
Fundamentally you can only do so much to make the speed of light faster! In cases like these where speed is important it's best to use a Dedicated plan to get infrastructure closer to where you intend to route traffic as well as increase timeouts where possible.