Hybrid automation - human in the loop

info

Hybrid automation is only available on our Enterprise plan, if you'd like to know more, get in touch! Full developer docs on hybrid automation here

Hybrid automation allows you to have humans intervene in an automation workflow. This is useful in case a user needs to input their credentials, handle 2FA or simply perform some actions on a website before resuming automation. This can be done with Puppeteer, Playwright and virtually any library that supports CDP connections.

How to stream a remote headless browsers

Browserless communicated with the browser at a CDP layer to return the Browserless.liveURL, which is a fully-qualified URL that doesn't require a token, which means you can share this with the end users. The URL can be opened in a new tab or displayed as an iFrame on your website where they will be able to click, type and interact with the browser.

You can also listen for the Browserless.captchaFound and Browserless.livecomplete events to identify when there's a captcha on the screen and also when a customers has completed their automation and closed the interactive tab.

Here's a sample snippet where you want end users to log in to Gmail with their credentials/2FA.

import puppeteer from 'puppeteer-core';

const login = async () => {
  const browser = await puppeteer.connect({
    browserWSEndpoint: 'wss://chrome.browserless.io?token=YOUR-API-TOKEN',
  });
  const page = await browser.newPage();

  await page.goto('https://www.gmail.com/');
  const cdp = await page.createCDPSession();
  const { liveURL } = await cdp.send('Browserless.liveURL');

  // Send this one-time link to your end-users.
  // This URL doesn't contain an API-token so there's no
  // secrets being leaked by doing so
  console.log(`Shareable Public URL:`, liveURL);

  //This event is fired when a captcha is found on the page.
  await new Promise((resolve) =>
    cdp.on('Browserless.captchaFound', () => {
      console.log('Found a captcha!');
      return resolve();
    }),
  );

  // This event is fired after a user closes the page.
  // Assuming the page is where it's supposed to be, we can
  // proceed with doing further automations
  await new Promise((r) => cdp.on('Browserless.liveComplete', r));

  // Implement your scraping, data collections or further automations here.

  // Don't forget to close!
  browser.close();
};

login().catch((e) => console.log(e));

Reusing the session after login

If you're using the hybrid automation for logging into a platform, you can reutilize the cookies, session and cache on subsequent browser sessions by using the &--user-data-dir flag.

puppeteer.connect({
  browserWSEndpoint: 'wss://chrome.browserless.io/?token=YOUR_API_KEY&--user-data-dir=~/browserless-cache-123',
});