Wait for navigation after a click

Clicking a link or button that triggers a page load requires Promise.all to guarantee the navigation wait is registered before the click fires. This pattern is essential for reliable session management with remote browsers.

Prerequisites

• A Browserless API token from your account dashboard

Steps

AI Agent

Use the Browserless MCP server to wait for navigation after a click from any MCP-compatible AI agent (Claude Desktop, Cursor, Windsurf, ChatGPT, etc.).

1. Connect the MCP server

Send this prompt to your AI agent to install the Browserless MCP server:

Go to https://github.com/browserless/browserless-mcp/blob/main/install.md
and follow the instructions to install the Browserless MCP server
for my client.

2. Navigate after a click

Use browserless_agent. The task requires interaction: clicking an element and waiting for the page to load.

Use the browserless_agent tool to go to
https://scraping-sandbox.netlify.app/products,
click the "Contact Us" link, wait for the new page to load,
then scrape the content

REST API

The /function endpoint runs your script inside the browser process, so the same Promise.all pattern applies. Use it to avoid the race condition even here.

cURL

View Full Code on GitHub

1. Send the request

curl -X POST \
  "https://production-sfo.browserless.io/function?token=YOUR_API_TOKEN_HERE" \
  -H "Content-Type: application/javascript" \
  --data-raw 'module.exports = async ({ page }) => {
    await page.goto("https://scraping-sandbox.netlify.app/products");

    // Register the navigation wait before clicking — if you click first,
    // the page load can complete before waitForNavigation() starts listening.
    await Promise.all([
      page.waitForNavigation({ waitUntil: "domcontentloaded" }),
      page.click("a[href=\"/contact-us\"]"),
    ]);

    const heading = await page.$eval("h1", el => el.textContent.trim());
    return { url: page.url(), heading };
  };'

2. Check the output

{
  "url": "https://scraping-sandbox.netlify.app/contact-us",
  "heading": "Contact Us"
}

JavaScript

View Full Code on GitHub

1. Send the request

const code = `module.exports = async ({ page }) => {
  await page.goto('https://scraping-sandbox.netlify.app/products');

  // Register the navigation wait before clicking — if you click first,
  // the page load can complete before waitForNavigation() starts listening.
  await Promise.all([
    page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
    page.click('a[href="/contact-us"]'),
  ]);

  const heading = await page.$eval('h1', el => el.textContent.trim());
  return { url: page.url(), heading };
};`;

const response = await fetch(
  'https://production-sfo.browserless.io/function?token=YOUR_API_TOKEN_HERE',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/javascript' },
    body: code,
  }
);

const result = await response.json();
console.log(result);

2. Check the output

Run with node navigate-after-click.mjs. You'll see the Contact Us page URL and its h1. This confirms the script landed on the page the link pointed to, not the products page.

{
  "url": "https://scraping-sandbox.netlify.app/contact-us",
  "heading": "Contact Us"
}

Python

View Full Code on GitHub

1. Install dependencies

pip install requests

2. Send the request

import requests

code = """
module.exports = async ({ page }) => {
  await page.goto('https://scraping-sandbox.netlify.app/products');

  // Register the navigation wait before clicking — if you click first,
  // the page load can complete before waitForNavigation() starts listening.
  await Promise.all([
    page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
    page.click('a[href="/contact-us"]'),
  ]);

  const heading = await page.$eval('h1', el => el.textContent.trim());
  return { url: page.url(), heading };
};
"""

response = requests.post(
    'https://production-sfo.browserless.io/function?token=YOUR_API_TOKEN_HERE',
    headers={'Content-Type': 'application/javascript'},
    data=code.encode('utf-8'),
)

print(response.json())

3. Check the output

Run with python navigate_after_click.py. You'll see the Contact Us page URL and its h1. This confirms the script landed on the page the link pointed to, not the products page.

{
  "url": "https://scraping-sandbox.netlify.app/contact-us",
  "heading": "Contact Us"
}

Java

View Full Code on GitHub

1. Send the request

import java.net.URI;
import java.net.http.*;

String token = "YOUR_API_TOKEN_HERE";
String endpoint = "https://production-sfo.browserless.io/function?token=" + token;

String code = """
    module.exports = async ({ page }) => {
      await page.goto('https://scraping-sandbox.netlify.app/products');

      // Register the navigation wait before clicking — if you click first,
      // the page load can complete before waitForNavigation() starts listening.
      await Promise.all([
        page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
        page.click('a[href="/contact-us"]'),
      ]);

      const heading = await page.$eval('h1', el => el.textContent.trim());
      return { url: page.url(), heading };
    };
    """;

HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create(endpoint))
    .header("Content-Type", "application/javascript")
    .POST(HttpRequest.BodyPublishers.ofString(code))
    .build();

HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());

2. Check the output

Run the class. The JSON body confirms navigation completed: url is the Contact Us page, not the products page.

{
  "url": "https://scraping-sandbox.netlify.app/contact-us",
  "heading": "Contact Us"
}

C#

View Full Code on GitHub

1. Send the request

using System.Net.Http;
using System.Text;

string token = "YOUR_API_TOKEN_HERE";
string endpoint = $"https://production-sfo.browserless.io/function?token={token}";

string code = @"module.exports = async ({ page }) => {
  await page.goto('https://scraping-sandbox.netlify.app/products');

  // Register the navigation wait before clicking — if you click first,
  // the page load can complete before waitForNavigation() starts listening.
  await Promise.all([
    page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
    page.click('a[href=""/contact-us""]'),
  ]);

  const heading = await page.$eval('h1', el => el.textContent.trim());
  return { url: page.url(), heading };
};";

using (HttpClient httpClient = new HttpClient())
{
    var content = new StringContent(code, Encoding.UTF8, "application/javascript");
    var response = await httpClient.PostAsync(endpoint, content);
    string result = await response.Content.ReadAsStringAsync();
    Console.WriteLine(result);
}

2. Check the output

Run the program. The JSON body confirms navigation completed: url is the Contact Us page, not the products page.

{
  "url": "https://scraping-sandbox.netlify.app/contact-us",
  "heading": "Contact Us"
}

Frameworks

Direct framework connections give you full control over timing, which means you own the navigation wait. This is where the bug most commonly surfaces. There's no wrapper to catch a missed waitForNavigation.

Puppeteer

View Full Code on GitHub

1. Install dependencies

npm install puppeteer-core

2. Connect and click

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'wss://production-sfo.browserless.io?token=YOUR_API_TOKEN_HERE',
});

try {
  const page = await browser.newPage();
  await page.goto('https://scraping-sandbox.netlify.app/products', { waitUntil: 'domcontentloaded' });

  // Wrong — navigation may finish between these two lines:
  // await page.click('a[href="/contact-us"]');
  // await page.waitForNavigation({ waitUntil: 'domcontentloaded' });

  // Right — Promise.all starts the wait before the click fires:
  await Promise.all([
    page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
    page.click('a[href="/contact-us"]'),
  ]);

  const heading = await page.$eval('h1', el => el.textContent.trim());
  console.log('URL:', page.url());
  console.log('Heading:', heading);
} finally {
  // Always close to release the session even on error.
  await browser.close();
}

3. Check the output

Run with node navigate-after-click.mjs. The console output shows the Contact Us page URL and h1 text. If you drop the Promise.all and see the products page heading instead, that's the race condition in action.

Playwright

View Full Code on GitHub

1. Install dependencies

npm install playwright-core

2. Connect and click

Playwright's page.click() auto-waits for the element to be actionable (visible and enabled), but it does NOT wait for any resulting navigation. That's still your responsibility. Two approaches work here:

• Promise.all with waitForNavigation — use when you don't know the destination URL in advance.
• waitForURL — use when you know the destination URL pattern. Because you click first and then await waitForURL, you skip the Promise.all wrapper entirely. Playwright polls for the URL match rather than listening for a navigation event, so the ordering doesn't matter.

import { chromium } from 'playwright-core';

const browser = await chromium.connect(
  'wss://production-sfo.browserless.io/chromium/playwright?token=YOUR_API_TOKEN_HERE'
);

try {
  const page = await browser.newPage();
  await page.goto('https://scraping-sandbox.netlify.app/products', { waitUntil: 'domcontentloaded' });

  // Approach 1 — works when you don't know the destination URL:
  await Promise.all([
    page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
    page.click('a[href="/contact-us"]'),
  ]);

  // Approach 2 — simpler when you know the destination URL pattern:
  // await page.click('a[href="/contact-us"]');
  // await page.waitForURL('**/contact-us');

  const heading = await page.$eval('h1', el => el.textContent.trim());
  console.log('URL:', page.url());
  console.log('Heading:', heading);
} finally {
  // Always close to release the session even on error.
  await browser.close();
}

3. Check the output

Run with node navigate-after-click.mjs. The console output shows the Contact Us page URL and h1 text. Both approaches correctly waited for the destination page before reading the DOM.

BQL

View Full Code on GitHub

1. Write the mutation

BQL executes actions sequentially, and the click action automatically waits for any resulting navigation to settle before moving to the next action. No Promise.all needed.

mutation NavigateAfterClick {
  goto(url: "https://scraping-sandbox.netlify.app/products", waitUntil: domContentLoaded) {
    status
  }
  clicked: click(selector: "a[href=\"/contact-us\"]") {
    x
    y
  }
  heading: text(selector: "h1")
}

After click, BQL waits for the page to settle before reading h1, so the text value reflects the destination page.

2. Run it

Paste into the BQL IDE and click Run.

3. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "clicked": { "x": 200, "y": 210 },
    "heading": "Contact Us"
  }
}