Scrape Google Shopping Results

Query Google Shopping and pull product names, prices, and links out of the results page. Google aggressively detects bots on this surface, so your approach matters. The REST tab is quick to try but prone to CAPTCHAs; the Frameworks and BQL tabs pair stealth mode with a residential proxy to look like a real user.

Prerequisites

• A Browserless API token from your account dashboard

Steps

REST API

Send the product grid selectors to the /scrape endpoint and get back matched text and position data. No browser code required.

note

Google Shopping runs bot detection that can serve a CAPTCHA or return an empty grid to automated requests. If you get incomplete results here, switch to the BQL tab, which routes through stealth mode and residential proxies to avoid that detection.

cURL

View Full Code on GitHub

1. Send the request

div.sh-dgr__grid-result h4 holds the product title; .a8Pemb holds the price Google renders inside each card:

curl -X POST \
  "https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=board+games&tbm=shop",
    "elements": [
      { "selector": "div.sh-dgr__grid-result h4" },
      { "selector": "div.sh-dgr__grid-result .a8Pemb" }
    ]
  }'

2. Check the output

The response groups results by selector. Each entry in results gives you the element's text, its rendered dimensions, and its position on the page. This is useful if you need to verify the right element matched:

{
  "data": [
    {
      "selector": "div.sh-dgr__grid-result h4",
      "results": [
        {
          "text": "Catan Board Game",
          "html": "Catan Board Game",
          "height": 20,
          "left": 16,
          "top": 220,
          "width": 180
        }
      ]
    },
    {
      "selector": "div.sh-dgr__grid-result .a8Pemb",
      "results": [
        {
          "text": "$44.99",
          "html": "$44.99",
          "height": 16,
          "left": 16,
          "top": 280,
          "width": 60
        }
      ]
    }
  ]
}

JavaScript

View Full Code on GitHub

1. Send the request

const response = await fetch(
  'https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      url: 'https://www.google.com/search?q=board+games&tbm=shop',
      elements: [
        { selector: 'div.sh-dgr__grid-result h4' },
        { selector: 'div.sh-dgr__grid-result .a8Pemb' },
      ],
    }),
  }
);

const { data } = await response.json();
const titles = data[0].results.map((r) => r.text);
const prices = data[1].results.map((r) => r.text);
console.log(titles.map((title, i) => ({ title, price: prices[i] })));

2. Check the output

Run with node scrape-google-shopping.mjs. You'll get an array of objects where each item pairs a product name with its price. The zip-by-index pairing is intentional: the /scrape endpoint returns results grouped by selector, so data[0].results[i] and data[1].results[i] always correspond to the same product card.

Python

View Full Code on GitHub

1. Install dependencies

pip install requests

2. Send the request

import requests

response = requests.post(
    'https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE',
    json={
        'url': 'https://www.google.com/search?q=board+games&tbm=shop',
        'elements': [
            {'selector': 'div.sh-dgr__grid-result h4'},
            {'selector': 'div.sh-dgr__grid-result .a8Pemb'},
        ],
    },
)

data = response.json()['data']
titles = [item['text'] for item in data[0]['results']]
prices = [item['text'] for item in data[1]['results']]
for title, price in zip(titles, prices):
    print(f'{title}: {price}')

3. Check the output

Run with python scrape_google_shopping.py. Each printed line is a product name followed by its price. The zip(titles, prices) pairing is intentional: the /scrape endpoint returns results grouped by selector, so the i-th title and i-th price always come from the same product card.

Java

View Full Code on GitHub

1. Send the request

import java.net.URI;
import java.net.http.*;

String token = "YOUR_API_TOKEN_HERE";
String endpoint = "https://production-sfo.browserless.io/scrape?token=" + token;

String payload = """
    {
      "url": "https://www.google.com/search?q=board+games&tbm=shop",
      "elements": [
        { "selector": "div.sh-dgr__grid-result h4" },
        { "selector": "div.sh-dgr__grid-result .a8Pemb" }
      ]
    }
    """;

HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create(endpoint))
    .header("Content-Type", "application/json")
    .POST(HttpRequest.BodyPublishers.ofString(payload))
    .build();

HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());

2. Check the output

Compile and run the class. The response body is JSON with data[0].results holding product names and data[1].results holding prices.

C#

View Full Code on GitHub

1. Send the request

using System.Net.Http;
using System.Text;
using System.Text.Json;

string token = "YOUR_API_TOKEN_HERE";
string endpoint = $"https://production-sfo.browserless.io/scrape?token={token}";

var payload = new
{
    url = "https://www.google.com/search?q=board+games&tbm=shop",
    elements = new[]
    {
        new { selector = "div.sh-dgr__grid-result h4" },
        new { selector = "div.sh-dgr__grid-result .a8Pemb" },
    },
};

using (HttpClient httpClient = new HttpClient())
{
    var jsonPayload = JsonSerializer.Serialize(payload);
    var content = new StringContent(jsonPayload, Encoding.UTF8, "application/json");
    var response = await httpClient.PostAsync(endpoint, content);
    string responseBody = await response.Content.ReadAsStringAsync();
    Console.WriteLine(responseBody);
}

2. Check the output

Run the program. The response JSON has two data entries: the first holds matched product names, the second holds prices. Each is an array of results objects.

Frameworks

Connect a headless browser via CDP, navigate to the Google Shopping results page, and pull product data directly out of the rendered DOM. We use stealth mode and a US residential proxy here because Google Shopping checks browser fingerprints and IP reputation. A datacenter IP will usually hit a CAPTCHA.

Puppeteer

View Full Code on GitHub

1. Install dependencies

npm install puppeteer-core

2. Connect and scrape

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint:
    'wss://production-sfo.browserless.io/stealth?token=YOUR_API_TOKEN_HERE&proxy=residential&proxyCountry=us',
});

try {
  const page = await browser.newPage();
  await page.goto(
    'https://www.google.com/search?q=board+games&tbm=shop',
    { waitUntil: 'networkidle2' }
  );

  const products = await page.evaluate(() =>
    Array.from(document.querySelectorAll('div.sh-dgr__grid-result')).map((card) => ({
      title: card.querySelector('h4')?.innerText?.trim() ?? '',
      price: card.querySelector('.a8Pemb')?.innerText?.trim() ?? '',
      link: card.querySelector('a')?.href ?? '',
    }))
  );

  console.log(products);
} finally {
  await browser.close();
}

3. Check the output

Run with node scrape-google-shopping.mjs. Each object in the array has title, price, and link. The link is a full Google Shopping product URL you can follow to get merchant details.

Playwright

View Full Code on GitHub

1. Install dependencies

npm install playwright-core

2. Connect and scrape

import { chromium } from 'playwright-core';

const browser = await chromium.connectOverCDP(
  'wss://production-sfo.browserless.io?token=YOUR_API_TOKEN_HERE&stealth&proxy=residential&proxyCountry=us'
);

try {
  const context = browser.contexts()[0];
  const page = await context.newPage();
  await page.goto(
    'https://www.google.com/search?q=board+games&tbm=shop',
    { waitUntil: 'networkidle' }
  );

  const products = await page.evaluate(() =>
    Array.from(document.querySelectorAll('div.sh-dgr__grid-result')).map((card) => ({
      title: card.querySelector('h4')?.innerText?.trim() ?? '',
      price: card.querySelector('.a8Pemb')?.innerText?.trim() ?? '',
      link: card.querySelector('a')?.href ?? '',
    }))
  );

  console.log(products);
} finally {
  await browser.close();
}

3. Check the output

Run with node scrape-google-shopping.mjs. Each object in the array has title, price, and link. The link is a full Google Shopping product URL you can follow to get merchant details.

BQL

View Full Code on GitHub

1. Write the mutation

Navigate to the Google Shopping URL, then use nested mapSelector calls to walk each product card and pull out title, price, and link in one pass. We use wait: true on the title and price selectors because Google Shopping renders those fields via JavaScript after the initial HTML loads. Without it you'd get empty strings:

mutation ScrapeGoogleShopping {
  goto(url: "https://www.google.com/search?q=board+games&tbm=shop", waitUntil: domContentLoaded) {
    status
  }

  results: mapSelector(selector: "div.sh-dgr__grid-result") {
    title: mapSelector(selector: "h4", wait: true) {
      innerText
    }
    price: mapSelector(selector: ".a8Pemb", wait: true) {
      innerText
    }
    link: mapSelector(selector: "a") {
      href: attribute(name: "href") {
        value
      }
    }
  }
}

2. Run it

Paste into the BQL IDE and click Run.

3. Check the output

Each entry in results corresponds to one product card. Title and price are arrays because mapSelector can match multiple child elements. In practice each card has one of each, so you'll read index [0]:

{
  "data": {
    "goto": { "status": 200 },
    "results": [
      {
        "title": [{ "innerText": "Catan Board Game" }],
        "price": [{ "innerText": "$44.99" }],
        "link": [{ "href": { "value": "https://www.google.com/shopping/product/1/specs?q=board+games&prds=..." } }]
      },
      {
        "title": [{ "innerText": "Ticket to Ride Board Game" }],
        "price": [{ "innerText": "$54.99" }],
        "link": [{ "href": { "value": "https://www.google.com/shopping/product/2/specs?q=board+games&prds=..." } }]
      }
    ]
  }
}

Next steps

• Automate Google Search — scrape Google web search results with BQL
• Scrape Structured Data — extract data from other sites using the /scrape endpoint
• Solving Cloudflare Challenges — bypass bot detection on heavily protected targets