Scrape Walmart product listings

Search Walmart and extract product names, prices, ratings, and review counts.

Prerequisites

• A Browserless API token from your account dashboard

Steps

Selector stability

Walmart updates its markup regularly. If selectors stop returning results, open the page in browser DevTools and inspect the current DOM structure to update them.

AI Agent

Use the Browserless MCP server to scrape product listings from Walmart from any MCP-compatible AI agent (Claude Desktop, Cursor, Windsurf, ChatGPT, etc.).

1. Connect the MCP server

Send this prompt to your AI agent to install the Browserless MCP server:

Go to https://github.com/browserless/browserless-mcp/blob/main/install.md
and follow the instructions to install the Browserless MCP server
for my client.

2. Scrape Walmart

Use browserless_smartscraper. Walmart has aggressive bot detection and smartscraper handles it automatically.

Use the browserless_smartscraper tool to scrape product listings
from https://www.walmart.com/search?q=laptop
and return the results as markdown

REST API

Send the BQL mutation over HTTP to the stealth endpoint. No browser library or BQL IDE required.

cURL

1. Send the request

curl -X POST \
  "https://production-sfo.browserless.io/stealth/bql?token=YOUR_API_TOKEN_HERE&proxy=residential&proxyCountry=us" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "mutation ScrapeWalmart { goto(url: \"https://www.walmart.com/search?q=iphone\", waitUntil: networkIdle) { status } waitForTimeout(time: 3000) { time } products: mapSelector(selector: \"[data-item-id]\") { name: mapSelector(selector: \"[itemprop='\''name'\'']\") { innerText } price: mapSelector(selector: \"[itemprop='\''price'\'']\") { innerText } rating: mapSelector(selector: \"span[itemprop='\''ratingValue'\'']\") { innerText } reviewCount: mapSelector(selector: \"[itemprop='\''ratingCount'\'']\") { innerText } link: mapSelector(selector: \"a[href*='\''\/ip\/'\'']\") { href: attribute(name: \"href\") { value } } } }",
    "variables": {},
    "operationName": "ScrapeWalmart"
  }'

2. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

JavaScript

1. Send the request

const query = `mutation ScrapeWalmart {
  goto(url: "https://www.walmart.com/search?q=iphone", waitUntil: networkIdle) {
    status
  }
  waitForTimeout(time: 3000) {
    time
  }
  products: mapSelector(selector: "[data-item-id]") {
    name: mapSelector(selector: "[itemprop='name']") {
      innerText
    }
    price: mapSelector(selector: "[itemprop='price']") {
      innerText
    }
    rating: mapSelector(selector: "span[itemprop='ratingValue']") {
      innerText
    }
    reviewCount: mapSelector(selector: "[itemprop='ratingCount']") {
      innerText
    }
    link: mapSelector(selector: "a[href*='/ip/']") {
      href: attribute(name: "href") {
        value
      }
    }
  }
}`;

const response = await fetch(
  'https://production-sfo.browserless.io/stealth/bql?token=YOUR_API_TOKEN_HERE&proxy=residential&proxyCountry=us',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ query, variables: {}, operationName: 'ScrapeWalmart' }),
  }
);

const { data } = await response.json();
const products = data.products.map((p) => ({
  name: p.name?.[0]?.innerText ?? '',
  price: p.price?.[0]?.innerText ?? '',
  rating: p.rating?.[0]?.innerText ?? '',
  reviewCount: p.reviewCount?.[0]?.innerText ?? '',
  link: p.link?.[0]?.href?.value ?? '',
}));
console.log(JSON.stringify(products, null, 2));

2. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

Python

1. Install dependencies

pip install requests

2. Send the request

import requests

query = """
mutation ScrapeWalmart {
  goto(url: "https://www.walmart.com/search?q=iphone", waitUntil: networkIdle) {
    status
  }
  waitForTimeout(time: 3000) {
    time
  }
  products: mapSelector(selector: "[data-item-id]") {
    name: mapSelector(selector: "[itemprop='name']") {
      innerText
    }
    price: mapSelector(selector: "[itemprop='price']") {
      innerText
    }
    rating: mapSelector(selector: "span[itemprop='ratingValue']") {
      innerText
    }
    reviewCount: mapSelector(selector: "[itemprop='ratingCount']") {
      innerText
    }
    link: mapSelector(selector: "a[href*='/ip/']") {
      href: attribute(name: "href") {
        value
      }
    }
  }
}
"""

response = requests.post(
    'https://production-sfo.browserless.io/stealth/bql',
    params={
        'token': 'YOUR_API_TOKEN_HERE',
        'proxy': 'residential',
        'proxyCountry': 'us',
    },
    json={'query': query, 'variables': {}, 'operationName': 'ScrapeWalmart'},
)

data = response.json()['data']
for product in data['products']:
    name = product['name'][0]['innerText'] if product['name'] else ''
    price = product['price'][0]['innerText'] if product['price'] else ''
    rating = product['rating'][0]['innerText'] if product['rating'] else ''
    print(f'{name} — {price} — ★{rating}')

3. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

Java

1. Send the request

import java.net.URI;
import java.net.http.*;

String token = "YOUR_API_TOKEN_HERE";
String endpoint = "https://production-sfo.browserless.io/stealth/bql?token="
    + token + "&proxy=residential&proxyCountry=us";

String query = """
    mutation ScrapeWalmart {
      goto(url: "https://www.walmart.com/search?q=iphone", waitUntil: networkIdle) { status }
      waitForTimeout(time: 3000) { time }
      products: mapSelector(selector: "[data-item-id]") {
        name: mapSelector(selector: "[itemprop='name']") { innerText }
        price: mapSelector(selector: "[itemprop='price']") { innerText }
        rating: mapSelector(selector: "span[itemprop='ratingValue']") { innerText }
        reviewCount: mapSelector(selector: "[itemprop='ratingCount']") { innerText }
        link: mapSelector(selector: "a[href*='/ip/']") { href: attribute(name: "href") { value } }
      }
    }
    """;

String payload = "{\"query\": " + com.google.gson.JsonParser.parseString(query) + ", \"variables\": {}, \"operationName\": \"ScrapeWalmart\"}";

HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create(endpoint))
    .header("Content-Type", "application/json")
    .POST(HttpRequest.BodyPublishers.ofString(payload))
    .build();

HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());

2. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

C#

1. Send the request

using System.Net.Http;
using System.Text;
using System.Text.Json;

string token = "YOUR_API_TOKEN_HERE";
string endpoint = $"https://production-sfo.browserless.io/stealth/bql?token={token}&proxy=residential&proxyCountry=us";

var payload = new
{
    query = @"mutation ScrapeWalmart {
      goto(url: ""https://www.walmart.com/search?q=iphone"", waitUntil: networkIdle) { status }
      waitForTimeout(time: 3000) { time }
      products: mapSelector(selector: ""[data-item-id]"") {
        name: mapSelector(selector: ""[itemprop='name']"") { innerText }
        price: mapSelector(selector: ""[itemprop='price']"") { innerText }
        rating: mapSelector(selector: ""span[itemprop='ratingValue']"") { innerText }
        reviewCount: mapSelector(selector: ""[itemprop='ratingCount']"") { innerText }
        link: mapSelector(selector: ""a[href*='/ip/']"") { href: attribute(name: ""href"") { value } }
      }
    }",
    variables = new { },
    operationName = "ScrapeWalmart",
};

using (HttpClient httpClient = new HttpClient())
{
    var content = new StringContent(
        JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");
    var response = await httpClient.PostAsync(endpoint, content);
    string body = await response.Content.ReadAsStringAsync();
    Console.WriteLine(body);
}

2. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

Frameworks

Connect through stealth mode and a residential proxy. This makes the browser fingerprint and IP look like a real US visitor, which is what gets past Walmart's bot detection. Then extract product data from the rendered page.

Puppeteer

View Full Code on GitHub

1. Install dependencies

npm install puppeteer-core

2. Connect and scrape

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint:
    'wss://production-sfo.browserless.io/stealth?token=YOUR_API_TOKEN_HERE&proxy=residential&proxyCountry=us',
});

try {
  const page = await browser.newPage();
  await page.goto('https://www.walmart.com/search?q=iphone', {
    waitUntil: 'networkidle2',
  });

  const products = await page.evaluate(() =>
    Array.from(document.querySelectorAll('[data-item-id]')).map((card) => ({
      name: card.querySelector('[itemprop="name"]')?.innerText?.trim() ?? '',
      price: card.querySelector('[itemprop="price"]')?.getAttribute('content')
        ?? card.querySelector('[itemprop="price"]')?.innerText?.trim() ?? '',
      rating: card.querySelector('span[itemprop="ratingValue"]')?.innerText?.trim() ?? '',
      reviewCount: card.querySelector('[itemprop="ratingCount"]')?.innerText?.trim() ?? '',
      link: card.querySelector('a[href*="/ip/"]')?.href ?? '',
    }))
  );

  console.log(JSON.stringify(products, null, 2));
} finally {
  await browser.close();
}

3. Check the output

Run with node scrape-walmart.mjs. Each object in the array contains name, price, rating, reviewCount, and link for one product.

Playwright

View Full Code on GitHub

1. Install dependencies

npm install playwright-core

2. Connect and scrape

import { chromium } from 'playwright-core';

const browser = await chromium.connectOverCDP(
  'wss://production-sfo.browserless.io?token=YOUR_API_TOKEN_HERE&stealth&proxy=residential&proxyCountry=us'
);

try {
  const context = browser.contexts()[0];
  const page = await context.newPage();
  await page.goto('https://www.walmart.com/search?q=iphone', {
    waitUntil: 'networkidle',
  });

  const products = await page.evaluate(() =>
    Array.from(document.querySelectorAll('[data-item-id]')).map((card) => ({
      name: card.querySelector('[itemprop="name"]')?.innerText?.trim() ?? '',
      price: card.querySelector('[itemprop="price"]')?.getAttribute('content')
        ?? card.querySelector('[itemprop="price"]')?.innerText?.trim() ?? '',
      rating: card.querySelector('span[itemprop="ratingValue"]')?.innerText?.trim() ?? '',
      reviewCount: card.querySelector('[itemprop="ratingCount"]')?.innerText?.trim() ?? '',
      link: card.querySelector('a[href*="/ip/"]')?.href ?? '',
    }))
  );

  console.log(JSON.stringify(products, null, 2));
} finally {
  await browser.close();
}

3. Check the output

Run with node scrape-walmart.mjs. Each object in the array contains name, price, rating, reviewCount, and link for one product.

BQL

View Full Code on GitHub

1. Write the mutation

Navigate to the Walmart search results URL and use mapSelector to pull structured data out of each product card. We use the stealth endpoint here because the plain BQL endpoint doesn't apply browser fingerprint masking. The waitForTimeout gives JS-rendered content time to settle after networkIdle fires.

mutation ScrapeWalmart {
  goto(url: "https://www.walmart.com/search?q=iphone", waitUntil: networkIdle) {
    status
  }

  waitForTimeout(time: 3000) {
    time
  }

  products: mapSelector(selector: "[data-item-id]") {
    name: mapSelector(selector: "[itemprop='name']") {
      innerText
    }
    price: mapSelector(selector: "[itemprop='price']") {
      innerText
    }
    rating: mapSelector(selector: "span[itemprop='ratingValue']") {
      innerText
    }
    reviewCount: mapSelector(selector: "[itemprop='ratingCount']") {
      innerText
    }
    link: mapSelector(selector: "a[href*='/ip/']") {
      href: attribute(name: "href") {
        value
      }
    }
  }
}

2. Run it

Paste into the BQL IDE and click Run.

3. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "waitForTimeout": { "time": 3000 },
    "products": [
      {
        "name": [{ "innerText": "Apple iPhone 15, 128GB, Black - Unlocked" }],
        "price": [{ "innerText": "$699.00" }],
        "rating": [{ "innerText": "4.6" }],
        "reviewCount": [{ "innerText": "2847" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-15/1752657432" } }]
      },
      {
        "name": [{ "innerText": "Apple iPhone 14, 128GB, Midnight - Unlocked" }],
        "price": [{ "innerText": "$499.00" }],
        "rating": [{ "innerText": "4.5" }],
        "reviewCount": [{ "innerText": "5120" }],
        "link": [{ "href": { "value": "https://www.walmart.com/ip/Apple-iPhone-14/633586395" } }]
      }
    ]
  }
}

Next steps

Scrape Etsy Product Listings

extract product data from another bot-protected marketplace

Scrape Glassdoor Job Listings

apply the same stealth pattern to job listings

Change Your Browser's IP Address Using Proxies

control which IP address target sites see