How do I scrape customer ratings from ecommerce sites?

Scraping customer ratings from ecommerce sites can be done, but the safest and most reliable approach is to use the site’s official API or export tools first. If scraping is allowed by the site’s Terms of Service and robots rules, here’s a practical way to do it.

1) Check legality and access rules

Before scraping:

• Review the site’s Terms of Service
• Check robots.txt
• Make sure you’re not collecting personal data you shouldn’t
• Use reasonable request rates to avoid disrupting the site

2) Prefer structured sources

Customer ratings often appear in places that are easier to extract than the visible page:

• JSON-LD / structured data in the HTML
• Embedded API responses loaded by the page
• Hidden scripts containing review data

Example of structured rating data in HTML:

<script type="application/ld+json">
{
  "@type": "Product",
  "aggregateRating": {
    "ratingValue": "4.6",
    "reviewCount": "128"
  }
}
</script>

3) Choose your scraping approach

A. Static HTML pages

Use Python with requests + BeautifulSoup.

import requests
from bs4 import BeautifulSoup
import json

url = "https://example.com/product/123"
headers = {"User-Agent": "Mozilla/5.0"}
html = requests.get(url, headers=headers).text
soup = BeautifulSoup(html, "html.parser")

# Try JSON-LD first
for script in soup.find_all("script", type="application/ld+json"):
    try:
        data = json.loads(script.string)
        if isinstance(data, dict) and "aggregateRating" in data:
            print(data["aggregateRating"])
    except Exception:
        pass

B. JavaScript-rendered pages

Use a browser automation tool like Playwright or Selenium if the ratings are loaded dynamically.

Example with Playwright:

from playwright.sync_api import sync_playwright

url = "https://example.com/product/123"

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto(url, wait_until="networkidle")
    html = page.content()
    print(html)
    browser.close()

4) Inspect the page for API calls

Often the ratings come from an XHR/fetch request:

• Open browser DevTools → Network
• Filter by Fetch/XHR
• Reload page and look for endpoints returning JSON review data

If you find such an endpoint, it may be much easier to request directly than parsing the page.

5) Extract what you need

Typical fields:

• average rating
• number of ratings/reviews
• individual review stars
• review text
• review date

Example selectors may vary by site:

rating = soup.select_one(".rating").get_text(strip=True)
review_count = soup.select_one(".review-count").get_text(strip=True)

6) Handle anti-bot measures

Be careful with:

• rate limiting
• CAPTCHAs
• login walls
• session cookies

Use:

• delays between requests
• retries with backoff
• caching
• rotating only if permitted and ethical

7) Store the data

Save to:

• CSV for simple analysis
• JSON for nested review structures
• a database for large-scale scraping

Example CSV output:

import csv

with open("ratings.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["product", "rating", "review_count"])
    writer.writeheader()
    writer.writerow({"product": "Widget", "rating": 4.6, "review_count": 128})

8) A safer alternative: use APIs or data providers

Many ecommerce platforms offer:

• official APIs
• partner feeds
• review export features

Examples:

• Shopify apps / admin APIs
• Amazon SP-API for authorized sellers
• Trustpilot / Bazaarvoice APIs where applicable

---

If you want, I can give you:

1. a Python script template for scraping ratings from a product page, or
2. a site-specific approach if you tell me which ecommerce site you’re targeting.