How can I collect SKU information from ecommerce sites?

There are a few common ways to collect SKU information from ecommerce sites, depending on whether you mean your own store, a partner’s site with permission, or public product pages.

1) Use the site’s official data sources first

Best option if available:

• Product feed / CSV export: Many platforms (Shopify, Magento, WooCommerce, BigCommerce) let you export products including SKU.
• Admin API: If you control the store or have access, use the platform API.
  • Shopify Admin API
  • WooCommerce REST API
  • Magento REST API
• Google Merchant Center / feeds: Sometimes includes SKU or merchant-specific ID.

2) Scrape product pages

If SKUs are visible on product pages, you can extract them by:

• Parsing the HTML for labels like SKU, Item #, Product code
• Looking at structured data:
  • JSON-LD (application/ld+json)
  • Microdata / schema.org Product markup
• Checking embedded scripts or page data objects

Typical flow:

1. Collect product URLs
2. Download page HTML
3. Search for SKU in:
   • visible text
   • meta tags
   • JSON-LD
   • JavaScript state blobs
4. Store the extracted SKU with the product URL

3) Use sitemap + crawl

If a site has a product sitemap:

• Get product URLs from sitemap.xml
• Crawl only product pages
• Extract SKU from each page

This is more efficient than crawling the whole site.

4) Use browser automation for dynamic sites

If the SKU is loaded by JavaScript:

• Use Playwright or Selenium
• Wait for the product page to fully render
• Extract the rendered DOM or network responses

5) Check hidden API/network calls

Many ecommerce sites load product details from JSON endpoints:

• Open browser dev tools
• Inspect XHR/fetch requests
• Find endpoints returning product JSON
• Extract SKU directly from the response

6) Use third-party data providers

If you need large-scale product data:

• Product data aggregators
• Web data APIs
• Price intelligence tools

These can be easier than building and maintaining crawlers.

---

Practical extraction examples

From JSON-LD

Many sites include:

<script type="application/ld+json">
{
  "@type": "Product",
  "name": "Example Product",
  "sku": "ABC-123"
}
</script>

From HTML

<div class="product-meta">
  <span>SKU:</span> <span>ABC-123</span>
</div>

From JavaScript state

<script>
  window.__INITIAL_STATE__ = {
    product: {
      sku: "ABC-123"
    }
  };
</script>

---

Example in Python for a simple page

import requests
from bs4 import BeautifulSoup
import json
import re

url = "https://example.com/product-page"
html = requests.get(url, headers={"User-Agent": "Mozilla/5.0"}).text
soup = BeautifulSoup(html, "html.parser")

# Try JSON-LD first
sku = None
for script in soup.find_all("script", type="application/ld+json"):
    try:
        data = json.loads(script.string or "{}")
        if isinstance(data, dict) and data.get("@type") == "Product":
            sku = data.get("sku")
            if sku:
                break
    except Exception:
        pass

# Fallback: regex search
if not sku:
    m = re.search(r'SKU[:\s]*([A-Za-z0-9\-_]+)', html, re.IGNORECASE)
    if m:
        sku = m.group(1)

print("SKU:", sku)

---

Important legal/ethical notes

• Check the site’s Terms of Service and robots.txt
• Avoid bypassing access controls or rate limits
• Prefer official APIs and feeds
• If data is private or behind login, get authorization first

If you want, I can also show you:

1. a Python scraper for SKU extraction,
2. a Playwright example for JavaScript-heavy sites, or
3. a Shopify/WooCommerce API approach.