What are the best datasets for price intelligence?

Finding the best dataset for price intelligence depends on your use case—whether you are training machine learning models for dynamic pricing, monitoring competitor retail prices in real time, or analyzing macro-level commodity and consumer price trends.

The top price intelligence datasets and data providers are broken down below by category.

---

1. Commercial Retail & E-Commerce Datasets (Enterprise Grade)

If you need structured, high-volume pricing feeds across major online platforms (Amazon, Walmart, Target, eBay, Google Shopping), these enterprise data providers offer raw feeds or pre-built datasets:

• Bright Data (Retail & E-Commerce Datasets): Offers pre-collected datasets as well as real-time web scraping feeds covering over 400M+ products. Includes historical and daily price updates, promotional flags, stock status, and shipping costs.
• Coresignal: Provides B2B e-commerce product and pricing datasets suited for hedge funds, brand managers, and retail analysts tracking multi-channel pricing behavior.
• GrowByData & Import.io (Aperture): Specialize in decision-ready price intelligence feeds that include localized/ZIP-code-level tracking, promo/discount tracking, and AI-driven SKU matching across non-standardized product titles.
• PriceEasy / WorldData.AI: Focuses heavily on daily store-level physical retail prices (tracking 700k+ global products) and hourly retail fuel pricing across 22+ countries.

---

2. Developer & API-Based Price Feeds

For teams building custom repricing engines, BI dashboards, or algorithmic trading bots, these API-first platforms provide live and historical product-level pricing:

• Keepa API: The industry standard for Amazon pricing intelligence. Tracks historical price charts, Buy Box dynamics, stock levels, and seller offers across millions of SKUs globally.
• Rainforest API: Converts Amazon product pages into structured JSON data. Delivers real-time and historical pricing, coupon codes, and seller lists across all major Amazon marketplaces.
• Apify / ScraperAPI: If you need bespoke price datasets from niche niche marketplaces or direct-to-consumer (D2C) brand sites, these platforms allow you to generate custom automated pricing scrapers and datasets on demand.

---

3. Open & Free Public Datasets (For ML, Prototyping, & Research)

If you need datasets for training machine learning models, conducting academic research, or testing pricing algorithms without paying subscription fees:

• Brazilian E-Commerce Public Dataset by Olist (Kaggle): One of the most famous public e-commerce datasets containing 100,000 real order records from 2016–2018 in Brazil. It includes product prices, freight rates, seller locations, and purchase timelines—ideal for training dynamic pricing models.
• Amazon Product & Pricing Datasets (Kaggle / Hugging Face): Public community datasets featuring snapshots of millions of Amazon products across categories (electronics, apparel, home), including list prices, discounted prices, ratings, and category trees.
• Flipkart Product Dataset (Kaggle): Contains ~20,000 product listings from India's largest e-commerce platform, including retail prices vs. discounted prices, brand, and specification trees.

---

4. Macro-Economic & Commodity Price Datasets

If your price intelligence needs extend beyond retail SKUs into raw materials, energy, or macro inflation trends:

• FRED (Federal Reserve Economic Data): Offers tens of thousands of open time-series datasets tracking the Consumer Price Index (CPI), Producer Price Index (PPI), and sector-specific wholesale prices.
• World Bank Commodity Markets ("Pink Sheet"): Free monthly global pricing data for energy (crude oil, natural gas), agricultural products, and metals/minerals.
• S&P Global Commodity Insights (Platts) & Argus Media: Premium enterprise benchmarks for energy, petrochemicals, metals, and shipping freight pricing.

---

5. B2B Data Marketplaces

If you want to discover, benchmark, and purchase proprietary pricing datasets across different sectors, consider searching B2B data marketplaces:

• Snowflake Data Marketplace & AWS Data Exchange: Both feature turn-key, queryable pricing datasets from financial institutions, retail aggregators, and satellite intelligence vendors.
• Datarade: A dedicated data provider directory where you can search, compare, and sample price intelligence data vendors (e.g., Knowtrex, Grepsr, Retail Scrape) based on region, coverage, and refresh frequency.

---

Key Factors to Evaluate in a Price Intelligence Dataset

When selecting a dataset, keep these criteria in mind:

1. Refresh Frequency: Do you need real-time/hourly price monitoring for automated repricing, or will daily/monthly historical snapshots suffice for strategy modeling?
2. Metadata Depth: A raw price point is rarely enough. Look for datasets that include shipping costs, coupons/discounts, stock status (in-stock vs. out-of-stock), and seller/Buy Box status.
3. SKU Matching Quality: If you are monitoring competitors, verify if the dataset provider uses exact SKU/UPC matching or AI-driven "similar item" matching to avoid comparing mismatched products.