Prompt
I'm building a demand forecasting app and need retail data with regular updates
Latest observation
For a demand forecasting app, you usually want retail data that is:
- Frequent: daily or weekly updates
- Consistent: same fields over time
- Granular: store/SKU-level if possible
- Reliable: minimal missing values
- Legally usable: clear licensing/API terms
Good options depend on whether you need public/free data, paid APIs, or your own retailer integration.
Good data source types
1. Public retail datasets Useful for prototyping and model development, but often not regularly updated.
- Kaggle retail datasets
- UCI / government datasets
- M5 Forecasting dataset
- Walmart sales datasets
2. Retail/commerce APIs Best if you need ongoing refreshes.
- Shopify API if you have access to a merchant store
- Amazon Selling Partner API
- eBay APIs
- Walmart Marketplace APIs
- POS/ecommerce connectors like Square, Clover, Lightspeed, Toast
3. Data providers / market intelligence vendors Better if you want external market-level retail signals.
- NielsenIQ
- Circana
- DataWeave
- Similarweb for traffic signals
- Scrape-free product/price intelligence vendors
4. Web scraping / price monitoring Can provide regular updates, but check site terms and legal constraints.
What to look for in a dataset
For forecasting, ideal columns include:
- date
- store_id
- product_id / SKU
- units_sold
- price
- promotions
- inventory / stockout status
- category
- region
- holidays / events
Recommendation
If you need regular updates, the best path is usually:
- transaction data from a retailer/ecommerce/POS system, or
- a paid retail data API/vendor.
If you just want to build and test the forecasting pipeline, start with:
- M5 Forecasting dataset
- Walmart sales datasets
- Favorita grocery sales dataset
If you want, I can help you with one of these next:
- a list of specific APIs with regular retail updates
- a free dataset shortlist
- a data schema for demand forecasting
- a data ingestion architecture for your app