Prompt
How do I find benchmark datasets for evaluating a forecasting model?
Latest observation
A good way to find benchmark datasets for forecasting is to start from the forecasting task and data type, then search in a few standard places.
1) Define what you need
Before looking for datasets, decide:
- Forecast horizon: short-term, medium-term, long-term
- Frequency: hourly, daily, weekly, monthly, yearly
- Domain: retail, energy, finance, weather, traffic, etc.
- Univariate or multivariate
- Single series or many series
- Need for exogenous variables: promotions, temperature, holidays
- Evaluation style: point forecasts, probabilistic forecasts, intermittent demand, hierarchical forecasting
This helps narrow down benchmark sets that are actually comparable to your model.
2) Use standard benchmark repositories
These are common places to find forecasting datasets:
General ML/data repositories
- Kaggle — many forecasting competitions and public datasets
- UCI Machine Learning Repository — classic datasets, sometimes useful for time series
- OpenML — searchable datasets and tasks
- Google Dataset Search — good for discovering datasets across the web
- Hugging Face Datasets — increasingly useful for time series and forecasting
Time-series-specific sources
- Monash Forecasting Repository — one of the best-known benchmark collections for forecasting
- M4 / M5 / M6 competition datasets — widely used benchmarks
- UCR/UEA Time Series Archive — more classification-focused, but worth checking
- NASA / NOAA / government open data portals — useful for weather, climate, and environmental forecasting
- World Bank / OECD / FRED — macroeconomic and financial time series
3) Search by benchmark papers
If you want datasets that are commonly accepted in research:
- Find recent forecasting papers in your domain
- Look at their “Experiments” or “Datasets” sections
- Search for the datasets they compare against
- Prefer datasets that are:
- publicly available
- well-documented
- used in multiple papers
Search terms like:
- “forecasting benchmark dataset”
- “time series forecasting benchmark”
- “probabilistic forecasting dataset”
- “multivariate time series forecasting benchmark”
4) Check competition datasets
Competitions often provide strong benchmark datasets and standard evaluation protocols:
- M-competitions: M4, M5, M6, etc.
- Kaggle forecasting competitions
- Domain-specific challenges in energy, retail, and traffic
These are especially useful because they often come with:
- train/test splits
- baseline methods
- official metrics
5) Look for built-in benchmarks in libraries
Some forecasting libraries include sample datasets or dataset loaders:
- GluonTS
- Darts
- sktime
- Nixtla / statsforecast / neuralforecast
- PyTorch Forecasting
These can save time if you want quick baselines and standardized preprocessing.
6) Evaluate the benchmark quality
A dataset is useful as a benchmark if it has:
- clear license and access
- enough historical length
- realistic forecasting challenge
- standard train/test split or rolling evaluation
- baseline results or prior papers using it
- minimal leakage issues
- enough samples if you want to compare models robustly
7) Practical search workflow
A simple workflow:
- Search your domain + “forecasting benchmark”
- Check major repositories like Monash and Kaggle
- Read recent benchmark papers
- Compare dataset frequency, horizon, and availability
- Pick 2–5 datasets with different characteristics
8) Good starting points
If you want well-known benchmarks, start with:
- M4 / M5
- Monash Forecasting Repository
- Kaggle forecasting datasets
- FRED / World Bank for macroeconomic series
- NOAA / NASA for environmental data
If you want, I can also give you:
- a curated list of benchmark forecasting datasets by domain
- or a step-by-step search strategy for your specific model/task