- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces PanelTS, a multi-domain benchmark dataset for panel-based time series forecasting. It adopts a unified representation that supports univariate, multivariate, and panel-based forecasting tasks from the same data source while preserving cross-unit structure. The benchmark comprises 1,844 datasets spanning synthetic data and four real-world domains (currency exchange rates, commodity ETFs, COVID-19 epidemiology, and equity markets), together with standardized data formats, task definitions, and evaluation protocols. Reference results are reported for 14 representative forecasting models on two illustrative scenarios (currency exchange rates and COVID-19), using a 100-step input, one-step-ahead, rolling-window protocol with both raw and scale-normalized RMSE/MAE metrics.
Critical strengths
-
The problem framing is legitimate and fills a real gap. The paper observes that existing forecasting benchmarks are organized either as collections of independent univariate series or as single-entity multivariate recordings, neither of which captures the "multiple related units with shared covariate definitions" structure common in real-world forecasting.
-
The unified data schema is a reasonable engineering contribution. Organizing targets and covariates so that the same data source can support univariate, multivariate, and panel tasks is clean and practically useful, and the scale-normalized error metrics address a real comparability problem across heterogeneous units.
Critical weaknesses
-
The benchmark's central claim is not empirically demonstrated. The paper's stated purpose is to enable the study of cross-unit dependencies, but (a) only two of the five dataset groups are evaluated; (b) nearly all baselines run in a target-history-only mode, so the panel structure and shared covariates the benchmark is built around are essentially never exercised; and (c) there is no panel-specific baseline or controlled ablation showing that exploiting the panel structure improves forecasting.
-
The independent contribution is thin relative to existing work. The core real-world data (exchange rates and COVID-19) overlaps substantially with the contemporaneous ESE work (Xu et al., 2026), and is largely repackaged from public sources; the synthetic generator's linear additive data-generating process is too simple to stress-test the nonlinear, time-varying cross-unit dependencies of real panel data.
The paper reads more as a repackaging/extension of the ESE data effort than as a self-contained benchmark contribution.
Questions for authors
- Could you provide a controlled comparison demonstrating that exploiting the panel structure improves accuracy?
- Please clarify the relationship between PanelTS and the data used in ESE (Xu et al., 2026).
- Why are the ETF and stock-market datasets absent from the reference experiments?