- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work presents Aurora-X: A time series foundation model for multivariate probabilistic forecasting with optional future covariates. The design includes: Temporal and cross-variate attention, a pattern-guided mixture of experts, an implicit quantile head that predicts any requested quantile, generates future patches in parallel, and a multi-stage training setup. This approach reports strong results on 5 established benchmarks, comparing both in-domain supervised models (e.g., iTransformer) and foundation models.
Critical strengths
-
Strong Evaluation: This work proves its robustness with strong results across various commonly evaluated benchmarks, which are challenging to surpass.
-
Implicit Quantile Head: An innovative and practical component that can predict quantiles on request and is shown to be more accurate than its alternatives.
-
In general, the work is easy to follow, with clean formulations.
Critical weaknesses
- Potential data leakage: There is a substantial risk of overlap between the pretraining corpora of Aurora-X, and the datasets used for evaluation. For example:
- LightGTS pretrains on datasets such as KDD-2018 and Solar, which are also included in GIFT-Eval.
- LOTSA incorporates datasets from sources such as LibCity, Kaggle, and Monash, which may overlap with datasets in fev-bench.
- Chronos pretraining includes datasets such as Electricity, Solar, and M4, which also appear in benchmarks such as GIFT-Eval and LTSF. Aurora-X's use of TimeBench, LOTSA, and Chronos pretraining data further complicates the assessment of potential overlap. These overlaps raise concerns about the reliability of the reported results and the fairness of comparisons. In particular, the use of large models with up to 1B parameters raises questions about potential memorization of training signals. The paper should provide a more detailed analysis of dataset overlap and its impact on evaluation.
-
Insufficient reproducibility: Providing code is generally an expected practice for time-series foundation models (TSFMs). The availability of code, together with sufficiently detailed information about the pretraining data, data processing, and evaluation protocols, is important for independently verifying the results and ensuring reproducibility.
-
Insufficient evaluation of probabilistic forecasting: The model is evaluated primarily using MASE, a point-forecast accuracy metric. This is insufficient to establish the value of the proposed Implicit Quantile Head, whose contribution should be assessed through probabilistic forecasting metrics. Evaluations using CRPS, quantile loss, or other distribution-sensitive metrics are necessary to demonstrate whether the proposed method improves predictive uncertainty estimation and probabilistic forecasting performance.
-
Limited methodological novelty: The paper identifies underexplored training potential and limited architectural versatility as key limitations of existing TSFMs. However, its proposed solutions largely combine established techniques, including mixture-of-experts (MoE), test-time scaling, and time-group modeling with covariate awareness, alongside increased model size and pretraining data. The Implicit Quantile Head appears to be the primary methodological contribution, but its effectiveness is difficult to assess independently given the concerns about data leakage, model scale, and experimental evaluation of probabilistic forecasting.
-
Insufficient component and scalability analysis: The reported improvements are difficult to attribute to specific methodological contributions given the simultaneous use of more data, larger models, and established architectural components. More comprehensive ablation studies are needed to isolate the effects of each component, particularly the Implicit Quantile Head. Further analysis of computational costs and scaling behavior would also help establish whether the proposed approach offers meaningful advantages over existing TSFMs.
Questions for authors
see above
5 comments
Response to W3 (Probabilistic forecasting evaluation)
We agree that MASE alone is insufficient to assess the Implicit Quantile Network (IQN). The original submission already reports distribution-sensitive metrics in Appendix A.7, although we did not emphasize them sufficiently in the main text.
The reported values are ratios to Seasonal Naive, aggregated geometrically; lower is better. In the GIFT-Eval table, bold denotes the best pretrained checkpoint; the fine-tuned Toto checkpoint is shown separately for context.
GIFT-Eval — Relative WQL
| Model | Score |
|---|---|
| Aurora-X | 0.4665 |
| TiRex-2 (PT) | 0.4669 |
| Toto-2.0 (2.5B, FT) | 0.4634 |
| Chronos-2 | 0.4854 |
| Falcon-2.0 | 0.4863 |
FEV-Bench — Relative SQL
| Model | Score |
|---|---|
| Aurora-X | 0.5168 |
| Chronos-2 | 0.5272 |
| TimesFM-2.5 | 0.5332 |
| TiRex-2 | 0.5450 |
| FlowState | 0.5837 |
Aurora-X is the best displayed TSFM on GIFT-Eval WQL, with the only lower entry shown there being an explicitly fine-tuned Toto checkpoint. On FEV-Bench, Aurora-X has the lowest displayed SQL. WQL and SQL average quantile losses rather than evaluating only the median, and SQL uses in-sample seasonal differences for scaling.
The IQN also provides a practical capability that fixed-grid heads do not: it accepts any requested at inference, without retraining a new output grid or sampling multiple trajectories. A single backbone pass produces the future states, after which arbitrary requested quantiles are decoded in parallel.
The following comparison supplements Figure 8 and evaluates all four forecasting heads after pretraining (Pre), before M1–M3 and Post. Figure 8 reports relative MASE; the table below adds relative WQL on GIFT-Eval and relative SQL on FEV-Bench to assess probabilistic forecasting performance. The backbone architecture, pretraining corpus, and evaluation protocol are shared, with a fixed sampling budget for the flow-matching and mixture-distribution heads. Lower scores are better. These are Pre-stage head comparisons, distinct from the final-model benchmark results reported above.
| Forecasting head | GIFT-Eval WQL | FEV-Bench SQL |
|---|---|---|
| IQN | 0.5381 | 0.6477 |
| Fixed quantile | 0.5656 | 0.6813 |
| Flow matching | 0.5734 | 0.7025 |
| Mixture distribution | 0.5866 | 0.7302 |
The complete head comparison appears in Appendix D.2.
Response to W4 (Methodological novelty)
We do not claim that MoE, IQN, multi-resolution modeling, or staged training were invented in isolation. Our contribution is their time-series-specific design and integration, in particular:
- Pattern-guided MoE. Load balancing controls how often experts are used, but not which temporal patterns share an expert. Aurora-X learns shallow patch similarities and uses a normalized co-assignment loss to guide deep routing through a straight-through estimator. The accompanying orthogonality regularizer prevents the pattern projection from collapsing. Appendix C gives the corresponding decomposition and gradient analysis, and the routing experiments show input-dependent expert preferences across depth.
- Continuous probabilistic head. IQN is an established idea; our contribution is its integration with a covariate-aware, parallel patch forecaster. The head conditions on both the future state and a continuous quantile level, so customized marginal quantiles can be queried in one forward pass without trajectory sampling.
- Variable-resolution Post. During Post, we train on aligned resampled history–target pairs. At inference, we vary the temporal span per token and restore predictions to the requested native grid without updating the trained model weights. This allows temporal coverage and token usage to be adjusted without retraining. Appendix B formalizes the equivalence between data-side resampling and the corresponding input/output projections.
- Capability-oriented training. The curriculum progressively introduces temporal pattern modeling, cross-variable dependencies, variable context and horizon lengths, future covariates, and resolution adaptation. We compare this schedule with a mixed-task baseline that includes all five task types from the start of training.
Response to Reviewer bLVt
We thank Reviewer bLVt for the careful and constructive comments. We address each concern below.
Response to W1 (Potential data leakage)
We agree that leakage would undermine a zero-shot evaluation. We therefore cleaned the training data before training and applied the same restriction throughout Pre, M1--M3, and Post.
Our audit comprised two steps:
The final Aurora-X corpus is source-disjoint from the five downstream benchmarks under this audit, including the one-billion-point subset used for Post. The examples in the review were included in the screening:
We have added the source manifest, matching rules, and removal records for independent inspection. We also describe this protocol in the main reproducibility discussion rather than leaving it only in the appendix. The detailed audit protocol is provided in Appendix D.1.
Response to W2 (Reproducibility and release)
We have updated the anonymous repository; it now includes the model architecture, training framework, and benchmark evaluation scripts. The implementation and reproducibility resources are summarized in Appendix D.7.
The training framework is adapted from publicly available Aurora/Sundial-style frameworks and uses standard Hugging Face components. Pre trains the initial model, and M1–M3 and Post continue from the preceding checkpoint using the supervised objectives specified in the paper. Post uses aligned resampled targets rather than targets on the original temporal grid.
The evaluation scripts follow each benchmark's official protocol and the evaluation guidance provided by its authors, including the prescribed data splits, input format, forecast horizons, quantile levels, and metric aggregation.
The paper already specifies the main model and optimization settings, including the 12-layer, 768-width backbone; 16 routed experts with top-4 routing; approximately 1.05B total parameters; AdamW with learning rate 5×10−5 and cosine annealing; FP32 training; and 160K total optimization steps (50K/25K/25K/50K/10K for Pre/M1/M2/M3/Post).