acceptodds
Under review as a conference paper at ICLR 2027

Aurora-X: Built for Extreme Time Series Forecasting

12 views

Abstract

Time series foundation models (TSFMs) enable cross-domain forecasting, but their development as general-purpose forecasters remains constrained by underexplored training potential and limited architectural versatility. To address these challenges, we introduce Aurora-X, a billion-scale TSFM with a progressive curriculum and a unified architecture. We first use channel-independent pretraining to learn temporal patterns, then introduce cross-variable dependencies, varied context and horizon lengths, and future covariates if available during midtraining. Variable-resolution post-training further enables an adjustable temporal span per token at inference. With fixed model weights, this supports longer histories under a fixed token budget or fewer tokens for the same history, enabling test-time scaling. With a versatile architecture, Aurora-X supports cross-variable modeling, covariate conditioning, and parallel decoding of future patches for probabilistic forecasting. These are supported by a novel pattern-guided mixture-of-experts that expands model capacity through sparse activation and uses shallow patch similarities to constrain deep-layer routing, guiding expert specialization across heterogeneous time series. Furthermore, we propose an implicit quantile network head that predicts arbitrary quantiles to characterize predictive distributions, enhancing probabilistic forecasting flexibility. Comprehensive experiments on GIFT-Eval, TIME, FEV-Bench, TFB, and DAG-Bench demonstrate state-of-the-art forecasting performance against pretrained TSFMs and task-specific supervised models.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bLVtOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work presents Aurora-X: A time series foundation model for multivariate probabilistic forecasting with optional future covariates. The design includes: Temporal and cross-variate attention, a pattern-guided mixture of experts, an implicit quantile head that predicts any requested quantile, generates future patches in parallel, and a multi-stage training setup. This approach reports strong results on 5 established benchmarks, comparing both in-domain supervised models (e.g., iTransformer) and foundation models.

Critical strengths
  • Strong Evaluation: This work proves its robustness with strong results across various commonly evaluated benchmarks, which are challenging to surpass.

  • Implicit Quantile Head: An innovative and practical component that can predict quantiles on request and is shown to be more accurate than its alternatives.

  • In general, the work is easy to follow, with clean formulations.

Critical weaknesses
  • Potential data leakage: There is a substantial risk of overlap between the pretraining corpora of Aurora-X, and the datasets used for evaluation. For example:
  1. LightGTS pretrains on datasets such as KDD-2018 and Solar, which are also included in GIFT-Eval.
  2. LOTSA incorporates datasets from sources such as LibCity, Kaggle, and Monash, which may overlap with datasets in fev-bench.
  3. Chronos pretraining includes datasets such as Electricity, Solar, and M4, which also appear in benchmarks such as GIFT-Eval and LTSF. Aurora-X's use of TimeBench, LOTSA, and Chronos pretraining data further complicates the assessment of potential overlap. These overlaps raise concerns about the reliability of the reported results and the fairness of comparisons. In particular, the use of large models with up to 1B parameters raises questions about potential memorization of training signals. The paper should provide a more detailed analysis of dataset overlap and its impact on evaluation.
  • Insufficient reproducibility: Providing code is generally an expected practice for time-series foundation models (TSFMs). The availability of code, together with sufficiently detailed information about the pretraining data, data processing, and evaluation protocols, is important for independently verifying the results and ensuring reproducibility.

  • Insufficient evaluation of probabilistic forecasting: The model is evaluated primarily using MASE, a point-forecast accuracy metric. This is insufficient to establish the value of the proposed Implicit Quantile Head, whose contribution should be assessed through probabilistic forecasting metrics. Evaluations using CRPS, quantile loss, or other distribution-sensitive metrics are necessary to demonstrate whether the proposed method improves predictive uncertainty estimation and probabilistic forecasting performance.

  • Limited methodological novelty: The paper identifies underexplored training potential and limited architectural versatility as key limitations of existing TSFMs. However, its proposed solutions largely combine established techniques, including mixture-of-experts (MoE), test-time scaling, and time-group modeling with covariate awareness, alongside increased model size and pretraining data. The Implicit Quantile Head appears to be the primary methodological contribution, but its effectiveness is difficult to assess independently given the concerns about data leakage, model scale, and experimental evaluation of probabilistic forecasting.

  • Insufficient component and scalability analysis: The reported improvements are difficult to attribute to specific methodological contributions given the simultaneous use of more data, larger models, and established architectural components. More comprehensive ablation studies are needed to isolate the effects of each component, particularly the Implicit Quantile Head. Further analysis of computational costs and scaling behavior would also help establish whether the proposed approach offers meaningful advantages over existing TSFMs.

Questions for authors

see above

5 comments

Response to Reviewer bLVt

Rebuttal Summary

  • Data leakage: We audited the TimeBench and LightGTS/ROSE-derived sources using lineage matching and series-level similarity, and removed overlapping or transformed benchmark sources before training, including Post.
  • Reproducibility: The anonymous repository now includes the model architecture, training framework, and benchmark evaluation scripts.
  • Probabilistic evidence: The submission already reports relative WQL of 0.4665 on GIFT-Eval and relative SQL of 0.5168 on FEV-Bench.
  • Novelty: We claim time-series-specific pattern-guided routing, continuous quantile forecasting, fixed-weight variable-resolution Post-training, and their integration—not generic novelty for MoE or IQN.
  • Attribution: We report three ablation studies after full training: pattern guidance, the progressive curriculum, and variable-resolution Post. Forecasting-head comparisons and parameter scaling are reported separately at the Pre stage. These studies use GIFT-Eval and FEV-Bench; the existing TIME results remain unchanged.

We thank Reviewer bLVt for the careful and constructive comments. We address each concern below.

Response to W1 (Potential data leakage)

We agree that leakage would undermine a zero-shot evaluation. We therefore cleaned the training data before training and applied the same restriction throughout Pre, M1--M3, and Post.

Our audit comprised two steps:

  1. Lineage and name matching. We matched the names, aliases, source information, and subdataset identifiers of GIFT-Eval, TIME, FEV-Bench, TFB, and DAG-Bench against the TimeBench and LightGTS/ROSE-derived candidate sources.
  2. Series-level similarity checking. For candidates identified in the first step, we computed series similarity after the relevant preprocessing and resampling. We removed exact matches, transformed or resampled versions, and high-similarity candidates rather than relying on filenames alone.

The final Aurora-X corpus is source-disjoint from the five downstream benchmarks under this audit, including the one-billion-point subset used for Post. The examples in the review were included in the screening:

  • KDD-2018 and Solar records potentially shared by LightGTS and GIFT-Eval were checked and removed when matched.
  • Candidate LibCity, Kaggle, and Monash records were checked against FEV-Bench lineage and series similarity, including renamed or transformed versions.
  • Chronos/LOTSA corpora were not used as additional Aurora-X training sources. The relevant question is whether the final Aurora-X manifest contains a benchmark series, not whether another baseline used an overlapping corpus.

We have added the source manifest, matching rules, and removal records for independent inspection. We also describe this protocol in the main reproducibility discussion rather than leaving it only in the appendix. The detailed audit protocol is provided in Appendix D.1.

Response to W2 (Reproducibility and release)

We have updated the anonymous repository; it now includes the model architecture, training framework, and benchmark evaluation scripts. The implementation and reproducibility resources are summarized in Appendix D.7.

The training framework is adapted from publicly available Aurora/Sundial-style frameworks and uses standard Hugging Face components. Pre trains the initial model, and M1–M3 and Post continue from the preceding checkpoint using the supervised objectives specified in the paper. Post uses aligned resampled targets rather than targets on the original temporal grid.

The evaluation scripts follow each benchmark's official protocol and the evaluation guidance provided by its authors, including the prescribed data splits, input format, forecast horizons, quantile levels, and metric aggregation.

The paper already specifies the main model and optimization settings, including the 12-layer, 768-width backbone; 16 routed experts with top-4 routing; approximately 1.05B total parameters; AdamW with learning rate 5×10−55\times10^{-5} and cosine annealing; FP32 training; and 160K total optimization steps (50K/25K/25K/50K/10K for Pre/M1/M2/M3/Post).

Response to W3 (Probabilistic forecasting evaluation)

We agree that MASE alone is insufficient to assess the Implicit Quantile Network (IQN). The original submission already reports distribution-sensitive metrics in Appendix A.7, although we did not emphasize them sufficiently in the main text.

The reported values are ratios to Seasonal Naive, aggregated geometrically; lower is better. In the GIFT-Eval table, bold denotes the best pretrained checkpoint; the fine-tuned Toto checkpoint is shown separately for context.

GIFT-Eval — Relative WQL

ModelScore
Aurora-X0.4665
TiRex-2 (PT)0.4669
Toto-2.0 (2.5B, FT)0.4634
Chronos-20.4854
Falcon-2.00.4863

FEV-Bench — Relative SQL

ModelScore
Aurora-X0.5168
Chronos-20.5272
TimesFM-2.50.5332
TiRex-20.5450
FlowState0.5837

Aurora-X is the best displayed TSFM on GIFT-Eval WQL, with the only lower entry shown there being an explicitly fine-tuned Toto checkpoint. On FEV-Bench, Aurora-X has the lowest displayed SQL. WQL and SQL average quantile losses rather than evaluating only the median, and SQL uses in-sample seasonal differences for scaling.

The IQN also provides a practical capability that fixed-grid heads do not: it accepts any requested τ∈(0,1)\tau\in(0,1) at inference, without retraining a new output grid or sampling multiple trajectories. A single backbone pass produces the future states, after which arbitrary requested quantiles are decoded in parallel.

The following comparison supplements Figure 8 and evaluates all four forecasting heads after pretraining (Pre), before M1–M3 and Post. Figure 8 reports relative MASE; the table below adds relative WQL on GIFT-Eval and relative SQL on FEV-Bench to assess probabilistic forecasting performance. The backbone architecture, pretraining corpus, and evaluation protocol are shared, with a fixed sampling budget for the flow-matching and mixture-distribution heads. Lower scores are better. These are Pre-stage head comparisons, distinct from the final-model benchmark results reported above.

Forecasting headGIFT-Eval WQLFEV-Bench SQL
IQN0.53810.6477
Fixed quantile0.56560.6813
Flow matching0.57340.7025
Mixture distribution0.58660.7302

The complete head comparison appears in Appendix D.2.

Response to W4 (Methodological novelty)

We do not claim that MoE, IQN, multi-resolution modeling, or staged training were invented in isolation. Our contribution is their time-series-specific design and integration, in particular:

  • Pattern-guided MoE. Load balancing controls how often experts are used, but not which temporal patterns share an expert. Aurora-X learns shallow patch similarities and uses a normalized co-assignment loss to guide deep routing through a straight-through estimator. The accompanying orthogonality regularizer prevents the pattern projection from collapsing. Appendix C gives the corresponding decomposition and gradient analysis, and the routing experiments show input-dependent expert preferences across depth.
  • Continuous probabilistic head. IQN is an established idea; our contribution is its integration with a covariate-aware, parallel patch forecaster. The head conditions on both the future state and a continuous quantile level, so customized marginal quantiles can be queried in one forward pass without trajectory sampling.
  • Variable-resolution Post. During Post, we train on aligned resampled history–target pairs. At inference, we vary the temporal span per token and restore predictions to the requested native grid without updating the trained model weights. This allows temporal coverage and token usage to be adjusted without retraining. Appendix B formalizes the equivalence between data-side resampling and the corresponding input/output projections.
  • Capability-oriented training. The curriculum progressively introduces temporal pattern modeling, cross-variable dependencies, variable context and horizon lengths, future covariates, and resolution adaptation. We compare this schedule with a mixed-task baseline that includes all five task types from the start of training.
Reviewer_grz7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Aurora-X, a billion-scale time series foundation model designed to support multiple forecasting capabilities within a unified architecture. The model combines temporal attention, cross-variable attention, and a pattern-guided mixture-of-experts mechanism. It also uses an implicit quantile head for arbitrary quantile prediction and parallel decoding of future patches. The authors propose a progressive training strategy that starts with channel-independent forecasting, then introduces cross-variable dependencies, flexible context and horizon lengths, and future covariates. A final multi-resolution post-training stage allows the model to adjust temporal resolution at inference time, enabling longer historical coverage or reduced token usage. Experiments across five forecasting benchmarks show strong performance compared with existing foundation models and supervised forecasting methods. The paper also provides architectural comparisons, training-stage analysis, and inference efficiency evaluations.

Critical strengths
  1. A comprehensive and capable forecasting foundation model. Aurora-X integrates multiple useful capabilities, including multivariate forecasting, future covariate conditioning, continuous quantile prediction, and variable-resolution inference. Developing these capabilities within a single model is valuable, and the progressive training strategy provides a reasonable framework for doing so.

  2. Strong empirical performance with fairly thorough evaluations. The model achieves competitive or leading results across five forecasting benchmarks. I also appreciate the controlled comparisons of different MoE mechanisms and forecasting heads, as well as the training-stage and inference efficiency analyses. These experiments provide useful support for the overall framework.

Critical weaknesses
  1. The unique contribution of the progressive training strategy is not fully established. The framework combines several existing techniques, including pattern-guided expert routing, implicit quantile prediction, and multi-resolution forecasting. While the authors provide useful architectural comparisons, many of these ideas have been studied before. The progressive curriculum appears to be a central contribution, but Figure 9 mainly shows that performance improves across successive training stages. Since each stage also introduces additional training steps and different task configurations, it is difficult to isolate the benefit of the proposed curriculum itself. A comparison with joint multi-task training using the same training tasks and a comparable computational budget would make this contribution more convincing.

  2. Some experimental comparisons are not fully controlled. The results are impressive, but some of the comparisons use substantially different input lengths. For example, on TFB, Aurora-X generally uses 2,880 historical observations, while eight supervised baselines use only 96. Although the authors acknowledge this difference, it is difficult to determine how much of the improvement comes from the proposed architecture and training strategy, rather than access to longer historical information. A comparison under matched context lengths would help clarify this point.

Questions for authors
  1. How much does the progressive curriculum contribute beyond additional training? Could the authors compare the proposed training schedule with joint multi-task training under a similar computational budget? This would help establish whether the order of introducing different capabilities is important.

  2. Does pattern-guided MoE remain advantageous after the complete training process? The MoE comparisons in Figure 7 are useful, but they are based on the pretraining stage. Do the advantages remain similar after midtraining and post-training? Also, could the authors provide stronger evidence that expert specialization reflects meaningful temporal patterns rather than mainly dataset differences?

  3. How much do the different context lengths affect the benchmark comparisons? On TFB, Aurora-X uses much longer histories than several supervised baselines. Could the authors provide results with matched input lengths or analyze how Aurora-X performs with shorter histories? Also, how are inference resolutions selected using benchmark feedback on GIFT-Eval and TIME?

  4. How is the inference resolution selected in practice? Appendix A.11 shows a trade-off between temporal detail, historical coverage, and token usage. The authors also mention selecting dataset-specific resolutions using benchmark feedback. Could they clarify how this selection is performed and how an appropriate resolution would be chosen for a new dataset without access to benchmark feedback?

5 comments

Response to Reviewer grz7

Rebuttal Summary

  • Curriculum attribution: We compared the progressive curriculum with mixed-task training that includes all five task types from the outset under a matched training budget.
  • Context control: We added a TFB evaluation with a 96-point history and compared it with PatchTST, a standard end-to-end supervised baseline using the same history length.
  • Post-training routing: We added a post-training pattern-guidance ablation and a within-dataset routing analysis to distinguish temporal-pattern specialization from dataset identity.
  • Resolution selection: We use 48 observations per token by default and adjust the span for histories shorter than 48, based on the available input length rather than benchmark scores.

We thank the reviewer for the positive assessment of Aurora-X's breadth and empirical evaluation. We address the two weaknesses and four questions below.

Response to W1 and Q1 (Progressive curriculum versus additional training)

Figure 9 shows improvements across training stages, but does not by itself separate the effect of the curriculum from that of additional training. We therefore compared the progressive curriculum with mixed-task training that includes all five task types from the outset, rather than introducing them stage by stage.

The five task types correspond to channel-independent forecasting, multivariate forecasting, forecasting with variable context and horizon lengths, future-covariate conditioning, and variable-resolution forecasting. The mixed-task baseline includes all five throughout training, with no stage-wise task introduction. Both settings use the same task definitions, cleaned training corpus, backbone, optimizer, and preprocessing, with matched total optimization steps and compute budgets.

Both models are evaluated after the full training budget. All scores are relative to Seasonal Naive; lower is better. Each entry reports MASE / WQL for GIFT-Eval and MASE / SQL for FEV-Bench.

Training strategyGIFT-Eval (MASE / WQL)FEV-Bench (MASE / SQL)
Progressive curriculum0.6588 / 0.46650.6170 / 0.5168
Mixed-task training0.6735 / 0.47840.6332 / 0.5321

This comparison assesses whether progressively introducing tasks improves performance over including all five task types from the outset under a matched training budget. The results are also included in Appendix D.3, Table 14.

Response to W2 (Matched context lengths on TFB)

We agree that the original TFB comparison uses different histories: Aurora-X receives 2,880 observations, whereas the eight supervised baselines receive 96. We evaluated the same final Aurora-X checkpoint with a 96-point history.

The matched-context evaluation keeps the four TFB horizons (96, 192, 336, and 720), data splits, normalization, forecast origins, and scoring protocol unchanged. We use native 48-point patches for all eight datasets, including Solar, so that the patch span does not exceed the matched history. The eight supervised baselines retain their published 96-point inputs.

To anchor the comparison to an existing end-to-end model rather than introduce a new supervised training regime, we use PatchTST as the comparator. PatchTST is trained directly on the TFB training splits with a 96-point history. We use the four-horizon averages from the existing full TFB table; PatchTST is a supervised reference, not a zero-shot model.

Each cell reports MSE / MAE averaged over the four horizons within each dataset; lower is better. Existing Aurora-X and PatchTST values and the measured 96-point Aurora-X results are shown.

TFB datasetAurora-X, 2,880 (zero-shot)Aurora-X, 96 (zero-shot)PatchTST, 96 (E2E)
ETTh10.390 / 0.4110.458 / 0.4410.469 / 0.455
ETTh20.353 / 0.3800.383 / 0.4100.387 / 0.407
ETTm10.335 / 0.3640.379 / 0.3920.387 / 0.400
ETTm20.268 / 0.3130.277 / 0.3240.281 / 0.326
Weather0.211 / 0.2450.247 / 0.2650.259 / 0.281
Solar0.222 / 0.2420.258 / 0.2940.270 / 0.307
Electricity0.170 / 0.2660.194 / 0.2980.216 / 0.304
Traffic0.413 / 0.2700.525 / 0.3420.555 / 0.362

The matched 96-point Aurora-X results are measured zero-shot results under the finalized inference protocol. The complete matched-context protocol and table are provided in Appendix D.4.

Response to Q2 (Post-training pattern-guidance)

Figure 7 evaluates MoE variants at the pretraining checkpoint. We agree that this does not establish whether the pattern-guided advantage survives midtraining and Post-training. We therefore added a full-curriculum pattern-guidance ablation that removes only Lpat\mathcal L_{\mathrm{pat}} throughout the five stages. The corpus, backbone, routing capacity, optimizer, stage budgets, and evaluation protocol remain unchanged.

Both point and probabilistic metrics are reported; lower is better.

Full-curriculum variantGIFT-Eval (MASE / WQL)FEV-Bench (MASE / SQL)
Aurora-X (full)0.6588 / 0.46650.6170 / 0.5168
w/o Lpat\mathcal L_{\mathrm{pat}}0.6745 / 0.49380.6427 / 0.5466

Testing temporal-pattern specialization beyond dataset identity

The existing routing probe uses TFB-based scenarios, but the scenario groups can also reflect dataset identity. To address this confound, we added a matched within-dataset probe using the same held-out windows and layer-wise routing extraction:

  1. sample equal numbers of windows from each dataset and temporal-characteristic group;
  2. compute expert-selection profiles for the same windows at the final checkpoint;
  3. measure separation between temporal-characteristic groups within each dataset, then average across datasets; and
  4. compare it with the corresponding separation after permuting characteristic labels within each dataset.

We use the normalized Jensen--Shannon (JS) distance because each expert-selection profile is a probability distribution. Under this normalization, 00 means identical routing profiles and larger values mean greater separation, with 11 as the maximum. The three analyses have distinct purposes:

  • Temporal pattern: measures whether windows with different temporal characteristics receive different routing profiles within the same dataset, thereby controlling for dataset identity.
  • Dataset identity: measures the routing separation between dataset groups and quantifies how much routing variation could be explained by source or dataset differences.
  • Null control: permutes the characteristic labels within each dataset and estimates the separation expected from sampling variation and the grouping procedure alone.
AnalysisGroupingNormalized JS distance
Temporal patternCharacteristics within each dataset0.12
Dataset identityDataset ID0.15
Null controlPermuted labels within each dataset0.03

The within-dataset temporal-pattern distance (0.120.12) is substantially above the permuted-label null (0.030.03), supporting pattern-dependent routing beyond chance. The larger dataset-identity distance (0.150.15) also shows that source identity contributes to routing; therefore, we do not claim that routing is independent of dataset identity. The revised appendix includes routing profiles across the full curriculum, which test whether these effects persist beyond pretraining. The post-training routing analysis and identity-controlled probe are provided in Appendix D.5.

Response to Q3 and Q4 (Context length and inference-resolution selection)

The matched-context TFB comparison above addresses the effect of history length; the results are also reported in Appendix D.4. Here we clarify the inference-resolution settings.

Resolution selection is based on the available input length, not on benchmark scores. The native patch length is 48. For histories of at least 48 observations, we use the default temporal span of 48 observations per token. When the history is shorter than one native patch, we apply a short-history rule to select a smaller span. These shorter spans are resampled to the model's native patch length, allowing the same model to process short inputs without changing its weights. The adjustment is intended to preserve the information available in the short history.

Available historyInference span
At least 48 observations48 (default)
Fewer than 48 observationsSmaller than 48, using the short-history rule

When a separate validation set is available, searching over inference resolutions may further improve forecasting accuracy beyond the default input-length-based rule. Such tuning is applicable only when validation data are available; the selected resolution is then fixed for test evaluation, without using test targets or test scores for selection.

The resolution sweep in Appendix A.11 is a separate sensitivity analysis to demonstrate the usage of budget control. The spans of 48, 96, 192, 336, and 480 were used to study temporal detail, historical coverage, and token usage. That sweep is not the procedure used to select inference spans for the reported GIFT-Eval and TIME results.

We have clarified this distinction and the input-length-based selection rule in Appendix D.6.

We thank the reviewer for requesting clarification of the evaluation protocol.

Reviewer_w12ZOpenReview ↗
Rating
4(+1)
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces Aurora-X, a time series foundation model with bilion-scale parameters that combines multivariate modeling, known future covariates, parallel forecasting, and continuous quantile prediction. Its main components are a pattern-guided mixture-of-experts architecture, an implicit quantile network head, and a progressive training curriculum. Variable-resolution post-training allows the model to adjust the temporal span represented by each token while keeping its weights fixed. Experiments cover five benchmarks, with additional analyses of expert routing, forecasting heads, training stages, and inference efficiency.

Critical strengths

S1. The model offers a coherent combination of practically useful forecasting capabilities.

S2. The pattern-guided routing mechanism is well motivated.

S3. The empirical evaluation is broad and generally encouraging.

Critical weaknesses

W1. Differences in historical context lengths. On TFB, Aurora-X uses 2,880 historical observations, whereas eight baselines use 96. A comparison with matched context lengths would help distinguish architectural gains from the benefits of additional historical information.

W2. Incomplete attribution of component and training gains. The current comparisons do not fully isolate the pattern-guidance loss or distinguish curriculum benefits from additional training. Removing only the pattern loss and comparing training strategies under matched budgets would strengthen the evidence.

W3. The methodological novelty could be clearer. The unified framework has practical value, but several capabilities build on established approaches. The paper could more clearly identify its central methodological advances, particularly how its resolution adaptation and post-training strategy extend prior work beyond system integration.

Questions for authors

See weaknesses.

3 comments

Response to Reviewer w12Z

Rebuttal Summary

  • Context-length control: We added a TFB evaluation with a 96-point history and compared it with PatchTST, a standard end-to-end supervised baseline using the same history length.
  • Attribution: We added focused controls removing only the pattern-guidance loss and comparing the proposed curriculum with joint multi-task training under a matched budget.
  • Novelty: We clarified that the methodological contributions are the time-series-specific routing objective, continuous quantile forecasting in the unified forecaster, and fixed-weight variable-resolution Post-training—not the isolated invention of MoE, IQN, or staged training.

We thank the reviewer for recognizing the practical value of Aurora-X and for identifying the remaining attribution and positioning gaps.

Response to W1 (Matched context lengths on TFB)

We agree that the original TFB comparison does not fully separate architectural gains from the benefit of a longer history. We therefore evaluated the same final Aurora-X checkpoint with a 96-point history.

The matched-context evaluation uses:

  • 96 observed points at each forecast origin;
  • the same four TFB horizons (96, 192, 336, and 720), data splits, normalization, and scoring protocol;
  • native 48-point patches for all eight datasets, including Solar, so that the patch span does not itself exceed the 96-point history; and
  • no change to the model weights or benchmark-specific preprocessing.

To anchor the comparison to an existing end-to-end model rather than introduce a new supervised training regime, we use PatchTST as the comparator. PatchTST is trained directly on the TFB training splits with a 96-point history. We use the four-horizon averages from the existing full TFB table; PatchTST is a supervised reference, not a zero-shot model.

The eight supervised baselines retain their published 96-point inputs. The table records the existing Aurora-X and PatchTST reference values together with the measured 96-point Aurora-X results. Each cell reports MSE / MAE averaged over the four horizons within each dataset; lower is better.

TFB datasetAurora-X, 2,880 (zero-shot)Aurora-X, 96 (zero-shot)PatchTST, 96 (E2E)
ETTh10.390 / 0.4110.458 / 0.4410.469 / 0.455
ETTh20.353 / 0.3800.383 / 0.4100.387 / 0.407
ETTm10.335 / 0.3640.379 / 0.3920.387 / 0.400
ETTm20.268 / 0.3130.277 / 0.3240.281 / 0.326
Weather0.211 / 0.2450.247 / 0.2650.259 / 0.281
Solar0.222 / 0.2420.258 / 0.2940.270 / 0.307
Electricity0.170 / 0.2660.194 / 0.2980.216 / 0.304
Traffic0.413 / 0.2700.525 / 0.3420.555 / 0.362

The matched 96-point Aurora-X results are measured zero-shot results under the finalized inference protocol. The complete matched-context protocol and table are provided in Appendix D.4.

Response to W2 (Component and training attribution)

We agree that the original stage-wise curves confound the benefit of the curriculum with the additional optimization steps and the changing task configuration. We added the following controlled comparisons:

  1. Pattern guidance. We removed only the pattern-guidance loss Lpat\mathcal L_{\mathrm{pat}} throughout Pre, M1–M3, and Post while keeping the experts, top-4 routing, backbone, corpus, optimizer, and training budget unchanged. The comparison uses the models after full training.
  2. Progressive curriculum. We compared the progressive schedule with mixed-task training, in which the five task configurations associated with Pre, M1, M2, M3, and Post are included together from the start of training. Unlike the curriculum, this baseline does not introduce tasks in successive stages. The training corpus, total number of optimization updates, and compute budget are matched.

Both comparisons are evaluated after the full training budget, with the final Aurora-X model as the common reference. All scores are relative to Seasonal Naive; lower is better. Each entry reports MASE / WQL for GIFT-Eval and MASE / SQL for FEV-Bench.

VariantGIFT-Eval (MASE / WQL)FEV-Bench (MASE / SQL)
Aurora-X0.6588 / 0.46650.6170 / 0.5168
w/o Lpat\mathcal L_{\mathrm{pat}}0.6745 / 0.49380.6427 / 0.5466
Mixed-task training0.6735 / 0.47840.6332 / 0.5321

These comparisons assess the contribution of the pattern-guidance loss and whether progressively introducing tasks improves performance over including all five task types from the outset under a matched training budget. The full controlled-attribution results are provided in Appendix D.3.

Response to W3 (Methodological novelty)

We agree that the introduction should distinguish methodological advances from established ingredients. We do not claim to have invented MoE, IQN, multi-resolution modeling, or staged training in isolation. Our claims are narrower and more specific:

  • Pattern-guided routing. Standard load balancing controls expert utilization, but does not specify which temporal patterns should share an expert. Aurora-X learns shallow patch similarities and uses a normalized co-assignment loss, propagated through a straight-through estimator, to guide deep routing. An orthogonality regularizer prevents the pattern projection from collapsing. The resulting routing behavior is input-dependent across depth rather than merely dataset-specific expert usage.
  • Continuous quantile forecasting. IQN is established. Our contribution is its integration with a covariate-aware, parallel patch forecaster: the head conditions on both the future state and a continuous quantile level, allowing arbitrary requested marginal quantiles in one backbone pass without sampling multiple trajectories.
  • Variable-resolution Post-training. During Post, we train on aligned resampled history–target pairs. At inference, we vary the temporal span per token and restore predictions to the requested native grid without updating the trained model weights. This allows temporal coverage and token usage to be adjusted without retraining. Appendix B formalizes the equivalence between data-side resampling and the corresponding input/output projections.
  • Capability-oriented curriculum. The curriculum progressively introduces temporal pattern modeling, cross-variable dependencies, variable context and horizon lengths, future covariates, and resolution adaptation. We compare this schedule with a mixed-task baseline that includes all five task types from the start of training.

We have revised the introduction to state these distinctions explicitly and to avoid presenting the underlying established components as standalone inventions.

We appreciate the reviewer’s constructive assessment. The added context-matched and attribution-controlled evaluations make the source of the reported gains more transparent. The inference-resolution selection policy is formalized in Appendix D.6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.