- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a post-hoc correction method for time-series forecasting without retraining the backbone model. It assumes that a frozen model generally underestimates or overestimates certain forecasts and therefore requires corrections (which are input-dependent and should not always be applied). The authors propose using a validation set to learn a regression model that predicts the error the base model will make based on the input context. This trained linear model (subject to low-rank constraints) is then applied during test time to estimate the base model's errors in advance and apply corrections (they evaluate two types of corrections: a pure correction and an adaptive one). Finally, at test time, they use delayed feedback to select the most suitable forecast.
Critical strengths
(i) The post-hoc corrections do not require retraining the backbone model. (ii) The proposed approach seems to yield satisfactory performance. (iii) Several ablation studies have been conducted to validate various design choices.
Critical weaknesses
(i) The method relies on a validation set, which is not always available (sometimes requiring a reduction in training set size, which hinders a fair evaluation and limits practical applicability). (ii) The method assumes the availability of delayed feedback, which is not always applicable in practice. Furthermore, the paper does not compare against existing methods that utilize delayed feedback (specifically test-time adaptation and online time-series forecasting approaches, even if those methods update the backbone). At a minimum, one would expect Pareto curves (memory vs. performance and compute vs. performance) comparing this framework to other online or test-time adaptation approaches. (iii) The low-rank constraint on the linear mapping from the input features needs to be explicitly ablated. Other algorithmic design choices also require stronger motivation. For example, why use both Pure and Adaptive corrections? Why not more variants? Why only these two, and how do they complement each other? These aspects remain unclear.
Here is a non exhaustive list of papers that will be nice to be compared to the papers at least from a performance vs. compute/memory point of view. I do not believe claiming retraining qualitatively is a reason to discard them from the baselines. We clearly need some quantitative fact (either by complexity analysis or a compute vs. performance curve for these methods and the one of the paper).
Lightweight Online Adaptation for Time Series Foundational Model Forecasts (ICML'25) Battling the non-stationarity in time series forecasting via test-time adaptation (AAAI'25) Adanodes: Test time adaptation for time series forecasting using neural odes. Shift-aware test time adaptation and benchmarking for time-series forecasting.
Also it exists several post training approaches for time series forecasting models (which uses augmented context and doesn't need retraining of the backbone which would be worthy to check)
Instance Adaptive Prompts as Context for Time series Foundational Models Adapt data to model: Adaptive transformation optimization for domain-shared time series foundation models.
And even more which can be found in this literature review (A Survey of Post-Training in Time Series Foundation Models)
Questions for authors
-
How can the method be applied in scenarios without a standard validation set? Should a portion of the training set be carved out for validation? Furthermore, for benchmark datasets with standardized validation sets, what if that set was already used for model selection or early stopping during backbone training? Is it still a reliable candidate for predicting residual errors?
-
How does the framework handle applications where delayed feedback is unavailable?
-
How does the method compare to test-time adaptation (TTA) or online adaptation approaches in terms of the performance vs. compute/memory trade-off? Could the authors provide 2D efficiency plots (e.g., performance vs. latency/memory) to illustrate the method's advantages over test-time approaches that require: (i) full retraining, (ii) fine-tuning, or (iii) neither (e.g., parameter-free methods utilizing context augmentation or output calibration) using delayed feedback (which is super additional information)?
Calibration Fairness, Adaptation Costs, and Rank Ablation
Calibration fairness. Base, Pure and SAR share the training split and checkpoint; Base's training set is not reduced. Appendix D.1 uses the first 25% of validation origins for checkpoint selection, an L+H gap before calibration, and an H-origin gap between fitting and assessment. The assessment block selects both step and initial action: its LCB guides selection, not an independent post-selection certificate (Appendix E.3). If early stopping used all validation data, use fresh chronological calibration; reuse is not independent. Without a validation set, reserve history or collect later labels, disclose this cost and match comparators' data budgets. Insufficient calibration means Base fallback.
Feedback. At decision t, only losses from origins j with j+H≤t inform selection. Earlier test outcomes can inform later decisions; weights, maps and past forecasts remain unchanged. Comparators must receive the same histories and maturity schedule. Without target feedback, only pre-test static choice or Base is available, with no online-selection gain. Extra dense forecasts in the foundation-model evaluation count toward its cost.
Adaptation and costs. The candidates share one backbone prediction, without backbone backpropagation. We added Chronos-Bolt Base and Moirai-2.0-R-small comparisons with one-time/periodic fine-tuning and TAFAS-GCM/full-feedback. The accuracy pilot covers six domains with seed2025. New costs cover energy (epf_np) and cloud (bizitobs_l2c_1H): four model-task cases, five separately run methods, one repeat, 457 issued origins and 20 official scored windows per case.
Totals include initialization, prediction and updates once; loading, input reads, scoring and audit I/O are excluded. Synchronization/profiling overhead remains. These are shared-GPU diagnostics, not isolated speed or minimal-RSS measurements. SAR's process-allocated GPU peaks are lower here, but it is not always faster. On Chronos/cloud, periodic FT lowers MASE by 8.51%, while SAR raises it by 0.58%.
FT uses at most 50 initial steps, pre-test checkpoint selection including zero updates, and five periodic steps every H origins. Moirai uses a declared quantile-loss reference. TAFAS-GCM omits native partial-label updates and retrospective adjustment; it cannot establish a win over native TAFAS. Of 20 measurements, 19 reproduce historical predictions within the declared tolerance. Moirai/cloud/TAFAS matches a fresh unchanged-recipe replay but not its historical artifact; the cause is unresolved and both versions are retained. Full retraining, ELF and other requested online/context baselines remain gaps. An isolated-device Pareto comparison remains outstanding; pending forecasts and audit arrays need separate memory accounting.
Rank and design choices. All 24 fixed-rank4/full jobs are complete: SimpleTM and MDMLP-EIA on ETTh1/H336 and PEMS08/H48, with three seeds. Checkpoints, data, splits and non-rank search grids match; selected ridge, step and Adaptive structure may differ.
Changes are 100×(rank4/full−1) after averaging three seed errors; negative favors rank4. Always-on Pure is the primary control: full is better in three of four combinations. We do not claim rank4 is necessary for accuracy. Small SAR differences do not establish statistical equivalence or optimal automatic rank selection.
Pure shares residual structure across steps/channels; Adaptive chooses spatial, pooled temporal or shared/channel-specific structure before testing; Base permits no correction. Separate 96-task, four-backbone, three-seed controls give mean MSE changes from Base of −4.56% for full SAR, −3.17% for Base+Pure and −3.64% without the issued-loss budget. This supports the additional candidate and accounting roles. Reserve removal and Base initialization remain competitive; we do not claim every threshold is necessary.
We will integrate verified results within the permitted revision window. As remaining experiments finish, we will complete the tables, plots, protocols, analysis and limitations, marking unfinished evidence and posting updates only within the allowed discussion period.