acceptodds
Under review as a conference paper at ICLR 2027

PRE: An In-Context Residual Estimation Paradigm for Plug-and-Play Time-Series Forecast Correction

Abstract

Time-series foundation models (TSFMs) are gaining popularity for their strong zero-shot forecasting capabilities and ease of deployment. However, recurring task-specific errors and distribution shifts create a need for adaptation during deployment, as frequent fine-tuning can be costly and requires careful hyperparameter selection and domain expertise. To address this challenge, we propose Pretrained Residual Estimator (PRE), which introduces a plug-and-play in-context learning (ICL) paradigm for post-hoc forecast correction. By conditioning on labeled historical examples, PRE infers residual corrections without task-specific online parameter updates. To support large-scale ICL pretraining, we synthesize diverse residuals and propose a two-stage pretraining strategy, first training PRE on synthetic residuals and then refining it on real residuals. PRE is independent of the base forecaster's internal architecture and supports variable context lengths and forecast horizons. Extensive evaluations demonstrate that PRE consistently improves the forecasting performance of unseen TSFMs on unseen tasks during its training and substantially outperforms supervised residual adapters, highlighting the potential of ICL as a generalizable paradigm for post-hoc forecast correction.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, time-series foundation models, residual estimation, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HNuVOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRE, a plug-and-play model that estimates and corrects the residuals of a base model without any parameter updates at deployment, by using past inputs, forecasts, and residuals as in-context examples. PRE builds a forecast memory consisting of past (input, forecast) scenarios and their residuals. Based on the similarity between the current scenario and the past scenarios, it computes a weighted aggregation of past residual information to predict a per-step residual distribution, and then corrects the forecast error by adding the median of this distribution to the current forecast. PRE is trained only on residuals from AutoARIMA, AutoTheta, and DLinear, using a two-stage strategy: it is first trained on synthetic residuals, obtained by adding combinations of eight types of interpretable error components to real residuals, and then further trained on real residuals that exhibit more complex patterns. When applied to 10 TSFMs, PRE reduces MASE by 2.69–10.89% in the in-domain setting and by 1.88–8.52% in the zero-shot setting. PRE also outperforms existing supervised residual correction methods such as COSA, ORCA, and δ\delta-Adapter, and yields further improvements when combined with them. Through ablation studies, the authors validate the effectiveness of the historical residual labels and of the two-stage training with synthetic residuals, and quantitatively report the performance and inference overhead with respect to the forecast memory size (K) and the forecast horizon (H).

Critical strengths
  • S1: The paper reformulates post-hoc forecast correction for TSFMs as an in-context residual regression problem, presenting a novel approach that uses past (input, forecast, residual) triples as in-context examples. Unlike existing residual correction methods, which rely on supervised parameter updates during deployment, the proposed method performs task-specific correction through contextual inference alone, without any parameter updates at deployment. As a result, it requires neither hyperparameter tuning nor expertise in model training.
  • S2: Since the proposed method uses only the inputs and outputs of the base model, it can be applied regardless of the base model's internal architecture. Although it is trained only on residuals from lightweight forecasters, it improves performance in both the in-domain and zero-shot settings across 10 TSFMs with diverse architectures, demonstrating generalization to both unseen models and unseen tasks.
  • S3: The paper addresses the scarcity of real residual data required for large-scale ICL pretraining through residual synthesis. A model trained solely on synthetic residuals still achieves substantial gains, confirming that the synthetic error patterns transfer to real forecasting errors.
Critical weaknesses
  • W1: The proposed model (approximately 10.7M parameters) is structurally much larger than the linear or small MLP-based adapters used as baselines, and it incurs both a pretraining cost and the cost of encoding K past scenarios for every forecast. However, no quantitative comparison with the online update cost of the baselines is provided, and Table 22 does not include the cost of generating base forecasts at past time steps, which is required to construct the memory (i.e., the cold-start cost). An end-to-end cost comparison with the baselines is needed to support the claim of "low deployment overhead."
  • W2: The paper presents the cost of fine-tuning and the difficulty of hyperparameter selection as its main motivation, yet it provides no comparison with TSFM fine-tuning (full or PEFT) in terms of either performance or cost. It also lacks comparisons with simple baselines, such as adding the mean of recent residuals or the most recent residual, or other residual-based statistical correction methods. Further evidence is therefore needed to justify the necessity of a model with approximately 10.7M parameters.
  • W3: The performance gains are limited to less than 0.5% in the Finance, Economics, Sales, and Healthcare domains, and performance even degrades on some datasets. To assess the practical value of the method, horizon-wise results and an analysis of the conditions under which the correction degrades performance are needed.
  • W4: Although PRE is designed to learn the residual distribution via an NLL objective, the quality of the predicted distribution (e.g., CRPS, coverage, calibration) is not evaluated, and it is not verified whether distributional modeling offers any advantage over direct point-estimate regression.
  • W5: The paper does not report results at the dataset × model level, which makes it difficult to verify the results in Tables 10 and 11. Moreover, all experiments are conducted with a single random seed, so the variability of the results cannot be assessed.
Questions for authors
  • Q1: Could you provide the end-to-end cost comparison described in W1?
  • Q2: Could you provide comparisons with TSFM fine-tuning and residual-based statistical correction methods, as described in W2?
  • Q3: Could you provide results and an analysis for the cases where performance degrades, as described in W3?
  • Q4: Could you provide an evaluation of the probabilistic forecasting performance described in W4, as well as results obtained using statistics other than the median (e.g., the mean)?
  • Q5: Could you add the complete dataset × model results described in W5? In addition, could you report confidence intervals based on experiments with at least three random seeds?
6 comments
AuthorsOpenReview ↗

Response to Reviewer HNuV (Part 1/5)

We sincerely appreciate your thoughtful and constructive review, and are particularly grateful for your recognition of PRE's core idea of formulating post-hoc forecast correction as an in-context residual regression problem.

We have carefully considered all your concerns regarding the sufficiency of our experimental results and conducted extensive additional experiments to address them. Specifically, we have added a deployment cost comparison with existing online adaptation methods (W1, Section 4.7 and Appendix D.6), comparisons with TSFM fine-tuning and training-free statistical correction methods (W2), and further analyses of domain-dependent correction performance (W3, Section 4.4). We have also evaluated the probabilistic forecasting quality of PRE and compared distributional residual estimation with direct point-estimate regression (W4, Appendix D). Furthermore, we have provided complete dataset-level results for all 10 TSFMs and conducted additional experiments with three random seeds (W5). These additional experiments provide further evidence of PRE's effectiveness, deployment efficiency, probabilistic modeling capability.


W1 and Q1: Discussion on deployment cost

Thank you for pointing out the potential additional cold-start costs during deployment. To address your concerns, we would first like to clarify that users do not need to train PRE during deployment, and its forecast memory Mt\mathcal{M}_t only maintains the most recent KK forecasting records, as detailed in Section 4.1. Therefore, the cold-start cost only involves generating KK forecasts using the base forecaster.

Cold-start cost. To further address your concern, we report the time required by statistical forecasters and foundation models to construct K=16K=16 historical forecasting records used in our main experiments, with context length L=512L=512 and forecast horizons H∈24,48,96H \in \\{24, 48, 96\\}.

ForecasterDeviceH=24 (ms)H=48 (ms)H=96 (ms)
AutoThetaCPU502.4559.2784.4
AutoARIMACPU3470.93554.24973.3
DLinearCPU0.30.30.4
TimesFM 2.5GPU46.848.048.4
Moirai 2.0GPU7.37.517.7
Chronos 2GPU14.314.514.6

As shown above, the cold-start cost for foundation models remains below 50ms, which is generally negligible. Even the slowest statistical forecaster, ARIMA, requires only 5.0 seconds to generate 16 forecasts with L=512,H=96L=512, H=96. Moreover, this cost is incurred only once at the beginning of a task. During deployment, Mt\mathcal{M}_t is maintained in a first-in-first-out (FIFO) manner, without reconstructing the entire memory for each forecast. We have further clarified the online memory update procedure in Section 4.1 to avoid potential confusion.

Online cost compared with baselines. Following your suggestion, in addition to the PRE deployment cost already reported in Figure 3 of Section 4.7 and Table 24 of Appendix D.6, we further compare PRE with the three online adaptation baselines under the same hardware conditions (as detailed in Appendix B.5) at H=96H=96. The results are reported below:

MethodOnline updating (ms)Inference (ms)Total deployment (ms)
PRE06.946.94
COSA5.480.235.71
ORCA8.350.468.81
δ-Adapter7.780.288.06

The results show that although online adapters have low inference costs, they require additional repeated computation for parameter updating during deployment. In contrast, our PRE shifts this computation to offline pretraining. Therefore, PRE achieves a comparable overall deployment cost and delivers substantially better performance with much larger model capacity. Overall, these results further support the effectiveness and low deployment cost of PRE. We hope these clarifications and additional results address your concerns, and sincerely thank you again for your constructive suggestions.

AuthorsOpenReview ↗

Response to Reviewer HNuV (Part 2/5)

W2 and Q2: Comparison with TSFM fine-tuning and statistical methods

We appreciate your valuable suggestions on TSFM fine-tuning and statistical residual correction. To address your concerns, we have conducted the following additional experiments.

Comparison with TSFM fine-tuning. Due to the limited time and the large number of models and tasks, it was impractical to design effective fine-tuning strategies for all 10 TSFMs across 85 tasks. We therefore select Chronos-2[1] and TimesFM-2.5[2] to finetune as their official implementations provide fine-tuning interfaces. We fine-tuned each model on 85 tasks using LoRA under their suggested fine-tuning settings. Furthermore, we applied the original, unmodified PRE to correct the forecasts of the fine-tuned base models.

MethodIn-domain MASEIn-domain GainZero-shot MASEZero-shot Gain
Chronos-20.914--0.886--
Chronos-2+PRE0.840+8.06%0.835+5.76%
Chronos-2 LoRA0.869+4.91%0.857+3.35%
Chronos-2 LoRA+PRE0.820+10.25%0.816+8.00%
TimesFM-2.50.853--0.834--
TimesFM-2.5+PRE0.830+2.69%0.818+1.88%
TimesFM-2.5 LoRA0.832+2.37%0.795+4.65%
TimesFM-2.5 LoRA+PRE0.819+3.89%0.788+5.51%

Results show that compared with fine-tuning alone, applying PRE to the original base models achieves better or comparable performance. More importantly, even after fine-tuning on specific tasks, the original, unmodified PRE can still provide further performance improvements: PRE reduces MASE by 5.64% and 4.78% on fine-tuned Chronos-2, and by 1.56% and 0.88% on fine-tuned TimesFM-2.5. These results highlight an important advantage of PRE as we discussed in Section 1: PRE provides effective plug-and-play correction without requiring users to understand the internal architectures of base models to conduct task-specific tuning. In contrast, fine-tuning requires additional model- and task-specific experimentation, and frequently updating large TSFMs online is computationally demanding.

Comparison with Statistical Methods. We evaluate four training-free statistical correction methods, including using the most recent residual (i.e., a zero-order hold), averaging historical residuals, and KNN. For KNN, we use the same memory size as PRE (K=16K=16) and evaluated two strategies: using the most similar residual and weighted averaging the similarity of residuals.

MethodIn-domain MASEIn-domain GainZero-shot MASEZero-shot Gain
Base TSFM0.915--0.901--
Last Residual1.246−36.17%1.189−31.84%
Mean Residual (K=16)0.966−5.63%0.945−4.80%
Nearest-neighbor Residual1.150−25.72%1.114−23.67%
KNN weighted (K=16)0.943−3.12%0.919−1.93%
PRE0.860+5.99%0.859+4.71%

The results show that these statistical methods fail to improve forecasting performance. For a well-specified forecasting model, the remaining residuals are expected to behave approximately as white noise, with little systematic structure left to exploit[3]. Our experimental analysis also reveals that, for strong base forecasters, useful patterns in the residuals are often obscured by noise. Consequently, directly aggregating raw residuals makes it difficult to identify beneficial correction structure. Considering the ability of deep learning models to suppress noise and extract informative patterns, as well as the preliminary success of existing studies in forecast correction (as discussed in Section 2.2), we further sought a generalizable mode to perform inference without task-specific training. These considerations motivated our choice of a pretrained model as the backbone of PRE. These additional experiments, together with the existing results in Section 4.2, demonstrate the effectiveness of using a pretrained model to perform residual inference through in-context learning.

We sincerely appreciate your suggestion for additional comparisons, which has helped us further demonstrate the value of our work. We will also incorporate comparisons with fine-tuning methods and their combinations with PRE into the revised manuscript. We hope these additional experiments and analyses fully address your concerns.

[1] Ansari, Abdul Fatir, et al. “Chronos-2: From Univariate to Universal Forecasting.” arXiv preprint arXiv:2510.15821 (2025).

[2] Das, Abhimanyu, et al. “A Decoder-Only Foundation Model for Time-Series Forecasting.” Proceedings of the 41st International Conference on Machine Learning (2024).

[3] Hyndman, Rob J., and George Athanasopoulos. Forecasting: Principles and Practice. 3rd ed. OTexts (2021).

AuthorsOpenReview ↗

Response to Reviewer HNuV (Part 3/5)

W3 and Q3: Analysis of applicability

Thank you for your thoughtful comments on the domain-wise performance of PRE. We appreciate the opportunity to further clarify the factors that may influence its effectiveness across different forecasting domains. Although we have discussed this issue in Section 4.4 and outlined related future research directions in Section 5, we provide a more detailed analysis using specific datasets to better explain the current applicability of PRE.

The effectiveness of historical residual correction depends on how well the underlying system dynamics can be inferred from the observed forecasting context. The future evolution of a dynamic system may be governed by its internal dynamics, as reflected in its historical states, or substantially influenced by external factors beyond the observed time series [4]. In the former case, historical observations can provide key information for inferring future changes. In the latter, if such external factors are not recorded, historical observations alone are insufficient to explain future system evolution, thereby weakening the transferability of historical scenario–residual relationships.

Examples in our experiments. As discussed in Section 4.4, datasets in Finance, Healthcare and Sales are substantially subject to external factors such as public policies, human behavior, and market interventions [5] [6]. Specifically, we further examined all four datasets on which PRE fails to achieve positive correction gains and found that their forecasting targets are influenced by broader external economic and social factors that are not captured by the available observations:

DatasetRecorded DataImportant External Factors
Port ActivityInternational cargo flowsGlobal trade demand, port operations
CryptoCryptocurrency pricesInvestor sentiment, regulatory policies
Uncertainty 1MEconomic uncertainty indicesMonetary and fiscal policies, macroeconomic indicators
epf electricity priceElectricity pricesElectricity demand, fuel costs, generation capacity

This analysis is consistent with the motivation behind PRE's architecture design: its effectiveness depends on whether historical forecasting scenarios contain sufficient information to explain residual variations. This also motivates a promising future direction discussed in Section 5: incorporating exogenous covariates that influence future system evolution into the forecast memory, with the aim of improving forecasting performance on tasks that are strongly affected by external factors.

[4] Cheng, Changqing, et al. “Time Series Forecasting for Nonlinear and Non-stationary Processes: A Review and Comparative Study.” IIE Transactions 47, no. 10 (2015): 1053–1071.

[5] Stock, James H., and Mark W. Watson. “Macroeconomic Forecasting Using Diffusion Indexes.” Journal of Business & Economic Statistics 20, no. 2 (2002): 147–162.

[6] Kraemer, Moritz U. G., et al. “The Effect of Human Mobility and Control Measures on the COVID-19 Epidemic in China.” Science 368, no. 6490 (2020): 493–497.

Reviewer_cdzPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PRE corrects forecasts from frozen models using attention over historical scenario–residual pairs. It is pretrained on synthetic and real residuals from 3 lightweight forecasters and evaluated on 10 unseen TSFMs across 85 tasks. Aggregate MASE falls by 5.99% in-domain and 4.71% on datasets held out from PRE pretraining.

Critical strengths
  1. Transfer from 3 lightweight training forecasters to 10 unseen TSFMs is convincing: aggregate MASE falls by 5.99% in-domain and 4.71% on held-out datasets (Table 7), with complementary gains alongside online adapters (Tables 8–9). This supports usefulness beyond the training forecasters.
  2. Black-box correction requires no deployment-time parameter updates, with median GPU overhead of 6.94 ms at horizon 96 (Tables 1 and 22). This makes PRE a practical plug-in for frozen forecasters.
Critical weaknesses
  1. Offline pretraining, model capacity, and historical residual labels change together, leaving the benefit of scenario–residual matching unclear. Same-memory residual baselines and label-reassignment controls are missing, limiting attribution to the proposed mechanism.
  2. One-seed results lack uncertainty estimates, and correlated sampling frequencies receive separate weights. Together with incompletely specified online evaluation protocols, this leaves the robustness and reproducibility of smaller gains on strong forecasters uncertain.
Questions for authors
  1. Mechanism: How does PRE compare with same-memory residual-averaging or similarity baselines, and with residual labels reassigned among historical scenarios?
  2. Uncertainty: Can you report paired uncertainty estimates accounting for temporal/source dependence and aggregation by underlying source?
  3. Protocol clarification: Please specify historical-forecast construction, chronological splits, normalization, recency bias, and baseline update/tuning settings.
Reviewer_kf7rOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

TSFMs sexhibit task-specific errors in deployment-time, while fine-tuning and existing online adapters require costly parameter updates. This paper proposes PRE, which corrects forecasts by estimating residuals via in-context learning over historical context-prediction–residual pairs. PRE is an in-context learning Transformer trainied by two-stage pretraining with synthetic residuals, and doesn't need deployment-time updates. Experiments cover in-domain and zero-shot settings on the TIME benchmark, multiple unseen TSFMs, comparisons with supervised adapters, ablations, and cost analysis. Overall, the paper is well written and the experiments are comprehensive, while several claims and the motivation behind design choices are not clear.

Critical strengths

1.The paper is written clearly and provides sufficient implementation details, which makes the method easy to reproduce and build upon.

2.The experiments are comprehensive, covering many TSFMs, multiple test-time adaptation baselines, in-domain and zero-shot forecasting settings, and analyses of memory size and efficiency, which support the performance superiority.

Critical weaknesses
  1. I think residual-based test-time adaptation may be not a very critical problem for TSFMs. Although it brings gains in forecasting accuracy, a ~5% improvement may not be crucial for advanced models. A more important problem may be how test-time adaptation can mitigate errors caused by distribution shifts that are rare but may lead to severely deviated forecasts.
  2. Although pretraining PRE with synthetic residuals avoids parameter updates at test time, it introduces pipeline complexity that makes the method not truly “plug-in”: one must first collect in-domain data and generate synthetic residuals, which incurs additional time cost (e.g., classical forecasters such as ARIMA can be slow). PRE may therefore only show its advantage under longer-term deployment.
  3. For the design of the in-context estimator, there may be simpler training-free alternatives, e.g., using KNN to select or aggregate the residuals of similar scenarios [1].
  4. In Eq. 11, why is the scenario representation fused with the residual representation, rather than fusing the forecast representation with the residual, or directly adding them without latent fusion?
  5. Residual fitting has been widely studied, and there are also quite a few in-context-learning-like approaches, such as ResMem [1]. While this may be new for time series forecasting, it is not very exciting.

[1] Yang, Zitong, et al. “ResMem: Learn what you can and memorize the rest.” Advances in Neural Information Processing Systems 36 (2023): 60768–60790.

Questions for authors
  1. How is the forecast memory constructed at the beginning of the test segment? Will the memory update the content ? How do you maintain the memory while testing ?
  2. Could the authors report the wall-clock cost of pretraining, including the synthetic data generation time ?
4 comments
AuthorsOpenReview ↗

Response to Reviewer kf7r (Part 1/4)

We sincerely appreciate your insightful and professional comments, which have greatly helped us improve the quality of our work. We are also grateful for your recognition of the paper's clarity and the comprehensiveness of our experiments.

In addition to your acknowledgment of the paper's strengths, we have carefully considered your constructive feedback and questions with more experiments. Specifically, we investigated PRE's performance under distribution shifts and added the results to Section 4.5 of the main text and Appendix D.7. We also provided a more detailed explanation of the initialization and updating of the forecast memory during evaluation in Section 4.1. To fully address your concerns, we provide detailed responses and additional experimental results below.


W5: Clarification on our novelty

We sincerely thank you for bringing the related work ResMem[1] to our attention. To address your concerns regarding the novelty of our work, we would like to clarify that ResMem and our proposed PRE differ fundamentally in their design principles and deployment convenience.

Differences in processing historical records. ResMem performs KNN over the entire training dataset and therefore requires storing the residuals of all training samples. For large-scale tasks, its cold-start cost grows linearly with the dataset size and can become non-negligible. For small-sample tasks, there may be insufficient historical references to construct an effective KNN memory. In contrast, PRE does not require access to the entire dataset during deployment. It only uses the most recent KK records (K=16K=16 in our experiments). Therefore, its cold-start cost is a constant regardless of the overall dataset size. As detailed in our response to W2, this cost is at most 5 seconds for the slowest base forecaster. This makes PRE applicable to both large-scale and small-sample tasks.

Differences in model dependency. ResMem does not compute similarity directly using the raw input data xx. Instead, it extracts intermediate features z=ϕ(x)z=\phi(x) from the base network ϕ\phi to calculate similarity weights. This requires users to have access to and understand the architecture of the base network, and design feature extraction strategies for different models. In contrast, PRE takes the raw context and the original outputs from the base model as inputs. Users neither need to retrain PRE nor need to understand or modify the internal architecture of the base forecaster.

Transferable residual inference. We believe that existing work such as ResMem demonstrates the value of residuals for improving predictions, and PRE further makes residual-based correction practical and transferable across tasks and models. To summarize, the core of PRE is not a simple combination of history, but transferable reasoning ability. This enables plug-and-play improvement across unseen models and tasks without any specific parameter updating.


W1: A closer look at distribution shifts

We appreciate your constructive comments on the research problem, and we also believe that correcting forecasting errors caused by distribution shifts is a key issue in post-forecast correction. To further investigate PRE's performance under distribution shifts, we refer to RevIN[2] and use mean shift and variance shift to identify distribution shifts. Specifically, at time step tt, we compare the most recent WW observed context xt−W+1:tx_{t-W+1:t} with seasonally aligned histories. A forecasting window is classified as distribution-shifted if either measure exceeds its corresponding 99th-percentile threshold of historical variations. we identify 995 shifted windows out of 31,879 test windows, approximately 3%. The correction results are reported below:

ScenarioForecast CasesBase MASECorrected MASECorrected Gain
Distribution Shift9950.7700.717+6.86%
Non-distribution Shift308840.9120.863+5.39%

PRE achieves a 6.86% improvement on distribution-shift cases, compared to 5.39% on non-shift windows, demonstrating its effectiveness under distribution shifts when the latest observed context exhibits distribution shifts.

We sincerely appreciate your insight into the importance of studying distribution shifts and have included these results into Section 4.5 in the main text and Appendix D.7. Thank you again for your valuable suggestions, which have helped us further improve our work.

[1] Yang, Zitong, et al. “ResMem: Learn what you can and memorize the rest.” Advances in Neural Information Processing Systems 36 (2023): 60768–60790.

[2] Kim, Taesung, et al. “Reversible Instance Normalization for Accurate Time-Series Forecasting Against Distribution Shift.” International Conference on Learning Representations (2022).

AuthorsOpenReview ↗

Response to Reviewer kf7r (Part 2/4)

W2: Clarification to deployment costs

Thank you for pointing out the potential additional costs of data preparation and deployment. To address your concerns, we would first like to clarify that both residual synthesis and PRE pretraining are not required for end users when deploying PRE. Users can directly apply our pretrained checkpoint to their own forecasters and tasks, without generating synthetic residuals or performing any additional training or parameter updates. We therefore refer to PRE as a plug-and-play model. The correction results in Section 4.2 on unseen modes and unseen tasks during pretraining can demonstrate the feasibility of this method.

Second, an important motivation for PRE comes from our observation of the gap between offline benchmark evaluation and real-world forecasting practices. Benchmarks typically evaluate models once on predefined test sets, whereas real-world applications, such as forecasting for weather, traffic flow, and electricity demand, require continuous rolling forecasts as new observations arrive. Once the observations become available, residuals can be naturally obtained at almost no additional cost, as it only requires computing the difference between the previous forecasts and the observations. The rolling forecasting is already part of the practical and needed forecasting workflow, rather than an additional operation introduced by PRE.

We acknowledge that, at initial deployment, if historical forecasts and observations are not available, PRE need to wait until sufficient forecasting experience has accumulated. To further address your concern, we report the time required by statistical forecasters and foundation models to construct K=16K=16 historical forecasting records used in our main experiments with context length L=512L=512 and forecast horizons H∈24,48,96H \in \\{24, 48, 96\\}.

ForecasterDeviceH=24 (ms)H=48 (ms)H=96 (ms)
AutoThetaCPU502.4559.2784.4
AutoARIMACPU3470.93554.24973.3
DLinearCPU0.30.30.4
TimesFM 2.5GPU46.848.048.4
Moirai 2.0GPU7.37.517.7
Chronos 2GPU14.314.514.6

As shown above, even the ARIMA mentioned in your comment requires only 5.0 seconds to generate 16 forecasts for L=512,H=96L=512, H=96. Moreover, this cost is incurred only once at the beginning of a task, as the forecasts can be stored for future use, and storing K=16K=16 numerical forecasting results imposes negligible storage overhead. We will further clarify the distinction between offline pretraining and online deployment, as well as the demand of real-world forecasting in our manuscript to avoid potential confusion.

AuthorsOpenReview ↗

Response to Reviewer kf7r (Part 3/4)

W3: Comparison with statistical methods

Thank you for your valuable suggestions on the correction methods. Our initial motivation for residual correction was intuitive: if a model overestimates future values, its predictions should be adjusted downward, and vice versa. We therefore initially tried directly using the residual from the previous forecast to correct the next forecast (i.e., a zero-order hold). However, experiments showed that this simple strategy is unreliable: the directional consistency between historical and current residuals is only around 60%, and even when the direction is consistent, its magnitude can be excessive that degrades forecasting performance.

Following your suggestion, we evaluate several training-free statistical correction methods, including using the most recent residual, averaging historical residuals, and KNN. For KNN, we use the same memory size as PRE (K = 16) and evaluated two strategies: using the most similar residual and weighted averaging the similarity of 16 residuals.

MethodIn-domain MASEIn-domain GainZero-shot MASEZero-shot Gain
Base TSFM0.915--0.901--
Last Residual1.246−36.17%1.189−31.84%
Mean Residual (K=16)0.966−5.63%0.945−4.80%
Nearest-neighbor Residual1.150-25.72%1.114−23.67%
KNN weighted Residual (K=16)0.943−3.12%0.919−1.93%
PRE0.860+5.99%0.859+4.71%

The results show that these statistical methods fail to improve forecasting performance, consistent with our early attempts. Our experimental analysis reveals that, for strong base forecasters, useful patterns in the residuals are often obscured by noise. Consequently, directly aggregating raw residuals makes it difficult to identify beneficial correction components. Considering the ability of deep learning models to suppress noise and extract informative patterns from complex data, as well as the preliminary success of existing studies in forecast correction (as discussed in Section 2.2), we further sought a model that could generalize across forecasting tasks and perform inference without task-specific training. These considerations motivated our choice of a pretrained model as the backbone of PRE. Our experimental results in Section 4.2 demonstrate that this approach achieves more effective forecast correction.

We hope these additional experiments and explanations clarify the motivation behind our design choices.


W4: Considerations for network architecture design

Thank you for your detailed discussion on the architecture design of PRE. Our design of Eq.(11) is based on two considerations. First, we add the residual evidence RtR_t to the current scenario representation StS_t, because the usefulness of historical residuals depends on the forecasting scenarios under which they occurred. Residuals alone cannot fully reflect their relevance to the forecasting task. Second, we subsequently apply an encoder to explicitly model temporal dependencies across forecast steps, as the cross-attention in Eq.(10) retrieves evidence independently for each forecast patch.

To validate these design choices, following your suggestion, we conduct two ablation experiments under the same training setting: (1) directly using St+RtS_t+R_t without the encoder, and (2) replacing StS_t with the forecast representation YtY_t, i.e., Enc⁡r(Yt+Rt)\operatorname{Enc}_r(Y_t+R_t). Results are reported in the following table.

Eq.(11)In-domain MASEIn-domain GainZero-shot MASEZero-shot Gain
Base TSFM0.915--0.901--
Enc⁡r(St+Rt)\operatorname{Enc}_r(S_t+R_t) (ours)0.860+5.99%0.859+4.71%
St+RtS_t+R_t0.862+5.79%0.863+4.33%
Enc⁡r(Yt+Rt)\operatorname{Enc}_r(Y_t+R_t)0.862+5.81%0.861+4.51%

The results show that both variants with the encoder outperform direct addition, suggesting that the decoupled design of historical residual retrieval and temporal dependency modeling is beneficial. using StS_t performs slightly better than using YtY_t, indicating that incorporating input context provides additional useful information. Thank you again for your thoughtful comments on our model architecture, and we hope these explanations and ablation experiments address your concerns.

AuthorsOpenReview ↗

Response to Reviewer kf7r (Part 4/4)

Q1: Initialization and maintenance of forecast memory

Thank you for your concern about the experimental details. At the beginning of evaluation, we initialize the forecast memory Mt\mathcal{M}_t using historical data preceding the test segment. We generate KK historical forecasts and compute the residuals to form the initial Mt\mathcal{M}_t in Eq.(5). If the historical data are insufficient to generate KK records, we initialize the memory with all available historical records.

During evaluation, Mt\mathcal{M}_t is updated with a first-in-first-out (FIFO) strategy. Once the observations corresponding to a previous forecast become fully available, we compute its residual and append it to Mt\mathcal{M}_t, and remove the oldest record to maintain a fixed memory size. Following your advice, we have provided a more detailed description for memory initialization and updating in Section 4.1 in the manuscript to help readers better understand our method.


Q2: Computational cost of pretraining

Thank you for your interest in the pretraining cost of PRE. To facilitate training and ablation studies, we save the base forecasts generated for the training data locally for repeated use. Therefore, the total pretraining cost of PRE can be divided into three parts: base forecast generation, synthetic residual generation, and the two-stage model training.

Under our current model architecture and training configuration (Appendices B.2 and B.3) and experimental hardware (Appendix B.6), the wall-clock costs are are summarized below:

Base forecast generationResidual SynthesisModel TrainingTotal Time
12h10min4h37min16h47min

The total cost is 16h47min, with most of the time spent generating base forecasts due to the large scale of the pretraining data. Importantly, this cost is incurred only once during offline pretraining and end users do not need to repeat this process. During deployment, they only need to construct the initial forecast memory, which incurs only a millisecond- or second-level cold-start cost based on the speed of base forecast, as reported in our response to W3. During subsequent forecast corrections, PRE incurs only 6.94ms of additional GPU inference time at a forecast horizon of 96, as reported in Section 4.6 of our manuscript.


We hope our responses have effectively addressed your concerns and we look forward to incorporating these insightful discussions in the camera-ready version of this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.