acceptodds
Under review as a conference paper at ICLR 2027

Remember, Don’t Descend: Online Retrieval Episodic Memory for Test-Time Adaptation in Time-Series Forecasting

Abstract

Time-series forecasting models often suffer performance degradation after deployment as the data distribution evolves over time. Existing test-time adaptation (TTA) methods for time-series forecasting address this issue by continually updating lightweight adaptation parameters using newly observed values. However, such updates persist feedback from a particular time point in the model parameters, so an inappropriate update may adversely affect subsequent predictions that are unrelated to the context in which the update was made. This raises a fundamental question: how can a forecasting model effectively adapt to a changing environment by exploiting the large amount of data that accumulates at inference time? We answer this question with a substantially simpler approach based on an online memory bank, demonstrating that it can outperform the prevailing paradigm of parameter-update-based TTA without requiring continual parameter optimization. Our findings suggest that time-series TTA can move beyond adaptation centered on per-step parameter updates toward a memory-centric adaptation paradigm, in which experience accumulated over time is explicitly retained and leveraged to continually respond to evolving distributions. Across five forecasting backbones, six datasets, and four forecast horizons, our method, OREM (Online Retrieval Episodic Memory), reduces MSE and MAE by and on average, respectively. OREM improves MSE over the frozen predictor in all 120 experimental settings. OREM also substantially reduces harmful corrections to the frozen backbone's forecasts compared with existing TTA methods, resulting in more reliable forecast adaptation.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Forecasting, Test-Time Adaptation, Online Adaptation, Episodic Memory, Retrieval-Based Adaptation, Distribution Shift, Non-Stationarity

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_hmXeOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose OREM for test-time adaptation of time-series forecasting after deployment. It adapts through retrieval while keeping the predictor frozen. It sets up two memory banks and also provides this evaluation algorithm. But I think the baselines it compares against are not appropriate.

Critical strengths
  1. The method does not update any parameters, so it is cheap to implement and very practical.
  2. The improvement over existing methods is significant.
  3. It is validated on commonly used time series datasets.
Critical weaknesses
  1. OneNet and FSNet are the standard references for online adaptation, and RAFT, PFRP and GTR are the retrieval-based forecasters the authors themselves admit are the closest work. There are so many models that should have been compared, but the authors compared none of them.
  2. The paper does not test multiple seeds. There is no variance, no repeated runs and no statistical analysis of the results, and that is not acceptable.
  3. The paper never discusses its limitations, and it never analyzes whether retrieval can have negative effects.
Questions for authors
  1. The authors treat OneNet and FSNet as the standard comparisons for online adaptation, and you say RAFT, PFRP and GTR are the closest retrieval-based forecasters, so why do none of them appear in the experiments? If you do not compare against the most direct competitors, how does your claim of beating existing methods hold up?
  2. The whole paper runs a single seed, with no variance, no repeated runs and no significance test. For settings that gain less than 1% or as little as 0.225%, why should I believe these are real improvements instead of noise? Can you provide multi-seed results and statistical tests?
  3. When does retrieval backfire? When the neighbours do not resemble the current window at all, can the correction make the forecast worse? The paper has no failure analysis and no limitations discussion, so can you explain when this method fails?
8 comments
AuthorsOpenReview ↗

Response to Reviewer hmXe (1/2): W1 / Q1

W1 / Q1

We appreciate the reviewer's careful evaluation and valuable suggestions, which motivated us to conduct additional experiments and investigate OREM's robustness and limitations more thoroughly. To address the reviewer's concern regarding the absence of comparisons with direct competitors, we additionally evaluated OneNet, FSNet, RAFT, and GTR across six datasets and four forecast horizons. For GTR, we considered OLS and TQNet backbones.

Table R1. Comparison with online adaptation and retrieval-based forecasters (MSE ↓\downarrow).

DatasetOneNetFSNetRAFTOLS+GTRTQNet+GTROLS+OREMTQNet+OREM
ETTh10.55420.67660.43230.45230.44630.41820.4166
ETTh20.95991.14760.38510.38050.37810.36980.3739
ETTm10.65900.70560.38170.42130.37350.37590.3624
ETTm20.51680.55930.28090.28740.27320.27980.2747
Weather0.29800.32470.27110.27930.24550.26000.2410
Electricity0.18660.22990.18400.22630.19230.18670.1585

Each value is the mean over H∈{96,192,336,720}H\in\{96,192,336,720\}. All methods use the same data splits and evaluation windows.

The results suggest that OREM generally achieves favorable forecasting performance relative to the compared methods.

Unlike OREM, which can be attached to a general-purpose forecasting backbone, OneNet, FSNet, and RAFT are standalone time-series forecasting models. We therefore conducted an additional experiment to examine whether applying OREM to these three methods would yield further performance improvements.

Table R2. Performance before and after applying OREM (MSE ↓\downarrow).

DatasetOneNet OriginalOneNet +OREMFSNet OriginalFSNet +OREMRAFT OriginalRAFT +OREM
ETTm10.4820.3720.4390.3460.32850.2953
Weather0.1730.1580.1980.1720.18760.1728
Electricity0.1620.1410.1890.1470.17380.1449

L=H=96L=H=96.

The additional experiments confirmed effective performance improvements from applying OREM to all three models. In particular, further improvements were observed for OneNet and FSNet while retaining their existing online adaptation mechanisms, and the same trend was found for the retrieval-based forecaster RAFT.

PFRP, unlike the methods above, is designed for univariate forecasting. For a fair comparison, we constructed univariate backbones by independently training each of the seven variables in the ETT datasets and provided OREM and PFRP with identical initial checkpoints.

Table R3. Comparison with PFRP on all seven ETT variables (MSE ↓\downarrow).

BackboneMethodETTh1ETTh2ETTm1ETTm2
TQNetFrozen0.46090.39660.38170.2779
PFRP0.45790.39330.37790.2773
OREM0.42840.39150.36590.2764
PatchTSTFrozen0.47310.41200.40720.3092
PFRP0.48530.40850.40720.3122
OREM0.43980.40210.37920.2981

In the comparison with PFRP, OREM also achieved the lowest MSE on every ETT dataset for both TQNet and PatchTST, without retraining the forecaster or training an additional retrieval-and-fusion module. We appreciate the reviewer's constructive suggestion, which enabled us to better understand OREM's advantages through a broader set of comparisons.

AuthorsOpenReview ↗

Response to Reviewer hmXe (2/2): W2 / Q2 and W3 / Q3

W2 / Q2

To assess the reproducibility and statistical reliability of the performance improvements, we conducted additional experiments focusing on ETTh2 (2.04%), ETTm2 (2.85%), and Weather (3.47%), which exhibited relatively small average MSE improvements in the original main experiments.

We evaluated 60 configurations, comprising three datasets, five backbones, and four forecast horizons, with three seeds per configuration. For each seed, Frozen and OREM shared exactly the same backbone weights. We calculated the three-seed mean ±\pm standard deviation (SD) and the two-sided 95% tt-confidence interval for the seed-wise MSE improvement rate.

Table R4. Multi-seed robustness analysis.

DatasetEvaluatedMean MSE improvedPositive 95% CI lower bound
ETTh2202017
ETTm2202020
Weather202020
Overall606057 (95%)

Positive CI lower bounds refer to the 95% tt-confidence interval of the seed-wise MSE improvement rate.

In particular, in the 0.225% improvement case highlighted by the reviewer (Weather--TQNet--H=336H=336), all three seeds showed lower MSE. The mean improvement was +0.458% ±\pm 0.005%, with a 95% CI of [+0.447%, +0.470%].

All 60 configurations reproduced a positive mean improvement, and 57 configurations had 95% confidence intervals entirely above zero. The remaining three configurations also had positive mean improvements, but their confidence intervals included zero. The complete multi-seed results will be incorporated into the revised manuscript.

W3 / Q3

To investigate the potential negative effects and limitations of retrieval, we analyzed the frequency of windows in which OREM increased the actual prediction error and the corresponding average MSE increase.

We define degradation as a test window in which the MSE after adaptation exceeds that of the Frozen backbone, and compare OREM with existing adaptation methods on DLinear and PatchTST for ETTm1, Weather, and Electricity (H=96H=96).

Table R5. Prediction degradation analysis (H=96H=96; lower is better).

DatasetBackboneOREMTAFASPETSACOSA
ETTm1DLinear22.84 / 0.037133.88 / 0.041837.16 / 0.020034.40 / 0.0318
PatchTST27.39 / 0.030439.43 / 0.010376.86 / 0.028239.13 / 0.0215
WeatherDLinear19.27 / 0.008030.02 / 0.005529.05 / 0.003634.07 / 0.0039
PatchTST27.62 / 0.008139.76 / 0.006371.96 / 0.006036.99 / 0.0037
ElectricityDLinear1.97 / 0.018124.28 / 0.003229.14 / 0.003837.04 / 0.0008
PatchTST5.11 / 0.015323.06 / 0.004188.29 / 0.019027.14 / 0.0009

Each entry is the degradation rate (%) / the mean MSE increase conditional on degradation.

OREM achieved the lowest window-level MSE degradation rate in all six evaluated configurations. While the conditional magnitude of degradation varied across methods, OREM consistently reduced the frequency of prediction-error increases relative to the frozen backbone. These results provide complementary evidence of OREM's reliability by directly measuring the occurrence of harmful corrections, beyond the direction-based analysis in the original Figure 4. We also examined actual failure cases from six datasets and found that even when input similarity is high, retrieval-based correction can increase error if the subsequent future patterns differ. We also observed cases where the future patterns suggested by retrieved episodes point in different directions, causing their combined result to deviate from the actual future. These limitations are also consistent with the design motivation for OREM's clipping mechanism, which bounds the magnitude of excessive corrections. We will discuss these limitations in the revised manuscript.

Thanks for the detailed response. The added baselines and multi seed results address most of my concerns, and the window level degradation analysis is helpful too. I’m more positive about the paper now. I still have a couple of questions.

  • I agree that OREM makes fewer harmful corrections in these comparisons. But for ETTm1/PatchTST, the degradation rate times the conditional MSE increase is about 0.0083 for OREM and 0.0041 for TAFAS. This only counts the losses, not the net improvement. Could you comment on this trade off and keep it clear in the reliability claims?

  • You mention failures where similar inputs lead to different futures. I was also asking about cases where there are no good neighbours in the memory. Are those queries more likely to get worse after correction?

Reviewer_1dEqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes OREM, an online adaptation method for time series forecasting. OREM is a RAG based approach to online adaptation where new time series episodes are stored in memory and then are retrieved to adjust forecasts for similar time series contexts. The method has the advantage over other weight-based online adaption methods in that it reacts quickly to distribution shift and that “bad” episodes are not learnt on and hence should have a lesser effect on performance. Experiments are conducted on standard time series datasets and show that OREM perform better than some existing online adaptation methods. Additionally, ablations and other controls/test are carried out to understand the behaviour of OREM.

Critical strengths
  • I like the insight that weight-based online adaption methods cannot adjust as quickly to distribution shift as RAG style methods and additionally that without some OOD detection they might be hurt by weird time series episodes. Also, I like that this we not just stated but evidenced in Table 4.

  • The paper is clearly written and does a good job of saying why the method is interesting, not just what the components are.

  • Performance of OREM is strong compared to other methods across the datasets and models looked at.

Critical weaknesses
  • My main concern is that as the authors point out in the related work there have been other RAG approaches suggested for time series forecasting. However, they are not tested against in the experiments. For example, how do their retrieval mechanisms compare to OREMs with everything else kept the same as OREM. Additionally, how well do they perform with just adding new episodes seen into their memory. These baselines need to be included for me to fully understand the novelty of the method and for me to consider adjusting my score.

  • My other major concern is that the proposed OREM is really doing kernel-weighted k-NN to predict the residual, a well known traditional method. There has be many works looking at such methods which need to be cited, also by reading them they might find new potential avenues for improvement. For example, given K=10, the authors could look at using local kernel machines, such as used in Haworth, James, et al. "Local online kernel ridge regression for forecasting of urban travel times." Transportation research part C: emerging technologies 46 (2014): 151-178. I think showing that the residual prediction method is already a well know and trusted method makes the paper stronger, not weaker.

  • Another weakness is the lack of extensive experiments, only ETT, Electricity and Weather is used. This is quite a small selection of datasets. Additionally, one of the main motivations of test-time adaptation methods is the updating of foundation models, so it would be good to see experiments with time series foundation models. For example, see Lee, Thomas L., et al. "Lightweight online adaption for time series foundation model forecasts.". This paper proposes a method that beats TAFAS and is tested on many datasets and foundation models. Also some additional baselines like naive seasonal forecasting would be nice to have.

Questions for authors
  • For tables 2 and Figure 4 it is unclear why the specific datasets and models used were chosen. Could you do the experiment over all datasets and models and if lacking space in main text, put other results in appendix?

  • How were the hyperparameters fitted, not stated in the text?

  • Can you explain βbin\beta^{bin} more clearly in the main text, unclear how the binning actually works.

  • Could you look at a \beta values across time plot, how does the weighting change.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 1dEq (1/4): W1–W2

W1

We sincerely thank the reviewer for their thoughtful and constructive feedback, which has helped us further clarify and strengthen our work. Following the reviewer's suggestion, we conducted additional comparisons with existing retrieval-based methods to further examine OREM's distinctive properties and practical benefits.

RAFT and PFRP are retrieval-based forecasting methods that use fixed memories constructed from training data. In contrast, OREM retrieves and reuses online experience accumulated after deployment, extending the adaptability of existing forecasters without retraining them.

We first evaluated an extension of RAFT's retrieval memory while keeping its learned weights fixed, and then assessed the effect of applying OREM to the original RAFT predictions.

Table R1. Online Experience and OREM Adaptation on RAFT (MSE ↓\downarrow)

DatasetRAFT (Train)RAFT (Train+Val)Online-RAFTRAFT+OREM
ETTm10.32850.32220.30430.2953
Weather0.18760.18590.17840.1728
Electricity0.17380.17010.16250.1449

RAFT+OREM reduced MSE relative to the original RAFT on all three datasets. More importantly, compared with Online-RAFT, which simply adds newly observed episodes to memory, RAFT+OREM achieved additional MSE reductions of 3.0%, 3.1%, and 10.8% on ETTm1, Weather, and Electricity, respectively.

This suggests that the performance gains cannot be explained by expanding online memory alone and that OREM's retrieval-based prediction correction provides additional benefits.

Table R2. Comparison with PFRP on All Seven ETT Variables (MSE ↓\downarrow)

BackboneMethodETTh1ETTh2ETTm1ETTm2
DLinearFrozen0.45330.37740.39960.2812
PFRP0.44030.37870.38860.2757
OREM0.42320.37340.37290.2791
TQNetFrozen0.46090.39660.38170.2779
PFRP0.45790.39330.37790.2773
OREM0.42840.39150.36590.2764
PatchTSTFrozen0.47310.41200.40720.3092
PFRP0.48530.40850.40720.3122
OREM0.43980.40210.37920.2981

In the comparison with PFRP, OREM achieved the lowest MSE in 11 of the 12 backbone--dataset combinations across DLinear, TQNet, and PatchTST, demonstrating generally stronger performance.

These results indicate that OREM can achieve more effective adaptation than existing retrieval-based forecasting methods without retraining the forecaster or learning an additional correction module.

W2

We appreciate the reviewer's constructive suggestion. OREM's similarity-weighted retrieval is closely related to classical kernel-weighted k-NN and nonparametric regression, and building on these established techniques supports the simplicity and practicality of OREM. We will add the suggested work by Haworth et al. (2014), Local Online Kernel Ridge Regression for Forecasting of Urban Travel Times, to the related-work discussion.

However, OREM's central contribution is not a new residual regression technique itself, but a framework that combines similarity-based retrieval of observed future values from past episodes with online episodic memory and feedback-based control of correction strength to adapt a frozen backbone after deployment. In particular, OREM constructs a correction signal from the retrieved future values relative to the frozen backbone's prediction and dynamically adjusts its strength based on realized feedback and retrieval similarity, rather than applying retrieved corrections unconditionally. As fully observed episodes accumulate during deployment, OREM selectively retrieves and reuses relevant experience, enabling online adaptation without additional model training or parameter updates.

AuthorsOpenReview ↗

Response to Reviewer 1dEq (2/4): Q1–Q2, W3 (Parts 1–2),

Q1

Thank you for this valuable suggestion. We conducted additional experiments to extend the analyses in Table 2 and Figure 4 to all six datasets and five backbones. The additional results have been incorporated into the appendix of the revised manuscript.

Q2

OREM uses one shared set of hyperparameters to avoid excessive tuning for individual datasets or backbones and to facilitate deployment across different environments.

Specifically, on the H=96H=96 validation splits of ETTh2 and ETTm2, we searched 10 combinations consisting of five decay values (ρ∈{0.99,0.995,0.997,0.999,1.0}\rho\in\{0.99,0.995,0.997,0.999,1.0\}) and two clipping thresholds (τ∈{0.5,1.0}\tau\in\{0.5,1.0\}). All other hyperparameters were fixed at their initial design values. The detailed hyperparameter settings are provided in the paper's Appendix under Implementation Details.

We applied the resulting single shared configuration to all 120 experiments comprising five backbones, six datasets, and four forecast horizons, achieving MSE improvements over the frozen model in every setting.

These results support OREM's practical advantage of providing consistent adaptation gains without fine-grained dataset- or backbone-specific hyperparameter tuning, making it straightforward to apply to diverse time-series forecasting models.

W3

We thank the reviewer for the helpful suggestions, which allowed us to examine OREM's advantages under a broader range of conditions.

1. Additional datasets

To address the concern about dataset diversity, we additionally evaluated three datasets: CzeLan, an ecological time series recording plant sap flow; Washington Bike Sharing, representing urban mobility demand; and Tetouan, representing urban infrastructure operations. These evaluations extend the experimental scope to ecological observations and urban infrastructure applications.

We evaluated DLinear with L=96L=96 and H∈{96,192,336,720}H\in\{96,192,336,720\}. Within each setting, all methods used the same Frozen backbone weights, data splits, normalization, and evaluation targets.

Table R3. Additional Dataset Evaluation (DLinear; MSE / MAE ↓\downarrow)

DatasetHHFrozenTAFASPETSACOSAOREM
CzeLan960.243 / 0.2920.237 / 0.2850.240 / 0.2860.236 / 0.2850.208 / 0.244
1920.273 / 0.3100.263 / 0.3020.267 / 0.3040.265 / 0.3040.241 / 0.270
3360.300 / 0.3300.290 / 0.3210.295 / 0.3240.298 / 0.3280.275 / 0.296
7200.358 / 0.3680.340 / 0.3580.349 / 0.3630.354 / 0.3670.334 / 0.343
Bike960.658 / 0.5560.633 / 0.5470.631 / 0.5460.635 / 0.5470.521 / 0.480
1920.645 / 0.5610.618 / 0.5520.618 / 0.5500.625 / 0.5550.531 / 0.495
3360.658 / 0.5750.631 / 0.5670.630 / 0.5640.648 / 0.5780.547 / 0.509
7200.707 / 0.6090.734 / 0.6250.744 / 0.6280.712 / 0.6140.614 / 0.552
Tetouan960.244 / 0.2930.230 / 0.3010.233 / 0.2910.228 / 0.2960.193 / 0.244
1920.282 / 0.3170.265 / 0.3270.268 / 0.3150.263 / 0.3190.236 / 0.274
3360.321 / 0.3400.296 / 0.3520.301 / 0.3400.298 / 0.3440.277 / 0.301
7200.361 / 0.3640.331 / 0.3810.338 / 0.3680.342 / 0.3710.319 / 0.329

All datasets use chronological 70/10/20 splits and training-set normalization. For Bike, missing observations were causally imputed, while evaluation used only observed target coordinates. The baselines retained their original partial-observation update protocols, whereas OREM used fully matured episodes.

These additional comparisons show that OREM generally achieves favorable forecasting performance across the evaluated datasets, supporting its applicability beyond the original benchmarks.

2. Additional naive and seasonal-naive baselines

Following the reviewer's suggestion, we additionally evaluated Naive and Seasonal-naive baselines on the six original datasets across four forecast horizons (H=96,192,336,720H=96,192,336,720). For each dataset, we report the average MSE and MAE across these horizons.

Table R6. Comparison with Naive and Seasonal-Naive Baselines (DLinear, MSE / MAE ↓\downarrow)

DatasetNaiveSeasonal-naiveDLinear + OREM
ETTh11.3211 / 0.73680.5996 / 0.47930.4215 / 0.4274
ETTh20.5393 / 0.48100.4825 / 0.43710.3708 / 0.3942
ETTm11.2710 / 0.69800.4890 / 0.42060.3763 / 0.3842
ETTm20.3850 / 0.39350.3577 / 0.35720.2799 / 0.3216
Weather0.3525 / 0.31930.3713 / 0.32350.2598 / 0.2813
Electricity1.6122 / 0.95820.3299 / 0.34140.1895 / 0.2780

Average over H∈{96,192,336,720}H\in\{96,192,336,720\}. Only Weather Seasonal-naive uses an observation context of 144 time steps to cover a full daily cycle.

AuthorsOpenReview ↗

Response to Reviewer 1dEq (3/4): W3 (Part 3)

3. Application to individually trained forecasters and foundation models

The primary objective of our study is to adapt already-trained individual time-series forecasters after deployment without updating their parameters. Although OREM is designed to improve such well-trained models, it is not tied to a specific backbone architecture. We therefore also evaluated its applicability to foundation models, as suggested by the reviewer.

Using the main evaluation protocol, we extended the comparison with ELF to the full benchmark comprising five individually trained forecasting backbones (DLinear, OLS, iTransformer, PatchTST, and TQNet), six datasets, and four forecast horizons, covering 120 configurations.

For concise presentation, Table R4 reports the average MSE and MAE across H∈{96,192,336,720}H\in\{96,192,336,720\} for each dataset--backbone combination.

Table R4. Individually Trained Forecasting Models (4-Horizon Average MSE / MAE ↓\downarrow)

BackboneMethodETTh1ETTh2ETTm1ETTm2WeatherElectricity
DLinearFrozen0.450 / 0.4390.375 / 0.3980.414 / 0.4080.285 / 0.3270.273 / 0.2910.226 / 0.309
ELF0.451 / 0.4400.381 / 0.4030.404 / 0.4020.288 / 0.3300.269 / 0.2940.229 / 0.316
OREM0.421 / 0.4270.371 / 0.3940.376 / 0.3840.280 / 0.3220.260 / 0.2810.189 / 0.278
OLSFrozen0.445 / 0.4320.374 / 0.3980.413 / 0.4070.286 / 0.3270.273 / 0.2910.215 / 0.292
ELF0.445 / 0.4330.380 / 0.4020.403 / 0.4010.287 / 0.3290.269 / 0.2940.218 / 0.301
OREM0.418 / 0.4230.370 / 0.3940.376 / 0.3840.280 / 0.3210.260 / 0.2810.187 / 0.273
iTransformerFrozen0.453 / 0.4480.401 / 0.4140.414 / 0.4140.307 / 0.3440.257 / 0.2790.189 / 0.277
ELF0.448 / 0.4440.387 / 0.4080.397 / 0.4020.290 / 0.3320.257 / 0.2820.193 / 0.285
OREM0.423 / 0.4330.383 / 0.4020.375 / 0.3900.291 / 0.3300.250 / 0.2720.172 / 0.262
PatchTSTFrozen0.453 / 0.4490.385 / 0.4080.390 / 0.4040.295 / 0.3340.257 / 0.2790.206 / 0.294
ELF0.447 / 0.4470.384 / 0.4080.383 / 0.3990.287 / 0.3310.257 / 0.2840.213 / 0.305
OREM0.424 / 0.4340.377 / 0.4010.371 / 0.3880.286 / 0.3270.251 / 0.2740.182 / 0.273
TQNetFrozen0.441 / 0.4330.378 / 0.4020.377 / 0.3930.278 / 0.3230.243 / 0.2690.164 / 0.258
ELF0.442 / 0.4350.382 / 0.4050.380 / 0.3940.279 / 0.3240.245 / 0.2740.169 / 0.268
OREM0.417 / 0.4240.374 / 0.3980.362 / 0.3820.275 / 0.3190.241 / 0.2670.158 / 0.252

Each entry is the arithmetic mean of the four horizon {96,192,336,720} Bold indicates the lower value of each metric.

The results in Table R4 show that OREM achieves favorable MSE and MAE compared with ELF across diverse individually trained forecasting backbones and datasets. These findings support the effectiveness of retrieval-based adaptation in improving existing forecasters without updating their parameters.

Table R5. Time-Series Foundation Models (Average MSE / MAE ↓\downarrow)

BackboneMethodETTh1ETTh2ETTm1ETTm2WeatherElectricity
Chronos-2Frozen0.4383 / 0.42450.3723 / 0.38580.4007 / 0.38120.2745 / 0.30990.2610 / 0.26510.1968 / 0.2693
ELF0.4336 / 0.42450.3714 / 0.38960.3962 / 0.38110.2745 / 0.31310.2523 / 0.26680.1961 / 0.2741
OREM0.4157 / 0.41980.3682 / 0.38500.3754 / 0.37540.2719 / 0.31040.2554 / 0.26510.1806 / 0.2639
TimesFM-2.5Frozen0.4293 / 0.42230.3876 / 0.39560.3792 / 0.38800.2852 / 0.32030.2537 / 0.27200.2043 / 0.2833
ELF0.4285 / 0.42240.3851 / 0.39550.3775 / 0.38730.2843 / 0.32060.2535 / 0.27310.2041 / 0.2843
OREM0.4138 / 0.41860.3816 / 0.39350.3637 / 0.37830.2807 / 0.31810.2490 / 0.26930.1832 / 0.2695

Results are equally averaged over four forecast horizons (H∈{96,192,336,720}H \in \{96,192,336,720\}). Bold indicates the lower value of each metric between OREM and ELF.

The results in Table R5 further demonstrate that OREM effectively improves foundation-model forecasts without parameter updates and achieves favorable MSE and MAE compared with ELF. Although foundation-model adaptation is not the primary focus of this study, these results support the applicability of OREM's retrieval-based correction mechanism beyond individually trained forecasting models. We sincerely thank the reviewer for encouraging us to explore this broader applicability of OREM.

Reviewer_JxPtOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Online Retrieval Episodic Memory (OREM), a gradient-free approach to test-time adaptation for time-series forecasting. OREM keeps a pretrained forecaster frozen and corrects its predictions by retrieving similar historical episodes from a fixed memory bank and an online bank that accumulates deployment episodes once their targets become fully observable. Similarity-based gating combines the retrieved forecasts, while correction strengths estimated from past feedback and magnitude clipping regulate their influence. Across five forecasting backbones, six datasets, and four forecast horizons, the authors report average MSE and MAE reductions of 5.43% and 3.37% relative to frozen predictors, with MSE improvements in all 120 settings. The central claim is that selectively retrieving accumulated deployment experience provides an effective alternative to continual parameter updates.

Critical strengths
  1. OREM incorporates deployment feedback through episodic retrieval without updating backbone parameters, providing a straightforward adaptation mechanism compatible with different forecasting architectures.

  2. The evaluation spans five backbones, six datasets, and four forecast horizons, with reported MSE improvements over frozen predictors in all 120 settings. Component ablations further support the usefulness of online memory.

Critical weaknesses
  1. The conceptual advance needs clearer justification relative to prior work. RAFT (Han et al., ICML 2025) and PFRP (Du et al., AAAI 2026) already exploit historical retrieval for forecasting, while AdaNPC (Zhang et al., ICML 2023) demonstrates memory-based test-time adaptation in classification. The paper acknowledges these connections. Its specific contribution lies in causally accumulating deployment episodes and calibrating forecast corrections, but the broader framing as a new adaptation paradigm appears stronger than the demonstrated conceptual advance.

  2. The experiments do not sufficiently isolate the value of the complete design. The online-bank and global/binned-coefficient ablations are useful, but leave unclear how much of the gain requires the full method. Comparisons with a standalone online nearest-neighbor predictor and a simpler retrieval baseline using a single combined memory bank are missing. Under matched data availability and memory budgets, these controls would help distinguish the benefits of access to deployment experience from those of the proposed memory organization and correction mechanisms.

  3. The practical deployment advantages remain insufficiently demonstrated. The bounded-memory study is useful, but the efficiency measurements cover retrieval rather than the complete adaptation pipeline. OREM also retains a fixed historical bank and additional online state, so avoiding gradient updates does not by itself establish a favorable trade-off between accuracy and resource use compared with lightweight parameter-update methods. It also remains unclear how the delay before new episodes enter memory affects recovery after a distribution change.

  4. The evidence for more reliable adaptation is incomplete. The stability analysis bounds prediction changes, while the broader reliability claim concerns whether those changes improve forecasts. The wrong-direction metric can overlook corrections that increase error by moving toward the target but overshooting. Furthermore, the recurring-shift experiment reports period-level averages while excluding boundary-crossing windows, leaving degradation and recovery around transitions insufficiently characterized. More direct evidence of how often adaptation increases error, and by how much, would strengthen the reliability claim.

Questions for authors
  1. How does OREM compare with a standalone online nearest-neighbor predictor and a simpler retrieval baseline using a single combined memory bank, under the same causal data availability and memory budget?

  2. With storage and maintenance of both memory banks included, how do OREM's accuracy, end-to-end latency, and total memory use compare with lightweight parameter-update baselines under constrained memory?

  3. Does OREM reduce the frequency and severity of forecast-error increases relative to the baselines? How does this behavior evolve immediately after a distribution change, before new episodes become fully observable?

5 comments
AuthorsOpenReview ↗

Response to Reviewer JxPt (1/3): W1

W1

We thank the reviewer for the opportunity to explain OREM's advantages more precisely.

RAFT and PFRP are related to OREM in that they retrieve past data for forecasting, but both primarily improve forecasting by using offline memories constructed from training data. More specifically, RAFT is a standalone trained forecasting backbone that integrates retrieved information, whereas PFRP uses a fixed Global Memory Bank and trainable retrieval and fusion modules to supplement existing forecasts. AdaNPC also proposes memory-based online TTA, but targets classification and performs nearest-neighbor classification using pseudo-labels. In contrast, OREM adds an episode to online memory only after its true future has been observed and uses these episodes to directly correct the outputs of a pretrained time-series forecaster without parameter updates.

OREM's central idea is to treat the online data continuously accumulated after deployment not merely as material for updating a model, but as an adaptation resource for directly correcting forecasts using observed experience. OREM requires neither a specific forecasting architecture nor an additional learned fusion module. It instead provides a simple yet effective mechanism that adjusts the strength and magnitude of corrections based on feedback from previous predictions. Thus, OREM offers the practical advantage of continually adapting diverse existing forecasters through accumulated deployment experience without model retraining or parameter updates.

To further validate these distinctions, we performed additional comparisons with RAFT and PFRP.

RAFT, unlike OREM, is a standalone time-series forecasting backbone. We conducted two comparisons to distinguish the benefit of using online experience itself from the additional benefit of OREM's correction mechanism. First, we evaluated Online-RAFT, which extends RAFT's retrieval memory online while keeping its learned weights fixed. Separately, we evaluated RAFT+OREM, which applies OREM to the original RAFT forecasts.

Table R1. Effect of Online Experience and OREM on RAFT (MSE ↓\downarrow)

DatasetRAFT (Train)RAFT (Train+Val)Online-RAFTRAFT+OREM
ETTm10.32850.32220.30430.2953
Weather0.18760.18590.17840.1728
Electricity0.17380.17010.16250.1449

The results show that directly leveraging post-deployment online experience, a central aspect of OREM, is beneficial even for an existing retrieval-based forecaster such as RAFT, and that the advantage becomes more pronounced when OREM is applied. In particular, RAFT+OREM reduced MSE relative to Online-RAFT by 3.0%, 3.1%, and 10.8% on ETTm1, Weather, and Electricity, respectively. Relative to the original RAFT, the corresponding reductions were 10.1%, 7.9%, and 16.6%. These results demonstrate both the value of accumulating online experience and the additional improvement obtained by applying OREM to an existing retrieval-based forecaster.

PFRP, unlike the other methods considered here, is applied to univariate forecasting. For a fair comparison, we independently trained univariate backbones for each of the seven variables in the ETT datasets and provided OREM and PFRP with the same initial checkpoints.

Table R2. Comparison with PFRP Across All Seven ETT Variables (MSE ↓\downarrow)

BackboneMethodETTh1ETTh2ETTm1ETTm2
DLinearFrozen0.45330.37740.39960.2812
PFRP0.44030.37870.38860.2757
OREM0.42320.37340.37290.2791
TQNetFrozen0.46090.39660.38170.2779
PFRP0.45790.39330.37790.2773
OREM0.42840.39150.36590.2764
PatchTSTFrozen0.47310.41200.40720.3092
PFRP0.48530.40850.40720.3122
OREM0.43980.40210.37920.2981

In this comparison, OREM achieved the lowest MSE on every ETT dataset for both TQNet and PatchTST, without retraining the forecaster or learning a retrieval-and-fusion module. Relative to PFRP, its maximum MSE reductions were 6.44% for TQNet (ETTh1) and 9.38% for PatchTST (ETTh1).

Together, these results show that OREM can effectively improve existing forecasters by directly exploiting online experience accumulated after deployment. In particular, OREM further improves a retrieval-based model such as RAFT and achieves high forecasting accuracy without the joint training required by PFRP. This empirically supports memory-centric test-time adaptation as an effective direction for reusing accumulated deployment data while keeping the weights of diverse forecasting models unchanged.

AuthorsOpenReview ↗

Response to Reviewer JxPt (2/3): W2 / Q1 and W3 / Q2

W2 / Q1

We appreciate the reviewer's suggestion to conduct controlled experiments that distinguish whether OREM's gains arise simply from storing and retrieving online experience or whether its proposed forecast correction provides additional benefits.

First, removing the Online Bank generally increased MSE in the original Table 2, while using only the Online Bank generally improved MSE over Frozen in Appendix Table 6. These findings support the practical contribution of experience accumulated after deployment to forecast adaptation.

We further examined whether storing and retrieving online experience alone would suffice, or whether OREM's complete design provides additional benefits, using matched causal data availability and memory storage budgets. Online kNN directly forecasts by computing a similarity-weighted average of the observed future values of similar episodes in online memory. Unified-20 merges the original Core Bank and Online Bank into a single bank and uses the 20 most similar episodes to correct the frozen backbone's predictions.

Table R3. Comparison with Simple Retrieval Baselines (Average MSE / MAE ↓\downarrow)

DatasetBackboneOnline kNNUnified-20OREM
ETTm1TQNet0.507 / 0.4770.369 / 0.3840.362 / 0.382
ETTm2TQNet0.452 / 0.4510.313 / 0.3380.275 / 0.319
WeatherTQNet0.487 / 0.4320.255 / 0.2740.241 / 0.267
ElectricityTQNet0.265 / 0.3360.163 / 0.2530.158 / 0.252

Results are equally averaged over four forecast horizons (H∈{96,192,336,720}H\in\{96,192,336,720\}). Each entry reports MSE / MAE.

OREM achieved lower average MSE and MAE than both baselines across all four datasets under matched causal data availability and memory budgets. Online kNN tests whether direct prediction from online experiences is sufficient, while Unified-20 evaluates a simpler single-bank correction approach. These results support the additional value of OREM's memory organization and feedback-guided correction beyond simpler retrieval-based alternatives.

W3 / Q2

To address the reviewer's concerns about deployment cost and memory maintenance, we conducted an additional end-to-end efficiency evaluation on an NVIDIA A100-SXM4-40GB using Electricity--DLinear (L=H=96L=H=96, 321 channels, 5,165 evaluation windows). We compared TAFAS, PETSA, COSA, and OREM under the same hardware environment, including versions of OREM with constrained Core and Online memory banks.

Processing time is reported in milliseconds per evaluation window based on batch throughput; OREM processes up to 96 windows per batch. Times are medians over three runs, while GPU and CPU memory values are the maxima across the three runs.

Table R4. End-to-End Deployment Efficiency (Electricity, DLinear, L=H=96L=H=96)

MethodCore / OnlineMSE ↓\downarrowms/window ↓\downarrowGPU MiB ↓\downarrowCPU MiB ↓\downarrow
TAFAS--0.2010.6971,8038,753
PETSA--0.2030.8961,8068,921
COSA--0.20738.8071,7428,931
OREM FullFull / Full0.1630.4264,7372,428
OREM 4K4K / 1K0.1660.1631,3642,411
OREM 1K1K / 1K0.1690.1159332,427

Times are three-run medians and memory values are three-run maxima. The ms/window measurements are based on batch throughput, not single-query latency.

With both banks limited to 1K, OREM maintained lower MSE than the compared adaptation methods while reducing processing time and GPU memory use relative to full OREM. These results suggest that OREM can maintain favorable forecasting accuracy with controllable memory and computational costs, without retraining the forecasting backbone.

AuthorsOpenReview ↗

Response to Reviewer JxPt (3/3): W4 / Q3

W4 / Q3

As the reviewer points out, an excessive correction may increase actual forecasting error even when the prediction moves in the correct direction. To evaluate this more directly, we extended the direction-based analysis in Figure 4 to window-level MSE degradation frequency and magnitude, and further analyzed adaptation immediately after distribution shifts.

1. Frequency and Magnitude of Actual Forecast Degradation

For each window, we measured the proportion of cases in which the MSE of OREM or a comparison method increased relative to the Frozen backbone (Degradation Rate) and the mean MSE increase among windows that actually deteriorated (Degradation Magnitude).

Table R5. Window-Level MSE Degradation: Rate (%) / Magnitude (↓\downarrow)

DatasetBackboneOREMTAFASPETSACOSA
ETTm1DLinear22.84 / 0.037133.88 / 0.041837.16 / 0.020034.40 / 0.0318
PatchTST27.39 / 0.030439.43 / 0.010376.86 / 0.028239.13 / 0.0215
WeatherDLinear19.27 / 0.008030.02 / 0.005529.05 / 0.003634.07 / 0.0039
PatchTST27.62 / 0.008139.76 / 0.006371.96 / 0.006036.99 / 0.0037
ElectricityDLinear1.97 / 0.018124.28 / 0.003229.14 / 0.003837.04 / 0.0008
PatchTST5.11 / 0.015323.06 / 0.004188.29 / 0.019027.14 / 0.0009

OREM had the lowest actual MSE degradation rate in all six settings. In particular, the degradation rates for DLinear and PatchTST on Electricity were 1.97% and 5.11%, respectively, markedly lower than those of the other TTA methods.

Although OREM does not always achieve the smallest conditional MSE increase when degradation occurs, it consistently reduces the frequency of harmful corrections across all six evaluated settings. This suggests that OREM's reliability advantage primarily stems from fewer degradation events, rather than smaller error increases whenever degradation occurs.

2. Adaptation Following Distribution Shifts

To examine adaptation immediately after distribution changes, we further divided the recurring-shift experiment in Section 4.6 into transition-boundary windows and subsequent forecasting intervals.

During the transition boundary (00--HH), the input from the previous dataset and the prediction targets from the new dataset overlap. Under the abrupt regime shifts considered in this experiment, both the pretrained forecaster and memory-based correction may temporarily exhibit degraded performance. This is because memory-based correction benefits from accumulating a certain amount of experience from the new regime to retrieve relevant past episodes. We will describe this limitation in greater detail in the revised manuscript. Importantly, however, OREM achieved lower MSE than Frozen from the first evaluation interval after the domain transition was complete (+H+H). In particular, when returning to the previously encountered ETTm2 domain in Period 4, OREM was already better at the transition boundary and consistently maintained lower forecasting error than the existing methods thereafter.

Table R6. Window-Level MSE Around Recurring Distribution Shifts (H=96H=96, ↓\downarrow)

PeriodDomainMethod0--H (Transition)+H+2H+3H
Period 2ETTm2Frozen4.5350.5330.5210.403
COSA6.8291.4721.0050.554
OREM6.1970.5160.5160.392
Period 3ETTm1Frozen5.0280.4340.2170.244
COSA5.0410.4360.2170.246
OREM5.3020.4310.1930.227
Period 4ETTm2Frozen2.6522.0561.0640.581
COSA2.6451.8981.0200.608
OREM2.5961.8440.8830.537

These findings show that OREM can improve forecasting performance by accumulating newly observed experiences in online memory after a distribution shift and selectively retrieving experiences relevant to the current input. In particular, when a previously encountered distribution reappears, past stored episodes can be reused without retraining the model or updating its parameters, which is an important advantage of OREM.

Reviewer_zNcTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OREM adapts frozen time-series forecasters by retrieving historical episodes from fixed and continually updated memory banks. It uses feedback-based correction coefficients and clipping to control the retrieved information’s influence, reporting improvements across five backbones, six datasets, and four forecast horizons.

Critical strengths

The paper presents a simple, model-agnostic approach with clear explanations, broad experimental coverage, and useful component ablations.

Critical weaknesses
  1. The novelty and comparative advantage remain insufficiently established. SOLID, ELF, and STEPS provide closely related adaptation mechanisms, yet Table 1 mainly compares gradient-based adapters. Matched-feedback comparisons would better establish what OREM adds beyond existing historical-data selection and output correction.

  2. The correction mechanism and reliability claims need stronger validation. Eq.6 fits coefficients before clipping, whereas Eq.5 clips the actual correction. The impact of this mismatch and of reusing coefficients from earlier memory states is unclear. Figure 4’s wrong-direction rate also misses harmful overshooting; actual per-window MSE degradation rates and magnitudes would be more informative.

  3. The robustness of the reported gains remains unclear. Small differences such as 0.374 versus 0.375 MSE lack uncertainty estimates accounting for temporal dependence. Although shared hyperparameters are a useful control, their selection procedure and the corresponding validation-based tuning of baselines should be clarified.

Questions for authors
  1. What substantial advance does OREM provide beyond SOLID, ELF, and STEPS, and how would these methods compare under matched feedback conditions?
  2. How much do fitting coefficients before clipping and reusing coefficients from earlier memory states affect performance? Can the authors report actual per-window MSE degradation rates and magnitudes?
  3. How were OREM and baseline hyperparameters selected, and are the smaller gains statistically reliable under uncertainty estimates accounting for temporal dependence?
6 comments
AuthorsOpenReview ↗

Response to Reviewer zNcT (1/4): W1 / Q1

W1 / Q1. Novelty and comparison with related adaptation methods

We thank the reviewer for the opportunity to clarify OREM's distinct contributions and its advantages over existing adaptation methods.

1. Key differences from existing methods

OREM explicitly retains online data accumulated after deployment and retrieves past experiences similar to the current input. It then uses the feedback-derived correction information associated with those experiences to directly adjust the output of a frozen backbone. The key distinction is not only the absence of parameter updates: OREM performs online adaptation by reusing individual past experiences and their associated correction information, yielding substantial and stable performance improvements. To examine these benefits more concretely, we compared OREM with the methods suggested by the reviewer.

SOLID uses similar historical examples but fine-tunes the prediction layer to exploit them. ELF continuously updates a Fourier-based online forecaster. STEPS uses a prefix of observed ground-truth values from the current forecast horizon to correct the remaining unobserved future.

Because SOLID and ELF can be compared with OREM under the main evaluation protocol, we evaluated them using the same data splits, evaluation windows, and forecast horizons.

Table R1. Comparison with Related Adaptation Methods (MSE / MAE$\downarrow$)

MethodETTh1ETTh2ETTm1ETTm2WeatherElectricity
SOLID0.592 / 0.4780.382 / 0.4040.401 / 0.4050.290 / 0.3310.261 / 0.2820.199 / 0.285
ELF0.447 / 0.4400.383 / 0.4050.394 / 0.4000.286 / 0.3290.259 / 0.2860.205 / 0.295
OREM0.421 / 0.4280.375 / 0.3980.372 / 0.3860.282 / 0.3240.252 / 0.2750.178 / 0.268

For STEPS, no public implementation was available, so we reimplemented the method according to the paper. Because STEPS uses a prefix observed after the initial forecast is issued, we used t+12t+12 as the common evaluation time for H=96H=96, when the first 12 target values are available, and evaluated the remaining 84 future values using information available at that time.

Table R2. Comparison with STEPS under a Common Observation Time (MSE$\downarrow$)

DatasetBackboneFrozen MSEOREM MSESTEPS MSE
ETTm1DLinear0.35230.30710.3210
ETTm1PatchTST0.32390.30130.3215
WeatherDLinear0.18840.17080.1795
WeatherPatchTST0.16730.15920.1673

In these comparisons, OREM achieved the lowest average MSE and MAE across six datasets relative to SOLID and ELF, and outperformed STEPS in all four configurations at the matched observation time.

Whereas existing approaches update a forecasting model or rely on a separately trained correction model to exploit online observations, OREM directly retrieves accumulated experiences from online memory to correct predictions without an additional learned corrector. These results indicate that reusing online experiences themselves can be an effective adaptation strategy without continual parameter updates.

AuthorsOpenReview ↗

Response to Reviewer zNcT (2/4): W2 / Q2 — Part 1

W2 / Q2. Correction mechanism, clipping, and forecast degradation

We thank the reviewer for the opportunity to clarify OREM's correction mechanism and stability. We address these questions through the design rationale and additional experimental results.

1. Rationale for Eqs. (5) and (6)

Eq. (6) estimates the correction coefficient β\beta from past feedback. However, a correction strength that was appropriate for a past episode does not necessarily guarantee a stable change to the current prediction. Eq. (5) therefore bounds the final correction to prevent excessively large prediction changes.

Indeed, in the w/o bounded corr. ablation in Table 2, removing clipping generally increased MSE. This demonstrates that estimating β\beta alone is insufficient and that bounding the final output correction plays an important role in improving correction accuracy.

2. Additional evaluation of clipping-aware coefficient fitting

To directly assess the discrepancy between Eqs. (5) and (6), we conducted additional experiments that incorporate clipping into the objective used to estimate β\beta.

We compared the original OREM with two clipping-aware variants across 60 configurations, comprising three datasets (ETTm1, ETTm2, and Weather), five backbones, and four forecast horizons. All three variants retain the same clipping operation in Eq. (5) at the final prediction stage; they differ only in whether clipping is incorporated during coefficient estimation.

Table R3. Effect of Clipping-Aware Coefficient Fitting (60 Configurations)

β\beta estimationΔ\DeltaMSE (%) ↓\downarrowΔ\DeltaMAE (%) ↓\downarrowΔ\Delta degradation rate (pp) ↓\downarrow
OREM (original)000
Eq. (6), clipping-aware-0.0075-0.0263+0.1912
Eqs. (6) and (8), clipping-aware-0.0127-0.0300+0.3812

From a stability perspective, incorporating clipping directly into the coefficient-fitting objective produces only negligible improvements in average prediction error, while slightly increasing the frequency of window-level degradation. These experiments help evaluate the effect of OREM's original separation between coefficient estimation and output clipping.

AuthorsOpenReview ↗

Response to Reviewer zNcT (3/4): W2 / Q2 — Part 2

3. Effect of reusing correction coefficients from past experiences

OREM retrieves past experiences similar to the current input and reuses correction coefficients that were effective for those experiences. It considers both similarity to past experiences and accumulated feedback statistics to determine an appropriate correction strength for the current prediction.

The original Table 2 ablation already examined this design. We additionally summarize the magnitude and frequency of the resulting performance differences.

Table R4. Effect of Retrieved Correction Coefficients (60 Configurations)

MethodMSE increase vs. OREM (%) ↓\downarrowConfigurations where OREM performs better
OREM (full)0.000---
Binned β\beta+0.95459 of 60
Global β\beta+1.68757 of 60

To directly examine whether reusing correction coefficients stored in past episodes is detrimental, we compared OURS, which reuses stored β\beta values, with a variant that re-estimates the retrieved episodes' coefficients using the current memory state. The current query's memory, retrieval, feedback statistics, and output clipping were kept unchanged.

Table R4a. Effect of Reusing Stored Correction Coefficients (H=96H=96, ↓\downarrow)

DatasetBackboneOURS (stored β\beta)Re-estimated β\beta
ETTm1DLinear22.84 / 0.037124.67 / 0.0424
PatchTST27.39 / 0.030430.65 / 0.0374
WeatherDLinear19.27 / 0.008023.50 / 0.0103
PatchTST27.62 / 0.008131.69 / 0.0103
ElectricityDLinear1.97 / 0.01811.99 / 0.0200
PatchTST5.11 / 0.01535.46 / 0.0164

Each cell reports degradation rate (%) / conditional mean MSE increase. Lower is better.

OURS achieved lower degradation rates and conditional harm in all six settings. These results show that reusing stored correction coefficients reduces both the frequency and magnitude of harmful corrections compared with re-estimating them from the current memory state, supporting OREM's coefficient-reuse design.

4. Window-level MSE degradation rate and magnitude

The original Figure 4 analyzes correction direction and overall performance improvement. However, as the reviewer notes, even a correction that moves in the correct direction can increase MSE if its magnitude is excessive.

To evaluate this effect directly, we compute for each test window

Δt=MSE⁡(y^t,yt)−MSE⁡(pt,yt).\Delta_t = \operatorname{MSE}(\hat y_t,y_t)-\operatorname{MSE}(p_t,y_t).

MSE degradation rate is the proportion of windows for which Δt>0\Delta_t>0. MSE degradation magnitude is the mean MSE increase conditional on degradation.

Table R5. Window-Level MSE Degradation (H=96H=96)

DatasetBackboneOREMTAFASPETSACOSA
ETTm1DLinear22.84 / 0.037133.88 / 0.041837.16 / 0.020034.40 / 0.0318
ETTm1PatchTST27.39 / 0.030439.43 / 0.010376.86 / 0.028239.13 / 0.0215
WeatherDLinear19.27 / 0.008030.02 / 0.005529.05 / 0.003634.07 / 0.0039
WeatherPatchTST27.62 / 0.008139.76 / 0.006371.96 / 0.006036.99 / 0.0037
ElectricityDLinear1.97 / 0.018124.28 / 0.003229.14 / 0.003837.04 / 0.0008
ElectricityPatchTST5.11 / 0.015323.06 / 0.004188.29 / 0.019027.14 / 0.0009

Each cell reports degradation rate (%) / conditional mean MSE increase.

This analysis evaluates OREM's stability not only through correction direction but also through the frequency of actual increases in prediction error. Although OREM does not always achieve the smallest conditional degradation magnitude, it consistently reduces the frequency of degradation events in these comparisons. This provides direct evidence that OREM's stability advantage extends beyond the direction-based analysis in the original Figure 4 to actual prediction error.

Overall, the additional ablation results support the role of output clipping in limiting excessive corrections and the importance of selectively reusing β\beta from similar past experiences. The window-level analysis further confirms that these design choices reduce the frequency of actual forecast degradation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated Manuscript: Summary of Revisions

We sincerely thank all reviewers for their constructive feedback and valuable suggestions. We have uploaded the revised manuscript, with revisions highlighted in blue. The major changes are summarized below.

Major Changes in the Revised Manuscript

In response to Reviewer zNcT:

  • Comparisons with related adaptation methods (Appendix E.2–E.3, Tables 16–21): Added comparisons with SOLID and ELF across 120 configurations, along with STEPS under a matched observation-time protocol, to better establish OREM's advantages over related adaptation methods.
  • Correction mechanism and stability (Appendix G.2, Table 28; Appendix H.1–H.2, Tables 32–33): Added clipping-aware coefficient-fitting experiments, comparisons between stored and re-estimated correction coefficients, and window-level analyses of actual prediction degradation, including its frequency and magnitude.
  • Hyperparameters and reproducibility (Appendix B; Appendix D.2 and D.6, Tables 8–11 and 13): Clarified the shared hyperparameter selection procedure, expanded sensitivity analyses, and added three-seed evaluations across 60 configurations, with positive mean improvements in all 60 and positive 95% confidence interval lower bounds in 57.

In response to Reviewer JxPt:

  • Controlled retrieval comparisons (Section 4.4, Table 3; Appendix E.4–E.5, Tables 22–23): Added experiments with RAFT memory extensions, PFRP, Online kNN, and Unified-20 to distinguish the benefits of accumulated online experience from those of OREM's memory organization and correction mechanism.
  • Deployment efficiency (Section 4.5, Figure 3; Appendix G.1, Table 27): Added end-to-end evaluations of processing time, GPU memory, and CPU memory, including constrained Core and Online Bank configurations, to characterize the accuracy–efficiency trade-off.
  • Reliability under distribution shifts (Appendix G.2 and G.4, Tables 28 and 31): Added analyses of actual forecasting-error increases and their net effects, together with prediction performance immediately after distribution transitions, including periods before sufficient new experience has accumulated.

In response to Reviewer 1dEq:

  • Relationship to existing retrieval methods (Section 2; Section 4.4, Table 3): Clarified OREM's connection to classical k-NN and online kernel regression, and strengthened comparisons with existing retrieval-based forecasting approaches.
  • Expanded evaluation scope (Appendix F.1–F.3, Tables 24–26): Added three datasets (CzeLan, Washington Bike Sharing, and Tetouan), Naive and Seasonal-naive baselines, and evaluations on two time-series foundation models, Chronos-2 and TimesFM-2.5.
  • Expanded ablations and correction analysis (Section 3.4; Appendix B and D.1–D.4, Table 7, Figures 6–9): Clarified the confidence-binned correction coefficient, extended component ablations and prediction-direction analyses to all six datasets and five backbones, and added visualizations of correction coefficients evolving over time.

In response to Reviewer hmXe:

  • Additional competitors and model compatibility (Section 4.4, Table 3; Appendix E.1 and E.4, Tables 15 and 22): Added comparisons with OneNet, FSNet, RAFT, GTR, and PFRP, and demonstrated the effects of applying OREM to existing online adaptive and retrieval-based forecasters.
  • Multi-seed robustness (Appendix D.2, Tables 8–11): Added complete three-seed results with mean, standard deviation, and seed-based confidence intervals to evaluate the reproducibility of forecasting improvements.
  • Failure cases and reliability trade-offs (Section 5; Appendix G.2–G.3, Tables 28–30): Added analyses separating harmful and beneficial correction contributions, as well as retrieval-similarity quartiles and difficult-to-retrieve queries. Clarified that fewer worsening events do not necessarily imply smaller harm magnitude, and incorporated a discussion of retrieval-related limitations into the Conclusion.

These revisions strengthen the empirical evaluation of OREM, clarify its contribution relative to existing adaptation and retrieval-based methods, and provide a more detailed characterization of its reliability, deployment costs, and limitations.

We appreciate the reviewers' thoughtful feedback and hope the revised manuscript addresses their concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.