acceptodds
Under review as a conference paper at ICLR 2027

Stabilizing Recursive Retrieval in Irregular Time-Series Forecasting

Abstract

Retrieval-augmented forecasting extends parametric predictors with historical trajectories retrieved from external temporal memory, yet existing methods largely adopt static memory, where retrieved evidence remains independent of the model's own predictions. This limitation can fail in continual forecasting, where previous forecasts are stored and later reused, creating a recursive feedback loop among memory, retrieval, and prediction. We present ECHO (Evolving Closed-loop Historical Observations), a framework for studying and controlling recursive retrieval-augmented forecasting with evolving temporal memory. ECHO combines predictive temporal retrieval with a fixed base forecaster and models memory evolution as a dynamical process. We identify a self-confirming retrieval loop, in which retrieval errors bias predictions, biased predictions contaminate memory, and the resulting memory reinforces future forecasting errors. We analyze when such perturbations decay or amplify, introduce grounding to anchor memory evolution to observed outcomes, and develop a reliability-aware adaptive memory update. Extensive experiments on four public benchmark datasets demonstrate that ECHO achieves competitive performance compared with a wide range of state-of-the-art methods.

Primary area:
learning on time series and dynamical systems
Keywords:
Recursive Retrieval, Irregular Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_54bcOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper uses RAG techniques for time series forecasting, with a particular focus on irregular time series. However, the novelty of the paper is not high enough, and it does not provide a good review of previous RAG research in time series. Therefore, I think the quality of this paper still needs to be improved.

Critical strengths
  1. The authors use RAG techniques for retrieval augmentation in time series forecasting, and the results are decent.
  2. The authors achieve a decoupling of this problem by freezing the parameters of the original predictor, and this is reasonable both in the experiments and in the comparison settings.
Critical weaknesses
  1. The only component actually designed for irregular sampling, the irregularity-specific similarity in Eq. 12 λirr\lambda_{\text{irr}}, has weight 0 in every experiment. It's never switched on.
  2. The prediction write-back loop, the stability analysis, the grounding, and the adaptive gating have nothing to do with whether sampling is irregular. Swap in evenly spaced series and all the conclusions still hold. So the paper is really about recursive retrieval control, not irregular time series.
  3. The experiments contain no analysis of irregularity at all. Different missing rates, different sampling intervals, nothing is reported, so readers can't see what ECHO is worth in irregular settings or where its limits are.
  4. The paper does cite retrieval-augmented forecasting works like TimeRAF and TS-RAG, but only in one sentence in the appendix. The main text never discusses how ECHO differs from them, and there is no retrieval-augmented baseline in the experiments. It flies the RAG-for-time-series banner but barely engages with that line of work.
Questions for authors
  1. The irregular similarity weight λirr\lambda_{\text{irr}} in Eq. 12 is 0 in every experiment. So what is this module even for? Does that mean the version without any irregular design is already good enough?
  2. The closed loop, the stability analysis, the grounding, the adaptive gating, all of that should hold on evenly spaced series as well, right? Can you give one example that fails only under irregular sampling but works fine under regular sampling?
  3. How does ECHO perform under different missing rates and different sampling intervals? Without even this analysis, why should anyone believe it is better suited to irregular settings than methods without retrieval?
  4. TimeRAF and TS-RAG are retrieval-augmented time-series forecasting works, but the main text never mentions them and the experiments never compare against them. So what is the actual difference between ECHO and them? Why is there not even one retrieval-augmented baseline?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 54bc

We thank the reviewer for the careful evaluation and constructive feedback. In response, we have clarified that ECHO's central contribution lies in recursive retrieval control, while irregular multivariate time-series forecasting serves as an important application setting. We have conducted additional experiments examining irregularity-specific similarity, recursive feedback under regular and irregular sampling, mask-aware retrieval aggregation, and robustness to varying missing rates and temporal sampling patterns. These analyses also clarify the limitations of retrieval augmentation under severe irregularity. Furthermore, we have expanded the discussion of TimeRAF, TS-RAG, and KUP-BI, and included TimeRAF and TS-RAG as retrieval-augmented baselines under a consistent IMTS evaluation protocol. Together, these revisions provide a clearer characterization of ECHO's methodological scope, empirical performance, and limitations. All new experimental results are included in the appendix of the revised manuscript. All revisions are highlighted in blue in the manuscript. We address each comment in detail below.

R2W1 & R2Q1: We acknowledge that the irregularity-specific similarity term in Eq. (12) was disabled in the original experiments (λirr=0\lambda_{\mathrm{irr}}=0). Varying its weight from 0 to 1 changes Top-KK retrieval sets for 97.2–100% of queries but yields negligible forecasting differences (Appendix B.12, Tables 14–15). We therefore clarify that this term is optional and does not explain ECHO's reported performance. Irregular inputs are handled by the pretrained irregularity-aware backbone and temporal representations. ECHO's central contribution instead concerns recursive prediction write-back, evolving memory, and reliability-aware grounding. We revised the Introduction to avoid overstating irregularity-specific novelty.

R2W2 & R2Q2: We agree that recursive feedback and grounding are not specific to irregular sampling. We evaluated ECHO on regularly sampled ETTh1 and under matched regular, random, and bursty sampling patterns. Under controlled retrieval coupling, grounding with γmin⁡=0.5\gamma_{\min}=0.5 reduces MSE relative to ungrounded recursion by 64–66% on ETTh1, while adaptive grounding achieves reductions of 2.9–4.1% on PhysioNet across sampling patterns (Appendix B.13, Tables 16–17). These benefits do not systematically increase with irregularity. However, incomplete retrieved trajectories require mask-aware aggregation: on Human Activity, masking reduces MSE from 0.8327 to 0.0883 compared with naive zero filling (Appendix B.13, Table 18). These findings establish that recursive feedback control is general, while irregular forecasting introduces additional missing-observation challenges.

R2W3 & R2Q3: We conducted missing-rate and sampling-pattern experiments using identical observation masks and frozen checkpoints for ECHO, static retrieval, and the non-retrieval backbone (Appendix B.14, Tables 19–21). Removing up to 70% of available observations increases forecasting errors across all methods. Under increasingly persistent missingness, ECHO's MSE rises from 0.0501 to 0.0999 on Human Activity and from 0.3564 to 0.4305 on PhysioNet. ECHO remains comparable to the baselines, but retrieval gains diminish under severe missingness and are occasionally negative on PhysioNet. We explicitly clarify that retrieval augmentation does not consistently improve performance as sampling irregularity increases.

R2W4 & R2Q4: We expanded comparisons with TimeRAF, TS-RAG, and KUP-BI (Appendix A.1) and added TimeRAF and TS-RAG as retrieval-augmented baselines under a consistent IMTS evaluation protocol. As shown in Table 1 in this response (Table 1 in the paper), ECHO achieves lower MSE and MAE than both methods across all four datasets. Relative to the strongest retrieval baseline, ECHO reduces MSE by 3.3% on Human Activity, 6.1% on USHCN, 5.6% on PhysioNet, and 3.2% on MIMIC. Unlike conventional retrieval augmentation, ECHO focuses on recursive forecast write-back into evolving memory and reliability-aware control of the resulting feedback. These results strengthen the empirical comparisons while clarifying our methodological distinction.

Table 1. Comparison with retrieval-augmented forecasting baselines.

MethodHuman ActivityUSHCNPhysioNetMIMIC
TimeRAF0.0426±0.0002 / 0.1161±0.00010.1581±0.0019 / 0.2710±0.00070.3263±0.0003 / 0.3632±0.00060.4167±0.0016 / 0.3945±0.0013
TS-RAG0.0431±0.0000 / 0.1152±0.00080.1647±0.0121 / 0.2656±0.01500.3270±0.0005 / 0.3690±0.00070.4032±0.0012 / 0.3916±0.0010
ECHO0.0412±0.0003 / 0.1142±0.00170.1485±0.0043 / 0.2605±0.00610.3080±0.0008 / 0.3620±0.00130.3904±0.0003 / 0.3769±0.0012
AuthorsOpenReview ↗

Additional Experimental Results

Appendix B.13 Table 16 Recursive feedback and grounding across sampling patterns.

ConditionRegularRandomBursty
Recursive write-back (γ=0\gamma=0)3.990±0.2253.929±0.3003.914±0.265
Adaptive ECHO1.951±0.5961.897±0.4521.809±0.334
Adaptive ECHO, γmin⁡=0.5\gamma_{\min}=0.51.341±0.1101.397±0.1671.382±0.180
Reduction vs. recursive (γmin⁡=0.5\gamma_{\min}=0.5)66%64%65%

Appendix B.13 Table 18 Mask-aware versus naive retrieval aggregation.

DatasetDensityMask-awareNaive
Human Activity0.250.0883±0.01370.8327±0.0602
PhysioNet0.480.4892±0.01700.4970±0.0104

Thanks for the detailed response. The new experiments address most of my concerns, and I appreciate that you have made the scope of the paper clearer. I still have a couple of questions.

  • I may have missed this, but did you directly vary the sampling intervals, or were the new experiments mainly about missing rates and different sampling patterns? If the intervals were not tested separately, I think it would be good to say that clearly.
  • The additional analyses show changes in memory composition, but the forecasting errors in the standard online evaluation are almost the same. Does this mean the effect of recursive feedback is mainly visible under the controlled-coupling setting?A short explanation would help me understand this better.

Overall, I’m satisfied with the revision.

Reviewer_H2P9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors study retrieval-augmented forecasting for irregular time series and derive a concrete framework for evolving retrieval memory. The problem formulation is refreshing, and the empirical results are encouraging. However, the empirical evidence does not yet convincingly establish the proposed recursive failure mechanism or demonstrate that the adaptive grounding strategy reliably resolves it.

Critical strengths

1、 The paper raises an interesting and practically relevant question about feedback loops in evolving retrieval memory. 2、 The distinction between static and recursively updated retrieval memory is conceptually clear. 3、 Delayed grounding is a reasonable way to use outcomes that become available after a forecast is made.

Critical weaknesses

1、 The central self-confirming retrieval loop is not directly demonstrated. The paper does not log or analyze round-by-round retrieval quality, memory contamination, or how an erroneous prediction is later retrieved and reinforces future errors. 2、 Memory evolves only on the validation split and is then frozen for test evaluation. Therefore, the test setting does not actually evaluate continual recursive retrieval and online grounding on the test stream. It instead evaluates a frozen memory state produced using validation outcomes. 3、 The adaptive grounding mechanism is only compared with one fixed grounding rate, γ=0.5\gamma=0.5. A broader comparison across fixed rates and a direct analysis of the selected adaptive grounding rates are needed to show that the controller, rather than a favorable fixed setting, is responsible for the gains. 4、 There is a duplicated forecasting equation in Eqs. (5) and (6), which should be corrected. 5、 The theoretical analysis is highly abstract. The scalar reliability state, the transition function, and the effective update coefficient are not estimated or connected to the actual retrieval-memory mechanism. As a result, the contraction condition does not provide a concrete stability guarantee for ECHO. 6、 Figure 1 motivates the method by showing how a retrieved historical trajectory and its subsequent evolution can help correct a forecast. This intuition appears closely related to prior work such as KUP-BI. The authors should discuss this connection and clarify which aspects of ECHO are specific to recursive memory updates and their stability, rather than to retrieval-augmented forecasting itself.

Questions for authors

My main questions and concerns are detailed in the Weaknesses section above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer H2P9

We thank the reviewer for the careful evaluation of our work and the constructive suggestions. We expanded the analyses of recursive retrieval feedback, online memory evolution, grounding strategies, and stability, clarified the distinction from KUP-BI, and corrected the duplicated equation. All revisions are highlighted in blue in the revised paper.

R2W1: We conducted a strictly chronological, round-wise analysis of memory composition, retrieval reuse, and retrieved-entry errors. Ungrounded recursive write-back accumulates model-generated entries, reaching 100% on Human Activity and 79.9% on PhysioNet. Adaptive grounding reduces these proportions to 80.0% and 55.3%, respectively, and lowers PhysioNet retrieved-entry error from 0.1306 to 0.0887 (Appendix B.10, Table 11 and Figure 8). Individual traces further show that contaminated retrieval can amplify subsequent forecasting errors by approximately 2× and 6× (Appendix B.7, Table 8 and Figure 7). These cases explicitly trace erroneous predictions through memory write-back and subsequent retrieval. The results provide direct evidence of recursive error feedback and its partial mitigation, without implying universal error amplification or superiority over fixed grounding.

R2W2: We conducted strictly chronological online evaluations in which forecasts use only previously available information and observation-based grounding occurs only after outcomes are revealed, preventing future-information leakage. Recursive write-back and adaptive ECHO achieve performance comparable to frozen-memory evaluation on Human Activity and PhysioNet, including bounded-memory and delayed-reveal settings (Appendix B.8, Table 9). Round-wise statistics further confirm that grounding reduces the accumulation and retrieval of model-generated memory entries (Appendix B.10, Table 11). Thus, comparable forecasting accuracy does not imply identical memory dynamics. These findings characterize continual memory evolution without establishing global stability or consistent forecasting gains over static memory.

R2W3: We expanded fixed-rate grounding ablations with a mean-matched baseline and analyzed online rate selection. All strategies use identical backbones, retrieval mechanisms, memory capacities, and Top-KK settings. Under standard evaluation, forecasting accuracy remains nearly unchanged across grounding policies. Under controlled coupling (λ=0.5\lambda=0.5, 30% biased write-backs), adaptive ECHO achieves 2.5% lower MSE than mean-matched fixed grounding (0.4758 vs. 0.4879) at the same average rate of 0.31, and performance comparable to fixed γ=0.5\gamma=0.5 with 38% less grounding (Appendix B.11, Table 12). Selected rates vary across samples, with 10th–90th percentiles of 0.146–0.296 on Human Activity and 0.206–0.470 on PhysioNet; their means remain similar across stream halves (Table 13). However, fixed γ=0.9\gamma=0.9 yields lower errors. These results suggest a potential grounding-efficiency benefit under matched budgets, without establishing statistical significance or consistent superiority. Importantly, the controlled-coupling setting is designed to expose grounding effects more clearly and should not be interpreted as representative of standard deployment conditions.

R2W4: We removed the duplicated content from the revised manuscript.

R2W5: We acknowledge that the scalar reliability model provides an abstraction of recursive feedback, not a stability guarantee for the complete ECHO system. We extended Sections 3.3–3.4 with a local sensitivity analysis linking memory perturbations to retrieval-based predictions and grounded write-back. Under local regularity and unchanged Top-KK selections, we derive a conditional prediction sensitivity bound and show that grounding attenuates expected write-back discrepancies by 1−γ1-\gamma under coupled updates sharing the same observation and grounding rate. Experiments show normalized forecast discrepancies below 0.001% for value perturbations and 0.005% for joint value–key perturbations, despite Top-KK overlap falling to 0.599. Amplified retrieval coupling increases the discrepancy to 0.29% (Appendix B.6, Table 6). These findings support limited prediction sensitivity under the tested conditions, but do not cover arbitrary retrieval-selection changes, establish global memory contraction, or imply controller-enforced stability.

R2W6: We acknowledge that retrieving historical trajectory continuations is shared with methods such as KUP-BI and is not unique to ECHO. Unlike KUP-BI's fixed historical library, ECHO recursively writes model-generated forecasts into evolving memory, allowing prediction errors to persist or propagate through subsequent retrievals. Our contribution therefore concerns the resulting closed-loop retrieval–forecasting–memory dynamics and reliability-aware adaptive grounding, rather than the retrieval of historical continuations itself. We clarified this distinction in the Introduction and Related Work.

AuthorsOpenReview ↗

Additional Experimental Results

Appendix B.7, Table 8 Trace of recursive error feedback on Human Activity under the controlled-coupling setting. Counterfactual errors are reported for subsequent queries without → with the retrieved memory entry.

CasePolicyOriginal errorStored asLater reuse (round: without → with)
A (round 3)Recursive (γ=0\gamma=0)5.16prediction (5.16)r12: 0.113 → 0.227; 0.031 → 0.188
Fixed γ=0.5\gamma=0.55.16prediction (5.16)r12: 0.113 → 0.227; 0.031 → 0.188
Adaptive ECHO5.16prediction (5.16)r12: 0.113 → 0.227; 0.031 → 0.188
Adaptive, γmin⁡=0.5\gamma_{\min}=0.55.16prediction (5.16)r12: 0.113 → 0.227; 0.031 → 0.188
B (round 12)Recursive (γ=0\gamma=0)6.36prediction (6.36)r18: 0.049 → 0.049
Fixed γ=0.5\gamma=0.56.36ground truth (0)not reused as contaminated
Adaptive ECHO6.36ground truth (0)not reused as contaminated
Adaptive, γmin⁡=0.5\gamma_{\min}=0.56.36ground truth (0)not reused as contaminated
All write-backsRecursive (γ=0\gamma=0)—contaminated: 218 / 218—
Fixed γ=0.5\gamma=0.5—contaminated: 108 / 218—
Adaptive ECHO—contaminated: 169 / 218—
Adaptive, γmin⁡=0.5\gamma_{\min}=0.5—contaminated: 108 / 218—

Appendix B.8, Table 9 Online evaluation with continual memory updates.

MethodHuman Activity (MSE / MAE)PhysioNet (MSE / MAE)
Backbone (no retrieval)0.0412±0.0024 / 0.1146±0.00820.3088±0.0007 / 0.3655±0.0024
Static memory (original protocol)0.0412±0.0004 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
Recursive write-back (γ=0\gamma=0)0.0412±0.0005 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
ECHO (adaptive grounding)0.0412±0.0005 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
ECHO, bounded memory0.0412±0.0005 / 0.1142±0.00180.3082±0.0007 / 0.3622±0.0012
ECHO, delayed reveal0.0412±0.0003 / 0.1142±0.00170.3080±0.0008 / 0.3620±0.0013

Appendix B.11, Table 12 Fixed versus adaptive grounding on PhysioNet under controlled retrieval coupling (λ=0.5\lambda=0.5) with 30% biased write-backs. Results are reported as MSE / MAE. The mean-matched fixed rate equals the average γt\gamma_t selected by the adaptive controller in this controlled-coupling setting (0.31); it therefore differs from the average under the standard setting reported in Table 3-5.

Grounding strategyMean γ\gammaMSE / MAE
Fixed γ=0\gamma=0 (recursive)0.000.5084±0.0279 / 0.5105±0.0170
Fixed γ=0.1\gamma=0.10.100.4987±0.0246 / 0.5052±0.0154
Fixed γ=0.3\gamma=0.30.300.4879±0.0165 / 0.4977±0.0110
Fixed γ=0.5\gamma=0.50.500.4772±0.0141 / 0.4895±0.0089
Fixed γ=0.7\gamma=0.70.700.4710±0.0096 / 0.4855±0.0060
Fixed γ=0.9\gamma=0.90.900.4644±0.0110 / 0.4812±0.0066
Fixed γ=1.0\gamma=1.0 (fully grounded)1.000.4669±0.0103 / 0.4826±0.0064
Mean-matched fixed γ\gamma0.310.4879±0.0158 / 0.4977±0.0109
Adaptive ECHO0.310.4758±0.0228 / 0.4892±0.0144

I am largely satisfied with the authors’ response. I have a few remaining comments on the writing and presentation that I hope the authors can address in the final revision.

  1. Please define and explain hyperparameters close to their first appearance. For example, the explanation of λp\lambda_p should immediately follow the equation where it is introduced, rather than appearing later.

  2. Figure 3 is quite blurry and difficult to read. I did not mention this in my first-round review, but it should be improved in the final version.

  3. The introduction repeats the same account of the one-way static-memory pipeline, recursive write-back, and self-reinforcing errors several times; a similar explanation appears again at the end of the related-work section. Retaining Eqs. (1)–(3) and one clear explanation of the mechanism would make the presentation more concise.

  4. Some of the writing issues below also need to be paid attention to

    · Use “Eq. (6)” rather than “Eq.6,” and apply this formatting consistently. · In Section 3.3, the reference immediately before Eq. (20) should be to Eq. (13), rather than Eq. (14). · In Appendix B.12, “Eq. equation 10” appears to be an editing error. Please also check the equation number: the irregularity-specific similarity term is introduced in Eq. (11). · In Appendix B.5, “rift topk” appears to be an unrevised code variable or placeholder. dian5 Appendix B.2 states that there are 14 baselines, but I count 16 listed methods.

AuthorsOpenReview ↗

Response to Follow-up Questions

We sincerely thank the reviewer for the positive assessment of our response and for the valuable suggestions to further improve the manuscript. We have carefully addressed all the remaining comments in the revised version. Specifically, we have clarified hyperparameter definitions near their first appearances, improved the clarity and readability of Figure 3, and streamlined the Introduction and Related Work sections to eliminate redundant explanations while retaining Eqs. (1)–(3). We have also standardized equation references, corrected the incorrect equation numbers, replaced the residual code-style variable with appropriate notation, and updated the baseline count from 14 to 16.

We greatly appreciate the reviewer's constructive feedback, which has helped us improve the clarity, consistency, and overall presentation of our manuscript.

Reviewer_UW1wOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces ECHO, a retrieval-augmented framework for irregular multivariate time-series forecasting with evolving memory. Its central motivation is similar to memory management in computer operation system in some degree (particularly the FIFO in line 329). To manage this “prediction memory,” ECHO combines global/local retrieval, utility-based re-ranking, and residual forecast fusion in a comprehensive framework. It dynamically updates the memory by probabilistically selecting either observed outcomes or model predictions for write-back. Experiments suggest that ECHO achieves competitive forecasting accuracy and computational efficiency compared with the baselines, with an additional study examining the impact of memory capacity on its performance.

Critical strengths
  1. The paper moves beyond static retrieval memory and studies how previous predictions can influence future retrieval. Identifying the resulting self-confirming feedback loop provides a valuable perspective on reliability in self-improving processes.
  2. The connection between evolving temporal memory and OS memory is interesting. The fixed forecaster can be deemed a computer, and the prediction can be deemed its generated data stream.
  3. ECHO combines global/local temporal matching, predictive utility-based re-ranking, and selective residual forecast fusion in a coherent framework. This design considers both historical similarity and forecasting usefulness, while preserving the base forecaster’s predictions.
  4. Across four benchmarks spanning healthcare, human activity, and climate, ECHO achieves five best and two second-best results among eight dataset–metric combinations. Its lower mean errors than fixed grounding across all four datasets also support the value of adaptive memory updates.
  5. The paper is well organized, with clear statements and logical explanations. The motivation, methodology, and experimental results are easy to follow, making the proposed framework accessible to readers.
Critical weaknesses
  1. Although ECHO manages the “prediction memory” in a comprehensive way, classical memory-management problems in operating systems, such as thrashing, may still occur. For example, if a bounded memory stores k distinct patterns while the prediction stream cycles through k+1patterns, a FIFO policy may repeatedly evict the pattern needed next. In this case, the relevant pattern would consistently fall outside the memory, potentially causing repeated retrieval misses and unstable forecasting performance (In theory).
  2. In Eqs. (8)–(12), the authors use centroid debiasing and global/local similarity matching to reduce similarity collapse. However, these operations do not explicitly preserve diversity within the “prediction memory,” which may still degenerate as similar predictions are repeatedly written back. Ideas from particle filtering, particularly diversity-preserving rejuvenation, may be helpful in addressing this issue.
  3. Lines 190—191 is repeated from Lines 192--193.
Questions for authors

see weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer UW1w

We thank the reviewer for the constructive feedback. In response, we have conducted additional experiments to investigate FIFO thrashing under bounded-memory constraints and clarified the limitations of existing retrieval and memory-management mechanisms, particularly regarding diversity preservation. We have also discussed potential diversity-aware extensions and removed the duplicated content. All revisions are highlighted in blue in the manuscript. We address each comment below.

R2W1: Thank you for this insightful observation. We agree that FIFO replacement is susceptible to thrashing under cyclic access patterns. Although ECHO employs unbounded memory by default, we conducted an additional stress test using bounded memory of capacity (C) and a cyclic stream of (C+1) distinct patterns.

As shown in Table 1 of this response (Table 10 in Appendix B.9 of the revised paper), FIFO yields zero pattern hits across all evaluated capacities, whereas utility-aware eviction achieves hit rates of 90.9–98.0%. Under amplified retrieval coupling, FIFO generally incurs higher forecasting errors, although higher hit rates do not consistently translate into lower MSE. These findings demonstrate FIFO's vulnerability to adversarial access patterns, complementing the limited memory-capacity sensitivity observed in Table 3 of the revised manuscript. We have clarified this limitation and identified utility-aware retention as a potential extension.

Table 1. FIFO thrashing and utility-aware eviction under bounded-memory constraints.

DatasetHA C=10HA C=20HA C=50PhysioNet C=10PhysioNet C=20PhysioNet C=50
Metrichit / MSEhit / MSEhit / MSEhit / MSEhit / MSEhit / MSE
FIFO0.000 / 0.05040.000 / 0.05670.000 / 0.07060.000 / 0.59080.000 / 0.41340.000 / 0.3944
Utility-aware eviction0.909 / 0.03990.952 / 0.04990.980 / 0.07070.909 / 0.43180.952 / 0.24130.980 / 0.2602
Unbounded memory1.000 / 0.03971.000 / 0.04921.000 / 0.06061.000 / 0.38791.000 / 0.26451.000 / 0.2432

R2W2: Thank you for this insightful suggestion. We agree that centroid debiasing and global/local similarity matching improve retrieval discrimination but do not explicitly preserve memory diversity. Repeated prediction write-back may therefore introduce redundancy and reduce the diversity of the evolving memory. Although ECHO employs adaptive grounding to regulate update reliability, it does not enforce diversity preservation. We have clarified this limitation in the revised manuscript and identified diversity-aware retention and particle-filtering-inspired rejuvenation as promising directions for future work.

R2W3: Thank you for pointing this out. We have removed the duplicated content in the revised manuscript.

I'm thankful to the authors for the additional experiments to solve my theoretical concern; I maintain my rating.

We sincerely thank the reviewer for acknowledging our additional experiments and for the thoughtful feedback. We appreciate the reviewer's time and consideration throughout the review process.

Reviewer_SSJiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ECHO, a framework for stabilizing retrieval-augmented forecasting with evolving temporal memory for irregular multivariate time series. It addresses the risk of error amplification when previous predictions are stored and reused in subsequent forecasting rounds. ECHO combines global and local temporal retrieval, utility-aware candidate ranking, residual forecast integration, and an adaptive grounding controller. A dynamical model is also introduced to analyze memory reliability and motivate the proposed stabilization mechanism. Experiments on four datasets—Human Activity, USHCN, PhysioNet, and MIMIC—show competitive performance, with ECHO ranking first on five of eight dataset–metric combinations and second on two others. However, the evidence for long-term stability and the connection between the theoretical analysis and the implemented controller require further scrutiny.

Critical strengths

S1. The paper addresses a practical problem in retrieval-augmented forecasting: previous predictions may be stored in memory and reused in subsequent rounds, potentially causing errors to accumulate over time. This is a meaningful issue in continual forecasting and extends the focus beyond conventional one-step prediction accuracy. S2. The proposed framework combines global and local temporal retrieval, utility-aware candidate ranking, residual forecast integration, and an adaptive grounding controller. These components provide a relatively complete approach to managing evolving memory and reducing the impact of unreliable retrieved information. The paper also introduces a dynamical model to analyze memory reliability and explain the stability of the proposed framework. S3. The authors evaluate ECHO on four datasets: Human Activity, USHCN, PhysioNet, and MIMIC. The results show competitive performance across different settings, with ECHO ranking first on five of eight dataset–metric combinations and second on two others. These experiments provide evidence of the method's effectiveness across multiple forecasting tasks.

Critical weaknesses

W1. The theoretical stability analysis is insufficiently connected to the implemented system. The paper derives a local stability condition using a simplified scalar reliability model, but does not establish that the actual retrieval and memory-update mechanisms satisfy the underlying assumptions or that the adaptive controller enforces this condition. Thus, the analysis offers useful intuition but does not provide a rigorous stability guarantee for ECHO. W2. The empirical evidence for recursive stability and adaptive grounding is limited. The main experiments primarily report forecasting errors, while the paper acknowledges that long-term memory contamination statistics are not logged per forecasting round. Furthermore, the held-out test set is evaluated with the memory state frozen after its evolution on the validation split, which does not fully establish robustness under continued recursive updates during deployment.

Questions for authors

Q1. What assumptions on the actual retrieval operator and memory-update policy are required for the scalar reliability dynamics to represent the implemented system? Can the authors establish a connection between the derived contraction condition and the practical adaptive grounding controller? Q2. Why is the memory state frozen before evaluating the held-out test set, rather than allowing memory to continue evolving as new observations become available? Can the authors provide a strictly chronological online evaluation in which predictions are generated before their corresponding outcomes are revealed, followed by subsequent memory updates?

1 comment
AuthorsOpenReview ↗

Response to Reviewer SSJi

We thank the reviewer for the constructive feedback. We strengthened the theoretical analysis and added online and memory-contamination experiments. All revisions are highlighted in blue. We respond below.

R2W1: Thank you for this insightful comment. We acknowledge that the scalar reliability model does not establish stability of the full ECHO system. We extended Sections 3.3–3.4 with a local sensitivity analysis linking memory perturbations to retrieval and grounded write-back. Under local regularity and fixed Top-(K) selections, we derive a prediction sensitivity bound and show that grounding attenuates expected write-back discrepancies by (1−γ1-\gamma) under coupled memory updates. Experiments show normalized forecast discrepancies below 0.001% for value perturbations and 0.005% for joint value–key perturbations, increasing to 0.29% under amplified coupling (Appendix B.6, Table 6). These findings support local robustness under the evaluated conditions, not global stability.

R2W2: Thank you for this comment. We address frozen-memory limitations through strictly chronological online evaluations and recursive error-feedback analyses. Continual updates perform comparably to frozen memory on Human Activity and PhysioNet, including bounded-memory and delayed-reveal settings (Table 1 in the response; Appendix B.8, Table 9 in the paper). Adaptive grounding reduces memory contamination from 38.6% to 29.9% and erroneous retrieval reuse from 33.0% to 22.4%, although fixed grounding performs better (Table 2 in the response; Appendix B.7, Table 7 in the paper). Case studies show error amplification of approximately 2× and 6× through contaminated retrieval and write-back (Appendix B.7, Table 8 and Figure 7 in the paper). These results demonstrate recursive error feedback and its partial mitigation through grounding without establishing global stability.

Table 1. Strictly chronological online evaluation.

MethodHuman Activity (MSE / MAE)PhysioNet (MSE / MAE)
Backbone (no retrieval)0.0412±0.0024 / 0.1146±0.00820.3088±0.0007 / 0.3655±0.0024
Static memory (original protocol)0.0412±0.0004 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
Recursive write-back (γ=0\gamma=0)0.0412±0.0005 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
ECHO (adaptive grounding)0.0412±0.0005 / 0.1142±0.00180.3080±0.0007 / 0.3620±0.0014
ECHO, bounded memory0.0412±0.0005 / 0.1142±0.00180.3082±0.0007 / 0.3622±0.0012
ECHO, delayed reveal0.0412±0.0003 / 0.1142±0.00170.3080±0.0008 / 0.3620±0.0013

Table 2. Recursive memory contamination and feedback on Human Activity.

MethodContamination rate ↓Erroneous reuse rate ↓MSE / MAE ↓
Static retrieval (no write-back)0.0000.0000.0886±0.0154 / 0.1850±0.0115
Recursive (γ=0\gamma=0)0.386±0.0000.330±0.1360.1308±0.0451 / 0.2103±0.0293
Fixed grounding (γ=0.5\gamma=0.5)0.191±0.0000.133±0.0800.1185±0.0398 / 0.2030±0.0259
Adaptive ECHO0.299±0.0090.224±0.1280.1283±0.0418 / 0.2078±0.0259
Adaptive ECHO, γmin⁡=0.5\gamma_{\min}=0.50.191±0.0000.145±0.0900.1156±0.0347 / 0.2000±0.0202

R2Q1: We thank the reviewer for this question. The scalar reliability model abstracts recursive memory evolution rather than exactly representing ECHO. We have extended Sections 3.3–3.4 to derive a local prediction sensitivity bound, (∣Y^t−Y^t′∣≤βLRd(Mt,Mt′)|\hat Y_t-\hat Y'_t|\leq\beta L_R d(\mathcal M_t,\mathcal M'_t)), under local regularity and unchanged Top-(K) selections. Grounding further attenuates expected write-back discrepancies by a factor of (1−γ1-\gamma) under coupled updates. However, these results do not establish global memory contraction, and the adaptive controller does not explicitly enforce the sufficient contraction condition. We have clarified these assumptions and limitations in the revised manuscript.

R2Q2: We thank the reviewer for this suggestion. We conducted strictly chronological online evaluations in which forecasts use only previously available information and grounding occurs after outcomes are revealed. Results (Table 1 in the response; Appendix B.8, Table 9 in the paper) show that recursive write-back and adaptive ECHO achieve performance comparable to frozen-memory evaluation on Human Activity and PhysioNet, including bounded-memory and delayed-reveal settings. These findings indicate limited sensitivity to continual memory evolution under the tested conditions, without establishing a general stability guarantee.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Discussion Summary

We thank the Area Chair and all reviewers for their constructive feedback. We substantially revised ECHO to strengthen evidence for recursive retrieval feedback, clarify its methodological scope, and address limitations of the stability analysis. Revisions are highlighted in blue, with new experiments in the appendix.

Reviewer SSJi: Recursive stability and online evaluation

The reviewer questioned the connection between our theoretical analysis and the implemented system, and the lack of continual-memory evaluation. We extended Sections 3.3–3.4 with a local sensitivity analysis linking memory perturbations, retrieval, and grounded write-back (Appendix B.6). We conducted chronological online evaluations, including bounded-memory and delayed-reveal settings (Appendix B.8), and analyzed memory contamination and erroneous retrieval reuse (Appendix B.7). Adaptive grounding reduces contamination from 38.6% to 29.9% and erroneous reuse from 33.0% to 22.4%. These findings support conditional local robustness, not a global stability guarantee.

Reviewer UW1w: Memory management and diversity

The reviewer raised concerns about FIFO thrashing and memory-diversity preservation. We conducted bounded-memory stress tests with cyclic access patterns (Appendix B.9). FIFO produces zero pattern hits, whereas utility-aware eviction achieves 90.9–98.0% hit rates. We clarify that adaptive grounding does not guarantee memory diversity and discuss diversity-aware retention as future work. The duplicated text was removed.

In the follow-up, the reviewer acknowledged the additional experiments and maintained their positive rating.

Reviewer H2P9: Recursive feedback, grounding, and theoretical justification

The reviewer requested direct evidence of recursive error propagation, online evaluation, broader grounding comparisons, and clearer theoretical limitations. We added round-wise memory analyses and error traces showing approximately 2× and 6× error amplification through contaminated retrieval (Appendices B.7 and B.10). Chronological evaluations confirm comparable performance under continual updates (Appendix B.8).

We expanded fixed-rate and mean-matched grounding comparisons. Under controlled retrieval coupling, adaptive ECHO achieves 2.5% lower MSE than mean-matched fixed grounding (Appendix B.11), although stronger fixed grounding can perform better. We clarified the assumptions underlying our stability analysis and distinguished ECHO from KUP-BI.

The reviewer stated that they were largely satisfied with our response. We addressed remaining presentation concerns, including figure readability, notation, equation references, and redundancy.

Reviewer 54bc: Irregularity-specific novelty and retrieval baselines

The reviewer questioned ECHO's irregularity-specific contribution, missingness analyses, and retrieval-augmented baselines. We clarified that ECHO's central contribution is recursive retrieval control, with irregular forecasting as an application setting.

We added experiments on irregularity-specific similarity, regular versus irregular sampling, missing rates, sampling patterns, and mask-aware aggregation (Appendices B.12–B.14). These show that feedback control is not exclusive to irregular sampling and that retrieval gains diminish under severe missingness.

We added TimeRAF and TS-RAG baselines. ECHO achieves lower MSE and MAE than both across four datasets, reducing MSE by 3.2–6.1% relative to the strongest retrieval baseline. We expanded the discussion of KUP-BI and distinguished static retrieval from evolving-memory feedback.

The reviewer acknowledged that the new experiments addressed most concerns and subsequently confirmed that both remaining questions were resolved.

Overall Assessment

The rebuttal resulted in improved reviewer assessments. Two reviewers increased their ratings, and all four now assign a score of 3 (Weak Acceptance). H2P9 and 54bc explicitly acknowledged that their concerns were largely addressed, while UW1w maintained a positive assessment.

The revised manuscript provides stronger evidence for recursive retrieval feedback, online memory evolution, grounding behavior, and retrieval robustness. We clarify that standard forecasting is relatively insensitive to memory perturbations, feedback amplification is more visible under controlled coupling, and adaptive grounding does not universally outperform fixed grounding.

ECHO's central contribution is investigating and controlling feedback when model-generated forecasts re-enter retrieval memory, extending retrieval-augmented forecasting beyond static historical libraries.

We hope the Area Chair will consider the revisions, new evidence, and positive reviewer consensus in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.