- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper uses RAG techniques for time series forecasting, with a particular focus on irregular time series. However, the novelty of the paper is not high enough, and it does not provide a good review of previous RAG research in time series. Therefore, I think the quality of this paper still needs to be improved.
Critical strengths
- The authors use RAG techniques for retrieval augmentation in time series forecasting, and the results are decent.
- The authors achieve a decoupling of this problem by freezing the parameters of the original predictor, and this is reasonable both in the experiments and in the comparison settings.
Critical weaknesses
- The only component actually designed for irregular sampling, the irregularity-specific similarity in Eq. 12 , has weight 0 in every experiment. It's never switched on.
- The prediction write-back loop, the stability analysis, the grounding, and the adaptive gating have nothing to do with whether sampling is irregular. Swap in evenly spaced series and all the conclusions still hold. So the paper is really about recursive retrieval control, not irregular time series.
- The experiments contain no analysis of irregularity at all. Different missing rates, different sampling intervals, nothing is reported, so readers can't see what ECHO is worth in irregular settings or where its limits are.
- The paper does cite retrieval-augmented forecasting works like TimeRAF and TS-RAG, but only in one sentence in the appendix. The main text never discusses how ECHO differs from them, and there is no retrieval-augmented baseline in the experiments. It flies the RAG-for-time-series banner but barely engages with that line of work.
Questions for authors
- The irregular similarity weight in Eq. 12 is 0 in every experiment. So what is this module even for? Does that mean the version without any irregular design is already good enough?
- The closed loop, the stability analysis, the grounding, the adaptive gating, all of that should hold on evenly spaced series as well, right? Can you give one example that fails only under irregular sampling but works fine under regular sampling?
- How does ECHO perform under different missing rates and different sampling intervals? Without even this analysis, why should anyone believe it is better suited to irregular settings than methods without retrieval?
- TimeRAF and TS-RAG are retrieval-augmented time-series forecasting works, but the main text never mentions them and the experiments never compare against them. So what is the actual difference between ECHO and them? Why is there not even one retrieval-augmented baseline?
6 comments
Additional Experimental Results
Appendix B.13 Table 16 Recursive feedback and grounding across sampling patterns.
| Condition | Regular | Random | Bursty |
|---|---|---|---|
| Recursive write-back () | 3.990±0.225 | 3.929±0.300 | 3.914±0.265 |
| Adaptive ECHO | 1.951±0.596 | 1.897±0.452 | 1.809±0.334 |
| Adaptive ECHO, | 1.341±0.110 | 1.397±0.167 | 1.382±0.180 |
| Reduction vs. recursive () | 66% | 64% | 65% |
Appendix B.13 Table 18 Mask-aware versus naive retrieval aggregation.
| Dataset | Density | Mask-aware | Naive |
|---|---|---|---|
| Human Activity | 0.25 | 0.0883±0.0137 | 0.8327±0.0602 |
| PhysioNet | 0.48 | 0.4892±0.0170 | 0.4970±0.0104 |
Thanks for the detailed response. The new experiments address most of my concerns, and I appreciate that you have made the scope of the paper clearer. I still have a couple of questions.
- I may have missed this, but did you directly vary the sampling intervals, or were the new experiments mainly about missing rates and different sampling patterns? If the intervals were not tested separately, I think it would be good to say that clearly.
- The additional analyses show changes in memory composition, but the forecasting errors in the standard online evaluation are almost the same. Does this mean the effect of recursive feedback is mainly visible under the controlled-coupling setting?A short explanation would help me understand this better.
Overall, I’m satisfied with the revision.
Response to Reviewer 54bc
We thank the reviewer for the careful evaluation and constructive feedback. In response, we have clarified that ECHO's central contribution lies in recursive retrieval control, while irregular multivariate time-series forecasting serves as an important application setting. We have conducted additional experiments examining irregularity-specific similarity, recursive feedback under regular and irregular sampling, mask-aware retrieval aggregation, and robustness to varying missing rates and temporal sampling patterns. These analyses also clarify the limitations of retrieval augmentation under severe irregularity. Furthermore, we have expanded the discussion of TimeRAF, TS-RAG, and KUP-BI, and included TimeRAF and TS-RAG as retrieval-augmented baselines under a consistent IMTS evaluation protocol. Together, these revisions provide a clearer characterization of ECHO's methodological scope, empirical performance, and limitations. All new experimental results are included in the appendix of the revised manuscript. All revisions are highlighted in blue in the manuscript. We address each comment in detail below.
R2W1 & R2Q1: We acknowledge that the irregularity-specific similarity term in Eq. (12) was disabled in the original experiments (λirr=0). Varying its weight from 0 to 1 changes Top-K retrieval sets for 97.2–100% of queries but yields negligible forecasting differences (Appendix B.12, Tables 14–15). We therefore clarify that this term is optional and does not explain ECHO's reported performance. Irregular inputs are handled by the pretrained irregularity-aware backbone and temporal representations. ECHO's central contribution instead concerns recursive prediction write-back, evolving memory, and reliability-aware grounding. We revised the Introduction to avoid overstating irregularity-specific novelty.
R2W2 & R2Q2: We agree that recursive feedback and grounding are not specific to irregular sampling. We evaluated ECHO on regularly sampled ETTh1 and under matched regular, random, and bursty sampling patterns. Under controlled retrieval coupling, grounding with γmin=0.5 reduces MSE relative to ungrounded recursion by 64–66% on ETTh1, while adaptive grounding achieves reductions of 2.9–4.1% on PhysioNet across sampling patterns (Appendix B.13, Tables 16–17). These benefits do not systematically increase with irregularity. However, incomplete retrieved trajectories require mask-aware aggregation: on Human Activity, masking reduces MSE from 0.8327 to 0.0883 compared with naive zero filling (Appendix B.13, Table 18). These findings establish that recursive feedback control is general, while irregular forecasting introduces additional missing-observation challenges.
R2W3 & R2Q3: We conducted missing-rate and sampling-pattern experiments using identical observation masks and frozen checkpoints for ECHO, static retrieval, and the non-retrieval backbone (Appendix B.14, Tables 19–21). Removing up to 70% of available observations increases forecasting errors across all methods. Under increasingly persistent missingness, ECHO's MSE rises from 0.0501 to 0.0999 on Human Activity and from 0.3564 to 0.4305 on PhysioNet. ECHO remains comparable to the baselines, but retrieval gains diminish under severe missingness and are occasionally negative on PhysioNet. We explicitly clarify that retrieval augmentation does not consistently improve performance as sampling irregularity increases.
R2W4 & R2Q4: We expanded comparisons with TimeRAF, TS-RAG, and KUP-BI (Appendix A.1) and added TimeRAF and TS-RAG as retrieval-augmented baselines under a consistent IMTS evaluation protocol. As shown in Table 1 in this response (Table 1 in the paper), ECHO achieves lower MSE and MAE than both methods across all four datasets. Relative to the strongest retrieval baseline, ECHO reduces MSE by 3.3% on Human Activity, 6.1% on USHCN, 5.6% on PhysioNet, and 3.2% on MIMIC. Unlike conventional retrieval augmentation, ECHO focuses on recursive forecast write-back into evolving memory and reliability-aware control of the resulting feedback. These results strengthen the empirical comparisons while clarifying our methodological distinction.
Table 1. Comparison with retrieval-augmented forecasting baselines.