acceptodds
Under review as a conference paper at ICLR 2027

ROAR: Retrieval Opportunity-Aware Refinement for Zero-Shot Time Series Forecasting

Abstract

Retrieval augmentation provides time series forecasters with historical continuations, yet even candidates that outperform the base forecast may fail to improve the final prediction. We propose ROAR, a Retrieval Opportunity-Aware Refinement framework for zero-shot time series forecasting. To better exploit these improvement opportunities, its training objective allocates additional emphasis across queries based on base-forecast difficulty and the relative improvement offered by retrieved candidates. Using this objective, ROAR first learns to aggregate aligned historical candidates and uses a learned gate to control their correction strength against a fixed base forecaster. It then jointly calibrates the forecasting module and gate to coordinate their contributions, while anchoring the combined prediction to the first-stage refined output. We derive exact decompositions of refinement gains and the opportunity-weighted training loss, and characterize additional gains from joint calibration under a local linearization. Experiments on seven benchmarks show that ROAR achieves the lowest average MSE among the compared methods. Further evaluations demonstrate average forecasting improvements across multiple backbone families and retrieval-augmented forecasters.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Retrieval-Augmented Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_v8ZzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper first shows that for RAG-based TS forecasters, such as TS-RAG and Cross-RAG on a frozen Chronos-Bolt backbone, a retrieved continuation that beats the base forecast often exists, but the final forecast rarely makes use of it. To address this issue, the authors propose ROAR, which is a refiner for TS forecasting which works in the prediction space. On seven TS benchmarks, ROAR reports the best average MSE and also improves several RAG methods and two other backbones.

Critical strengths

[1] Diagnostic study (Appendix A) is the most useful part of the paper to me.

The section includes controls that are often missing in this TS research area (e.g., split-horizon check for regression to the mean (Table 7) and the shuffled-neighbor control (Table 5)). It also reports the results that do not fit the authors' story, for example that Cross-RAG's retrieved candidates are no better than random draws and that no observable feature separates queries where retrieval helps from those where it does not. This makes me think that other researchers working on RAG-based methods using TSFMs would find it useful.

[2] The refiner is cheap and transfers across hosts and backbones.

Stage I adds only about 160 trainable parameters, and the same module improves three RAG hosts (Fig. 3) and two other backbones (Table 3). The backbone transfer result is more convincing to me than the main table, since the gains there are larger than the gaps between methods in Table 2.

Critical weaknesses

[1] Main mechanism (opportunity-aware weighting) is never isolated.

The ablation study shown in Table 4 removes Stage I as a whole, so the scorer, gate, aligned candidates and convex refinement all disappear together. Thus, the gain only tells us that a gated mixture of aligned candidates helps. It does not show that the opportunity term helps compared with plain weighting (beta = 0), or that the gate loss matters (lambda_gate is never set to 0 in Fig. 5). Simple non-learned refiners are also missing.

This is important as Eq. 13 already shows that simply combining the base and candidate predictions can improve the performance, even w/o considering the opportunity. Also, results in Appendix A.4 shows that the features used by the gate do not provide much information about even when retrieval is actually helpful. Therefore, I believe the improvement may come mainly from combining 20 aligned predictions, and not from the proposed mechanism.

[2] The gains are small, reported without uncertainty, and Stage II doubles inference cost.

While results are averaged over 5 runs, standard deviations or significance tests are not provided. The gap to Cross-RAG is 0.193 vs. 0.187 average MSE, and Cross-RAG is better than ROAR's own base in Table 4, which is why I believe the gains are small. Stage II improves the average MSE only by 0.0018, but Table 11 shows that it almost doubles the latency, FLOPs, and number of parameters, as the frozen reference branch needs to run together with the updated branch. Also, with lambdaref=15lambda_ref = 15, the paper’s own analysis shows that Stage II can improve only about 12% of the remaining error. For these reasons, I am concerned whether Stage II is worth the additional cost compared to simply averaging two separately trained modules.

The evaluation is also narrow.

  • It uses a single horizon (H = 64),
  • four of the seven datasets are ETT variants,
  • and there are no probabilistic metrics even though the method shifts quantiles.

The sweeps in Fig. 5 are run on the test splits, so if K and lambda_ref were chosen this way, the zero-shot protocol is weakened.

[Smaller issues]

The main text says retrieval uses the TS-RAG subset of the Chronos corpus, but Appendix A.1 says each variable retrieves from its own training split, and this changes how zero-shot the setting is. PFRP and RATL are adapted to zero-shot without describing what was changed. There are also a few broken figure and equation references.

Questions for authors

Q1) Could you report the performance of ROAR with β=0\beta = 0, λgate\lambda_{gate} = 0, and a fixed-gate uniform-average refiner over the same 20 aligned candidates? If beta = 0 performs within noise of the full model, the framing of the paper would need to change, and my rating depends mostly on this.

Q2) What are the standard deviations over the 5 runs for Tables 2 and 4, including the reproduced baselines? This would tell me whether the gaps to Cross-RAG are larger than seed variance.

Q3) Which retrieval database is used at test time in the main experiments, the Chronos-corpus subset or each target dataset's training split? Also, were K, lambda_ref and beta selected on the evaluation datasets? Both affect how zero-shot the setting really is.

Q4) Does reference anchoring in Stage II add anything beyond a cost-matched baseline, such as an ensemble of two forecasting modules or Stage I on top of a module fine-tuned with plain MSE? Given the 2x inference cost, I would like to see this comparison.

1 comment

I thank the authors for their additional experiments during the rebuttal period. However, I still have some concerns regarding the generalizability of the proposed method and the contribution of the opportunity-aware mechanism. Although the new results have addressed some of my questions, I am not fully convinced that the proposed method gives sufficient value. I will maintain my current score, but I will discuss the paper with other reviewers and consider their reviews & opinions.

Reviewer_1wZLOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies retrieval-augmented zero-shot time series forecasting. The authors observe that retrieved historical candidates often outperform the base forecast, yet existing retrieval-augmented methods frequently fail to exploit them. To address this, they propose ROAR, which refines a frozen foundation model forecast by aggregating aligned retrieved candidates with a learned scorer and an input-dependent gate. Training proceeds in two stages: Stage 1 learns the scorer and gate with an opportunity-weighted loss that emphasizes queries where the base forecast is poor and a better candidate exists, and Stage 2 jointly calibrates the forecasting module and gate while anchoring predictions to the Stage 1 output. The paper also provides an exact decomposition of the refinement gain and analyzes joint calibration under local linearization. Experiments on seven benchmarks show that ROAR achieves the lowest average MSE among the compared TSFMs and retrieval-augmented methods, and it improves performance when applied to other backbones and retrieval-augmented forecasters.

Critical strengths
  1. The problem is well motivated with an empirical diagnosis. The paper first shows empirically that useful retrieved candidates often exist but are not exploited. The proposed opportunity-weighted objective follows directly from this diagnosis, and Figure 4 also confirms that ROAR's gains grow with available opportunity.

  2. The experiments are comprehensive and the improvements are consistent. ROAR achieves the lowest average MSE among five retrieval-augmented methods under a shared backbone and retrieval protocol. It also works as a plug-in to improve RAF, TS-RAG and Cross-RAG, and it works with Moirai2.0 and TimesFM2.5 as backbones. Although the per-dataset gains are modest, their consistency across backbones and baselines suggests the approach is general rather than tuned to one setting.

Critical weaknesses

All experiments are conducted on seven standard long-term forecasting benchmarks. Although these datasets are widely used in prior work, the coverage of domains and frequencies is limited. Evaluation on broader benchmarks would make the claims more convincing.

Questions for authors
  1. Results are averaged over five runs, but no standard deviations are reported. Could the authors report mean ± std for results and indicate whether the improvement over baselines is statistically significant?
  2. Since the paper targets zero-shot forecasting, could the authors evaluate ROAR on a more diverse benchmark (e.g., GIFT-Eval) or on additional datasets from domains and frequencies not covered by the current seven? Even a subset would help assess generality.
  3. In Table 5, the share of samples with a better candidate is almost identical for Cross-RAG’s retrieved top-K (35.3%) and for random draws from the library (35.1%), suggesting that its retrieval does not find better candidates more often than chance. However, replacing the original neighbors with shuffled ones degrades Cross-RAG substantially, what is the possible reason of this result?
Reviewer_j6uoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper starts from the observation that, although some retrieved candidate patches do help, they are not properly used. To address this, the paper trains a gate that determines the weight between the base prediction and the retrieved results. In addition, the paper increases the loss of “difficult queries with a good candidate”, which it calls opportunity-aware weighting.

Critical strengths
  1. The core idea is simple and makes sense. Giving more weight to meaningful queries is reasonable.

  2. The reported numbers are strong. The comparison with other TSFMs and RAG-based methods shows that ROAR outperforms them, and using ROAR as a plug-in consistently improves the performance.

Critical weaknesses
  1. The novelty is unclear. Each component, including the input-dependent fusion gate (TS-RAG) and learning based on candidate usefulness (RATL), already appears in existing works, so the main contribution of this paper becomes combining them.

  2. The design choices behind the training weight w_i (Eq. 7) are unclear. (1) g_i already represents the usefulness, so it is unclear why an additional threshold is needed. (2) Why is 1 added? If it is to prevent a zero weight, it could be any other arbitrary constant. (3) g_i is a ratio between errors and thus scale-free, so why is it multiplied by s_i^2? The text states that this multiplication sets the baseline weighting of the raw-scale forecasting loss, but the raw-scale MSE is already proportional to s_i^2, which would make the loss proportional to s_i^4.

  3. The motivation of the paper is not delivered directly in the Abstract and Introduction. More specifically, the terms candidate, neighbor, proposal, reference, base forecast and refined forecast are used together and read as jargon.

  4. In Appendix A, the performance of classifying whether a query is a “good query” is around 0.5, which makes it difficult to trust the decision of the gate.

  5. There is no ablation study with and without opportunity weighting.

  6. ROAR requires a large efficiency trade-off. In Appendix C.3, it requires double the latency, FLOPs and memory (total parameters), which cannot be described as “moderate and predictable”.

  7. The equations, especially Eq. 13, Eq. 14 and Eq. 16, are fine as a mathematical description of the process, but they are too trivial to be a contribution bullet described with “derive” or “characterize”. These equations neither justify the framework design nor provide a verifiable guarantee.

Questions for authors
  1. Please clearly summarize the novelty of this work distinguished from existing works.

  2. Please clarify the design choices that might be ambiguous. The training weight w_i was a one example that was most ambiguous.

  3. Please suggest the evidence that the gate can work well even with 0.5 AUROC.

  4. Please suggest the isolated effect of opportunity weighting.

  5. Please clarify the significance or insight of equations that leads to a contribution.

1 comment

Thank you for the response and the additional experiments. Several concerns are addressed. Three points remain.

  1. Contribution (W1, W5). The response places the novelty in the opportunity-aware weighting and Stage II. In Table R2, however, the largest part of the improvement comes from the gate supervision, while these two components contribute only a small part.

  2. Gate (W4). Although Appendix A does not evaluate the gate itself, there is still no explicit evidence that the proposed method can distinguish the queries where retrieval is useful.

  3. Scale factor (W2). As clarified, the Stage-I loss is already proportional to the scale s_i. It therefore remains unclear why the additional factor s_i^2 is needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.