acceptodds
Under review as a conference paper at ICLR 2027

HIRA: GRADIENT-FREE RESIDUAL RETRIEVAL FOR LONG-HORIZON TIME-SERIES FORECASTING

Abstract

Can a frozen time-series foundation model adapt to long-horizon forecasting without additional gradient training? We introduce Hidden-state Residual Adaptation (HIRA), which turns the backbone’s own context representations into an index of historical forecast errors. Suffix-pooled final tokens retrieve observed residuals, and horizon-wise configurations convert them into corrections without training an encoder, retriever, or fusion network. Under chronological selection with memory and configurations frozen before test, HIRA yields positive mean gains on four backbones at both 336 and 720 steps. On Chronos-2, its agreement-gated variant improves all seven datasets at the frozen-protocol 720-step horizon, reducing MSE by 13.15% on average, compared with 12.61% for the tested matched SARAFstyle control. The ungated recipe achieves long-horizon mean reductions of 4.54%, 7.00%, and 4.74% on TimesFM, Moirai 2.0, and Sundial, respectively. Mechanism controls and a recoverable-value analysis connect these gains to query-accessible historical errors and clarify transfer limits. These results establish residual memory as a competitive route to long-horizon adaptation without additional gradient training.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, foundation models, retrieval-based adaptation, residual correction, hidden representations, distribution shift

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_t4GqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HIRA, a gradient-free adaptation method for frozen time-series foundation models. HIRA stores historical forecast residuals indexed by suffix-pooled hidden representations from the frozen backbone. At inference time, it retrieves residual trajectories from similar historical contexts and adds their weighted aggregation to the base forecast. Retrieval configurations and correction strengths are selected separately for different forecast-horizon blocks using chronological validation. The method is evaluated across TimesFM, Chronos-2, Moirai 2.0, and Sundial.

The paper is clearly written, carefully executed, and unusually transparent about development versus confirmation evidence. However, I have a fundamental concern about novelty relative to the previously released RATL paper. In my view, the current submission does not establish a sufficiently substantive contribution beyond RATL to warrant acceptance.

Critical strengths

The motivation is intuitive: a frozen forecaster may make recurring, context-dependent errors that can be reused as an adaptation memory. The proposed method is simple and does not require training an additional encoder, retriever, or fusion network. The chronological availability constraints are carefully designed to avoid direct use of unrevealed future targets. The evaluation covers four foundation-model backbones, and the authors commendably retain negative-transfer cases and explicitly distinguish developmental, confirmation, and post-hoc evidence.

Critical weaknesses

My primary concern is the substantial conceptual and algorithmic overlap with the previously released paper “RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting,” which appeared on arXiv on September 3, 2026 (arXiv:2609.03937). RATL is neither cited nor discussed in the current submission.

The overlap concerns the central idea rather than a broad similarity in topic. Both methods freeze a base forecaster, run it on historical forecasting windows, compute model-specific residual trajectories, store historical context representations as retrieval keys and the corresponding residuals as values, retrieve residuals from similar contexts under causal availability constraints, aggregate the retrieved residuals, and add the resulting correction to the base forecast. Both methods also account for variation across forecast blocks and use validation to control correction strength. Thus, both papers make the same central conceptual move: replacing the retrieval of historical future trajectories with the retrieval of base-model-specific historical forecast errors.

HIRA differs from RATL in several implementation details. It uses native TSFM hidden states, suffix pooling over the last one, three, or five tokens, cosine top-k retrieval, validation-selected block-specific configurations rather than a learned set-aware router, and an optional agreement gate. It also provides a recoverability analysis. However, these differences appear to be choices of representation, router simplification, hyperparameter selection, and analysis around the same residual-memory framework, rather than a substantively different method.

The paper’s own ablations reinforce this concern. Hidden-state retrieval improves the mean gain over raw-input retrieval by only approximately 0.80 percentage points. Horizon-wise configuration improves over a global configuration by approximately 0.37 percentage points, and selecting among last-1/3/5 representations improves over last-1 by approximately 0.21 percentage points. These results suggest that most of the benefit comes from query-dependent residual retrieval itself, which is precisely the idea already developed by RATL, rather than from HIRA’s specific hidden-state or horizon-wise design.

I am not making a claim about whether the work was developed independently, since this cannot be determined from the manuscript. Nevertheless, novelty must be assessed relative to prior work available at submission time. The residual-retrieval idea is conceptually simple and straightforward to reproduce once stated. The burden is therefore on the current paper to distinguish itself through a direct comparison with RATL, a genuinely different technical contribution, or a capability not already covered by the existing framework. The manuscript currently does not meet this burden. Given the absence of any discussion or comparison, I cannot determine whether HIRA constitutes an independent substantive advance or a relatively direct gradient-free instantiation of the RATL residual-memory principle.

Questions for authors

How does HIRA differ substantively from RATL beyond replacing a learned router with validation-selected nonparametric retrieval?

Can the authors provide a direct comparison with RATL under the same backbone, datasets, residual memory, and label-availability protocol?

Reviewer_ipmbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes HIRA, a gradient-free adapter for frozen time-series foundation models. It uses suffix-pooled hidden states to retrieve historical forecast residuals and applies similarity-weighted corrections, with configurations selected separately for three horizon blocks through purged chronological validation. A cross-view agreement gate is additionally used for Chronos-2. The theoretical analysis distinguishes oracle correction headroom from information recoverable through the query and establishes limitations under ambiguity. Experiments cover four backbones and seven benchmark target series, with positive mean gains at horizons 336 and 720 for each backbone. Gated Chronos-2 achieves a 13.15% mean MSE reduction at horizon 720. Additional validation includes component ablations, statistical uncertainty estimates, and a 73-series evaluation at horizon 96.

Critical strengths

A simple adaptation mechanism with useful cross-backbone results. Using a forecaster’s existing representations to index its historical errors avoids training an additional encoder, retriever, or fusion network. Positive long-horizon mean gains across four backbones support the usefulness of this approach. In particular, ungated HIRA improves all 14 long-horizon Moirai settings, providing evidence beyond the Chronos-2 variant.

Careful evaluation and transparent reporting. The protocol explicitly enforces historical-label availability, freezes memory before testing, and distinguishes development results from subsequent frozen-protocol evaluations. The paper also reports negative transfer, unsuccessful extensions, and stronger competing methods. These disclosures, together with controls for representation and calibration and uncertainty estimates accounting for overlapping windows, make the empirical evidence easier to assess.

Critical weaknesses

The distinctive methodological contribution has only modest empirical support. Nearest-neighbor correction of forecast errors is already established in Rectify (Ben Taieb and Hyndman, 2012). HIRA’s contribution therefore rests largely on frozen hidden-state retrieval and its configuration scheme. Across the TimesFM matrix, hidden retrieval exceeds raw-context retrieval by 0.80 percentage points in mean gain, span selection adds 0.21 points over last-1, and horizon-wise configuration adds 0.37 points over a global configuration. Paired uncertainty for these increments is not reported. The theoretical results provide a valid interpretation of residual correction, but they apply broadly to correction rules and do not establish when HIRA’s particular representation and retrieval estimator should succeed.

The long-horizon validation remains narrow and does not fully isolate the source of the gains. The main experiments use seven target channels and a fixed 160-point context, while the expanded 73-series evaluation covers only horizon 96. The frozen-protocol evaluations also reuse previously inspected datasets and potentially overlapping target timestamps, limiting their independence from development. Furthermore, the strongest Chronos-2 results include an agreement gate without corresponding ungated and gate-matched comparisons. These limitations leave uncertainty about the breadth of the long-horizon gains and how much of the Chronos-2 improvement can be attributed to the proposed retrieval mechanism.

Questions for authors

Component attribution: Can you provide paired confidence intervals for hidden versus raw-context retrieval, span selection versus last-1, and horizon-wise versus global configuration? These comparisons directly determine the added value of the proposed components.

Gate ablation: What are the ungated Chronos-2 results at horizons 192 and 720, and how do comparable residual baselines perform when given the same agreement gate?

Additional evidence for the current claims: Do the gains at horizon 720 persist on the predeclared expanded channel set and with longer input contexts? A focused evaluation would materially strengthen the evidence for long-horizon generalization.

Reviewer_v9ujOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Hidden-state Residual Adaptation (HiRA), which uses suffix-pooled hidden states from a frozen time-series foundation model to retrieve historical forecast residuals and apply weighted, horizon-block corrections. The method selects configurations using purged chronological validation and includes an exact zero-correction fallback. Across TimesFM, Chronos-2, Moirai 2.0, and Sundial, it reports positive long-horizon macro gains, with different horizons and variants designated as development or frozen-protocol evidence. The paper also gives a squared-loss decomposition of oracle headroom into query-identifiable and irreducible components, and includes retrieval, regression, calibration, and matched-mechanism controls.

Critical strengths

1.The experimental protocol takes temporal leakage seriously. Historical residuals are included only after their targets would be observable, validation is purged by the forecast horizon, test targets are excluded from memory and configuration selection, and the exact backbone forecast remains available through zero shrinkage.

2.The paper is unusually candid about evidence status and negative results. It separates development from frozen-protocol evaluations, reports negative-transfer cases, notes the failure of its validation-only warning (AUROC 0.50), and clearly labels matched mechanism controls as non-native reproductions. This makes the empirical claims easier to assess.

Critical weaknesses

1.The methodological novelty is somewhat incremental: HiRA is cosine nearest-neighbor regression over frozen model states with a weighted residual vote and validation-selected shrinkage. The paper provides evidence that this is a useful training-free adapter, but its strongest positioning should remain empirical rather than implying that residual retrieval is uniformly competitive with learned adaptation. In fact, the matched future-retrieval mixer is stronger on TimesFM overall and on Moirai's long-horizon mean, while the matched controls are not native reproductions and differ in fitting protocol.

2.Configuration selection searches a nontrivial grid (view span, neighbor count, temperature, and correction scale) independently for each of three horizon blocks, using two validation folds. Purging and the nonnegative-fold eligibility rule are sensible safeguards, and the later frozen horizons help, but the paper would benefit from a clearer account of selection variance and sensitivity to reducing this grid or using a nested/held-out selection procedure. This matters most for smaller gains and the limited number of independent datasets.

Questions for authors

1.How sensitive are the gains to the size of the configuration grid and to the two-fold eligibility rule? Could you report the distribution of selected configurations or a nested/held-out selection check on a representative subset, especially for small-gain cells?

2.Could the main claims be framed explicitly around positive average gains rather than broad dataset-level superiority, given the seven-dataset sign-flip result and the negative-transfer cells? What independent dataset or deployment-shift evaluation would be needed to support stronger generalization claims?

3.For comparisons with learned retrieval/fusion methods, can the paper provide a more tightly matched comparison or explain which differences in fitting and selection budgets are most likely to account for the ranking changes across backbones?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.