acceptodds
Under review as a conference paper at ICLR 2027

DMRC: Residual-Manifold Memory Adaptation for Frozen Time-Series Forecasters

Abstract

Retrieval augmentation methods have achieved notable success in time series forecasting by reusing historical input information. However, when events such as trend drifts or changes in volatility alter the prediction errors associated with similar inputs, these methods may struggle to retrieve historical correction information applicable to the current forecast.To address this limitation, we propose Dynamic Memory-enhanced Residual Correction (DMRC), which adapts frozen forecasters by dynamically adjusting the retrieval of historical residual prototypes according to their relevance to current forecasts. DMRC constructs hierarchical residual prototypes offline and retrieves complementary principal-direction and orthogonal residual components for forecast correction. During online forecasting, multi-view and temporal-overlap consistency checks provide label-free feedback to update retrieval priorities for the principal-direction component, reducing reliance on input similarity alone when historical error patterns become less applicable. This mechanism enables adaptive use of historical correction information while keeping both the forecaster and memory contents frozen. Across seven datasets, four forecast horizons, and six frozen forecasters, DMRC consistently outperforms the frozen baseline, with MSE reductions of up to 15.10%.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Retrieval Augmented Forecasting, Residual Manifold Memory, Label-Free Dynamic Retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_HvMVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Dynamic Memory-enhanced Residual Correction (DMRC), an external adaptation framework for frozen time-series forecasters. Offline, it constructs hierarchical residual prototypes using a learned residual-aware representation and decomposes their values into principal-subspace and orthogonal components. During deployment, fixed gates combine retrieved residuals with the base forecast, while label-free consistency signals and leave-one-out prototype credit update channel-wise retrieval priorities through KL-regularized mirror descent. The forecaster and memory contents remain frozen. The main contributions are residual-aware hierarchical retrieval and online adaptation of residual selection without future labels.

Critical strengths

Practical adaptation setting. DMRC adapts frozen forecasters without online parameter optimization or future labels, making it relevant when retraining and immediate supervision are unavailable.

Broad empirical coverage. Across seven datasets, six forecasters, and four horizons, DMRC improves MSE in 146 of 168 settings. This breadth supports the practical usefulness of residual correction across different forecasting architectures.

Critical weaknesses

The external baseline comparisons are difficult to verify. Appendix D.1 describes TQNet as retrieving historical segments and their futures, whereas the cited method uses learnable temporal queries within attention. How it was adapted to the shared frozen-checkpoint setting is unspecified, undermining confidence in the comparative evaluation.

The proposed uncertainty feedback appears ineffective for Chronos. Its interval-width term is unchanged when a residual prototype is removed, so it cancels in the leave-one-out credit (Eqs. 15 and 21). As written, only temporal-overlap feedback can drive updates. Without a Chronos-specific static-versus-dynamic ablation, the reported gains do not establish the contribution of the proposed online adaptation mechanism.

Questions for authors

Clarification: How were TQNet and KUP-BI implemented under the shared frozen-checkpoint protocol? What modifications were made, and which additional parameters were trained?

Clarification: For Chronos, how can removing a residual prototype change the interval-width term in Eq. 15? Does this term contribute any nonzero leave-one-out credit?

Evidence needed for assessment: Can you provide Chronos results comparing static residual memory, overlap-only updates, and full DMRC under identical settings? This would establish whether its gains depend on the proposed online adaptation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer HvMV (Part 1/2)

We apologize for the delayed response, as we have been occupied with several time-sensitive commitments over the past few days.

We thank the reviewer for the careful and detailed feedback. We first address the implementation of the baseline methods raised in W1 and Q1, and then clarify the Chronos feedback mechanism and its empirical contribution in response to W2, Q2, and Q3.

W1 and Q1: Implementation and Integration of the Baselines

We thank the reviewer for noting that the original description of the baseline implementations was insufficient. We clarify that we did not directly reproduce the original end-to-end training pipelines of TQNet and KUP-BI. Instead, we adapted them as external enhancement methods under the shared frozen-forecaster protocol. The revised manuscript now provides the corresponding implementation details.

For TQNet-Post, we first use the fully frozen base forecaster to produce the base forecast. In parallel, the temporal-query, cross-variable attention, and MLP pathways of TQNet generate a residual prediction from the historical window, which is then added to the base forecast through learnable channel-wise gates. During training, we optimize only this residual plugin, including the temporal query, attention projections, input projection, residual MLP, output projection, and channel-wise gates. All parameters of the base forecaster remain frozen throughout.

For KUP-BI-Frozen, we retain the variable-wise retrieval mechanism based on history--target--continuation chains constructed from the training set. The retrieved information is used to construct an auxiliary historical window, xauxx_{\mathrm{aux}}, after which the same frozen forecaster separately computes f(x)f(x) and f(xaux)f(x_{\mathrm{aux}}). The two forecasts are fused at the output level using the original GatedFusion module. Under the current dynamic_gate=False setting, only the channel-wise static gating parameters are trained, while both the retrieval library and the base forecaster remain fixed.

For both adaptations, plugin parameters are optimized only on the training set, checkpoints are selected using the validation set, and all parameters remain fixed during testing. The revised manuscript now explicitly lists the retained components of the original methods, the modifications required by the frozen-forecaster protocol, the trainable parameters, and the training settings. These results should therefore be interpreted as comparisons between adapted methods under a unified frozen-forecaster protocol, rather than as direct reproductions of the original end-to-end configurations.

W2 and Q2: Interval-Width Feedback for Chronos

We thank the reviewer for identifying this issue. The reviewer is correct that the raw prediction-interval width

1HC∑h,c(Qt,h,c,0.9−Qt,h,c,0.1)\frac{1}{HC}\sum_{h,c}\left(Q_{t,h,c,0.9}-Q_{t,h,c,0.1}\right)

is invariant to a common additive residual correction. If removing a prototype removes the same additive term from the lower and upper quantiles, the interval width is unchanged before and after removal. Consequently, the raw interval width cannot itself produce nonzero leave-one-out credit. The original Eq. 15 and its accompanying description conflated this diagnostic uncertainty statistic with the cross-view stability term used in the implementation.

In the implementation, we treat Q0.1Q_{0.1}, Q0.5Q_{0.5}, and Q0.9Q_{0.9} as three probabilistic forecast views. Each view forms its own retrieval query and separately passes through the Adapter, Retrieval, and Gating modules, producing a corrected quantile forecast Q~t,h,c,p\widetilde Q_{t,h,c,p}. The view loss used for online updating is the dispersion among these corrected quantile forecasts:

Qˉt,h,c=13∑p∈{0.1,0.5,0.9}Q~t,h,c,p,\bar Q_{t,h,c}=\frac{1}{3}\sum_{p\in\lbrace0.1,0.5,0.9\rbrace}\widetilde Q_{t,h,c,p},

\left(\widetilde Q_{t,h,c,p}-\bar Q_{t,h,c}\right)^2.$$ Because the three quantile views use different queries, retrieval weights, and candidate-prototype participation patterns, removing a prototype need not remove the same residual correction from all three views. Therefore, the implemented quantity $$\mathcal L_{V,t}^{(-i)}-\mathcal L_{V,t}$$ can be nonzero. We directly measured this quantity in the new diagnostic. On ETTm1 at $H=336$, the corrected-quantile view credit was nonzero in 24.74%, 39.57%, and 21.32% of candidate leave-one-out evaluations for the three random seeds, respectively. The corresponding mean absolute values were $4.796\times10^{-4}$, $1.361\times10^{-3}$, and $1.561\times10^{-3}$. These measurements confirm that, unlike the raw interval width, the corrected-quantile view term used in the implementation does not cancel identically and can provide nonzero prototype-level feedback for mirror descent.
AuthorsOpenReview ↗

Response to Reviewer HvMV (Part 2/2)

W2 and Q2 (continued): Clarification of the Update Term

To remove this ambiguity, we now explicitly identify the raw interval width as a diagnostic quantity only and no longer describe it as part of the leave-one-out update. We have also corrected Eq. 15 to reflect the term used in the implementation, namely the corrected quantile-view dispersion.

W2 and Q3: Static and Dynamic Residual Memory for Chronos

Following the reviewer's suggestion, we conducted a strictly controlled comparison among Static residual memory, Overlap-only updates, and Full DMRC. We report three-seed results on ETTm1 at H=336H=336.

SeedStatic MSE / MAEOverlap-only MSE / MAEFull DMRC MSE / MAEMSE Improvement of Full over Static
20210.497988 / 0.4446600.497988 / 0.4446600.497890 / 0.444568+0.0197%
20220.497955 / 0.4448880.497955 / 0.4448880.497613 / 0.444620+0.0686%
20230.496658 / 0.4440550.496658 / 0.4440550.496606 / 0.444037+0.0103%
Mean over three seeds0.497533 / 0.4445340.497533 / 0.4445340.497370 / 0.444408+0.0329%

Full DMRC achieves lower MSE than Static for all three seeds. The mean MSE decreases from 0.497533 to 0.497370, corresponding to an average improvement of 0.0329%; the mean MAE also decreases from 0.444534 to 0.444408, an improvement of 0.0284%. Under the same setting, the frozen Chronos model has an MSE of 0.510745. Static residual memory therefore already provides the main improvement of approximately 2.59%, while Full DMRC further increases the total improvement to approximately 2.62%. These results indicate that hierarchical residual memory itself provides the primary and more stable benefit, while online residual-selection adaptation can offer an additional improvement on top of the static correction.

Importantly, Overlap-only is numerically equivalent to Static for all three seeds, whereas Full DMRC further reduces MSE for every seed. Because the corrected-quantile view term is the only difference between the two dynamic variants, this controlled comparison attributes the additional improvement in this setting to the quantile-view feedback rather than to the temporal-overlap term or to differences in the base forecaster, memory contents, or gating parameters. Together with the nonzero leave-one-out credit diagnostic above, the results establish a complete evidence chain: the corrected-quantile view term produces nonzero prototype credit, changes online retrieval priorities, and yields a consistent accuracy improvement across seeds on ETTm1 at H=336H=336.

We also conducted extended diagnostics on other horizon--dataset combinations. The incremental benefit of online updating is setting-dependent and does not consistently exceed Static under all conditions. We have therefore narrowed our conclusion as follows: static hierarchical residual memory is the primary and more stable source of the Chronos gains, whereas label-free online residual-selection adaptation provides a small but reproducible additional benefit in some settings. This more precise conclusion both distinguishes the contributions of the two DMRC components and clarifies the applicability boundary of the current surrogate feedback.

Accordingly, we have corrected the definition of Eq. 15 and revised the Methods, ablation study, and Limitations sections to clarify the feedback term used in the implementation, add the controlled comparison among the three settings, and state the setting dependence of the online benefit.

We again thank the reviewer for helping us identify the ambiguity between the original equation and the implementation, and for motivating a controlled comparison that distinguishes the contribution of static residual memory from that of online adaptation.

AuthorsOpenReview ↗

Response to Reviewer HvMV

Thank you again for your thorough and detailed review. We fully acknowledge the depth of your concerns and have been working carefully through each point. We would very much appreciate your thoughts on our rebuttal when you have a chance — even a brief response would be invaluable to us. Looking forward to your reply and further discussion.

Reviewer_YoTvOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes DMRC, a method that improves the forecasting performance of time series models through an external residual memory. Without modifying the forecasting model’s parameters, the method constructs a memory bank from historical data. During forecasting, it checks whether the current forecast matches historical data and uses the historical information to correct the forecast.

Critical strengths

S1. The research problem has practical value. The paper investigates a relatively lightweight approach to improving forecasting accuracy without modifying model parameters. This has practical relevance in settings where the details of the forecasting model are unknown or the model cannot be modified. S2. The method has a degree of general applicability. Since it requires no changes to model parameters and relies only on historical data for correction, it does not depend on the specific implementation of the forecasting model and can be applied to different types of forecasters.

Critical weaknesses

W1. The writing lacks clarity and coherence. Many passages contain repetitive wording, abrupt logical transitions, and vague explanations, making the paper difficult to understand. W2. The central hypothesis is insufficiently justified. The paper introduces a key hypothesis in the opening paragraph and designs the method around it. However, the hypothesis is supported by only two experiments, which do not provide sufficient justification.

Questions for authors

Q1. The reasoning in the introduction is unclear. How should it be understood? How do the limitations of large-scale pretraining in eliminating deployment non-stationarity and of retrieval in identifying reusable errors motivate the so-called error manifold hypothesis? What inspired this hypothesis? Q2. The paper lacks a dedicated related work section and mentions related methods only in the second paragraph of the introduction. Furthermore, the experiments do not compare against these baseline methods, and the main text only compares forecasts without any adjustment against those obtained using the proposed method. Why is this the case? Q3. Could Section 2.2 provide a more formal justification? Evidence from a limited set of experiments is insufficient to guarantee that residuals can effectively correct forecasts under all circumstances. This concerns the theoretical basis of the method. Q4. Figure 1 shows an Encoder and a latent variable zz being used for querying, but Section 3.2 seems to describe a different procedure. What is the difference between the two, or is the description inconsistent? Appendix C.3 does not clearly explain any of the settings either. Q5. Is there a theoretical basis for the formulation of the final prediction in Section 3.3, namely Equation 5? Q6. What exactly does Section 3.5 do? Is it part of the stability feedback mechanism? Why is it presented as a separate section? Its current content does not fully correspond to Figure 1. Q7. Section 3 contains abrupt logical transitions. The overview in Section 3.1 does not clearly explain the relationships among the following four subsections and only mentions forecast correction. The transition from Section 3.4 to Section 3.5 is too abrupt and confusing. The purpose and role of source normalization in Section 3.5 are unclear. There are also some inconsistencies between Sections 3.3 and 3.5. Q8. The paper does not specify numerical values for many hyperparameters, including the smoothing constant ε\varepsilon in Equation 9, the ratio clipping upper bound Rmax⁡R_{\max}, and the uniform smoothing coefficient δ\delta. Even for some parameters whose settings are reported, such as BB, KK, and τ\tau, the paper does not explain how their values are selected. How are these parameters set and selected, and what is the rationale? Q9. There are issues with the experiments. First, the datasets cover an overly narrow range of domains, with almost all coming from the energy domain. Furthermore, although many forecasting models are evaluated, few competing baseline methods are included, and the main text does not present any corresponding comparison results.

Reviewer_MFFROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Dynamic Memory-enhanced Residual Correction (DMRC), a post-hoc adaptation framework for frozen time-series forecasters under non-stationarity without deployment-time weight updates or online ground-truth target labels. Offline, a ridge-regression "residual-aware projector" predicts base-model residuals r^reg∈RH\widehat{\mathbf{r}}^{\mathrm{reg}} \in \mathbb{R}^H from hand-crafted features (history xt\mathbf{x}_t, base forecast y^0,t\widehat{\mathbf{y}}_{0,t}, and local statistics) and extracts a shared low-dimensional orthonormal temporal subspace U\mathbf{U}. Normalized subspace coordinates q=ν−1U⊤r^reg\mathbf{q} = \nu^{-1}\mathbf{U}^\top \widehat{\mathbf{r}}^{\mathrm{reg}} organize per-channel training residuals into coarse groups (via the first two coordinates) and B=192B = 192 fine-grained prototypes, where each prototype stores the mean ground-truth residual decomposed into a subspace component vb∥=UU⊤vb\mathbf{v}_b^\parallel = \mathbf{U}\mathbf{U}^\top \mathbf{v}_b and an orthogonal remainder vb⊥=vb−vb∥\mathbf{v}_b^\perp = \mathbf{v}_b - \mathbf{v}_b^\parallel. Online, the forecaster, projector, memory bank M0\mathcal{M}_0, and validation-tuned channel-wise scalar gates g∥,c,g⊥,c∈[0,0.8]g_{\parallel,c}, g_{\perp,c} \in [0, 0.8] remain fixed; only per-channel retrieval distributions πt,c∈ΔB\boldsymbol{\pi}_{t,c} \in \Delta^B (biasing within-group top-KK selection of vb∥\mathbf{v}_b^\parallel, while vb⊥\mathbf{v}_b^\perp uses static content matching) are updated via KL-regularized online mirror descent (exponentiated gradient). Credit is computed via top-KK leave-one-out differences on a label-free stability surrogate—combining cross-view perturbation dispersion for point forecasters (or Q0.9−Q0.1Q_{0.9} - Q_{0.1} interval width for probabilistic forecasters) with temporal-overlap consistency—reweighted by source-normalized access ratios. Evaluated across 77 long-horizon benchmarks, 44 horizons (H∈96,192,336,720H \in \\{96, 192, 336, 720\\} at L=512L = 512), and 66 frozen backbones (DLinear, PatchTST, MICN, FreTS, iTransformer, Chronos-Bolt-Base), DMRC lowers MSE relative to the un-augmented Frozen checkpoint in 146/168146/168 cells (1212 ties, 1010 degradations; up to −15.10-15.10\\%), with external retrieval comparisons (KUP-BI, TQNet) in Appendix D.2 and DLinear ablations in Section 4.3.

Critical strengths
  1. Practical black-box deployment framing with safe gated fallback. Requiring only forward-pass access while freezing the forecaster, projector, and prototype values addresses a realistic deployment constraint for pretrained and API-served time-series models. Decomposing prototype residuals into a dynamically prioritized principal subspace (vb∥\mathbf{v}_b^\parallel) and a purely content-matched orthogonal remainder (vb⊥\mathbf{v}_b^\perp), safeguarded by validation-calibrated scalar gates g∥,c,g⊥,c∈[0,0.8]g_{\parallel,c}, g_{\perp,c} \in [0, 0.8], provides a clean fallback mechanism that demonstrably reverts to y^0,t\widehat{\mathbf{y}}_{0,t} when validation support is absent (e.g., DLinear and iTransformer on ETTm2), while the causal predict-then-update protocol (App. A.1, C.1) is clearly structured and avoids future-target leakage at step tt.

  2. Plausible utility of offline residual-coordinate indexing across diverse model families. Evaluating modular residual correction across six distinct architectures (linear, patch-based, convolutional, frequency-MLP, inverted Transformer, and a zero-shot probabilistic foundation model) shows broad plug-in compatibility. Within the DLinear ablation (Table 4), replacing the residual-aware query representation produces the largest error increase (0.3051→0.31730.3051 \to 0.3173 average MSE), and static gated residual memory alone yields clear improvements on tasks such as DLinear–Weather (0.2421→0.22630.2421 \to 0.2263 MSE), supporting the core intuition—stated with commendably restrained theoretical scope in Appendix B.4—that indexing historical experience by predictable error structure can be more informative than raw input similarity.

Critical weaknesses
  1. W1. The online label-free dynamic update—the paper's central novelty—contains degenerate components, lacks empirical validation of its surrogate mechanism, and omits core mathematical specifications needed for reproducibility.

    • Inert view-consistency feedback across both probabilistic and point forecasters: For Chronos-Bolt-Base (δF=1\delta_F = 1), Eqs. (6) and (15) define LV,t=1H∑h=1H(Qt,h,0.9−Qt,h,0.1)\mathcal{L}_{V,t} = \frac{1}{H}\sum_{h=1}^H (Q_{t,h,0.9} - Q_{t,h,0.1}) without input perturbations, while Appendix C.2 (lines 965–968) states that Q0.5Q_{0.5} is corrected for point evaluation and outer quantiles are used only to form LV,t\mathcal{L}_{V,t}. Because prototypes store single point-residual vectors vc,b∈RH\mathbf{v}_{c,b} \in \mathbb{R}^H, leave-one-out removal of prototype ii in Eq. (10)/(21) either leaves Q0.9,Q0.1Q_{0.9}, Q_{0.1} unchanged or shifts both by an identical additive offset that cancels out—making LV,t(−i)−LV,t≡0\mathcal{L}_{V,t}^{(-i)} - \mathcal{L}_{V,t} \equiv 0 identically for all ii and leaving only the tiny λo=10−4\lambda_o = 10^{-4} overlap term active, yet Chronos exhibits some of the largest claimed gains (≈−9.5\approx -9.5\\% on ETTm1, ≈−14.5\approx -14.5\\% on Weather) with no static-vs.-dynamic ablation. Equally concerning, for point forecasters (δF=0\delta_F = 0), Figure 4(c) shows that setting V=1V = 1—where LV,t=11⋅H∣yt(1)−yˉt∣22≡0\mathcal{L}_{V,t} = \frac{1}{1 \cdot H}\\|\mathbf{y}_t^{(1)} - \bar{\mathbf{y}}_t\\|_2^2 \equiv 0 identically—yields virtually indistinguishable MSE and MAE from V=3V = 3, even though V=3V = 3 requires multiple extra forward passes of the frozen backbone per window (and no runtime/latency overhead is reported anywhere, despite K×V×CK \times V \times C leave-one-out evaluations on datasets such as Electricity with C=321C = 321).
    • Unvalidated surrogate credit, confounded DLinear-only ablations, and vacuous step-size bounds: Appendix D.4 only checks whether window-level Lstab,t\mathcal{L}_{\mathrm{stab},t} stratifies top-1010\\% MSE windows on a single setting (AUROC=0.709\mathrm{AUROC} = 0.709 on iTransformer–ETTh1-96); it never tests whether per-prototype leave-one-out stability credit Gt,c,iG_{t,c,i} actually correlates with true leave-one-out MSE reduction MSEt,c(−i)−MSEt,c\mathrm{MSE}_{t,c}^{(-i)} - \mathrm{MSE}_{t,c} (or why penalizing dispersion/overlap discrepancy would not simply suppress sharp corrections during genuine regime shifts). In Table 3 (DLinear only), Absolute update (0.3134440.313444) and Source-normalized update (0.3133780.313378) both omit stability credit and both slightly underperform Static memory (0.3133390.313339), while DMRC w/o source normalization (r~t,c(i)≡1\tilde{r}_{t,c}(i) \equiv 1 with Gt,c,iG_{t,c,i} active) is never tested; moreover, dynamic gains over Static memory (0.30510.3051 vs. 0.31330.3133) arise almost entirely on Exchange (0.3881→0.35050.3881 \to 0.3505, vs. Frozen 0.38720.3872) and ETTh2 (0.2991→0.28760.2991 \to 0.2876, vs. Frozen 0.29930.2993), while w/o Coarse-to-fine (0.31320.3132) and w/o Memory Evolution (0.31330.3133) produce nearly identical numbers on every dataset. Meanwhile, Appendix B's textbook Hedge/mirror-descent derivation requires η=2log⁡B/(TGmax⁡2)≲10−1\eta = \sqrt{2\log B / (T G_{\max}^2)} \lesssim 10^{-1} for non-trivial regret, whereas experiments fix η=1000\eta = 1000 with Gmax⁡=1G_{\max} = 1, rendering the single-step log-weight bound in Eq. (33) (2ηGmax⁡=20002\eta G_{\max} = 2000) vacuous.
    • Missing core specifications ("phantom" Appendix C.3): Section 3.2 (line 185) states "Feature construction, basis estimation, and normalization details are provided in Appendix C.3", but Appendix C.3 contains only three qualitative prose paragraphs and zero equations. Left undefined across the entire paper are: the explicit ridge-regression features and penalty; subspace rank dd of U\mathbf{U} (note also that a linear PCA-style subspace does not substantiate the title's "Residual-Manifold" framing); coarse-group count, clustering algorithm, and handling of within-group K=32K = 32 or empty-set weight renormalization at K=1K = 1 (App. A.4); exact view perturbations Tv\mathcal{T}_v and how non-invertible "masking" (line 660) admits Tv−1\mathcal{T}_v^{-1}; main-text definition of ℓc,b\ell_{c,b} in Eq. (4), the role of stop-gradient sg(...) in a gradient-free finite difference (Eqs. 6–7), and values of ρ,Rmax⁡,δ,ε\rho, R_{\max}, \delta, \varepsilon.
  2. W2. The empirical evaluation relies on underperforming frozen checkpoints, omits essential baselines (while mischaracterizing TQNet), and suffers from validation-split ambiguity.

    • Substantially underperforming base checkpoints and sub-persistence Exchange gains: At L=512L = 512, the Frozen checkpoints leave abnormally large errors relative to standard published benchmarks—far beyond minor drop_last differences—which inflates systematic, easily correctable bias: Frozen PatchTST reaches MSE 0.4880.488 (ETTh1-96), 0.7740.774 (ETTh1-720), and 0.4050.405 (ETTm1-96) versus ≈0.370,0.447,\approx 0.370, 0.447, and 0.2900.290 in Nie et al. (2023); Frozen DLinear on ETTh1-96 is 0.4330.433 (vs. ≈0.375\approx 0.375 in Zeng et al., 2023); and iTransformer on ETTh1-720 is 0.7950.795, with backbone training details (epochs, learning rate, RevIN, early stopping) omitted from Appendix C.2. Similarly, on Exchange, Zeng et al.'s naive last-value Repeat baseline achieves ≈0.081/0.167/0.305/0.823\approx 0.081 / 0.167 / 0.305 / 0.823 across H∈96,192,336,720H \in \\{96, 192, 336, 720\\}; Frozen DLinear (0.0859/0.1809/0.3406/0.94140.0859 / 0.1809 / 0.3406 / 0.9414) is worse than persistence, and DMRC-DLinear (0.0825/0.1536/0.3060/0.86000.0825 / 0.1536 / 0.3060 / 0.8600) largely pulls a sub-persistence linear extrapolation back toward persistence, making the headline −15.10-15.10\\% reduction weak evidence of dynamic adaptation.
    • Omitted baselines, mischaracterization of TQNet, and mixed competitiveness on stronger backbones: Four natural baselines are absent: (a) directly applying gated ridge predictions y^0,t+g,r^treg\widehat{\mathbf{y}}_{0,t} + g\\,\widehat{\mathbf{r}}_t^{\mathrm{reg}} with no memory bank; (b) causal delayed-label online residual adaptation using realized targets yt−H\mathbf{y}_{t-H} (and partially observed prefixes xt−s+1:tx_{t-s+1:t}), which are available at time tt in streaming forecasting without future leakage or backbone gradients—meaning "strictly label-free" is an unbenchmarked design restriction rather than an operational necessity; (c) time-series test-time/output-space adapters cited in Section 1 (TAFAS, Kim et al., 2025; PETSA, Medeiros et al., 2025; COSA, Im & Kwon, 2026; Proceed, Zhao & Shen, 2025); and (d) RAFT (Han et al., 2025, cited in Sec. 2.2). Meanwhile, Appendix D.1 (lines 1061–1068) describes TQNet (Lin et al., 2025, arXiv:2505.12917) as retrieving "historical segments similar to the current input and us[ing] their associated future segments" from a database, whereas Lin et al.'s TQNet is a standalone parametric network with learnable periodic queries and no retrieval database. Even among the four Appendix D.2 methods (Frozen, KUP-BI, TQNet, DMRC—omitted from main-text Table 1), DMRC achieves lowest/tied-lowest error in only 91/16891/168 (54.254.2\\%) MSE cells and 75/16875/168 (44.644.6\\%) MAE cells, losing on overall average MSE and MAE to TQNet on PatchTST (0.38890.3889 vs. 0.38730.3873 MSE; best in 10/2810/28 MSE, 5/285/28 MAE) and to both KUP-BI (0.37380.3738) and TQNet (0.37680.3768) on iTransformer (0.37770.3777 MSE; best in 5/285/28 MSE, 3/283/28 MAE, with 88 ties and 66 degradations vs. Frozen, including all four Exchange horizons). Against Frozen itself, DMRC degrades MAE in 30/16830/168 cells, and many MSE wins are −0.01-0.01\\% to −0.10-0.10\\% without seed standard deviations across the 33 memory seeds.
    • Inconsistent validation-split usage and table discrepancy: Section 4.2 (line 322) and Appendix D.1 (line 1066) state that memory items come from training and validation intervals, whereas Appendix C.3 (line 1000) says memory is built on training and gates are tuned on validation. If validation residuals are inside M0\mathcal{M}_0, DMRC gains both optimistic in-sample gate selection on validation and privileged recent pre-test ground-truth supervision unseen by Frozen (and 8080\\% target supervision against zero-shot Chronos-Bolt-Base). Finally, the ETTm2 row in Tables 2–4 (0.2040200.204020 MSE / 0.2953620.295362 MAE) does not match Table 7 (0.2044020.204402 / 0.2965470.296547), despite claiming exact fallback to Frozen.
Questions for authors

Questions Whose Answers Could Materially Affect My Assessment:

  1. Backbone checkpoint quality and Exchange persistence: How were the five supervised backbones trained (epochs, optimizer/learning rate, batch size, drop_last, RevIN, early stopping), and why are Frozen errors on ETTh1/ETTm1 (PatchTST, DLinear, iTransformer) substantially worse than published benchmarks at comparable look-backs? Does DMRC still provide statistically significant gains (reporting mean ±\pm std across seeds) on properly tuned checkpoints, and how does it compare against the naive last-value Repeat baseline on Exchange?
  2. Missing baselines (ridge-only, delayed-label, TTA, and RAFT) & TQNet instantiation: Please report comparisons against: (a) direct gated ridge correction y^0,t+g,r^treg\widehat{\mathbf{y}}_{0,t} + g\\,\widehat{\mathbf{r}}_t^{\mathrm{reg}} without memory retrieval; (b) a simple causal delayed-label online residual baseline using realized yt−H\mathbf{y}_{t-H} (e.g., EMA residual bias or online ridge update); and (c) at least one cited output-space TTA adapter (COSA / PETSA / TAFAS) plus RAFT. In addition, please clarify how TQNet (Lin et al., 2025)—which is a parametric network with learnable periodic queries rather than a historical-segment retrieval method—and KUP-BI were instantiated as wrappers around frozen backbones.
  3. Degenerate LV,t\mathcal{L}_{V,t} behavior (Chronos and V=1V = 1) & multi-backbone ablations: (a) For Chronos-Bolt-Base, does removing prototype ii change LV,t\mathcal{L}_{V,t} at all, or are online updates driven solely by λoLT,t\lambda_o \mathcal{L}_{T,t} (λo=10−4\lambda_o = 10^{-4})? (b) Given that V=1V = 1 (LV,t≡0\mathcal{L}_{V,t} \equiv 0) matches V=3V = 3 in Figure 4(c), what does multi-view perturbation add beyond latency? Please report Static memory vs. DMRC (and DMRC w/o source normalization, keeping Gt,c,iG_{t,c,i} with r~t,c(i)≡1\tilde{r}_{t,c}(i) \equiv 1) on Chronos-Bolt-Base and at least one non-linear Transformer (PatchTST or iTransformer).
  4. Direct test of stability credit & πt,c\boldsymbol{\pi}_{t,c} dynamics: Can you compute post hoc the correlation between per-prototype leave-one-out stability credit Gt,c,iG_{t,c,i} and true leave-one-out MSE improvement MSEt,c(−i)−MSEt,c\mathrm{MSE}_{t,c}^{(-i)} - \mathrm{MSE}_{t,c}, and plot πt,c\boldsymbol{\pi}_{t,c} entropy/concentration over time to explain why dynamic updates help on Exchange/ETTh2 (DLinear) but are flat on ETTm1/Electricity?
  5. Validation-split targets in M0\mathcal{M}_0: Please reconcile lines 322/1066 ("training and validation intervals") with line 1000 ("training interval"). Were validation ground-truth residuals stored in M0\mathcal{M}_0 while tuning g∥,c,g⊥,cg_{\parallel,c}, g_{\perp,c} on validation, and what happens when M0\mathcal{M}_0 uses strictly train residuals (or when Frozen also sees val)?

Clarification Requests: 6. Omitted Appendix C.3 details, runtime, and ETTm2 mismatch: Please provide exact definitions for the ridge features/penalty, subspace rank dd, coarse-group count and clustering method (plus K=1K = 1 renormalization and K=32K = 32 group-size handling), ρ\rho, ℓc,b\ell_{c,b}, Rmax⁡R_{\max}, δ\delta, ε\varepsilon, view transforms Tv\mathcal{T}_v / Tv−1\mathcal{T}_v^{-1}, sg(...) in Eq. (6), the exact "oracle gain" formula in Fig. 2(a), per-window wall-clock/memory overhead (especially on Electricity, C=321C = 321), and the discrepancy in ETTm2 between Tables 2–4 (0.2040200.204020) and Table 7 (0.2044020.204402).

Minor Suggestions (Not Affecting Score):

  • Replace "residual manifold" with "residual subspace" to reflect the linear projection U\mathbf{U}, reconcile "consistently outperforms" (OpenReview metadata) with "mostly outperforms" (PDF abstract), and fix minor bibliography errors (Liu Chong →\to Chong Liu for KUP-BI; 2025 →\to 2012 for Shalev-Shwartz).
7 comments
AuthorsOpenReview ↗

Response to Reviewer MFFR (Part 1/6)

We apologize for the delayed response, as we have been occupied with several time-sensitive commitments over the past few days.

We thank the reviewer for the careful examination of the method, experimental protocol, and reproducibility details. We first address the label-free online feedback mechanism and its empirical contribution in response to W1, Q3, and Q4, and then provide the method definitions and reproducibility details requested in W1 and Q6. We subsequently respond to W2, Q1, Q2, and Q5 regarding the frozen checkpoints, the persistence baseline on Exchange, the requested baselines, and the validation protocol. To avoid repetition, concerns in W1/W2 that overlap with the subsequent questions are addressed together. In the revised manuscript, we have corrected equations and descriptions that did not accurately reflect the implementation and narrowed claims that extended beyond the available evidence. To answer the reviewer directly without substantially expanding the manuscript, we additionally report in this response the requested Repeat and gated ridge-only comparisons, together with a controlled static/dynamic memory comparison for Chronos.

W1 and Q3: Label-Free Online Feedback and the Contributions of Static and Dynamic Memory

The reviewer correctly notes that, if one directly uses the raw interval width stated in the original manuscript,

1HC∑h,c(Qt,h,c,0.9−Qt,h,c,0.1),\frac{1}{HC}\sum_{h,c}\left(Q_{t,h,c,0.9}-Q_{t,h,c,0.1}\right),

and applies exactly the same additive correction to the lower and upper quantiles, this quantity is invariant to the common shift and the leave-one-out difference cancels. We agree with this mathematical observation, but respectfully disagree with the further inference that the online feedback used by the Chronos implementation must therefore be identically zero. The original manuscript conflated the raw interval width used for diagnosis with the corrected quantile-view dispersion used for online updating.

In the implementation, Q0.1Q_{0.1}, Q0.5Q_{0.5}, and Q0.9Q_{0.9} are treated as three probabilistic forecast views. Each view forms a separate query and passes through the Adapter, Retrieval, and Gating modules to produce a corrected quantile forecast Q~t,h,c,p\widetilde Q_{t,h,c,p}. The online update uses

Qˉt,h,c=13∑p∈{0.1,0.5,0.9}Q~t,h,c,p,\bar Q_{t,h,c}=\frac{1}{3}\sum_{p\in\lbrace0.1,0.5,0.9\rbrace}\widetilde Q_{t,h,c,p},

\left(\widetilde Q_{t,h,c,p}-\bar Q_{t,h,c}\right)^2.$$ Because the three quantile views can have different queries, retrieval weights, and candidate-prototype participation patterns, removing the same prototype need not induce an identical shift in all three views. Thus, $\mathcal L_{V,t}^{(-i)}-\mathcal L_{V,t}$ is not required to be zero. We directly measured this quantity on Chronos–ETTm1 at $H=336$. Across the three random seeds, the corrected quantile-view credit was nonzero in 24.74%, 39.57%, and 21.32% of the candidate leave-one-out evaluations, with corresponding mean absolute values of $4.796\times10^{-4}$, $1.361\times10^{-3}$, and $1.561\times10^{-3}$. These results confirm that the reviewer’s cancellation argument applies to the raw interval width, but not to the credit used in the implementation. The revised manuscript now identifies the raw interval width as a diagnostic quantity and corrects the relevant equation to show the actual update term above. To determine whether the nonzero credit also affects the predictions, we conducted a strictly controlled comparison among Static, Overlap-only, and Full DMRC on Chronos–ETTm1 at $H=336$, which we report here in direct response to the reviewer. Within each seed, the three settings share the same frozen checkpoint, offline memory, gates, retrieval parameters, and test stream. The online feedback term is the only difference. | Seed | Static MSE / MAE | Overlap-only MSE / MAE | Full DMRC MSE / MAE | MSE improvement of Full over Static | | :---: | :---: | :---: | :---: | :---: | | 2021 | 0.497988 / 0.444660 | 0.497988 / 0.444660 | **0.497890 / 0.444568** | +0.0197% | | 2022 | 0.497955 / 0.444888 | 0.497955 / 0.444888 | **0.497613 / 0.444620** | +0.0686% | | 2023 | 0.496658 / 0.444055 | 0.496658 / 0.444055 | **0.496606 / 0.444037** | +0.0103% | | Mean ± sample standard deviation | 0.497533 / 0.444534 | 0.497533 / 0.444534 | **0.497370 / 0.444408** | +0.0329% |
AuthorsOpenReview ↗

Response to Reviewer MFFR (Part 2/6)

W1 and Q3 (continued): Controlled Memory Comparison and Point-Forecaster Views

Under the same setting, the Frozen MSE is 0.510745. Static already provides the main improvement of approximately 2.59%, while Full increases the total improvement to approximately 2.62%. Overlap-only is numerically identical to Static, whereas Full further reduces MSE for all three seeds. This controlled result shows that the corrected quantile-view term produces a nonzero update and yields a small but seed-consistent additional benefit in this setting. We do not overgeneralize this result: the primary Chronos gain comes from static hierarchical residual memory, whereas online residual-selection adaptation provides a smaller improvement that depends on the dataset and forecasting setting.

For point forecasters, the cross-view dispersion term is zero when V=1V=1, but the temporal-overlap term can remain active. Thus, V=1V=1 corresponds to Overlap-only rather than to the complete multi-view update. The similar aggregate results for V=1V=1 and V=3V=3 in Fig. 4(c) indicate that average accuracy is not highly sensitive to VV; by themselves, they show neither that multi-view feedback is universally beneficial nor that it is universally ineffective. We have therefore revised the figure caption and the accompanying text and no longer present this curve as evidence that multiple views are necessary.

In the existing DLinear Static–Full comparison, the average MSE decreases from 0.313394 to 0.305180, although the gains are concentrated on Exchange and ETTh2. The Chronos comparison reported above shows a smaller dynamic contribution. The available evidence therefore supports the conclusion that online updating is effective for some backbones and datasets, but not the stronger claim that all nonlinear backbones benefit consistently. We have narrowed the corresponding claims and no longer interpret the six-backbone Frozen–DMRC table as a six-backbone dynamic ablation.

W1 and Q4: Stability Credit, Source Normalization, and the Step-Size Analysis

The reviewer suggests a post hoc correlation analysis between Gt,c,iG_{t,c,i} and the true leave-one-out MSE improvement. Such an analysis would be a useful additional diagnostic, but we respectfully disagree that a monotonic prototype-level correlation is a necessary condition for the surrogate mechanism to be meaningful. The purpose of GG is to provide local, relative stability feedback when online labels are unavailable, not to estimate the true MSE. The true leave-one-out MSE can only be audited offline after the future targets have become available. Our direct evidence for the effect of the update is the Static–Overlap-only–Full comparison with the remaining components fixed, rather than entropy reduction or a single correlation coefficient. We now also state explicitly in the Limitations that stability and accuracy may diverge during genuine regime shifts, and that gating and static fallback can reduce, but not eliminate, this risk.

The reviewer also raises an important point about the attribution in Table 3. Absolute update and Source-normalized update both omit GG; their difference therefore compares two access-frequency update rules and cannot isolate the role of source normalization in the complete DMRC update. We have removed this overly strong attribution. Until a separate comparison that retains GG while setting only r~t,c(i)=1\widetilde r_{t,c}(i)=1 is available, we also refrain from claiming that source normalization has been independently validated.

Regarding η=1000\eta=1000, the reviewer is correct that this empirical step size cannot be substituted into the classical small-step regret bound to obtain a non-vacuous performance guarantee. This observation, however, does not imply that the empirical update is necessarily ineffective. The mirror-descent derivation in the appendix describes the update form under bounded surrogate utility and a fixed comparator; it does not guarantee a monotonic reduction in the true MSE. We have restricted the interpretation of Eq. (33) accordingly: it describes the probability-simplex update form rather than providing an empirical performance guarantee for η=1000\eta=1000. Empirical effectiveness is assessed by the controlled comparisons reported above.

AuthorsOpenReview ↗

Response to Reviewer MFFR (Part 3/6)

W1 and Q6: Method Definitions and Reproducibility Details

We appreciate the reviewer’s request for additional implementation details. These quantities are explicitly defined in the implementation rather than being unspecified executable components. To remove ambiguity and answer the reviewer directly, we provide the exact settings below:

  • Features and ridge regression: For each channel, the state vector contains a 16-bin adaptive average pooling of the 512-step history after removing the local mean, an 8-bin pooling of the base forecast after removing the same mean, the local mean, and an intercept, for a total of 26 dimensions. An eight-dimensional summary of overlapping-forecast disagreement is an optional feature, with its use selected on the validation set. Features are standardized using training-set statistics. Ridge regression is fitted only to training residuals, with λ=0.1\lambda=0.1.
  • Subspace and clustering: The subspace rank is d=4d=4, and U\mathbf U contains the four leading eigenvectors of the second-moment matrix of the ridge-predictable residuals. The memory capacity is B=192B=192. We first perform eight-group K-means clustering in the first two normalized query coordinates, and then perform fine-grained K-means clustering within each group using the full query coordinates, with n_init=10.
  • Top-KK boundary cases: If a group contains fewer than KK candidates, all real candidates are used and padded positions receive zero weight. If K=1K=1 and the sole candidate is removed in the leave-one-out branch, the counterfactual readout is defined as zero correction; no softmax is applied to an empty set.
  • Priority term: ℓc,b=log⁡(πt,c(b)+ϵπ)\ell_{c,b}=\log(\pi_{t,c}(b)+\epsilon_\pi) and ρ=0.1\rho=0.1. The numerical values are ϵ=10−6\epsilon=10^{-6}, Rmax⁡=10R_{\max}=10, δ=0.01\delta=0.01, Gmax⁡=1G_{\max}=1, and ϵπ=10−8\epsilon_\pi=10^{-8}.
  • Views for point forecasters: The reported experiments use the original input and two Gaussian-jitter views, with the noise standard deviation set to 0.03 times the historical standard deviation of each channel, giving V=3V=3. These transformations do not alter the temporal coordinates, so Tv−1\mathcal T_v^{-1} is the identity alignment. Masking was mentioned only as a candidate design and was not used for the reported results; this statement has been removed from the revised manuscript.
  • Stop-gradient: sg(·) indicates only that the mean view and historical overlapping forecasts are treated as reference quantities without backpropagation. The leave-one-out credit is a finite difference of function values and does not obtain gradients through GG.
  • Oracle gain: For the test residual rt=yt−y^0,tr_t=y_t-\hat y_{0,t}, the diagnostic oracle selects

vt∗=arg⁡min⁡v∈M0∥rt−v∥22,v_t^*=\arg\min_{v\in\mathcal M_0}\lVert r_t-v\rVert_2^2,

and reports the oracle gain (%) as

{\sum_t\lVert r_t\rVert_2^2}\right).$$ This diagnostic uses test targets and represents only the correctable upper bound provided by the historical residual set. It is not a deployable method and does not contribute to any primary result. “Residual-Manifold” was intended as a descriptive name for the residual structure, not as a claim that the method learns a nonlinear manifold. To avoid an unintended geometric implication, we have replaced it with the more precise terms “predictable residual subspace” and “residual-subspace memory.”
Reviewer_XqemOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DMRC, a retrieval-augmented framework for enhancing frozen time-series forecasting without instantaneous labels during inference. The motivation is that similarity in the X space does not necessarily imply similarity in forecasting residuals.

DMRC uses an offline residual-aware memory where they use ridge-based residual projection to create the low-dimensional representations. Residuals are then organized into coarse-to-fine prototypes, which are retrieved to correct the predictions. During online inference, retrieval priorities are adjuted dynamically through a label-free objective based on multi-view prediction consistency and temporal overlapping.

The proposed method is evaluated on multiple time-series datasets and achieved significant improvement compared with the baseline.

Critical strengths

1- I like the motivation, which is reasonable to me. Adapting frozen forecasting models under distribution shifts is a practical problem. 2- Residual-aware retrieval is something interesting. Similarities in the historical inputs may not necessarily imply similarities in the forecast space. 3- The combination of residual prototypes, coarse-to-fine retrieval, and the online priority update forms a solid design framework. 4- Empirical evaluation indicates an overall improvement, and the component ablation provides good evidence that the proposed design contributes to the performance.

Critical weaknesses

1- Multi-view consistency and temporal overlap consistency are used for memory evolution. It is unclear why prediction consistency necessarily implies prediction accuracy. How to avoid the scenario where multiple forecasts are highly consistent while sharing the same prediction bias structure?

2- I like the idea of residual-manifold motivation, but more discussion would benefit from empirical validation. The author argues that the residual-aware representations provide a more meaningful representation for retrieving historical errors.

  1. The residual-manifold motivation would benefit from more direct empirical validation. More direct comparison could be interesting - comparing residual distances of the nearest neighbors through learned representations versus those selected using the original X space. Certain examples to highlight different representations and more direct empirical comparison between these two approaches can better showcase the significance of the proposed approach
Questions for authors

Since the residual prototypes remain frozen during inference, how does DMRC behave when the test distribution exhibits residual patterns that are not adequately represented in the offline memory?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.