acceptodds
Under review as a conference paper at ICLR 2027

MORA: Correcting Time Series Forecasts with Coarse Future Signals

Abstract

Recent architectural advances have improved time-series forecasting. However, our analysis of forecasters with diverse architectures reveals a common pattern: a substantial share of residual energy is concentrated in a small number of low-frequency components. Simple linear probes further show that the forecasters' historical inputs still contain predictive information about errors in the forecast mean. Building on these findings, we propose Model-Output Residual Adaptation (MORA), a model-agnostic residual adapter that refines the outputs of frozen forecasters without requiring additional inputs. MORA estimates segment-wise means of the target sequence through structural extrapolation and learned estimation, rather than directly predicting backbone-specific residuals. It then converts discrepancies between these estimates and the corresponding statistics of the backbone forecast into stagewise corrections, scaling each update according to estimation reliability. Extensive experiments on twelve benchmarks with six forecasting backbones show average MSE reductions of for long-term forecasting and for short-term forecasting, with lower MSE in of the 288 evaluated cases. The benefits also extend to three time-series foundation models evaluated at different context lengths. Together, these results demonstrate that predictable coarse-scale residuals can be effectively exploited to improve existing forecasts. Code repository: https://anonymous.4open.science/r/MORA-B0B0/.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Forecast Correction, Residual Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9ZupOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The residual energy of forecasting models with different architectures tends to concentrate in a small number of low-frequency components, while historical inputs still contain signals that can predict future mean biases. Motivated by this observation, the paper proposes MORA, a model output residual adaptation method that exploits underutilized information in historical data to refine the predictions of existing time series forecasting models. Experiments demonstrate improvements across multiple datasets and models.

Critical strengths
  1. The motivation is novel, and the design of MORA is supported by extensive experiments and statistical analyses. These analyses reveal systematic residual patterns in existing forecasting models and motivate a mechanism for predicting these residuals to correct the forecasts.
  2. MORA is a plug-in module that can be integrated with different forecasting models to improve their performance.
Critical weaknesses
  1. Some experimental comparisons raise concerns. In Table 2, the results for MORA are obtained from local experiments, whereas the baseline results are taken from the cited literature.
  2. Compared with single-stage forecasting, the more complex two-stage prediction and correction pipeline introduces additional computational overhead.
Questions for authors
  1. Given that Figure 1 shows that the residuals are dominated by low-frequency components, why not directly predict and correct the spectral components instead of using segment-wise mean correction?
  2. How would jointly training the predictor and corrector end to end compare with training them separately?
  3. Why does a systematic offset occur in the forecasts? Is this phenomenon prevalent across most datasets and models? This observation seems rather surprising.
7 comments
AuthorsOpenReview ↗

Response to Reviewer 9Zup (Part 1/4): Baseline Protocol in Table 2 (W1)

Thank you for your constructive review and for recognizing MORA's novel motivation, extensive statistical analyses of residual patterns, and plug-in utility across diverse forecasting backbones. Below and in Parts 2–4, we address your concerns and questions in detail, starting with Weakness 1 (W1) in Part 1. We hope these clarifications and additional results resolve your concerns.

1. Why does Table 2 include literature-reported baseline scores alongside local runs, and does MORA hold under strictly paired comparisons? (W1)

1. Across the 96 underlying baseline settings summarized in Table 2 (detailed in Table 15), 90 settings (93.8%) are in fact our own local reproductions; only 6 entries (†\dagger) use author-reported scores because our local reproductions on those 6 settings had noticeably higher errors.

  • Because MORA refines a frozen checkpoint fθf_\theta via Y^=fθ(X)+Δϕ(X,Y^(0))\widehat{\mathbf{Y}} = f_\theta(\mathbf{X}) + \Delta_\phi(\mathbf{X}, \widehat{\mathbf{Y}}^{(0)}) (Eq. 1–2), paired local evaluation is our default protocol throughout the paper. Among the 96 dataset–backbone–horizon settings in Table 15 (App. F), 52 settings were not reported in the original baseline papers (the PEMS transfer benchmarks for PatchTST, DLinear, and VPNet, plus TimeFilter at H=96H=96; App. D.2), and for 38 of the 44 settings with author-reported PEMS reference scores, our local reproductions using the official repositories closely match the author-reported results (covering all 12/12 referenced TimeFilter settings, 11/16 iTransformer settings, and 15/16 CGTFra settings).
  • However, for the remaining 6 settings (marked with †\dagger in Table 15), our local reproductions using the official repositories still yielded noticeably higher errors than the author-reported numbers (by 5.8%–32.0% in MSE) after reproduction and configuration checks.
  • Crucially, MORA was still applied to our local checkpoints fθf_\theta in all 6 cases. To avoid benchmarking against weaker local reproductions and retain a demanding comparison against author-reported performance, we used the stronger (lower-error) reference scores (†\dagger) for these 6 entries in Tables 2 and 15. This choice lowered rather than inflated MORA's displayed improvements:
DatasetBackboneHHAuthor-Reported (†\dagger)Local Baseline (fθf_\theta)Local Baseline fθf_\theta + MORA
PEMS03iTransformer480.125 / 0.2360.135 / 0.2470.126 / 0.230
PEMS03CGTFra480.119 / 0.2280.126 / 0.2340.110 / 0.217
PEMS04iTransformer480.120 / 0.2330.131 / 0.2450.096 / 0.200
PEMS04iTransformer960.150 / 0.2620.165 / 0.2760.109 / 0.213
PEMS08iTransformer480.186 / 0.2350.205 / 0.2930.134 / 0.227
PEMS08iTransformer960.221 / 0.2670.292 / 0.3240.221 / 0.254

2. Under strictly paired local comparisons across all 96 settings, all 6 settings improve and MORA achieves even stronger overall gains.

  • In the table above, against the corresponding local checkpoints fθf_\theta, MORA improves both MSE and MAE in all 6 of 6 cases. Even against the stronger author-reported references (†\dagger), MORA achieves lower MAE in all 6 of 6 cases and lower MSE in 4 of 6 cases (with one additional case matching at three-decimal precision at 0.221).
  • Across the entire 96-setting PEMS matrix (App. D.5), using 100% strictly paired local comparisons increases the MSE win count from 90/96 to 92/96 and yields larger mean error reductions:
Evaluation Protocol (All 96 PEMS Settings in Table 2)Avg. MSE ReductionAvg. MAE ReductionWin Count (MSE / MAE)
Table 2 Protocol (90 local + 6 stronger reported †\dagger)28.10%18.47%90/96 / 94/96
Strictly Paired Local Protocol (all 96 local checkpoints)28.88%19.09%92/96 / 94/96
  • Even if those 6 settings are excluded altogether, MORA still improves MSE in 86 of 90 (95.6%) settings and MAE in 88 of 90 (97.8%) settings. We will also include the 6-setting side-by-side comparison table above and clarify this selection rationale directly in Appendix D.2 of the final manuscript.
AuthorsOpenReview ↗

Response to Reviewer 9Zup (Part 2/4): Computational Overhead of the Two-Stage Pipeline (W2)

2. How much computational, memory, and latency overhead does MORA introduce? (W2)

1. At inference time, MORA requires only one forward pass of the frozen backbone, no additional deep forecasting network, and no test-time parameter updates—adding only +0.33 to +1.11 ms per single query (B=1B=1) on task-specific backbones and <0.56% latency on foundation models across all 8 datasets.

  • Rather than running a second deep network at test time, MORA executes only compact deterministic history feature extraction (ϕc\boldsymbol{\phi}_c), closed-form linear ridge projection, two low-rank matrix–vector updates (GeKcedce,GℓKcℓdcℓ\mathbf{G}^e\mathbf{K}_c^e\mathbf{d}_c^e, \mathbf{G}^\ell\mathbf{K}_c^\ell\mathbf{d}_c^\ell, rank k=64k=64), and linear reconstruction via the precomputed basis USc\mathbf{U}\mathbf{S}_c (Sec. 2.2, Eq. 3–11; Table 8).
  • We profiled end-to-end inference overhead across all 8 long-term benchmarks, horizons H∈{96,720}H\in\{96, 720\}, batch sizes B∈{1,32}B\in\{1, 32\}, and 3 representative backbones—two recent SOTA task-specific models (TimeFilter [ICML 2025] and CGTFra [ICML 2026] at L=96L=96) and a 200M foundation model (TimesFM 2.5 at L=512L=512)—on an RTX 4090 (24 GB). Results across channel dimensions C∈{7,137,321,862}C\in\{7, 137, 321, 862\} (Base → +MORA) at H=96H=96 and H=720H=720 are shown below:

Inference Overhead at H=96H=96:

BackboneDataset (CC)+GFLOPsFLOPs Inc.+Mem (MiB)B=1B=1 Latency (ms)B=32B=32 Latency (ms)
TimeFilterETTh1 (7)0.00060.24%9.053.34 → 4.186.62 → 7.54
TimeFilterSolar (137)0.01202.03%28.833.37 → 4.425.17 → 6.89
TimeFilterECL (321)0.01750.23%17.043.50 → 3.8542.03 → 43.04
TimeFilterTraffic (862)0.04540.26%27.532.68 → 3.0549.35 → 51.33
CGTFraETTh1 (7)0.000417.02%8.542.43 → 2.882.49 → 3.03
CGTFraSolar (137)0.012013.42%29.043.53 → 4.423.56 → 5.22
CGTFraECL (321)0.01752.30%16.483.61 → 3.934.19 → 5.20
CGTFraTraffic (862)0.04540.17%27.206.02 → 6.3953.81 → 55.94
TimesFM 2.5ETTh1 (7)0.00150.0006%10.46400.30 → 401.66514.84 → 516.79
TimesFM 2.5Solar (137)0.02870.0006%53.14445.45 → 447.049314.79 → 9319.38
TimesFM 2.5ECL (321)0.02660.0002%17.43688.66 → 689.3321581.3 → 21589.4
TimesFM 2.5Traffic (862)0.06960.0002%28.801984.11 → 1985.0354124.7 → 54149.7

Inference Overhead at H=720H=720:

BackboneDataset (CC)+GFLOPsFLOPs Inc.+Mem (MiB)B=1B=1 Latency (ms)B=32B=32 Latency (ms)
TimeFilterETTh1 (7)0.00401.41%9.753.34 → 3.956.67 → 7.92
TimeFilterSolar (137)0.069510.26%65.173.37 → 4.485.15 → 8.31
TimeFilterECL (321)0.16501.99%32.043.48 → 4.4042.39 → 47.28
TimeFilterTraffic (862)0.43852.39%64.002.69 → 3.7450.33 → 65.27
CGTFraETTh1 (7)0.00304.63%9.643.53 → 4.003.63 → 4.63
CGTFraSolar (137)0.069615.45%65.173.63 → 4.593.66 → 6.84
CGTFraECL (321)0.165012.86%32.343.60 → 4.554.97 → 9.93
CGTFraTraffic (862)0.43851.58%63.005.95 → 7.0054.74 → 69.99
TimesFM 2.5ETTh1 (7)0.00570.0024%22.13390.59 → 392.07519.38 → 522.13
TimesFM 2.5Solar (137)0.11100.0024%132.20438.39 → 440.189118.31 → 9129.27
TimesFM 2.5ECL (321)0.17400.0016%40.44692.15 → 693.6221448.8 → 21472.1
TimesFM 2.5Traffic (862)0.46280.0013%70.362010.32 → 2012.6354783.3 → 54855.0

2. Across all 48 configurations (all 8 datasets ×\times 3 backbones ×\times H∈{96,720}H\in\{96, 720\}), MORA adds minimal cost on high-dimensional series and requires only one-time offline fitting.

  • Task-specific backbones (32 settings on TimeFilter & CGTFra at L=96L=96): Across all 8 datasets, single-query (B=1B=1) latency increases by +0.33 to +1.05 ms at H=96H=96 and +0.46 to +1.11 ms at H=720H=720, with 8.54–65.17 MiB of additional memory. At B=32B=32 (whole-batch latency across B×CB\times C series: on small-CC datasets like ETTh1, B=1B=1 and B=32B=32 have similar wall-clock times because 224 series still fit within one GPU wave, whereas on large-CC datasets like Traffic, B=32B=32 processes 27,58427{,}584 series per batch and scales with throughput), MORA adds only +0.54 to +15.25 ms per batch (<0.48<0.48 ms per sample). On high-dimensional benchmarks (ECL, Traffic), our factorized estimator (df=32d_{\mathrm{f}}=32, App. C.2) keeps the FLOPs increase at H=96H=96 to 0.17%–2.30%.
  • Foundation models (16 settings on TimesFM 2.5 at L=512L=512): Because a foundation model runs a deep stack of Transformer layers whereas MORA only computes lightweight history statistics and small matrix projections on the output without re-running the backbone, its relative overhead is negligible—adding 0.0002%–0.0024% FLOPs and <0.56%<0.56\% latency at both B=1B=1 and B=32B=32, with zero OOF backbone replicas trained (App. C.5). All adapter parameters are fitted once offline via closed-form ridge/moment estimation and validation calibration, and we will add the full 48-setting benchmark in Appendix D.3.
AuthorsOpenReview ↗

Response to Reviewer 9Zup (Part 3/4): Segment-Wise Means (Q1) and Separate vs. Joint Training (Q2)

3. Why use segment-wise mean supervision rather than directly predicting spectral components? (Q1)

1. MORA already reconstructs its correction in a spectral basis. The segment means decide what is estimated from the history, and the spectral basis decides how the correction is reconstructed.

  • In Sec. 2.2.2 and App. C.3, the correction for channel cc is reconstructed in a tapered low-order IDCT basis U∈RH×k\mathbf{U}\in\mathbb{R}^{H\times k}:

Δc=USc(z^ce+z^cℓ)\Delta _ c = \mathbf{U}\mathbf{S} _ c(\widehat{\mathbf{z}} _ c^e + \widehat{\mathbf{z}} _ c^\ell)

  • The coarse estimators are trained to predict segment-wise future means qc=Ayc\mathbf{q} _ c = \mathbf{A}\mathbf{y} _ c from the history alone (Eq. 4–6), while the subsequent residual mappings are fitted to spectral residual coordinates zc\mathbf{z} _ c. MORA therefore separates coarse future estimation from backbone-specific residual correction. The two steps are linked by the discrepancy dc=mc−Ay^c(0)\mathbf{d} _ c = \mathbf{m} _ c - \mathbf{A}\widehat{\mathbf{y}} _ c^{(0)} (Eq. 7). Direct residual in Table 5 skips the first step and regresses zc\mathbf{z} _ c directly from the history features.
  • We use segment means for the first step because they are local in time and keep the level scale of the target. A level change late in the horizon affects only the last entries of qc\mathbf{q} _ c, whereas it generally affects multiple global DCT coefficients. Since A1H=1Q\mathbf{A}\mathbf{1} _ H = \mathbf{1} _ Q, centering at the last observation (q~c=qc−xL,c1Q\widetilde{\mathbf{q}} _ c = \mathbf{q} _ c - x _ {L,c}\mathbf{1} _ Q, Eq. 5) and periodic template alignment (Eq. 25) apply directly to qc\mathbf{q} _ c.

2. In Table 5, regressing the spectral residual coordinates directly lowers MSE but does not improve MAE on any dataset, while MORA gives the lowest MAE on all three datasets and reduces MSE in every case.

  • Direct residual uses the same history features ϕc\boldsymbol{\phi} _ c and basis U\mathbf{U}, with a parameter count within 2% of MORA, and regresses zc\mathbf{z} _ c from the OOF training pairs (Sec. 3.3, App. E). MSE / MAE, best in bold:
TimeFilter, L=H=96L=H=96ETTm1WeatherSolar
TimeFilter (Base)0.313 / 0.3540.153 / 0.1990.199 / 0.239
+ Affine calibration0.314 / 0.3570.153 / 0.2280.200 / 0.239
+ Direct residual0.300 / 0.3570.150 / 0.2110.181 / 0.239
+ δ\delta-Adapter (O)0.370 / 0.4150.156 / 0.2240.192 / 0.243
+ δ\delta-Adapter (I+O)0.312 / 0.3560.153 / 0.2010.171 / 0.234
+ MORA0.304 / 0.3500.148 / 0.1980.173 / 0.231
  • Direct residual has the lowest MSE on ETTm1, but its MAE is worse than the base model on ETTm1 and Weather and unchanged on Solar. MORA beats it in MAE on all three datasets and in MSE on Weather and Solar. Unlike the reduced-rank ridge fit in Direct residual, MORA scales each coordinate by an OOF reliability gain (Ge,Gℓ\mathbf{G}^e, \mathbf{G}^\ell, Eq. 37–38), which downweights coordinates with low estimated reliability, and the final scales Sc\mathbf{S} _ c are calibrated on validation data (Eq. 39).

4. How does separate training compare with joint end-to-end training? (Q2)

1. MORA keeps the deployed backbone checkpoint unchanged and needs no joint fine-tuning of the backbone and corrector. This matters most for foundation models.

  • MORA is fitted on the backbone's forecasts (out-of-fold forecasts for task-specific models, App. C.5) and leaves the deployed checkpoint as it is. Joint training would instead re-optimize each backbone with the corrector and change the pretrained weights of foundation models. MORA targets the setting where the deployed model stays fixed, and our comparisons are made under the same constraint.
  • With pretrained weights unchanged, MORA reduces MSE in all 48 dataset–model–context averages of Chronos-2, Sundial, and TimesFM 2.5 (Table 3; average reductions of 20.8%–28.8% at L=96L=96 and 6.0%–8.5% at L=512L=512) and in 187 of 192 (97.4%) individual horizon settings (Tables 13–14).

2. Separate fitting also gives fixed targets for closed-form estimation, and MORA outperforms a gradient-trained adapter on the same frozen backbones.

  • Freezing the backbone fixes the residual targets rc=yc−[fθ(X)]:,c\mathbf{r} _ c = \mathbf{y} _ c - [f _ \theta(\mathbf{X})] _ {:,c}. Once the first-stage mapping is fitted and fixed, the remaining residual coordinates likewise provide fixed targets for second-stage estimation. The residual mappings and reliability gains are then solved in closed form from regularized OOF second-order moments (Eq. 35–38), without iterative joint optimization.
  • On the same frozen backbones, MORA outperforms the gradient-trained δ-Adapter (I+O) (Liang et al., 2026) in 7 of 8 settings on MSE and 6 of 8 on MAE (Tables 5 and 11), while the output-only δ-Adapter (O) degrades the backbone on 2 of 3 datasets.
Reviewer_MnpZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper analyzes the residual patterns of various neural forecasters and subsequently proposes a new model-agnostic residual adapter, MORA, to correct them. At its core, MORA focuses on correcting the coarse patterns of the backbone's forecasts using a stagewise design. The experimental results show that MORA improves overall accuracy metrics across both end-to-end models and time-series foundation models in most settings.

Critical strengths
  • The motivation of the paper is clear, and residual learning is a classic topic in time series forecasting.

  • The proposed stagewise design and coarse pattern supervision seem sensible.

  • The experiments also evaluate MORA's effects on time-series foundation models.

Critical weaknesses

In my opinion, there are several critical issues in this study:

The general motivation is clear and valid, as residual learning is a well-studied topic in the forecasting community. However, the core premise of this study—that the residuals of neural forecasters converge to low-frequency patterns—is not well supported. The paper only demonstrates this using three examples (DLinear, iTransformer, and TimeFilter), which are not representative of the broader landscape of forecasting architectures. In my view, this conclusion does not easily generalize to many other models; for instance, direct forecasting and autoregressive models may exhibit entirely different error patterns. Consequently, the core motivation of this study risks being invalid.

The methodology section does not clearly state whether MORA is learnable. If it is, the supervision signals and loss functions are not explicitly defined in the manuscript, which is a critical omission for a residual learning study.

It confused me that there are no visualizations of actual forecasting curves anywhere in the paper. Since the motivation claims that errors mainly lie in low-frequency components, it should be straightforward to visually demonstrate how MORA refines predictions in these coarse trends. However, the authors only provide overall accuracy metrics, which fail to directly verify the specific effects of the proposed module.

Similarly, overall numerical improvements alone cannot verify whether MORA has actually learned the intended low-frequency corrections, and Figure. 3 is confusing and obscure for this intention. Simply achieving metric improvements over the selected baselines is not sufficient proof of the hypothesis, as many prior works have already shown that generic residual learning can easily boost performance.

Questions for authors
  1. Why do the foundation models shown in the results generally perform much worse than the selected end-to-end models? In my opinion, poorer baseline performance often leaves much more room for improvement.

  2. Why not demonstrate the effects of MORA under longer input settings, such as lookback windows of 512–720 for end-to-end neural forecasters? It appears that most of the selected baselines do not represent the true SOTA on this benchmark.

  3. Can you provide more direct evidence showing that the improvements achieved by MORA actually stem from reducing low-frequency errors?

12 comments
AuthorsOpenReview ↗

Response to Reviewer MnpZ (Part 1/3): Core Motivation & 9-Model Residual Analysis (W1)

Thank you for your review. We reply in 3 parts: Part 1 (here): W1; Part 2: W2–W4 & Q3; Part 3: Q1–Q2.

W1: Do low-frequency residuals generalize across architectures (including direct vs. autoregressive models)?

1. The pattern persists across nine models and eight datasets

We extend the Fig. 1(a) and Appendix G (Table 16) diagnostic to all nine evaluated backbones. For each frozen model, we compute the test-set residual energy (%) in the first 8 orthonormal DCT-II modes (8.33% of modes, including the constant mode):

F8=100×∑i,c∥P8ri,c∥22∑i,c∥ri,c∥22,ri,c=yi,c−y^i,c(0).F_ 8 = 100\times\frac{\sum_ {i,c}\Vert P_ 8 r_ {i,c}\Vert_ 2^2}{\sum_ {i,c}\Vert r_ {i,c}\Vert_ 2^2},\quad r_ {i,c}=y_ {i,c}-\hat{y}_ {i,c}^{(0)}.

DatasetDLinearPatchTSTiTransformerTimeFilterCGTFraVPNetChronos-2SundialTimesFM 2.5
ETTh148.1950.0250.0650.5650.5750.7851.7449.9951.62
ETTh281.7379.0277.1378.7579.1379.1480.2679.7480.37
ETTm188.0487.4386.8387.2087.3887.1392.4490.6894.44
ETTm284.8682.9282.7582.2382.4282.9585.0484.6387.95
Weather83.9882.0481.8579.5479.5079.6586.6183.3984.81
ECL61.5459.2557.4455.5755.8056.8764.6160.9761.57
Traffic32.9930.5029.3229.6029.6029.0832.2131.5231.61
Solar94.5393.1292.7992.4992.4892.0696.7297.1398.07

The key result is cross-architecture consistency within each dataset: F8F_ 8 is 79.50–86.61% on Weather, 92.06–98.07% on Solar, and 29.08–32.99% on Traffic (∼3.7×\sim 3.7\times the uniform 8.33% share). Cross-dataset variation far exceeds within-dataset model variation, confirming this pattern across architectures.

2. Why this structure can recur across architectures

Assuming finite second moments, let II be the query information and μ(I)=E[Y∣I]\mu(I)=E[Y\mid I]. Any fixed forecaster ff has residual rf=Y−f(I)=ϵ+bf(I)r_ f=Y-f(I)=\epsilon+b_ f(I), with ϵ=Y−μ(I)\epsilon=Y-\mu(I) and bf(I)=μ(I)−f(I)b_ f(I)=\mu(I)-f(I). Since E[ϵ∣I]=0E[\epsilon\mid I]=0, conditional orthogonality gives E∥Pqrf∥2=E∥Pqϵ∥2+E∥Pqbf∥2E\Vert P_ q r_ f\Vert^2=E\Vert P_ q\epsilon\Vert^2+E\Vert P_ q b_ f\Vert^2 for any fixed orthogonal projector PqP_ q:

Fq(f)100=E∥Pqrf∥2E∥rf∥2=(1−λf)pq(ϵ)+λfpq(b)(f),\frac{F_ q(f)}{100}=\frac{E\Vert P_ q r_ f\Vert^2}{E\Vert r_ f\Vert^2}=(1-\lambda_ f)p_ q^{(\epsilon)}+\lambda_ fp_ q^{(b)}(f),

withpq(ϵ)=E∥Pqϵ∥2E∥ϵ∥2,pq(b)(f)=E∥Pqbf∥2E∥bf∥2,λf=E∥bf∥2E∥rf∥2∈[0,1].\text{with}\quad p_ q^{(\epsilon)}=\frac{E\Vert P_ q\epsilon\Vert^2}{E\Vert\epsilon\Vert^2},\quad p_ q^{(b)}(f)=\frac{E\Vert P_ q b_ f\Vert^2}{E\Vert b_ f\Vert^2},\quad \lambda_ f=\frac{E\Vert b_ f\Vert^2}{E\Vert r_ f\Vert^2}\in[0,1].

The decomposition identifies a shared contribution from conditional data dynamics and a model-dependent contribution from forecast bias. The expanded diagnostic shows that, despite the latter differences, low-frequency concentration remains consistent across architectures. Under yt+1=yt+ut+1y_ {t+1}=y_ t+u_ {t+1} with zero-mean i.i.d. shocks independent of history (Var(ut)=σ2\text{Var}(u_ t)=\sigma^2), ϵh=∑j=1hut+j\epsilon_ h=\sum_ {j=1}^h u_ {t+j} has Cov(ϵh,ϵk)=σ2min⁡(h,k)\text{Cov}(\epsilon_ h,\epsilon_ k)=\sigma^2\min(h,k), putting >97.3% of expected residual energy in the first 8 DCT modes for H∈[96,720]H\in[96,720].

3. We explicitly tested autoregressive generation

On Weather (H=720H=720), Chronos-2 uses direct multi-patch projection, Sundial generates a 720-step block, and TimesFM 2.5 feeds generated patches back into its decoder:

Weather, H=720H=720Output generationF8F_ 8, L=96L=96F8F_ 8, L=512L=512
Chronos-2Direct multi-patch73.75%74.73%
SundialSingle-block flow generation73.36%73.91%
TimesFM 2.5Autoregressive patch feedback77.42%77.07%

Crucially, the observation persists under actual autoregressive rollout: the first 8 DCT modes account for 77.42% and 77.07% of TimesFM 2.5's residual energy across both contexts, so direct vs. autoregressive decoding preserves the coarse-scale residual structure.

4. Spectral concentration and history-based correction

Any correlation between the residual and a history-derived signal comes from the forecast deviation bf(I)b_ f(I), rather than the unpredictable innovation ϵ\epsilon. For any unit direction uu and history signal s(I)s(I), Cov(u⊤rf,s(I))=Cov(u⊤bf(I),s(I))\text{Cov}(u^\top r_ f,s(I))=\text{Cov}(u^\top b_ f(I),s(I)). This connects the spectral analysis in Fig. 1(a) with the history-probe evidence in Fig. 1(b), further reported in Appendix G: Proposition 1 (Appendix B, Eqs. 12–23) shows that, whenever Cov(u⊤rf,s)≠0\text{Cov}(u^\top r_ f,s)\neq 0, the optimal affine correction along uu achieves a strictly lower expected MSE than the best constant correction along uu, reducing MSE by Cov(u⊤rf,s)2/[HVar(s)]\text{Cov}(u^\top r_ f,s)^2/[H\text{Var}(s)].

Direct evidence across 9 models, 8 datasets, and autoregressive rollout thus resolves the concern that our motivation rests on three models. With our spectral decomposition and Proposition 1, this establishes a solid basis for MORA: coarse-scale errors recur across architectures, and their history-dependent component is a principled target for correction.

AuthorsOpenReview ↗

Response to Reviewer MnpZ (Part 2/3): Supervision, Forecast Visualizations & Low-Frequency Error Analysis (W2–W4, Q3)

Following Part 1 (App. G.1, Fig. 5 in the revised PDF; all revisions in blue), Part 2 addresses W2–W4 & Q3.

1. MORA is fitted offline with explicit supervision and objectives (W2)

MORA is a learnable offline adapter on a frozen backbone, with all supervision targets and objectives defined in Sec. 2.2 and Appendix C. MORA fits coarse estimators, residual mappings, and calibration sequentially:

  • Learned Coarse Estimator (WcvW_ c^v, Sec. 2.2.1 & App. C.1, Eq. 26): Ridge regressors map standardized historical features Φcv\Phi_ c^v to future segment means qc=Ayc∈RQq_ c = Ay_ c \in \mathbb{R}^Q (Eq. 3, targeting future states, not backbone residuals) for views v∈{abs,rel}v\in\{\text{abs},\text{rel}\}:

Wcv=arg⁡min⁡W∥Qcv−ΦcvW∥F2+λv∥W∥F2.W_ c^v = \arg\min_ W \Vert Q_ c^v - \Phi_ c^v W\Vert_ F^2 + \lambda_ v \Vert W\Vert_ F^2.

  • Stagewise Residual Mappings & Calibration (Sec. 2.2.2–2.2.3 & App. C.4–C.5): With zc=U⊤(yc−y^c(0))z_ c = U^\top(y_ c - \hat{y}_ c^{(0)}) (Eq. 8), Stage 1 uses pairs (dce,zc)(d_ c^e, z_ c) and Stage 2 targets zcℓ=zc−z^cez_ c^\ell = z_ c - \hat{z}_ c^e against the Stage-1-updated forecast:

(dce, zc),(dcℓ, zcℓ=zc−z^ce),dcℓ=mcℓ−A(y^c(0)+Uz^ce).(d_ c^e,\, z_ c), \qquad (d_ c^\ell,\, z_ c^\ell = z_ c - \hat{z}_ c^e), \qquad d_ c^\ell = m_ c^\ell - A(\hat{y}_ c^{(0)} + U\hat{z}_ c^e).

Kce,KcℓK_ c^e, K_ c^\ell are fitted from regularized second-order moments (Eqs. 35–36), with out-of-fold reliability gains Ge,GℓG^e, G^\ell (Eqs. 37–38). Validation scales ScS_ c minimize Eq. (39); all parameters remain fixed at test time:

Lcal=14∑b=14[0.1MSEbMSEb(0)+0.9MAEb,ϵMAEb(0)].\mathcal{L}_ {\text{cal}} = \frac{1}{4}\sum_ {b=1}^4 \left[0.1\frac{\text{MSE}_ b}{\text{MSE}_ b^{(0)}} + 0.9\frac{\text{MAE}_ {b,\epsilon}}{\text{MAE}_ b^{(0)}}\right].

2. Visualizations of actual forecasting curves and coarse trends (W3)

We added trajectory visualizations in Sec. 3.4 (Fig. 4) and Appendix G.3 (Figs. 6–7) across four backbones and four datasets.

  • Fig. 4 (Sec. 3.4) shows TimeFilter on Weather, ETTm1, Solar, and ECL: Panel (a) plots history, ground truth, and forecasts before/after MORA; Panel (b) plots the corresponding DCT-8 reconstructions.
  • Figs. 6–7 (Appendix G.3) extend this to TimeFilter, iTransformer, DLinear, and TimesFM 2.5 on the same windows and vertical limits.

These plots make the correction effect explicit: on Weather and Solar, MORA reduces future-level and peak-magnitude errors, aligning forecasts with the ground truth in both full trajectories and DCT-8 reconstructions.

3. Direct frequency-localized evidence of MORA's improvements (W4 & Q3)

1. Fig. 3 is an energy-preserving representation of low-frequency error. For each test window, e∈R8Ce\in\mathbb{R}^{8C} concatenates the first 8 residual DCT coefficients across channels (normalized by HC\sqrt{HC}). With validation-fitted unit direction vv, coordinates a=v⊤ea=v^\top e and b=∥e−av∥2b=\Vert e-av\Vert_ 2 satisfy a2+b2=∥e∥22a^2+b^2=\Vert e\Vert_ 2^2, so squared distance to ★ equals low-frequency MSE. On Solar (Sec. 3.4, Fig. 3), MORA cuts low-frequency MSE by 41.0% (DLinear) and 13.9% (TimeFilter), alongside 38.9% and 12.8% total MSE reductions.

2. Full-test frequency decomposition attributes 94.29%–100.55% of net MSE gains to the first 16 DCT modes across four datasets. Auditing TimeFilter (L=H=96L=H=96) with r(0)=y−y^(0)r^{(0)}=y-\hat{y}^{(0)} and r(M)=y−y^(M)r^{(M)}=y-\hat{y}^{(M)} across all test windows and channels, orthonormality gives Δtotal=Δq+Δ>q\Delta_ {\text{total}} = \Delta_ q + \Delta_ {>q}:

Δq=1NHC∑i,c(∥Pqri,c(0)∥22−∥Pqri,c(M)∥22).\Delta_ q = \frac{1}{NHC}\sum_ {i,c}\Big(\Vert P_ q r_ {i,c}^{(0)}\Vert_ 2^2 - \Vert P_ q r_ {i,c}^{(M)}\Vert_ 2^2\Big).

Dataset (TimeFilter)Total MSE: Before → AfterModes 1–8 (Δ8/Δtotal\Delta_ 8/\Delta_ {\text{total}})Modes 1–16 (Δ16/Δtotal\Delta_ {16}/\Delta_ {\text{total}})
ETTh10.369161 → 0.35849337.48%94.29%
ETTm10.313376 → 0.303798101.09%100.55%
Weather0.153036 → 0.147672100.11%100.09%
Solar0.198832 → 0.173376100.09%100.02%

Low-frequency error reduction accounts for nearly the entire net MSE gain (94.29%–100.55% within modes 1–16). On ETTm1, Weather, and Solar, modes 1–8 alone contribute 100.09%–101.09%: because Δtotal=Δq+Δ>q\Delta_ {\text{total}}=\Delta_ q+\Delta_ {>q}, ratios slightly above 100% simply reflect a tiny offsetting net error increase (Δ>q<0\Delta_ {>q}<0, 10−610^{-6} to 10−410^{-4} MSE) in the remaining modes alongside the dominant low-frequency gain. On ETTh1, substantial gains in modes 9–16 bring the 16-mode share to 94.29%.

3. Controlled comparisons confirm MORA's structured design over generic residual fitting. In Table 5, Direct residual regression uses the same features and basis UU (parameter count matched within ~2%), yet MORA achieves the lowest MAE on all three datasets while improving both MSE and MAE. Across Tables 5 and 11, MORA beats input–output δ\delta-Adapter in MSE on 7/8 settings and MAE on 6/8 settings, and Table 4 shows that removing either coarse source or stagewise refinement increases both errors in all three settings.

AuthorsOpenReview ↗

Response to Reviewer MnpZ (Part 3/3): Baseline Strength Across Context Lengths & Lookback Protocols (Q1–Q2)

Following Parts 1 and 2, Part 3 addresses Q1 and Q2 regarding SOTA baseline representation and lookback protocols.

1. Why do foundation models have higher errors at L=96L=96, and does MORA rely on weak baselines? (Q1)

1. We evaluate the latest SOTA foundation models—Chronos-2, Sundial, and TimesFM 2.5; their higher error at L=96L=96 reflects zero-shot inference under a short window, whereas at L=512L=512 they outperform the strongest supervised baselines on 6 of 8 datasets.

  • In Table 1, the six task-specific models are supervised and trained on each target dataset. In Table 3, the unadapted (Base) foundation models are evaluated zero-shot with frozen pretrained weights (Sec. 3.2 & App. D.2).
  • Under a short lookback of L=96L=96 (e.g., 24 hours on 15-minute ETTm1), zero-shot models have limited context to infer dataset-specific level, scale, and multi-period structure, leaving larger uncalibrated coarse errors that MORA reduces by 20.8%–28.8% on average.
  • However, at L=512L=512 in Table 3, these frozen foundation models are strong SOTA baselines—on 6 of 8 datasets, at least one unadapted foundation model already achieves lower MSE than the best supervised backbone in Table 1 (ETTh1, ETTh2, ETTm1, ECL, Traffic, and Solar; entries show Base → +MORA):
DatasetBest Supervised SOTA (Table 1, L=96L=96)Chronos-2 (L=512L=512)Sundial (L=512L=512)TimesFM 2.5 (L=512L=512)
ETTh10.418 (TimeFilter)0.418 → 0.402 (-3.9%)0.397 → 0.387 (-2.5%)0.421 → 0.403 (-4.4%)
ETTh20.359 (VPNet)0.352 → 0.345 (-2.0%)0.344 → 0.340 (-1.0%)0.372 → 0.365 (-1.9%)
ETTm10.376 (VPNet)0.387 → 0.338 (-12.8%)0.366 → 0.338 (-7.5%)0.381 → 0.341 (-10.4%)
ETTm20.271 (TimeFilter)0.280 → 0.257 (-8.2%)0.275 → 0.252 (-8.2%)0.291 → 0.273 (-6.1%)
Weather0.238 (VPNet)0.258 → 0.213 (-17.5%)0.244 → 0.213 (-12.9%)0.263 → 0.215 (-18.3%)
ECL0.164 (TimeFilter/VPNet)0.163 → 0.154 (-5.0%)0.148 → 0.143 (-2.9%)0.164 → 0.156 (-5.0%)
Traffic0.408 (TimeFilter)0.392 → 0.389 (-0.8%)0.458 → 0.441 (-3.7%)0.382 → 0.379 (-0.7%)
Solar0.204 (VPNet)0.187 → 0.187 (-0.2%)0.229 → 0.208 (-9.2%)0.281 → 0.222 (-21.2%)

2. MORA consistently improves both these top-performing L=512L=512 foundation models and the strongest recent supervised SOTA backbones.

  • On the L=512L=512 foundation baselines above, MORA reduces MSE in 93 of 96 (96.9%) individual horizon settings and all 24 of 24 model–dataset pairs (App. D.5 & Table 14), with average MSE reductions of 6.3% (Chronos-2), 6.0% (Sundial), and 8.5% (TimesFM 2.5)—including 12.9%–18.3% on Weather and 7.5%–12.8% on ETTm1. On every dataset, the MORA-adapted L=512L=512 foundation models achieve lower MSE than all unadapted and MORA-adapted L=96L=96 models in Table 1.
  • Among supervised models (Tables 1–2), MORA also improves the strongest recent SOTA backbones (TimeFilter [ICML 2025], CGTFra [ICML 2026], and VPNet [ICLR 2026]), e.g., lowering MSE by 4.4% and 4.3% for TimeFilter and CGTFra across the eight long-term datasets (up to 12.9% on Solar and 5.5%–7.4% on Weather) and by 12.1%–18.6% across the four PEMS benchmarks.

2. Why use L=96L=96 for end-to-end models, and do our baselines represent true SOTA? (Q2)

1. Our end-to-end backbones include the latest SOTA models (up to ICLR/ICML 2026) under the field's standard L=96L=96 unified benchmark protocol.

  • Beyond classical baselines (DLinear, PatchTST, iTransformer [ICLR 2024]), we evaluate recent SOTA architectures: TimeFilter (ICML 2025), CGTFra (ICML 2026), and VPNet (ICLR 2026). Across these works, L=96L=96 with H∈{96,192,336,720}H\in\{96,192,336,720\} is the standard unified benchmark protocol (Sec. 3.1 & App. D.2), ensuring all six backbones receive the same historical context length under their authors' validated SOTA configurations.
  • Moreover, all baseline–MORA comparisons in Tables 1 and 3 are strictly paired: each pair shares the same frozen checkpoint, historical query, and forecast horizon, isolating the effect of output-space adaptation rather than per-backbone lookback searching.

2. Long-context SOTA (L=512L=512) is explicitly evaluated via the latest foundation models without confounding backbone retraining.

  • Pretrained foundation models natively support extending the input from L=96L=96 to L=512L=512—a standard, widely adopted context length in foundation-model evaluations (Ansari et al., 2025; Google Research, 2025; Liu et al., 2025)—while keeping their pretrained weights fixed.
  • Both our Weather diagnostic at L=512,H=720L=512, H=720 in Part 1 (F8=73.91–77.07F_ 8 = 73.91\text{–}77.07%) and Table 3 above confirm that extending lookback to L=512L=512 does not eliminate predictable coarse-scale errors, and that MORA remains effective on long-context SOTA forecasts.
Reviewer_niJoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

MORA is a post-hoc adapter for a frozen forecaster. It estimates the mean of the future in Q contiguous segments, once by extrapolating the history and once with ridge regression on DCT and statistical features of the history, compares these estimates with the segment means of the backbone forecast, and maps the difference through a learned gain matrix onto a low-order cosine correction subspace. Two such updates are applied in sequence, and a validation-fitted scale is applied to their sum. It is tested on six backbones over twelve datasets and on three foundation models.

Critical strengths
  1. The paper starts from a measured observation, that forecast residuals concentrate in low DCT modes (Fig. 1, App. G), and the resulting method is simple, linear and easy to reproduce.
  2. The evaluation is broad: six backbones, eight long-horizon and four PEMS datasets, and three foundation models at two context lengths, with the backbone frozen throughout.
  3. Gains are reported per cell, including the cases where MORA hurts (CGTFra on ETTh1 and VPNet on ETTh2 in Table 1; MAE of foundation models at L = 512 in Table 3).
Critical weaknesses
  1. MORA is regularized linear regression on features of the history, and the paper's own comparison shows the added machinery does not pay off. In Table 5 (TimeFilter, H = 96), plain residual regression on the same features and correction basis has a lower MSE than MORA on ETTm1 (0.300 vs 0.304), the input-output δ-Adapter (Liang et al., 2026), already published and cited, has a lower MSE on Solar (0.171 vs 0.173), and the margin on Weather over plain regression is 0.002. The stagewise update that the paper presents as its key design improves on independent updates by 0.001 to 0.005 MSE (Table 4: 0.422 vs 0.427, 0.376 vs 0.379, 0.190 vs 0.191), measured on one backbone with one seed, and App. E says that these examples are "a retrospectively selected subset of the full study". Post-hoc correction of a frozen forecaster is an existing idea, and I could not find in this paper a component that is both new and shown to matter.

  2. The large headline gains come from weak backbones and short contexts. The averages of 7.2% (long-term) and 28.1% (short-term) are driven by DLinear and PatchTST, whose MSE falls by 14.9% and 9.3% on the long-term sets and by up to 66% on PEMS, yet after adaptation they remain worse than the unadapted TimeFilter (PEMS03 MSE 0.127 for PatchTST and 0.137 for DLinear with MORA, against 0.101 for TimeFilter alone, Table 2). For the strongest backbones the average gain is 4.4% (TimeFilter) and 4.3% (CGTFra). The foundation models improve by 21 to 29% only at a context of 96, which handicaps them; at 512 the gain falls to 6 to 8.5%, and MAE gets worse on ETTh1 (Chronos-2 -0.75%, TimesFM -0.87%) and on Solar (TimesFM -4.5%).

  3. The cost of MORA is not reported. "Frozen backbone, no additional inputs" leaves out the fitting procedure: to obtain out-of-fold residuals, App. C.5 retrains task-specific backbone replicas, one for each of four training blocks, then fits ridge regressors, cross-fits two gain matrices and calibrates scales on the validation set. The paper gives no training time, memory or inference cost for any of this. Every comparison in Tables 1 to 3 is between a backbone and the same backbone plus this pipeline, and without the cost a gain of 4% on a strong backbone cannot be weighed.

  4. The noise level is unknown and the motivating analysis is weak. Four of the eight long-term datasets (ETTh1, ETTm1, ETTm2, Weather) are single runs (App. D.4), so differences such as the +0.7% of VPNet and the -0.5% of CGTFra on ETTh1 cannot be separated from seed variation. Fig. 1(a) reports that the first eight of 96 DCT modes hold about 80% of the residual energy on Weather, with no reference (for example the spectrum of the target or of a naive forecast's error), so it cannot show that this is a property of these backbones rather than of forecast errors in general. The pattern is not uniform either: the share is over 92% on Solar but only 29 to 33% on Traffic (App. G.1), and on Traffic the gains of the four strongest backbones are 1.6 to 2.8% (Table 1).

Questions for authors

See weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer niJo (Part 1/3: Summary)

Dear Reviewer niJo,

Thank you for reviewing our paper. We disagree with the conclusion that MORA’s design "does not pay off" and with the Soundness: 1 / Rating: 1 assessment. Below we summarize our direct responses and core evidence for Weaknesses 1–4, followed by detailed analyses in Parts 2 and 3.


Weakness 1 (Claim: MORA's added design "does not pay off")

The review draws this conclusion from isolated cells in Table 5, whereas the full ablation and comparison results across the paper validate MORA:

  • 96.7% Global Win Rate: MORA reduces MSE in 277/288 task-specific (96.2%) and 187/192 foundation-model (97.4%) cases—totaling 464/480 (96.7%).
  • 7/8 MSE wins over δ\delta-Adapter (Tables 5 & 11): The review cites only Table 5 while overlooking Table 11 on Page 24 (on the same page quoted by the review). Across all 8 settings in Tables 5 and 11, MORA beats input–output δ\delta-Adapter (I+O) in 7/8 settings on MSE and 6/8 on MAE, while output-only δ\delta-Adapter (O) degrades the backbone on 2/3 datasets.
  • Full ablations (Tables 4, 5, & 10): Apart from δ\delta-Adapter, Table 5 (Sec. 3.3) tests our constructed controls (Affine calibration, Direct residual): MORA achieves the lowest MAE on all 3 datasets while improving MSE in every case (Direct residual worsens backbone MAE on 2/3 datasets). Beyond Table 4, Table 10 (Page 24) validates MORA's extrapolation signals across 40 template-enabled settings .

Weakness 2 (Claim: Gains come from weak backbones and short contexts)

MORA's gains are not confined to simple models or short windows; it consistently improves recent SOTA backbones and long-context (L=512L=512) foundation models:

  • Gains on recent SOTA backbones: With weights frozen, MORA reduces mean long-term / short-term MSE on TimeFilter (ICML'25) by 4.44% / 12.10% (47/48 settings), CGTFra (ICML'26) by 4.34% / 18.59% (47/48), VPNet (ICLR'26) by 2.76% / 18.43% (44/48), and iTransformer (ICLR'24) by 7.37% / 11.57% (43/48)—averaging 4.73% (long-term) and 15.17% (PEMS) excluding DLinear and PatchTST.
  • Same-backbone evaluation: MORA is a post-hoc adapter for a given frozen model. Comparing DLinear + MORA with TimeFilter compares different architectures; applied to TimeFilter itself, MORA reduces PEMS03 MSE from 0.101 to 0.098 and improves 47/48 settings.
  • Robustness at L=512L=512: Alongside the task-specific default (L=96L=96, 20.8%–28.8% MSE reduction), we evaluated L=512L=512 (a common long-context setting for foundation models). Even with 5.3×5.3\times more history at L=512L=512, MORA still reduces MSE in 93/96 settings (96.9%) and all 24/24 dataset–model averages (by 6.0%–8.5%), proving robustness across context lengths.

Weakness 3 (Claim: Computational cost of MORA is not reported)

MORA adds minimal deployment overhead and requires zero OOF retraining on foundation models:

  • Low deployment overhead: Inference runs only one original frozen backbone plus MORA's fixed linear projections with no test-time updates, adding 0.923 ms (+0.165 GFLOPs, +32 MiB) on ECL (C=321C=321) and 1.051 ms (+0.439 GFLOPs, +64.0 MiB) on Traffic (C=862C=862) (B=1,H=720B=1, H=720).
  • Zero foundation-model replicas: For all three Foundation Models, zero OOF replicas are trained (weights remain frozen; App. C.5). Task-specific 4-block OOF replicas are used only offline; we will add full offline timings in the revision.

Weakness 4 (Claim: Seed noise is unknown and motivating analysis lacks reference)

Multi-seed stability and out-of-sample error predictability from history are already verified in the paper:

  • Multi-seed stability (93/96 wins): Across the four 3-seed benchmarks (ETTh2, ECL, Traffic, Solar—96 settings, 288 runs; App. D.4), MORA improves 3-seed mean MSE in 93/96 settings (96.9%, including 72/72 on ECL/Traffic/Solar), matching the single-seed win rate (94/96, 97.9%). We can add multi-seed runs for the remaining datasets in the revision.
  • Spectral reference & predictability: Relative to a uniform reference (8/96=8.3%8/96 = 8.3\%), low-frequency error share is 48.2%–94.5% on 7 datasets and 29.3%–33.0% on Traffic (3.5×–4×3.5\times\text{--}4\times above uniform; Table 16). Crucially, validation-fitted linear probes (Fig. 1b, Prop. 1, Table 16) confirm positive test correlation from history across all datasets (rmed=0.240–0.393r_{med}=0.240\text{--}0.393 on Traffic, where MORA improves 24/24 settings).

In summary, we believe the current assessment (Soundness: 1 / Rating: 1) does not accurately or fairly reflect our paper's actual contributions and full experimental results. All reported results are fully reproducible, and our code is open-sourced (linked in the Abstract). Detailed evidence for Weaknesses 1–4 is provided in Parts 2 and 3 below.

AuthorsOpenReview ↗

Response to Reviewer niJo (Part 2/3: Detailed Responses to Weaknesses 1 & 2)

Detailed Responses to Weaknesses 1 & 2

1. Core Innovation, Overall Gains, and Selective Citation in Ablations (Weakness 1)

(1) Core innovation and distinction from δ\delta-Adapter (Liang et al., 2026): Figure 1 and Appendix G empirically identify low-frequency residual structure and predictive signals in historical inputs. Proposition 1 establishes that, under its stated covariance condition, there exists a history-dependent affine correction that improves expected MSE beyond the best constant correction along the corresponding direction. MORA turns this into an explicit coarse-future-estimation and residual-correction formulation: it combines structural extrapolations with a learned predictor supervised on future segment means q=AYq=AY, then maps their discrepancies d=m(X)−AY^(0)d = m(X) - A\hat{Y}^{(0)} into a tapered cosine subspace via regularized second-order moments and cross-fitted reliability gains (Ge,GℓG^e, G^\ell) in two stages. Unlike δ\delta-Adapter—which trains neural modules via gradient descent on the forecasting loss and (δ\delta-Adapter I+O) jointly optimizes input perturbations and output edits—MORA operates strictly in the output space. Fitting the adapter requires no backpropagation through the deployed backbone. MORA derives its corrections from coarse future estimates scaled by out-of-fold reliability.

(2) Overall gains vs. selective citation in the review:

  • Overall effectiveness (Tables 1–3, 12–15): MORA reduces MSE in 277/288 task-specific (96.2%) and 187/192 foundation-model (97.4%) settings, totaling 464/480 (96.7%). These results establish the breadth of the adaptation benefits; the value of the specific design is examined by the controlled comparisons in Tables 4, 5, 10, and 11.
  • Selective citation of δ\delta-Adapter (Table 5 vs. Table 11 on Page 24): The review cites Solar MSE in Table 5 (0.171 vs. 0.173) while ignoring Table 11 on Page 24 (the exact page quoted by the review), which tests δ\delta-Adapter (I+O) (official code, matched protocol) on 5 more settings at L=H=96L=H=96, all 5 of which MORA wins in MSE. Across all 8 settings in Tables 5 & 11, MORA wins 7/8 on MSE and 6/8 on MAE (with lower MAE on Solar: 0.231 vs. 0.234), while output-only δ\delta-Adapter (O) worsens both metrics on 2/3 datasets (ETTm1: 0.370 vs. 0.313; Weather: 0.156 vs. 0.153).
  • Selective citation of our own ablations (Tables 4, 5, & 10): Apart from δ\delta-Adapter, Table 5 (Sec. 3.3) tests two controls we constructed (Lines 443, 1278): Affine calibration (improving neither metric) and Direct residual (using MORA's features, basis, and OOF pairs at matched parameters, predicting residuals directly). The review cites ETTm1 MSE (0.300 vs. 0.304) while omitting that Direct residual increases MAE relative to the unadapted TimeFilter backbone on 2/3 datasets (ETTm1: 0.357 vs. 0.354; Weather: 0.211 vs. 0.199), whereas MORA improves both metrics in every case and achieves the lowest MAE on all 3 datasets (beating Direct residual by 1.61% MSE / 6.01% MAE on Weather and 4.05% MSE / 3.41% MAE on Solar). Likewise, the review quotes Page 24 and cites the three TimeFilter settings in Table 4 (where full MORA still achieves the best MSE and MAE) while omitting Table 10 on Page 24, which evaluates the extrapolation signals over 40 matched template-enabled settings.

2. Gains on Recent SOTA Backbones and Long-Context Foundation Models (Weakness 2)

(1) Substantial gains on strong recent SOTA backbones:

Frozen BackboneLong-Term MSE Imp.PEMS MSE Imp.Settings Improved
TimeFilter (ICML'25)4.44%12.10%47/48 (97.9%)
CGTFra (ICML'26)4.34%18.59%47/48 (97.9%)
VPNet (ICLR'26)2.76%18.43%44/48 (91.7%)
iTransformer (ICLR'24)7.37%11.57%43/48 (89.6%)

Excluding DLinear and PatchTST, MORA cuts mean MSE on these four SOTA backbones by 4.73% (long-term) and 15.17% (PEMS). Comparing DLinear + MORA with TimeFilter conflates backbone architecture with adapter gains; on TimeFilter itself, MORA cuts PEMS03 MSE from 0.101 to 0.098 and improves 47/48 settings.

(2) Gains across both L=96L=96 and L=512L=512 foundation-model contexts: Table 3 tests each foundation model with and without MORA at the same context length and on the same queries under L=96L=96 (the task-specific default, cutting MSE by 20.8%–28.8%) and L=512L=512 (a common long-context setting for foundation models). Even at L=512L=512 (5.3×5.3\times more history), MORA still cuts MSE in 93/96 settings (96.9%) and all 24/24 dataset–model averages (by 6.30%, 5.99%, and 8.51% for Chronos-2, Sundial, and TimesFM 2.5, with lower mean MAE for each). Gains under both settings (187/192 cases, 97.4%) demonstrate that MORA's gains persist across both evaluated context lengths.

AuthorsOpenReview ↗

Response to Reviewer niJo (Part 3/3: Detailed Responses to Weaknesses 3 & 4)

Detailed Responses to Weaknesses 3 & 4

3. Offline Preparation and Deployment Costs (Weakness 3)

We will add an explicit computational cost table in the revision separating offline preparation from deployment:

  • Deployment Cost (One Frozen Backbone + Fixed Adapter): The four task-specific OOF replicas (App. C.5) are used strictly offline to construct leak-free training signals. At deployment, inference runs only one original frozen backbone followed by MORA's fixed coarse estimators and residual mappings, with zero test-time updates. Measured on an RTX 4090 (B=1,L=96,H=720B=1, L=96, H=720) with TimeFilter on our two largest long-term datasets (latency and peak allocated memory are measured; GFLOPs are operator-level estimates):
Dataset (CC channels)Latency: Base →\to +MORA (ms)Added Latency (ms)Peak Memory: Base →\to +MORA (MiB)Added GFLOPs
ECL (C=321C=321)3.475→4.3973.475 \to 4.397+0.923+0.923101.385→133.423101.385 \to 133.423 (+32.038+32.038)0.16500.1650
Traffic (C=862C=862)2.685→3.7362.685 \to 3.736+1.051+1.05187.607→151.60787.607 \to 151.607 (+64.000+64.000)0.43850.4385
  • Offline Fitting Cost: For all three Foundation Models, pretrained weights remain frozen throughout and zero OOF backbone replicas are trained (App. C.5, Line 979). For task-specific backbones, generating blocked OOF forecasts trains four replicas on training partitions offline; once OOF predictions are cached, fitting MORA itself requires only ridge regression, regularized second-order moments, and at most eight 4-variable box-constrained IRLS steps per channel for validation calibration. We will report full offline OOF and adapter-fitting wall-clock times in the revision. These measurements complement the fitting protocol already specified in Appendix C.5 and provide an explicit account of the accuracy–cost trade-off.

4. Repeated-Backbone Evaluation and Motivating Residual Analysis (Weakness 4)

  • Gains persist under repeated-backbone evaluation: First, every local baseline–MORA pair is evaluated on the exact same frozen checkpoint and test windows, directly measuring MORA's correction effect without checkpoint confounding. Second, across ETTh2, ECL, Traffic, and Solar, evaluated with three backbone seeds (96 settings, 288 runs; App. D.4), MORA reduces the seed-averaged MSE in 93/96 settings, including 72/72 on ECL, Traffic, and Solar. These results show that the improvements extend beyond the single-run benchmarks. We will provide the corresponding per-seed paired improvements and standard deviations in the revision so that their variability can be assessed directly.
  • Spectral reference and predictability: Figure 1 and Appendix G (reporting 3 representative backbones in the text due to space limits) establish two complementary facts:
    1. Spectral concentration relative to a uniform baseline (Fig. 1a, App. G.1): Because the 96 orthonormal DCT modes partition total squared error (MSE=∑k=095Ek\text{MSE} = \sum_{k=0}^{95} E_k), a uniform spectrum places 8/96=8.33%8/96 = 8.33\% of error energy in the first 8 modes. Across Table 16, observed F8F_8 shares are 48.2%–94.5% on seven datasets and 29.3%–33.0% on Traffic (still 3.5×–4×3.5\times\text{--}4\times above uniform).
    2. Out-of-sample predictability from history (Fig. 1b, Prop. 1, App. G.2): Because spectral concentration alone does not imply predictability (App. G.2, Line 1752), we fit linear ridge probes on validation windows and evaluate them on unseen test queries using only the 8 historical DCT features. Table 16 confirms positive median test correlations (rmedr_{med}) across all settings for the three analyzed backbones at L=H=96L=H=96—including rmed=0.393,0.285,and 0.240r_{med} = 0.393, 0.285, \text{and } 0.240 on Traffic—aligning with MORA improving all 24/24 settings on Traffic (Table 12).

We respectfully ask you to reassess the "does not pay off" conclusion and the Soundness: 1 / Rating: 1 scores in light of the full evidence above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.