acceptodds
Under review as a conference paper at ICLR 2027

Variance-Guided Time-Series Correction

Abstract

Post-hoc correction offers a lightweight paradigm to mitigate residual errors in frozen models. The central challenge is deciding which relationships finite calibration data can support, as similar residual energies often correspond to fundamentally different corrective values. Through multi-seed spectral diagnostics, we reveal that residual errors conflate persistent compensable patterns with stochastic training-run variations, motivating a correction rule that dynamically adapts its dependency structure. To resolve this, we formulate this choice as covariance-resolution selection, ranging from coordinate-wise scaling to channel, phase, and neighboring-mode interactions. Theoretically, our analysis characterizes linearly recoverable error, decomposes the benefit of expanding structure into additional recoverable signal and fitted-operator excess, and bounds corrective-gain estimation error uniformly over norm-bounded classes. The resulting framework fits closed-form candidates and selects their resolution, ridge multiplier, and fitting source using cross-fitted validation. Across an extensive 320-case evaluation matrix, our method improves settings with an average relative MSE reduction of , compared with for the best fixed resolution. It also improves outputs already enhanced by post-hoc adapters or training losses. These results establish structural resolution as an effective decision variable, showing that adapting complexity to calibration support enables reliable correction from a single frozen checkpoint.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Forecasting, Post-Hoc Correction, Spectral Analysis, Model-Agnostic Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AkNKOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The work introduces a post-hoc correction framework designed for frozen time-series forecasters. Structured linear residual correctors are fitted in the frequency domain, while their structural complexity, regularization strength, and fitting source are determined via validation, together with block-wise gating. A theoretical analysis delineates the recoverable error and the associated estimation trade-offs. Across eight datasets, ten backbone models, and four prediction horizons, the method yields improvements in 279 out of 320 configurations, achieving an average relative MSE reduction of 6.34%.

Critical strengths
  • The manuscript tackles a practically motivated question: how to enhance a frozen forecaster while leaving the backbone untouched.
  • The structured correction framework is logically coherent, and the theoretical treatment sheds light on the recoverable error as well as the tension between approximation and estimation.
  • The empirical study encompasses a range of datasets, architectures, forecasting horizons, and random seeds, thereby furnishing ample evidence of its practical value.
  • The method's usefulness is further corroborated by the comparative analyses, composition experiments, and ablation studies. The decision to report negative outcomes and metric trade-offs adds to the clarity of the exposition.
Critical weaknesses
  • The practical necessity of gating over and above ridge regularization remains insufficiently substantiated. A dedicated ablation study would help pinpoint the failure mode it is intended to address and quantify its actual contribution.
  • Benchmarking against alternative structures under matched regularization and selection budgets would further bolster the case for the proposed G2–G5 family.
  • The link between the multi-seed variance diagnostics and the deployed structural choices remains indirect.
Questions for authors

Please see weakness

3 comments
AuthorsOpenReview ↗

Reply to Reviewer AkNK (1/2) — Role of Block Gating and Matched-Budget Structural Comparisons

Dear Reviewer AkNK,

Thank you for your constructive review. In Part 1, we isolate the role of block gating beyond ridge regularization and evaluate alternative parameterizations under matched budgets (W1.1, W1.2). Part 2 details how offline multi-seed diagnostics connect to single-checkpoint deployment decisions (W1.3).

W1.1 — Why block gating is needed alongside ridge regularization

In spectral post-hoc correction, a single ridge multiplier faces an inherent trade-off across frequencies. Matrix classes G3–G5(3) tune one global penalty λ\lambda shared across all modes via aggregate validation loss (Alg. 1; G2 has no ridge grid). When strong low-frequency improvements dominate that aggregate score, the chosen λ\lambda can remain too small for volatile high-frequency modes, letting local overfit slip through. Block gating resolves this mismatch by zeroing out individual harmful modes without over-shrinking informative ones.

To separate gating's contribution from ridge tuning and model reselection, we ran new ablations on frozen ETT and Weather checkpoints:

  1. Pruning mode-level overfit at fixed ridge: Holding class (G3–G5) and selected λ\lambda identical across 144 candidate pairs on ETT, gating reduces test MSE in 18/24 (G3), 16/24 (G4), and 16/24 (G5) training-fitted operators—which account for 582/960 of our benchmark selections (Table 36)—by removing mode-specific train-to-validation overfit (versus 5–6/24 on validation fits).
  2. End-to-end protection on ETT: On ETTh2/iTransformer/H96 (baseline 0.3091), removing the gate from the selected G2 operator degrades test MSE from 0.3023 to 0.3899, and full ungated reselection only recovers to 0.3054. Across the ETT panel, gating improves all iTransformer means over ungated reselection, with minor trade-offs on DLinear.
  3. Staying non-intrusive on clean multivariate signals: Conversely, on Weather/iTransformer/H96, global ridge alone already stabilizes T:G5(3) (+8.23% ungated vs. +8.16% gated). Gating thus acts as a conditional safeguard—pruning noisy modes when one global λ\lambda is too coarse, while remaining transparent when global regularization suffices.

W1.2 — Evaluating alternative structures under matched budgets

A natural question is whether our gains stem from the aligned G2–G5 dependency hierarchy itself, rather than simply having a strong regularization and validation pipeline. In the paper, Tables 3 and 15 already compared published post-hoc adapters on identical splits, while Fig. 5a tested matched-count spectral supports. To evaluate alternative parameterizations under identical ridge grids, cross-fitting folds, block gates, and selection budgets, we added a 1,080-run synthetic benchmark (comparing aligned local supports against equal-count permuted supports) alongside real-data G5 variants:

Benchmark (Matched Budget)Proposed G2–G5Alternative Structure
Synthetic control (New, 1,080 runs)0.7962 risk (N=32); 0.5816 mean0.8998 (permuted local); 0.6051 mean
Post-hoc adapters (Paper Tables 3, 15)+7.61% mean (77/80 wins, H=96)Socket+Plug +6.41%, δ\delta-Adapter +3.17%, UEC-STD +2.16%
Spectral supports (Paper Fig. 5a)Wins 72/74 (+4.09 pp) & 59/60 (+5.71 pp)Permuted coords & random channel supports

On Weather/iTransformer/H96 (where validation actively selects G5(3) across all seeds), we further tested permuted frequency neighborhoods and a pre-fixed rank-10 G5 candidate under the exact same 64 ridge values, two fitting sources, and block gates (3-seed means):

Weather / iTransformer (H=96)Mean MSE ↓Retained GainMean Coeffs
Uncorrected baseline0.17321——
Dense G5(3)0.15907100.0%259,308
Permuted-frequency G5(3)0.1593997.7%259,308
Fixed rank-10 G5(3)0.1628773.1%82,320
Selector (w/ rank-10 G5)0.1617081.4%41,846 (−83.9%)

True frequency adjacency outperforms permuted neighborhoods across all seeds (by a modest 0.20%, since both retain the aligned within-mode G4 channel–phase block driving most of the ~8% gain). Meanwhile, replacing dense G5 with a fixed rank-10 G5 candidate in the selector picks rank-10 G5 once and G3 twice across the three seeds—retaining 81.4% of dense gain while reducing average nominal representation coefficients by 83.9%.

AuthorsOpenReview ↗

Reply to Reviewer AkNK (2/2) — Bridging Multi-Seed Diagnostics to Single-Checkpoint Deployment

W1.3 — Linking multi-seed diagnostics to single-checkpoint deployment

Why do we analyze multi-seed runs in Section 3 when practical deployment provides only one frozen model? In real-world deployment, retraining a forecaster across multiple random seeds solely to measure forecast variance is computationally prohibitive; we are given only a single frozen checkpoint. Our framework bridges offline diagnostics and single-checkpoint deployment through a direct two-stage logic:

  1. Offline multi-seed diagnostics identify why residual magnitude fails (Paper Sec. 3): Training multiple seeds offline serves as an analytical lens to uncover forecast residual ambiguity—showing that modes with comparable residual energy can hide starkly different run-to-run instability (29% median seed variance in weak-skill bands vs. 0.2% in high-skill bands across 65 cells, Fig. 1 & Table 5). This explains why correction rules based on residual magnitude alone fail: large residuals often reflect high training-run volatility rather than recoverable systematic bias.
  2. Single-checkpoint risk estimation replaces seed variance at deployment (Theorems 1–3 & Corollary 4): Because cross-seed variance is unobservable when only one checkpoint is given, our theory establishes how to assess structural risk from that single model alone (detailed step-by-step in our Reply to Reviewer JzC7, Part 1, W3). Theorem 1 (Eqs. 8–9) shows how latent seed volatility leaks into a single checkpoint's empirical forecast–error cross-moment alongside true bias, and why dampening seed variance can inflate mean prediction error. Theorems 2–3 and Corollary 4 (Eqs. 11–14) then prove that realized joint moments and cross-fitted validation gain on that one checkpoint directly determine whether expanding from coordinate scaling (G2) to cross-channel or neighboring-mode couplings (G3–G5) captures genuine signal beyond finite-sample estimation error.
  3. Empirical alignment across 960 real runs and 1,080 synthetic runs (Paper & New): Without ever accessing another seed, our single-checkpoint selector reproduces the exact structural choices that multi-seed diagnostics would prescribe. Across the paper's 960 single-seed runs (Fig. 4b), it contracts to coordinate damping (G2) in 435/480 (90.6%) runs on high-volatility ETT datasets (weak-band seed variance 34%–56%, Table 5), while expanding to G3–G5(3) in 432/480 (90.0%) runs on lower-volatility multivariate datasets (Weather, Solar, ECL, Traffic; 9%–21%). Consistently, in our new 1,080-run synthetic benchmark, single-sample selection recovers the population-best fitted candidate in 943/960 signal-bearing runs (mean regret 2.71×10−42.71\times 10^{-4}).

Thank you for the additional experiments, which address my first two concerns. The link between multi-seed variance diagnostics and deployment-time structure selection remains indirect. I maintain my rating of Weak Accept.

Reviewer_ggb3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a post-hoc correction framework that applies a linear corrector to the outputs of a frozen time-series forecasting backbone. Its main contribution is a covariance-resolution hierarchy for selecting a checkpoint-specific structure, spanning coordinate-wise scaling, cross-channel interactions, real-imaginary coupling, and neighboring-mode interactions. Through multi-seed spectral analysis, it examines the relationship between forecast skill and training-run variation. Its theoretical analysis distinguishes residual energy from recoverable error and characterizes the approximation-estimation trade-off associated with expanding the correction structure. The framework uses ridge regularization, dimension guards, validation-based block gating, and cross-fitted validation to select the covariance resolution and fitting source. Deployment requires a single checkpoint and a fixed corrector. Across 320 combinations, the method improves 279 settings and achieves an average relative test-MSE reduction of 6.34%.

Critical strengths
  1. The paper introduces forecast residual ambiguity to explain how similar residual energies can conceal different mixtures of compensable bias and stochastic variation from training runs. Its multi-seed spectral analysis shows higher normalized cross-seed variance in weak-skill modes. This supports considering forecast-error dependencies rather than residual energy alone when assessing corrective value, since expanding the dependency structure can fit spurious correlations and degrade performance
  2. The paper organizes correction-operator complexity into a covariance-resolution hierarchy and establishes a selection principle: expanding the structure is beneficial when the additional recoverable error exceeds the added fitted-operator excess. It implements this principle through cross-fitted validation to select a resolution supported by the available calibration data.
  3. Experiments show additional test-MSE reductions when the method is composed with existing post-hoc adapters and spectral training objectives. These results support the complementary value of covariance-resolution selection alongside existing methods.
Critical weaknesses
  1. Despite validation-based block gating and fitting-source selection, Table 1 and Table 25 show systematic degradation on ETTh1 in long-horizon forecasting. In particular, at H=720H=720, test MSE increases for all backbones, with an average relative MSE reduction of -8.08%. This indicates that corrections learned from calibration data may not transfer reliably to deployment and warrants further analysis of distribution shift.
  2. For high-dimensional multivariate time series, wider covariance resolutions require substantial coefficient storage and additional computation, which may limit the practical benefits of lightweight post-hoc correction. For example, on Traffic, G5(3)G_5(3) requires approximately 436.9M coefficients and 1666.7 MiB of dense FP32 storage. Appendix F.2 and Table 38 describe streamed accumulation and report measured fitting time and inference latency, but the relative overhead compared with the backbone’s own storage and latency remains unclear. A direct comparison of the performance gains and additional costs, particularly for lightweight backbones, would help clarify this trade-off.
Questions for authors
  1. Could the authors further investigate the test performance degradation on ETTh1 at H=336H=336 and H=720H=720 despite validation-based block gating and fitting-source selection? In particular, it would be helpful to examine how the relationship between forecasts and residual errors changes from validation to test, and whether these changes explain why corrective gain decreases or becomes negative.
  2. Could the correction costs reported in Appendix F.2 and Table 38 be directly compared with the parameter storage and inference latency of lightweight backbones such as DLinear and FITS? It would also be useful to discuss whether low-rank or channel-group constraints reduce these costs while preserving most of the test-MSE improvement.
  3. Since adjacent validation windows across the two cross-fitting folds share up to H−1H-1 target time points, could the authors examine whether introducing a temporal gap between the folds affects the selected covariance resolution and the resulting test-MSE improvement?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer ggb3 — ETTh1 Shift Analysis, Target-Disjoint Folds, and Lightweight Overhead

Dear Reviewer ggb3,

Thank you for your constructive review. Part 1 examines ETTh1's long-horizon degradation and target-disjoint validation gaps (W1.1, Q1, Q3); Part 2 compares overhead against lightweight backbones and evaluates compact alternatives (W1.2, Q2).

W1.1 & Q1 — Why ETTh1 degrades at H=336 and H=720

On ETTh1, our selector already contracts to coordinate-wise G2 in all 120/120 runs (Fig. 4b), ruling out wide-matrix overfitting. Why do validation-selected G2 fits still hurt test MSE? Two factors interact at long horizons:

  1. Few independent validation horizons: Because ETTh1 is hourly, H=720 spans 30 days—leaving <4 non-overlapping horizons in the 4-month validation split (vs. 7.5 days on 15-min ETTm1/m2, where we gain +2.78% and +2.06%, Tables 1 & 8). Dense stride-1 overlap within that single season creates high local consistency that keeps G2 gates open.
  2. Seasonal moment shift: Under ETTh1's 12/4/4-month split (App. C.2), training spans a full year, while validation and test occupy distinct 4-month seasons where long-horizon error correlations shift or flip sign (App. A.6; δ\delta-Adapter likewise degrades 9/10 ETTh1 backbones even at H=96, Table 16).

To pinpoint how forecast–residual relationships change, we replayed 24 frozen checkpoints (DLinear and iTransformer on ETTh1 H=96/336/720 and ETTh2 H=96). Writing residual e=y−y^e=y-\hat{y} and correction d=y~−y^d=\tilde{y}-\hat{y} gives ΔMSE=E−2A\Delta\mathrm{MSE}=E-2A, with correction energy E=E[d2]E=\mathbb{E}[d^2] and alignment A=E[ed]A=\mathbb{E}[ed] (ΔMSE>0\Delta\mathrm{MSE}>0 means higher error). Evaluating each fixed deployed operator on validation vs. test (3-seed means), alongside opposite-fold validation gains:

ETTh1 SettingCross-Fit Val GainVal AAVal EETest AATest EETest ΔMSE\Delta\mathrm{MSE} ↓
DLinear / H=336+9.93%0.13590.13580.00080.0354+0.0339
DLinear / H=720+9.92%0.14810.1478−0.01070.0337+0.0551
iTransformer / H=336+20.31%0.29800.29780.03390.0791+0.0114
iTransformer / H=720+25.54%0.42290.42250.03660.0949+0.0216

Even on held-out opposite folds, G2 achieves +9.9% to +25.5% MSE reductions. On test, however, DLinear loses nearly all alignment at H=336 and reverses direction at H=720 (A<0A<0), while iTransformer's alignment attenuates below E/2E/2 (2A<E2A<E). Scaling target-disjoint validation size 8×8\times within that 4-month split (1/8 to full across all 24 runs) leaves mean relative error flat (−1.43% to −1.20%), confirming that cross-season distribution shift—rather than within-season window count—drives the test degradation.

Q3 — Temporal gap between cross-fitting validation folds

Stride-1 windows across the validation midpoint share up to H−1H-1 target steps, which can make cross-fitting optimistic at long horizons. To separate target overlap from sample-count reduction, we refitted all 24 checkpoints under Original, Purged (an (H−1)(H-1)-step midpoint gap making fold targets disjoint), and Outer equal-count (dropping the same window count from split ends while keeping midpoint overlap):

Setting (3-seed mean MSE ↓)BaselineOriginalPurged (disjoint)Outer equal-count
ETTh1 DLinear / H=960.39630.39760.39970.3979
ETTh1 DLinear / H=3360.48700.52090.51650.5241
ETTh1 DLinear / H=7200.51320.56840.55300.7851
ETTh1 iTransformer / H=960.38980.38130.38270.3806
ETTh1 iTransformer / H=3360.48790.49930.49210.5123
ETTh1 iTransformer / H=7200.51190.53350.50780.5670
ETTh2 DLinear / H=960.34580.3117 (V:G3)0.3427 (V:G2)0.3037 (V:G3)
ETTh2 iTransformer / H=960.30910.30230.30230.3013
  1. ETTh1 long horizons: Class and source stay unchanged in 21/24 runs. Purging improves all four ETTh1 long-horizon cases (tightening V:G2 gating and beating baseline on iTransformer/H720), whereas Outer deletion worsens them—ruling out sample count alone and showing midpoint overlap inflated optimism, though seasonal shift leaves 3/4 cases above baseline.
  2. Shorter-split trade-off: Dropping H−1H-1 central windows shortens fold span; on ETTh2/DLinear/H96 (the only 3/24 runs changing class), Purged contracts from V:G3 (0.3117, +9.8%) to V:G2 (0.3427, +0.9%), while Outer keeps V:G3 (0.3037). We will add both diagnostics to Appendix C.6.
AuthorsOpenReview ↗

Reply to Reviewer ggb3 (2/2) — Overhead vs. Lightweight Backbones and Compact Alternatives

W1.2 & Q2a — Direct cost–gain comparison with lightweight backbones (DLinear & FITS)

When a host forecaster is itself ultra-lightweight—such as DLinear (18,624 FP32 parameters) or FITS (1,984 complex64 parameters)—a dense high-dimensional corrector changes the deployment footprint substantially. To make this trade-off explicit beyond Table 38, we ran new matched-hardware profiling on Traffic (H=96, C=862, batch size 1, 200 CUDA-synchronized runs on a single A100 GPU), pairing forward latency and weight/coefficient storage directly with test-MSE gains and selection shares from Tables 13–14 and Fig. 4b:

ComponentLatency (ms)StorageGain & Share (Tables 13–14)
DLinear forward0.2740.071 MiBBaseline MSE: 0.6484
FITS forward0.3430.015 MiBBaseline MSE: 0.6540
+G2 correction+0.1040.161 MiB0 / 120 runs
+G3 correction+0.334138.89 MiB63 / 120 runs (e.g., +12.67% on DUET)
+G4 correction+0.475555.56 MiB28 / 120 runs (e.g., +5.66% on TimeMixer)
+G5(3) dense+1.3081,666.68 MiB29 / 120 runs; +14.32% (DLinear), +15.83% (FITS)

Two practical conclusions emerge from this comparison:

  1. Accuracy–cost trade-off on lightweight hosts: Dense G5(3) adds 4.77×4.77\times DLinear's and 3.81×3.81\times FITS's forward time (~1.6 ms total per window) alongside 1.67 GiB of deployed coefficients (note that Appendix F.2's streaming reduces fitting workspace rather than deployed coefficient storage). In return, it unlocks 14.3%–15.8% test-MSE reductions by capturing cross-sensor and neighboring-mode error couplings that channel-independent linear/spectral backbones omit by design.
  2. Adaptive selection avoids defaulting to G5(3): Across all 120 Traffic runs, our selector picks dense G5(3) in only 29/120 runs (primarily for simpler backbones at shorter horizons) and chooses lighter V:G3 (+0.334 ms, 12×12\times smaller storage) in 63/120 runs—including 9/10 backbones at H=720 (Table 14).

Q2b — Preserving gains at lower cost: candidate caps, low-rank maps, and channel groups

When memory or latency budgets rule out dense G5(3) on high-channel series, most of the MSE reduction can be retained at a fraction of the storage and compute cost through three straightforward options:

  1. Cap candidate width at G4 (Paper Table 24): Excluding G5(3) from the candidate pool cuts peak coefficient storage by 3×3\times (to 555.6 MiB) and Traffic correction latency from 1.308 ms to 0.475 ms, while preserving 86.9% of our full benchmark gain across all 320 settings (+5.51% mean MSE reduction vs. +6.34%, improving 280/320 cells).
  2. Low-rank G5 candidates (New experiment): To test whether low-rank constraints preserve cross-mode gains without adding rank-tuning overhead, we replaced dense G5(3) with a pre-fixed rank-10 G5 candidate on Weather/iTransformer/H96 under the exact same 64 ridge values, two fitting sources, and block gates (3-seed means):
VariantMean MSE ↓Retained GainMean Coeffs
Uncorrected baseline0.17321——
Dense G5(3)0.15907100.0%259,308
Fixed rank-10 G5(3)0.1628773.1%82,320
Selector (w/ rank-10 G5)0.1617081.4%41,846 (−83.9%)

Allowing the selector to choose among G2–G4 and rank-10 G5 picks rank-10 G5 once and G3 twice across the three seeds—retaining more gain than fixed rank-10 G5 (81.4% vs. 73.1%) while cutting average representation parameters by 83.9% (evaluated via reconstructed maps here; factorized storage and inference scale as O(MCr)\mathcal{O}(MCr) for rank r≪Cr \ll C). 3. Channel-group block-diagonals: Partitioning CC variables into BB correlated groups defines a linear subspace between coordinate-wise G2 and full-channel G3/G4. Theorem 3 and our closed-form ridge solver apply directly per block, reducing both storage and O(MC2)\mathcal{O}(MC^2) multiply-adds by B×B\times. We will incorporate the matched-GPU benchmark, the low-rank results, and this compact-subspace discussion into Appendix F.2.

AuthorsOpenReview ↗

kindly remind

Dear Reviewer ggb3,

Thank you again for your thoughtful feedback, which helped us improve our paper. As the discussion period draws to a close, we hope you have a chance to review our response and additional results. If you feel they address your concerns, we would be grateful for your reconsideration of the paper. We would be happy to clarify any remaining points.

Best regards,

Authors

I thank the authors for the additional experiments and clarifications. Considering the paper's contributions and experimental results, I will maintain my original score.

Reviewer_JzC7OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies post-hoc correction of time series models. It proposes a new method that fits a correcting adapter using a supervised calibration dataset on top of a frozen backbone model.

Critical strengths

The idea of compensating for variance across seeds itself seems interesting and novel in this context.

The empirical results presented in the experiments section do seem to show significant improvements over the baseline across all tests.

There is a plausible comparison to prior work in Table 3, although this table does seem to show only composability with prior work, rather than actually showing that the proposed method improves upon or beats prior work.

Critical weaknesses

The paper is very densely written and its motivation is hard to follow. E.g. there are a lot of sentences like "We answer this question by treating the allowed dependency structure, or covariance resolution, as a selection variable. A weighted projection identity characterizes the error recoverable within each structure. Expanding a nested structure adds recoverable directions, while a uniform moment perturbation bound describes the estimation error relevant to those directions." Paragraphs like this just list a bunch of things that are true about the work without explaining why or connecting results to other results.

It's not really clear why this problem in particular is interesting or what we are supposed to learn from it: of course one can improve the performance of a frozen model by training an adapter! The fact that there is an improvement here by itself should not be surprising, right?

Similarly, it's not clear why the reader should care about any of the theoretical results. None of the theorems are compared against theory from prior work. There should be some text around the theorems that unpacks for the reader what the theorem says and why it is important. For example, Theorem 1 is a straightforward result with a half-page proof that just involves expanding out some expected values: why was it important to draw attention to this result in particular?

Questions for authors

What is the reader expected to take away from this result? Is there anything here that the broader machine learning community could learn from, beyond just people working on post-hoc correction of time series models?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer JzC7 (1/2) — Why Post-Hoc Correction is Non-Trivial & Unified Theoretical Framework

Dear Reviewer JzC7,

Thank you for your candid feedback. Part 1 covers why post-hoc correction is non-trivial and our single-seed theory; Part 2 covers broader ML takeaways and clear prose.

W2 — Why post-hoc correction is non-trivial & standalone wins

Training an adapter on a frozen forecaster does not "of course" improve test error. Unlike cross-task transfer, every backbone here was already trained on the training split and early-stopped on validation under MSE (Sec. 5.1, App. C.2). Automatic gain provably fails here: combining Theorem 3 / Corollary 4 (Eq. 14) with App. A.4 shows that a fitted adapter strictly worsens MSE whenever finite-sample estimation excess exceeds leftover recoverable bias (Geman et al., 1992; James & Stein, 1961)—and when forecast–error cross-moment Γ=0\Gamma=0 (App. A.4, p=1/2p=1/2), recoverable error is zero despite unit residual MSE, so any nonzero fit hurts risk. Preventing negative transfer is why safe post-hoc correction is an active frontier (Liang et al., 2026; Hoang Nguyen et al., 2026; Xie et al., 2025), as Reviewers ggb3 and AkNK noted:

  1. Naive and published adapters often hurt test MSE: Unregularized linear maps (G3–G∞) fit up to 99.9% in-sample yet worsen cross-fitted MSE by −47.3% to −2,964.6% (Table 37; even on the full training split, wider maps lose on 39/40 cells at C≤7C \le 7 and 14/14 at C≥321C \ge 321, App. E.2). Even with norm bounds and early stopping, δ\delta-Adapter worsens 18/80 cells at H=96 (9/10 on ETTh1, Tables 15–16), UEC-STD gives +0.53% median gain, and FreDF / PS Loss average −11.72% / −0.80%.
  2. Ridge-tuned fixed structures still fail (Table 4, App. E.4): Under the same 64-ridge search, gates, and source selection, fixed G3/G4/G5(3) worsen mean MSE at H=720 (−4.38% / −2.67% / −1.22%) vs. +4.64% for ours, while scalar G2 averages just +1.42% overall (vs. +6.34% across 320 cells) and misses multivariate signal on Solar (+2.66% vs. +21.35%).
  3. Standalone superiority in Table 3 (clarifying Strengths): The "Alone vs. plain" row of Table 3 (and Table 15) benchmarks standalone adapters across all 80 cells at H=96:
Standalone MethodMean ↑Median ↑WinsWorst ↑
Ours (standalone)+7.61%+3.91%77/80−0.73%
Socket+Plug+6.41%+3.23%75/80−1.53%
δ\delta-Adapter+3.17%+2.46%62/80−3.03%
UEC-STD+2.16%+0.53%67/80−0.52%

Composability (+10.34%, 80/80 wins after Socket+Plug) is an extra gain.

W3 — Gap filled by our theory & unified single-seed system

Our theory solves a concrete deployment gap left open by prior work: given only one frozen checkpoint and small calibration data, how complex an error-dependency structure (G2–G5) can be fitted without overfitting training noise?

  1. Gap vs. prior theory (Sec. 2): Bias–variance and ensemble analyses (Geman et al., 1992; Krogh & Vedelsby, 1994) require multiple training runs unavailable at deployment. Conversely, Wiener, James–Stein, and reconciliation rules (Wickramasuriya et al., 2019) assume a fixed structure (coordinate scaling or known hierarchy), giving no rule to select which cross-channel, phase, or mode couplings finite data can support.
  2. Why Theorem 1 matters (exposing the single-seed trap): Multi-seed runs show equal-energy errors mix bias with training noise (29% vs. 0.2% seed variance across skill bands, Fig. 1 & Table 5). Despite its short proof, Theorem 1 (Eqs. 8–9) shows why single-seed fitting gets fooled: run-to-run noise Σξ\Sigma_\xi leaks directly into a single checkpoint's forecast–error cross-moment, mimicking true bias. Also, canceling seed variance (K→IK \to I) opposes ridge (K→0K \to 0) and can bias mean forecasts—why correction cut seed variance in 8/8 diagnostic settings yet worsened test MSE in 5/8 (DLinear/ETTh1/H720: variance −64.7%, MSE +10.7%).
  3. How Theorems 2–3 estimate risk from one seed: Without extra seeds, Theorem 3 & Corollary 4 use single-checkpoint moments to prove widening (G2 →\to G5) lowers MSE iff extra cross-channel/mode signal beats added finite-sample fitting excess (Eq. 14), while Theorem 2 (Eq. 11) shows narrower subspaces project out off-diagonal moment noise on small splits. We will unpack this intuition around each theorem.
AuthorsOpenReview ↗

Reply to Reviewer JzC7 (2/2) — Broader ML Takeaways & Presentation Improvements

Q1 — Takeaways for the broader machine learning community

Beyond time-series forecasting, calibrating or probing a frozen pretrained model on a small labeled split (e.g., linear probes, post-hoc calibration heads, or lightweight output adapters) faces the exact same small-sample structural trade-off. Our study establishes three general lessons:

  1. Error magnitude and parameter count do not determine post-hoc recoverability: Two error modes—or two frozen models—with identical residual MSE can carry zero or substantial recoverable signal depending on prediction–residual cross-moments (Figure 1a, Appendix A.4). Even at identical parameter counts, aligning an adapter's allowed interactions with domain structure beats permuted coordinates on 72/74 real cells (+4.09 pp median) and random channel supports on 59/60 cells (+5.71 pp median, Figure 5a & Appendix E.2; likewise achieving 0.7962 vs. 0.8998 population risk at N=32 in a 1,080-run matched-budget synthetic control).
  2. Damping run-to-run variance is not equivalent to lowering prediction error: Stochastic optimization leaves substantial seed-to-seed volatility in weak modes of frozen neural networks (Figure 1b, Table 5). Yet aggressively suppressing volatile outputs perturbs the mean prediction and can worsen test MSE even while variance drops (Theorem 1)—a practical caution for post-hoc shrinkage and test-time output calibration.
  3. Treat interaction structure—not just scalar weight decay—as a selection variable: On finite calibration splits (where held-out operator disagreement scales as ∼n−1\sim n^{-1}, Figure 5b & Table 33), tuning a scalar ridge penalty on one fixed dense head cannot stop noisy cross-feature terms from degrading transfer. Selecting which dependency resolution (G2–G5) the calibration sample can support prevents both small-sample overfitting (H=720, Table 4) and narrow-adapter underfitting (Solar, Appendix E.4), while adding +4.4% to +12.1% secondary MSE reductions on top of existing post-hoc adapters and training losses (Table 3).

W1 — Streamlining dense prose & connecting ideas clearly

We appreciate your concrete critique of our writing density. Packing diagnostic, theoretical, and algorithmic terms into compact summary sentences made the motivation harder to follow than necessary. In the revision, we will make three targeted improvements across the manuscript:

  1. Motivate before formalizing: Reorganize the Abstract, Section 1, and Section 3 around a clear causal thread (why residual magnitude is ambiguous across seeds →\to why single-checkpoint deployment cannot observe seed variance →\to how cross-fitted validation selects structural resolution), explaining why simpler approaches fail before naming technical components.
  2. Unpack every formal result in plain English: Add intuitive lead-ins and takeaway remarks around Theorems 1–3 and Corollary 4 so readers immediately see what practical misconception each result prevents without reconstructing links across sections.
  3. Replace dense term-listing paragraphs: As a concrete example, we will rewrite the Section 1 passage you flagged (Lines 065–069) as follows:

Original (Lines 065–069): "We answer this question by treating the allowed dependency structure, or covariance resolution, as a selection variable. A weighted projection identity characterizes the error recoverable within each structure. Expanding a nested structure adds recoverable directions, while a uniform moment-perturbation bound describes the estimation error relevant to those directions."

Revised: "Because deployment provides only one frozen checkpoint, residual size alone cannot tell whether fitting a wide cross-channel adapter will fix systematic bias or overfit training noise. We therefore use cross-fitted validation to select the adapter's dependency structure itself—from per-channel scaling (G2) to cross-channel and neighboring-frequency maps (G3–G5). Our analysis shows why this works: wider structures capture extra error signal (Theorem 3), but also expose small calibration splits to larger moment-estimation noise (Theorem 2), making expansion beneficial only when newly captured signal outweighs finite-sample fitting error (Corollary 4)."

Dear Reviewer JzC7,

Thank you again for your thoughtful feedback, which helped us improve our paper. As the discussion period draws to a close, we hope you have a chance to review our response and additional results. If you feel they address your concerns, we would be grateful for your reconsideration of the paper. We would be happy to clarify any remaining points.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.