acceptodds
Under review as a conference paper at ICLR 2027

MATE: Delayed Credit Assignment for Test-Time Adaptation in Time-Series Forecasting

Abstract

Distribution shifts can undermine time-series forecasting accuracy after deployment. Test-time adaptation for time-series forecasting (TSF-TTA) can mitigate these errors, but a single correction mechanism can struggle with complex and abrupt shifts. Its effectiveness also varies across forecast leads and variable channels. Delayed target availability further complicates learning where and how strongly to apply each correction. We introduce MATE (Matured-Feedback Adaptive Test-Time Error Correction) to address these challenges. MATE separates correction generation from credit allocation under a full-window feedback protocol. It keeps the pretrained backbone frozen and constructs complementary corrections through a structured residual dictionary. As feedback matures, MATE accumulates credit statistics to guide lead–channel credit weights and regulate each correction’s contribution. It also updates input and output calibrators using matured historical forecast errors to capture distribution shifts. Comparisons across datasets, forecasting backbones, and shift scenarios demonstrate broad forecasting gains. Mechanism ablations further support the effectiveness of position-specific correction control driven by matured feedback.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, test-time adaptation, delayed feedback, residual correction, credit assignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2UJdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MATE, a test-time adaptation framework for time-series forecasting under delayed feedback. MATE keeps the pretrained forecasting backbone frozen and constructs five residual correction candidates using local and global output structures, historical forecast versions, input calibration, and recent observations. The proposed credit controller uses accumulated second-order statistics from fully matured historical predictions to estimate correction weights. To account for differences across forecast horizons and variables, it combines lead-specific and channel-specific weights regularized toward a global reference.

The method is evaluated on six datasets, four forecasting backbones, and five distribution-shift scenarios. MATE achieves the lowest MSE in 396 out of 460 configurations (86.1%), with additional ablation studies examining the contribution of credit allocation granularity and individual residual sources.

Critical strengths
  1. Consistent empirical improvements across diverse settings.

The paper demonstrates consistent forecasting improvements across multiple datasets, backbone architectures, prediction horizons, and distribution-shift scenarios. MATE achieves the lowest average MSE in all five scenarios and outperforms the compared methods in a large majority of configurations. The broad evaluation provides reasonably strong empirical evidence that the approach is effective across different forecasting settings rather than being tailored to a particular backbone or dataset.

  1. A reasonable and well-structured approach to correction allocation under delayed feedback.

Separating residual generation from credit allocation is a sensible design choice. The use of accumulated second-order statistics allows the controller to account for interactions among correction candidates, while the global reference provides regularization for lead- and channel-specific weights when feedback is limited. The full-window feedback protocol also ensures that adaptation relies only on available historical targets. Overall, the proposed framework is technically coherent and addresses a relevant challenge in time-series test-time adaptation.

Critical weaknesses
  1. The significance of the proposed methodological contribution is not fully established.

While the overall framework is technically reasonable, its main components are relatively straightforward, combining multiple residual corrections, regularized quadratic optimization, and additive lead–channel weight decomposition. The selection of the five residual sources is also primarily motivated by intuition, without a systematic analysis of their complementarity or necessity.

More importantly, the ablation results raise questions about the incremental contribution of the proposed lead–channel allocation, which appears to be a central methodological contribution of the paper. In Table 4, the full method achieves only a marginal improvement over channel-only allocation (0.2258 vs. 0.2263 in average MSE, approximately 0.22%). Although the full method performs best on average, the results do not clearly establish whether the additional lead-dependent allocation provides a sufficiently meaningful advantage over simpler alternatives.

Thus, while the empirical results are encouraging, it remains unclear to what extent the performance gains stem from the proposed credit allocation mechanism itself, rather than the availability of multiple complementary correction candidates. This makes the significance of the methodological contribution somewhat difficult to assess.

Questions for authors
  1. [Clarification / Additional evidence] The improvement of full lead–channel allocation over channel-only allocation is relatively small in Table 4. Could the authors elaborate on when lead-dependent allocation provides meaningful additional benefits? Are these benefits consistent across datasets, backbones, and forecasting horizons?

  2. [Clarification] What motivates the particular selection of the five residual correction sources? Is there evidence that they provide sufficiently distinct or complementary correction directions? A clearer understanding of their interactions would help assess the importance of the proposed residual dictionary.

  3. [Clarification] The credit statistics accumulate historical evidence while the residual-generating calibrators continue to evolve. How does the method account for the potential mismatch between historical residual candidates and those generated by updated calibrators? Additionally, how does accumulating statistics without explicit forgetting affect adaptation to abrupt distribution shifts?

3 comments
AuthorsOpenReview ↗

Thank you for pointing out that the original manuscript needed to distinguish more clearly between the roles of credit allocation and residual combinations. In response, we added credit controls with fixed candidates, experiments with individual and incrementally combined sources, and analyses of candidate evolution and forgetting. We also revised the associated discussion. We show the revised material in blue text to help you locate the changes. Please see the revision for details.

Thank you for pointing out that the original manuscript needed to distinguish more clearly between the roles of credit allocation and residual combinations. In response, we added credit controls with fixed candidates, experiments with individual and incrementally combined sources, and analyses of candidate evolution and forgetting. We also revised the associated discussion. We show the revised material in blue text to help you locate the changes. Please see the revision for details.

Direct Questions

Q1. When does lead-dependent credit help?

(1) Which configurations benefit. We hold the residual candidates fixed and use Channel-only as the control to assess the value of distinguishing forecast leads in addition to channels. Joint allocation reduces mean paired MSE by 0.27%. The mean gain is positive for all four backbones. It is largest on ETTm1, while the two methods are close on Exchange. See Section 5.3, Table 4, and Appendix C.9, Table C14 of the revision.

(2) Forecast horizons and the correspondence between leads and weights. Across the same 80 dataset–backbone–scenario combinations at each horizon, joint allocation yields an average gain at all four horizons. The gain is larger at shorter horizons, as shown below. With fixed candidates, averaging the lead component while retaining the channel component raises MSE. This also supports varying correction strength by lead. See Appendix C.5, Table C11.

Supplementary table. MSE reduction of Full MATE relative to Channel-only at each horizon

Forecast horizonMean paired MSE reductionConfigurations won by Full MATE
960.46%73/80
1920.36%76/80
3360.26%71/80
7200.17%58/80

Q2. Why choose these five residual sources, and do they complement one another?

(1) Rationale for the five sources. Local uses relationships among nearby forecast leads and across channels. Global uses smooth structure across the forecast window. Vintage combines earlier forecasts for the same target from different origins. Input captures the effect of input calibration on the forecast, and Context uses the persistence of recent observations. Section 4.2 describes how each source is constructed.

(2) Standalone effects. We allow each of the five residual sources to adapt independently. Each source yields lower mean MSE than the frozen backbone in all four scenarios. Relative to Vintage-only, the single source with the lowest overall mean MSE, Full MATE reduces mean paired MSE by 3.45% and has lower error in 366 of 368 configurations. See Section 5.3, Table 5 for the single-source comparison.

(3) Gains from combining sources. We fix the residual candidates and re-estimate the credit weights after each addition. In the sequence tested in Table 5, each new source lowers mean MSE in all four scenarios. For example, adding Vintage still improves the forecast when Local and Global are already present. This indicates that overlapping historical forecasts provide information that complements output structure. See Section 5.3, Table 5 for MSE and Appendix C.6, Table C13 for MAE.

Table 5: Mean MSE for independent single-source adaptation and fixed candidate subsets. Each scenario has 92 configurations and three test seeds per configuration. Bold marks the lowest displayed MSE within each adaptation group for each scenario.

ExperimentSettingNaturalLevelDynamicProfile
ReferenceFrozen0.32280.33370.33510.3296
Independent adaptationLocal-only0.31790.32920.32990.3249
Global-only0.31610.32790.32800.3235
Vintage-only0.31610.32730.32820.3228
Input-only0.31730.32890.32900.3241
Context-only0.32070.33140.33440.3286
Fixed candidatesLocal0.31840.32960.33040.3253
Local + Global0.31370.32570.32550.3213
Local + Global + Vintage0.30840.32060.32010.3158
Local + Global + Vintage + Input0.30710.32000.31870.3150
Full MATE (all five sources)0.30510.31770.31800.3141

Q3. Do historical candidates differ from current candidates, and how does accumulated credit affect adaptation after an abrupt shift?

(1) How historical candidates enter credit updates. When full-window feedback arrives, MATE updates the credit statistics with the candidates saved at prediction time and the corresponding base forecast errors. As the calibrators change, the accumulated statistics still contain evidence from earlier candidates. We recalculated Local, Global, and Input on the same historical inputs using the current calibrators and observed changes in the candidates. See Sections 4.1 and 4.4 and Appendix C.7.

(2) How forgetting affects adaptation after a shift. We compared cumulative credit with two forgetting rates. Faster forgetting increased full-stream error. In Level, slower forgetting reduced mean MSE by 0.452% over the first 256 steps after the first complete post-shift feedback arrived. The gain was close to zero when the observation window was extended to 1,024 steps. We retain cumulative credit in the main experiments based on the full-stream results. See Appendix C.7 and Table C15.

Table C15: Full-stream error reductions of credit forgetting relative to cumulative credit. Positive values favor forgetting.

Half-life (matured origins)MSE reductionMAE reductionMSE wins
512−0.273%−0.077%74/368
2,048+0.003%+0.029%176/368

Thank you for the detailed response. I believe the additional experiments adequately address my concerns. In particular, the findings on Vintage and the complementarity of residual sources are interesting.

However, the performance gains remain relatively modest, especially for the proposed lead–channel allocation and the incremental improvements from adding some residual sources (e.g., Input and Context). I also find the methodological novelty somewhat limited. While the rebuttal strengthens the empirical evidence, I remain unconvinced that the overall contribution is sufficiently significant. Therefore, I will maintain my rating.

Reviewer_wfGVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes MATE, a method that corrects the output of a frozen forecasting model by combining five candidate residuals through credit-based weighting. The residuals in MATE consist of Local, Global, Vintage, Input, and Context. The Local residual represents a correction for localized forecast errors arising from short-range dependencies between neighboring forecast leads and from interactions among variables (channels). The Global residual represents a correction for systematic biases that appear smoothly across the entire forecast horizon (e.g., overall level shifts or trend distortions). The Vintage residual represents the correction that the consensus of past forecasts, issued at different origins for the same future time point, contributes to the current forecast. The Input residual represents a correction driven by input-side adaptation, obtained as the change in prediction when the way the frozen model interprets recent inputs is adapted to the distribution shift. The Context residual represents a correction that reflects the persistence of the most recent observation in proportion to how stable the current series is. The credit weight of each residual represents how much each correction should be trusted at each forecast lead and variable, and is determined from matured historical forecast errors. The modules are updated using matured historical feedback, which becomes available once all targets of a past forecast have arrived. The input calibrator is trained with a pseudo-Huber loss on predictions recomputed from calibrated inputs, and the output calibrator is trained with an MSE loss on the Local and Global corrections, each with one AdamW step per matured batch. Credit statistics and vintage statistics are updated by accumulation without gradient descent. As a result, MATE achieves an average MSE reduction of 5.06% over the frozen model (no TTA) and 2.65% (TAFAS) to 4.88% (PETSA) over existing TTA methods, attaining the lowest MSE in 396 of 460 configurations (86.1%).

Critical strengths
  • S1: The paper formulates forecast correction as two separate stages: generating candidate residuals and allocating credit that determines how much each residual should be trusted. It proposes a framework that keeps the backbone frozen and learns lead- and variable-specific correction strengths solely from matured historical errors.
  • S2: In addition to the original benchmark, the method is evaluated on TTFBench and on synthetic scenarios with controlled types of distribution shift, which concretely demonstrates its effectiveness under diverse shift conditions.
  • S3: MATE attains the lowest MSE in 86.1% of 460 configurations spanning four backbones and five scenarios.
Critical weaknesses
  • W1: Neither source code nor supplementary material is provided, so the reproducibility of the reported results cannot be verified.
  • W2: The structure of the input calibrator is described in only a single sentence ("analogous to the output calibrator"), leaving its functional form, kernel size, low-rank component design, and initialization unspecified.
  • W3: The scope and design of the mechanism ablation are limited.
    • It is conducted on only 36 of the 460 configurations (ETTh2/ETTm2/Weather, PatchTST/DLinear, Natural/Level). The TTFBench, Dynamic, and Profile scenarios as well as the FreTS and iTransformer backbones are excluded, so it is unclear whether the effect of each component generalizes across the main experimental conditions.
    • Only leave-one-out removal is used. Without single-residual or cumulative-addition experiments, the standalone effect of each residual cannot be separated from redundancy among residuals. For example, it is difficult to tell whether the small effect of removing Global (+0.04%) means that Global is unnecessary or that its role overlaps with Local.
  • W4: The evaluation is limited to benchmarks with relatively few channels and short lengths (ETT, Weather, Exchange), so the effectiveness and computational cost on large-scale datasets remain unverified.
  • W5: No efficiency metrics (runtime, memory usage, number of trainable parameters) are reported. In particular, the input residual requires an additional backbone forward pass for every prediction, and training the input calibrator requires forward and backward passes through the frozen backbone, which is expected to incur substantial overhead; however, this is not analyzed. Moreover, each prediction requires solving 1 + H + C box-constrained QPs and maintaining a cache of past forecast versions as well as lead- and channel-wise credit statistics, so the cost may grow with the horizon and the number of channels.
Questions for authors
  • Q1: Do the authors plan to release the source code in the final version to enable reproduction of the results?
  • Q2: Could the authors describe the implementation of the input calibrator in detail? In particular, please specify its overall functional form (whether it uses a residual structure), the axis and kernel size of the convolution, the design of the channel and low-rank components, and the initialization scheme (whether it starts as an identity mapping).
  • Q3: Could the authors provide the additional ablation results discussed in W3?
  • Q4: Could the authors report performance and computational cost (time, memory) on datasets with many variables, such as Traffic and Electricity?
  • Q5: Could the authors report the per-prediction and per-update runtime, GPU memory usage, and number of trainable parameters for MATE and the compared methods? In particular, please show the share of total cost attributable to the additional forward/backward passes for the input residual and to credit allocation (QP solving), and how the cost scales with the horizon and the number of channels.
3 comments
AuthorsOpenReview ↗

Thank you for your helpful comments on reproducibility, the input calibrator, and computational cost. We considered your suggestions and revised the manuscript using additional experiments and clearer explanations. The revision includes more implementation detail, experiments on residual combinations, and a resource analysis. We show the revised material in blue text to make the changes easier to locate. Please see the revision for details.

Thank you for your helpful comments on reproducibility, the input calibrator, and computational cost. We considered your suggestions and revised the manuscript using additional experiments and clearer explanations. The revision includes more implementation detail, experiments on residual combinations, and a resource analysis. We show the revised material in blue text to make the changes easier to locate. Please see the revision for details.

Q1 / W1. Will the code and reproducibility materials be released?

We will publicly release the MATE implementation and experiment scripts upon acceptance. The revised manuscript includes a reproducibility statement. Appendix A describes the data splits, scenario construction, feedback scheduling, and implementation parameters.

Q2 / W2. How is the input calibrator implemented?

The input calibrator has the following residual form.

Cin(X)=X+DWConv⁡(X)+X⊙s+b+XMoff+Δlr(X).C_{\mathrm{in}}(X)=X+\operatorname{DWConv}(X)+X\odot s+b+XM_{\mathrm{off}}+\Delta_{\mathrm{lr}}(X).

The depthwise convolution runs along the time axis with a kernel length of 17. Here, ss is a per-channel scaling factor, bb is a positional bias, and MoffM_{\mathrm{off}} has a zero diagonal. The term Δlr\Delta_{\mathrm{lr}} computes Uc(Vc⊤Xc)U_c(V_c^\top X_c) separately for each channel cc, with rank 8. We initialize the left factor with an orthogonalized discrete Legendre basis and update it during training. The right factor and the remaining correction parameters start at zero, so the input calibrator is initially the identity. The Input residual is the forecast on the calibrated input minus the base forecast from the same frozen backbone. See Section 4.2 and Appendix A.2.

Q3 / W3. Ablation coverage and the standalone effect of residual sources

(1) Expanded ablation coverage. We expanded from 36 to 368 configurations across six datasets, four backbones, and four scenarios, using three seeds for each configuration. The expansion includes the Dynamic and Profile scenarios, the FreTS and iTransformer backbones, and the Exchange dataset. Relative to Channel-only, the strongest single-axis control, Full MATE has lower mean paired MSE in every scenario. See Section 5.3, Table 4, and Appendix C.5. TTFBench remains part of the main performance comparisons.

(2) Testing individual residual sources. We retain one source at a time and let it adapt independently. All five sources produce lower mean MSE than the frozen backbone in each of the four scenarios. Full MATE improves on Vintage-only, the single source with the lowest overall mean MSE, by 3.45% in mean paired MSE and has lower error in 366 of 368 configurations.

(3) Testing the gain from adding each source. We fix the candidate residuals and re-estimate credit weights every time a new source is added. Each addition in the order shown in the table lowers mean MSE in all four scenarios. These results support combining sources that complement one another. Section 5.3, Table 5 reports MSE for both comparisons. Appendix C.6, Table C13 reports MAE.

Table 5: Mean MSE for independent single-source adaptation and fixed candidate subsets. Each scenario has 92 configurations and three test seeds per configuration. Bold marks the lowest displayed MSE within each adaptation group for each scenario.

ExperimentSettingNaturalLevelDynamicProfile
ReferenceFrozen0.32280.33370.33510.3296
Independent adaptationLocal-only0.31790.32920.32990.3249
Global-only0.31610.32790.32800.3235
Vintage-only0.31610.32730.32820.3228
Input-only0.31730.32890.32900.3241
Context-only0.32070.33140.33440.3286
Fixed candidatesLocal0.31840.32960.33040.3253
Local + Global0.31370.32570.32550.3213
Local + Global + Vintage0.30840.32060.32010.3158
Local + Global + Vintage + Input0.30710.32000.31870.3150
Full MATE (all five sources)0.30510.31770.31800.3141

(4) The contribution of Global. In all four scenarios, Global alone yields lower mean MSE than the frozen backbone. With the candidates fixed, adding Global to Local also lowers mean MSE in each scenario. For example, mean MSE drops from 0.3184 to 0.3137 in Natural. This shows that Global contributes additional gains when local corrections are already available (Table 5 of the revision).

Q4 / W4. Accuracy and cost on Traffic and Electricity

Thank you for suggesting an evaluation on datasets with many variables. Given the time available for this revision, we leave end-to-end accuracy, runtime, and memory measurements on Traffic and Electricity for future work. We added a resource analysis on the datasets used in the current study.

Q5 / W5. Cost of prediction, updates, and credit estimation

(1) Trainable parameters and peak memory. In Natural, we compared 16 matched configurations on ETTh2 and Weather, covering four backbones and horizons of 96 and 720. MATE has the lowest median trainable parameter count and peak GPU memory among the four methods. Lightweight baselines use fewer parameters and less memory in some PatchTST configurations. See Section 5.3, Appendix A.3, Table A1, and Appendix E, Table E1 for the per-configuration data. Table E2 summarizes the results across configurations.

Table E2: Median trainable adaptation parameters and peak allocated GPU memory over the 16 matched configurations in Natural.

MethodTrainable parameters (thousands)Peak GPU memory (MiB)
TAFAS455.13356.79
PETSA71.66322.74
DynaTTA463.65518.20
MATE55.59207.04

(2) Cost of prediction and feedback updates. Component-wise timing shows that both prediction and updates take longer for MATE in the measured configurations. At prediction time, constructing the Input residual requires an additional forward pass of the frozen backbone on the calibrated input. When feedback matures, training the input calibrator requires forward and backward passes through that backbone. MATE also maintains a cache of previous forecasts. See Sections 4.2 and 4.4 for the prediction and update procedures.

(3) Credit estimation and scaling. In the timing analysis, solving the credit quadratic programs is a major source of additional cost. Each refresh solves 1+H+C1+H+C five-variable problems. The weights can be reused while the credit statistics stay fixed. In controlled operator tests, longer horizons increase the time to recompute weights, and more channels increase the time to update the credit statistics. See Section 4.3.

(4) Future improvements. This study focuses on generating corrections and allocating credit under matured feedback. We plan to prioritize reducing the cost of recomputing credit weights and to assess accuracy, runtime, and memory on real datasets with more channels. Section 6 discusses these costs and directions for future work.

Thank you for the detailed response and the revised manuscript. I also appreciate the added reproducibility statement and the commitment to release the code (W1).

However, some of the key experiments I raised as concerns have been deferred to future work. In particular, the evaluation on datasets with many variables (W4), is not included in the revision.

In addition, the following details are still missing from the response and the revision.

  • W5: The revision reports trainable parameter counts and peak GPU memory, but no runtime measurements. The response states that component-wise timing shows longer prediction and update times for MATE, yet these measurements do not appear in the revised manuscript. The share of total cost attributable to the additional forward/backward passes for the input residual and to the credit QPs, as well as the controlled tests on scaling with the horizon and the number of channels mentioned in the response, are also not reported. The claim regarding 20 matched dataset–backbone pairs (Section 5.3 and Appendix E) is not supported by a table, since Table E1 covers only ETTh2 and Weather. Moreover, for PatchTST, MATE uses more memory than TAFAS and PETSA (e.g., 801.36 MiB vs. 564.60 and 556.93 MiB on Weather at H = 720). So the deffered experiments with many variables would be important to explain the scalability.
  • W3: Tables 4 and 5 report only means and win counts, without seed-level standard deviations or significance tests, and the cumulative-addition experiment is conducted in a single order, so the contribution of each source may depend on the order of addition. In addition, the MAE results in Table C12 show that removing Local (−0.03%, 45.7% wins) or Input (+0.04%, 58.4% wins) has almost no effect, which suggests that the contribution of these two sources is negligible in terms of MAE.

Therefore, I maintain my current score.

Reviewer_2rtZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes MATE a test-time adaptation method for time-series forecasting under distribution shift. MATE keeps the pretrained forecasting model frozen, applies several distinct residual corrections from a predefined dictionary and generates corrections then it learns how much to use each correction for every lead (prediction horizon) and channel only when the complete ground truth labels are available. It furthermore trains input and output calibrators using to adapt to changing data distributions.

Critical strengths
  • The paper studies a critical limitation and an interesting problem in TTA for time series forecasting that can have real-world impacts
  • While some aspects of MATE are not novel per se, I found the overall combination and design novel enough for ICLR and its audience
Critical weaknesses
  • It was not easy for me to follow the paper. I think the presentation and flow of the paper can be improved particularly in section 4.

  • Experiments can be improved if extended to the Monash repository datasets as well e.g., M4 dataset (https://forecastingdata.org/) or M5 the a recent version of it (it’s separate from Monash).

  • It is completely understandable why authors excluded FAC from empirical evaluation however, this method seems the closest to the idea of MATE. So, I encourage authors to do their best and have a fair comparison with this method. Please note that while I list this under weakness it’s more of a suggestion and I will not hold ignoring it against authors.

  • Some of the results are based on a single run and there is no statistical analysis on those that were repeated 3 times. This is particularly important for the main results in Table C1: Credit-allocation ablation MSE averaged by dataset where differences are small and not always in favour of the full MATE.

  • Exchange seems to be the weakest dataset among the studied set and was not included in the ablation study. Overall, I found that ablation study can be improved for example the shared global reference, regularization toward that reference, the use of second-order Gram matrix, the FTRL formulation, or the additive factorization. I am not expecting authors to do all of these but justifying some of these design choices and their impacts either empirically or by discussion would be beneficial.

  • While it is understandable why authors modified implementations of the TTA baselines, it is not clear to me if MATE’s gains persist when partial feedback is available meaning employing the setting that some baselines were originally designed to operate at.

Questions for authors
  • What exactly is being studied in w/o Local, Global, or Input? Does this mean the calibration is removed (e.g., output calibration)?

  • Why does MATE struggle for long horizon forecasting?

  • Why only Natural and Level settings were studied for the ablations?

  • Why authors did not study sensitivity to the smoothness penalties and their coefficient? It would be interesting to see or at least discuss what would happen if we remove one of the penalties or set the coefficient to zero as well as a small sweep over ranges of coefficient.

3 comments
AuthorsOpenReview ↗

Thank you for your helpful comments on the method presentation, the scope of the ablations, and the results. After considering your suggestions and questions, we reorganized Section 4 and added expanded ablations, three-seed statistics, and partial-feedback comparisons. Because the revisions span several sections, we use blue text only for the headings of revised subsections and the captions of updated tables to help you locate the changes.

Thank you for your helpful comments on the method presentation, the scope of the ablations, and the results. After considering your suggestions and questions, we reorganized Section 4 and added expanded ablations, three-seed statistics, and partial-feedback comparisons. Because the revisions span several sections, we use blue text only for the headings of revised subsections and the captions of updated tables to help you locate the changes.

Q1. What do w/o Local, w/o Global, and w/o Input remove?

(1) w/o Local removes the local residual from the output calibrator. The global residual remains active during prediction and updating.

(2) w/o Global removes the global residual from the output calibrator. The local residual remains active during prediction and updating.

(3) w/o Input skips the additional forecast on the calibrated input and removes its data-fitting loss.

The remaining branches and credit weights continue to update online. Across the 368 configurations, Full MATE reduces mean paired MSE by 0.27%, 0.48%, and 0.30% relative to these three variants, respectively. See Section 5.3, Table 4, and Appendix C.

Q2. Why are the gains smaller at longer horizons?

At all four tested horizons, MATE has positive median MSE reductions relative to both the frozen backbone and TAFAS (Figure 4). The table below compares the mean MSE reductions at horizons 96 and 720 across the same 100 dataset–backbone–scenario combinations.

Comparison across matched forecast horizons

ReferenceReduction at 96 stepsReduction at 720 steps
Frozen backbone5.23%5.04%
TAFAS3.65%2.02%

The gain relative to TAFAS narrows at the longer horizon, while the gain relative to the frozen backbone stays similar. Longer horizons delay the full-window feedback needed for credit updates. How much this delay contributes to the smaller gain remains to be tested separately. See Appendix B for the matched results.

Q3. Why did the original manuscript include ablations only in Natural and Level?

(1) Original settings. The original manuscript used 36 configurations to examine the unmodified data stream (Natural) and a controlled level shift (Level).

(2) Expanded evaluation. We expanded the credit allocation and residual removal ablations to 368 configurations across six datasets, four backbones, and four scenarios, with three seeds per configuration. The expanded evaluation includes Dynamic and Profile and adds the Exchange dataset.

Table 4: Mechanism ablations across four scenarios. Gains are mean paired MSE reductions of Full MATE relative to each variant. Wins compare configuration means across three test seeds. Each entry shows gain (%) / wins (share).

VariantNaturalLevelDynamicProfileOverall
Uniform+4.32 / 90 (97.8%)+3.70 / 88 (95.7%)+3.73 / 89 (96.7%)+3.58 / 89 (96.7%)+3.83 / 356 (96.7%)
Global-only+1.06 / 82 (89.1%)+0.98 / 75 (81.5%)+0.83 / 82 (89.1%)+0.90 / 83 (90.2%)+0.94 / 322 (87.5%)
Lead-only+0.78 / 77 (83.7%)+0.70 / 77 (83.7%)+0.53 / 71 (77.2%)+0.59 / 76 (82.6%)+0.65 / 301 (81.8%)
Channel-only+0.26 / 77 (83.7%)+0.28 / 76 (82.6%)+0.26 / 78 (84.8%)+0.28 / 78 (84.8%)+0.27 / 309 (84.0%)
w/o Local+0.32 / 75 (81.5%)+0.22 / 65 (70.7%)+0.28 / 74 (80.4%)+0.25 / 62 (67.4%)+0.27 / 276 (75.0%)
w/o Global+0.58 / 89 (96.7%)+0.37 / 80 (87.0%)+0.55 / 88 (95.7%)+0.42 / 83 (90.2%)+0.48 / 340 (92.4%)
w/o Vintage+1.60 / 83 (90.2%)+1.50 / 78 (84.8%)+1.58 / 85 (92.4%)+1.60 / 84 (91.3%)+1.57 / 330 (89.7%)
w/o Input+0.37 / 64 (69.6%)+0.11 / 56 (60.9%)+0.43 / 65 (70.7%)+0.28 / 57 (62.0%)+0.30 / 242 (65.8%)
w/o Context+1.08 / 75 (81.5%)+1.11 / 76 (82.6%)+0.37 / 72 (78.3%)+0.50 / 74 (80.4%)+0.76 / 297 (80.7%)

(3) Credit allocation. Full MATE reduces mean paired MSE by 3.83% relative to Uniform. Compared with Channel-only, the strongest single-axis control, the mean reduction is positive in all four scenarios. The overall reduction is 0.27%, with lower MSE in 309 of the 368 configurations. This supports varying the correction strength across leads in addition to across channels.

(4) Residual removal. We remove the five residual sources one at a time. Removing Vintage has the largest effect in each of the four scenarios. Full MATE has 1.57% lower mean paired MSE than the variant without Vintage.

(5) Results on Exchange. Relative to Uniform, Full MATE has lower MSE in 45 of 48 configurations and reduces mean paired MSE by 2.73%. Full MATE and Channel-only have similar mean MSE on Exchange. See Section 5.3, Table 4, and Appendix C.5 of the revision. Appendix C.9, Table C14 gives the single-axis comparison for Exchange.

Q4. Do the smoothness penalty and its coefficient affect the results?

We compared four coefficients, 0, 10−510^{-5}, 10−410^{-4}, and 10−310^{-3}, across 368 configurations and three seeds. At 10−410^{-4}, we also tested the convolution-kernel penalty and the positional-bias penalty separately. Relative to the default setting, the largest absolute change in configuration-mean MSE or MAE was below 1.90×10−71.90\times 10^{-7}. Forecast accuracy was insensitive to the tested penalty settings. See Appendix C.8.

Concerns Raised in the Review

W1. Organization and presentation of Section 4

We rewrote Section 4 to follow the prediction and update procedure in Figure 1. Section 4 of the revised manuscript gives the full description.

(1) Section 4.1 explains how the base forecast and candidate residuals are combined into the final forecast.

(2) Section 4.2 describes what information each of the five residual types uses and how it is constructed.

(3) Section 4.3 explains how we accumulate statistics, estimate global and single-axis credits, and combine them into weights for individual forecast positions.

(4) Section 4.4 specifies which statistics are updated when a historical batch matures and how the two calibrators are trained.

W2. Adding external datasets such as M4 or M5

Thank you for the suggestion. The current main evaluation covers 460 configurations across six datasets, four backbones, and five scenarios. Given the time available for this revision, we leave evaluation on M4 and M5 for future work.

W3. A fair comparison with FAC

Thank you for suggesting a comparison with FAC, which also uses full-window feedback. The current study compares TAFAS, PETSA, and DynaTTA under the same feedback constraints. We have yet to complete a comparison with FAC within this revision period. We plan to use the same data splits, backbone checkpoints, and feedback protocol for that comparison.

W4. Single runs and statistics across three seeds

(1) Completing the single-seed experiments. We extended the stress tests from one seed to three. The main experiments and stress tests now report three-seed means and sample standard deviations. See Appendices B and D of the revision.

(2) Checking the small difference in Table C1. For the 12 ETTh2 configurations, we paired runs by configuration and seed, then summarized the differences across the three seeds. The paired MSE difference (Lead-only minus Full MATE) was (−0.3479±0.5274)×10−3(-0.3479\pm 0.5274)\times 10^{-3}. Lead-only had slightly lower MSE on two seeds, and Full MATE had slightly lower MSE on the third. See Appendix C.1, Table C1 and its accompanying paired analysis.

(3) Checking the expanded evaluation. Full MATE has lower mean MSE and MAE than Lead-only on five of the six datasets. ETTh2 is the exception. The paired MSE and MAE comparisons for both single-axis controls are reported below and in Appendix C.9, Table C14 of the revision.

Table C14: Single-axis credit controls by dataset. Each cell reports the mean paired error difference (control minus Full MATE) followed by the sample SD across three test seeds. Both values are in 10⁻³ units. Positive differences favor Full MATE.

DatasetNLead-only MSELead-only MAEChannel-only MSEChannel-only MAE
ETTh164+2.6971 ± 0.2717+2.0843 ± 0.1793+0.3749 ± 0.0109+0.4750 ± 0.0117
ETTh264−0.7537 ± 0.2865−1.1285 ± 0.2444+0.4738 ± 0.0135+0.5159 ± 0.0095
ETTm164+0.4932 ± 0.2015+0.8164 ± 0.0876+3.4483 ± 0.0417+2.1873 ± 0.0180
ETTm264+0.9600 ± 0.1952+0.5024 ± 0.1441+0.3855 ± 0.0216+0.5871 ± 0.0254
Weather64+4.5605 ± 0.1880+4.3484 ± 0.1765+0.4968 ± 0.0500+0.9433 ± 0.0300
Exchange48+2.4028 ± 0.3067+1.4797 ± 0.2054−0.0140 ± 0.0459+0.1472 ± 0.0248

W5. Ablation coverage and credit design

(1) Expanded ablation coverage. As described in Q3, we expanded the credit allocation and residual removal ablations to 368 configurations, including 48 on Exchange. See Section 5.3, Table 4, and Appendices C.5 and C.9 of the revision.

(2) Testing lead-specific weights. With the candidate residuals fixed, we either average the lead credit component across leads or reverse the full weights along the lead axis. Full MATE has 0.23% and 0.69% lower mean paired MSE than these two controls, respectively. This supports matching weights to their forecast leads. See Appendix C.5, Table C11.

Table C11: Lead-specific controls. Gains are mean paired MSE reductions of Full MATE relative to each control. Each entry shows gain (%) / wins (share).

VariantNaturalLevelDynamicProfileOverall
Lead-averaged+0.22 / 66 (71.7%)+0.22 / 68 (73.9%)+0.23 / 70 (76.1%)+0.24 / 69 (75.0%)+0.23 / 273 (74.2%)
Lead-reversed+0.73 / 78 (84.8%)+0.70 / 76 (82.6%)+0.66 / 83 (90.2%)+0.67 / 78 (84.8%)+0.69 / 315 (85.6%)

(3) Testing whether residual sources complement one another. Relative to Vintage-only, the strongest single-source variant, Full MATE reduces mean paired MSE by 3.45% and has lower MSE in 366 of 368 configurations. With candidates fixed, each additional source in the tested sequence lowers mean MSE in all four scenarios. See Section 5.3, Table 5, and Appendix C.6.

(4) Rationale for the credit design. Section 4.3 explains how global credit provides a shared reference for estimating lead-specific and channel-specific weights, and how the Gram matrix retains cross-terms between candidates. It also explains how FTRL solves the credit problem from accumulated statistics and how the lead and channel credits are combined.

(5) Remaining controls. Given the time available, separate ablations of the global reference, shrinkage regularization, Gram cross-terms, the FTRL formulation, and the additive combination remain for future work. The added ablations support the effectiveness of lead–channel credit allocation. We plan to start with separate tests of the global reference, shrinkage regularization, and Gram cross-terms.

W6. Performance when baselines use partial feedback

(1) Comparison setup. We ran a matched comparison on 92 Natural configurations using three seeds. MATE used full-window feedback. TAFAS, PETSA, and DynaTTA could update using the target prefix observed at the time. These updates affected only subsequent forecasts.

Table B11: MATE with full-window feedback compared with baselines using partial feedback in Natural.

Partial-feedback baselineMSE reductionMAE reductionMSE wins
TAFAS2.64%1.62%85/92
PETSA5.49%2.52%92/92
DynaTTA4.97%3.84%86/92

(2) Results. When the three baselines use partial feedback, MATE still reduces mean paired MSE by 2.64%, 5.49%, and 4.97%, respectively, in the tested Natural settings. We leave the use of partial feedback within MATE for future work.

(3) Where to find the results. Section 3 and Appendix A.2 of the revised manuscript describe the full-window feedback protocol. The main Natural results are in Section 5.2, Table 1. The partial-feedback comparison is in Appendix B, Table B11.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.