- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
MELO is an online calibration framework for time-series forecasting that combines recursive least-squares experts with varying forgetting factors and covariance inflation settings, using MLpol to aggregate their predictions alongside those of the backbone models. The key intuition is that models forecasting the same target often capture a shared predictive direction but differ in their calibration errors and responses to distribution shifts. To address this, MELO balances the retention of historical information with adaptation to changing conditions. Covariance inflation enables the experts to adjust more rapidly when predictive relationships shift, while different forgetting factors provide complementary memory horizons. By aggregating multiple experts, MELO aims to improve forecasting robustness and generalization under distribution shift.
Critical strengths
-
Well-supported design and transparent implementation: The ablation studies show that removing covariance inflation substantially degrades performance and clarify the contributions of memory selection, intercept correction, and ensemble composition. These results help explain the mechanism behind the performance gains. The supplied code is consistent with Algorithm 1 and includes extensive experiments corroborating the paper’s claims, strengthening confidence in the results and supporting reproducibility.
-
Practical efficiency with clearly defined applicability: The method operates as a lightweight calibration layer on predictions from frozen backbones, requiring no additional backbone fine-tuning. This makes it straightforward to integrate into existing forecasting pipelines with limited computational overhead. The paper also explicitly acknowledges that calibration cannot recover information absent from the original forecasts, giving readers a clear understanding of the method’s capabilities and limitations.
Critical weaknesses
-
Insufficient comparative evidence to establish MELO’s practical contribution.
The evaluation omits relevant adaptation methods, including OMPB [1], ELF [2], TAFAS [3], SOLID [4], and OneNet [5]. OMPB is particularly relevant to calibration of a fixed backbone, while ELF and OneNet provide important comparisons for online aggregation. The experiments also rely mainly on small ensembles of conventional forecasting models; the appendix’s tabular foundation-model experiments on a shorter electricity window do not establish consistent benefits for time-series foundation models. Comparisons should align forecast horizons, available information, and label-arrival timing, report computational costs, and distinguish methods that update the backbone from those that preserve it. Without broader comparisons or clear explanations of incompatibility, the current evidence does not adequately establish MELO’s advantage over existing adaptation and ensembling approaches.
-
Insufficient evidence of robustness across different types of distribution shift.
The electricity and taxi experiments use plausible event-based regimes, but these do not establish the type or severity of shift or distinguish marginal changes from changes in the conditional input–target relationship. MELO updates an affine combination of frozen forecasts: covariance inflation facilitates adaptation without expanding the available representation. A large demand collapse may therefore be manageable through level or scale corrections, leaving robustness to feature-dependent changes, altered temporal relationships, and shifts in relative predictor quality insufficiently tested. Controlled experiments and diagnostics that account for seasonality and temporal dependence are needed to identify when calibration succeeds or fails, especially because wrapping an already adaptive predictor can worsen performance. Clearer links between the idealized recursion, covariance-inflated analysis, and implementation choices would also help readers assess which guarantees apply to the evaluated method.
References
[1] Chenfeng Huang, Zixuan Ma, and George Michailidis. Model-Agnostic Online Certificate-Driven Calibration for Time Series Forecasting Under Distribution Shift. UAI, 2026. (OMPB)
[2] Thomas L. Lee, William Toner, Rajkarn Singh, Artjom Joosen, and Martin Asenov. Lightweight Online Adaption for Time Series Foundation Model Forecasts. ICML, 2025. (ELF)
[3] HyunGi Kim, Siwon Kim, Jisoo Mok, and Sungroh Yoon. Battling the Non-stationarity in Time Series Forecasting via Test-time Adaptation. AAAI, 2025. (TAFAS)
[4] Mouxiang Chen, Lefei Shen, Han Fu, Zhuo Li, Jianling Sun, and Chenghao Liu. Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shift. KDD, 2024. (SOLID)
[5] Yi-Fan Zhang et al. OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling. NeurIPS, 2023.
Questions for authors
Based on the weakness, I have the following questions:
-
Comparison with relevant adaptation methods: How does MELO compare with OMPB, ELF, TAFAS, SOLID, and OneNet under matched forecast horizons, information access, and label-arrival timing? Please report computational costs, distinguish methods that update the backbone from those that preserve it, and explain any incompatibilities that prevent a fair comparison.
-
Applicability to modern time-series forecasting models: Does MELO consistently improve deep learning forecasters, including recent time-series foundation models, across multiple datasets and shift conditions compared with the same models without calibration? Under what conditions does wrapping an already adaptive predictor degrade performance?
-
Shift characterization and limits of affine calibration: What evidence distinguishes changes in the input distribution from changes in the conditional input–target relationship in the electricity and taxi experiments, after accounting for seasonality and temporal dependence? Can controlled experiments involving feature-dependent changes, altered temporal relationships, or changes in relative predictor quality clarify when MELO’s affine calibration succeeds or fails?
-
Robustness to longer feedback delays: How does MELO perform when the label-feedback delay exceeds one time step? Can the authors evaluate forecasting accuracy and adaptation speed across several delay lengths, particularly following distribution shifts, while ensuring that all compared methods receive labels at the same time?