- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Memory-Guided Test-Time Calibration (TTC), a prediction-level calibration framework for frozen time-series forecasters. The work is motivated by the observation that forecasting models selected according to aggregate validation performance may still exhibit structured, context-dependent residual patterns that provide useful information for subsequent predictions. TTC constructs an external memory that associates variable-wise representations of historical look-back windows with their corresponding forecasting residual trajectories. At test time, the method retrieves relevant historical patterns and aggregates their residuals to estimate a local correction, with a conservative fallback mechanism to limit excessively large adjustments. The framework is evaluated across eight benchmark datasets, five heterogeneous forecasting backbones, and multiple prediction horizons. The paper also includes comparisons with representative test-time adaptation methods, ablation and sensitivity studies, and evaluations of computational efficiency and scalability.
Critical strengths
- TTC uses historical forecasting residuals to correct subsequent predictions. The variable-wise retrieval mechanism selects relevant historical residuals for each variable, and the retrieved residuals are aggregated to calibrate the current forecast. The method provides a practical approach to improving the predictions of frozen forecasting models.
- TTC introduces an external calibration mechanism that can be integrated with existing forecasting models. Its variable-wise retrieval representation is constructed directly from observed temporal patterns, enabling a common calibration procedure across heterogeneous forecasting architectures. The combination of context-dependent retrieval, similarity-weighted residual aggregation, and conservative correction is straightforward and technically coherent.
- The paper compares TTC with TAFAS and SOLID using the same validation-selected forecasting checkpoints. The ablation studies examine different residual correction strategies and show the effectiveness of context-aware retrieval and multi-neighbor residual aggregation. The paper also reports parameter sensitivity, inference latency, and memory overhead..
Critical weaknesses
- The complete retrieval representation combines statistical summaries, normalized temporal shape, and first-order differences. Although the existing comparison between retrieval representations is informative, it does not fully isolate the contribution of each component. A leave-one-block-out ablation would provide a clearer understanding of their individual roles and complementary.
- The default residual memory is constructed from both training and validation windows. While this provides broader historical coverage, a train-only memory comparison would help clarify whether TTC retains its calibration benefits with a more restricted memory source and how the additional validation windows influence performance.
- The overall methodology is understandable, but some figures contain relatively dense information, and certain methodological details, particularly the fixed-backbone calibration setting, are explained repeatedly across sections. A more concise presentation and clearer visual hierarchy would make the main ideas and empirical findings easier to follow.
Questions for authors
-
Could the authors provide a leave-one-block-out ablation of the complete retrieval representation by individually removing statistical summaries, normalized temporal shape, and first-order differences? Results on representative model–dataset settings would help clarify the relative importance of these components.
-
How does TTC perform when the residual memory is constructed using only the training split? A comparison with the default training+validation memory, while keeping the forecasting checkpoint and other calibration settings unchanged, would help clarify the influence of historical memory coverage on calibration performance.