Past Errors, Better Forecasts: Memory-Guided Calibration for Frozen Time Series Predictors

Time series forecasting supports decision making in many data-intensive applications, where forecasting models typically use a fixed-length historical window to predict a future sequence. In standard forecasting pipelines, a single checkpoint is typically selected according to aggregate validation performance, yet strong aggregate performance does not guarantee uniformly small errors across samples, variables, or temporal contexts. Consequently, a frozen forecaster may exhibit structured, input-dependent residual patterns on historical labeled windows, which can provide reusable evidence of how the same forecaster tends to deviate under similar temporal conditions. Motivated by this observation, we propose Memory-Guided Test-Time Calibration (TTC), a non-parametric residual-memory framework that calibrates frozen time-series forecasters at the prediction level. After model selection, TTC constructs an external residual memory that associates compact variable-wise representations of historical look-back windows with their corresponding true-minus-predicted residual trajectories. At inference time, the current look-back window is used to retrieve historically similar variable patterns, and their residual trajectories are similarity-weighted and aggregated to compensate the base prediction. To reduce harmful over-correction, TTC applies a correction-ratio fallback that preserves the base prediction when the relative magnitude of the candidate correction is excessively large. The forecasting backbone remains frozen throughout calibration, and TTC never accesses the future labels of the current test window. Experiments across eight benchmark datasets and five heterogeneous forecasting backbones, together with comparisons against representative test-time adaptation methods and extensive ablation and sensitivity analyses, demonstrate the effectiveness and robustness of TTC across diverse forecasting settings.