Learning from Historical Overlap Errors: Adaptive Pointwise Reweighting for Time Series Forecasting

Long-term time series forecasting models are commonly trained with sliding windows, so nearby input windows predict many of the same future timestamps. Standard training optimizes these overlapping forecasts independently, leaving the errors of preceding forecasts unused as timestamp-specific evidence for determining the optimization priority of current predictions. We propose Historical Overlap Reliability Weighting (HORW), a model-agnostic method that compares each prediction point’s current error with a statistical reference derived from historical prediction errors of preceding overlapping windows for the same target timestamp, and uses the resulting error gap to adjust the optimization emphasis on each prediction point. Specifically, after each training epoch, HORW holds the epoch-end model parameters fixed during a gradient-free sequential pass over the training set to construct a historical error memory. For each prediction point in the next epoch, it uses the larger of the mean and median errors from preceding windows forecasting the same target timestamp as a conservative reference. When the current error exceeds this reference, HORW maps the normalized gap to a point-wise loss weight. A single signed scalar coefficient specifies the response to this gap: its sign determines whether the corresponding point is relatively emphasized or attenuated, while its magnitude controls the response strength. HORW preserves the original model architecture, applies to diverse forecasting objectives, and adds no inference-time cost. Experiments on seven datasets with three architecturally diverse forecasting models show improvements in the vast majority of evaluated settings, with additional gains when HORW is combined with auxiliary training objectives. Ablations further support the value of timestamp-aligned historical references over global historical statistics and current-error references.