From Windows to Regions: Regional Gradients for Controlling Forecasting Trajectories

Time-series forecasters train on overlapping sliding windows, where localized temporal segments repeatedly contribute to parameter updates. We reveal that while recurring high-loss regions provide constructive immediate gradients, their directional alignment with the remaining data collapses over training, causing multi-step momentum accumulation to exert severe trajectory drag. Physical data withholding confirms that this phenomenon is strictly anchored to temporal location, yet physically discarding observations risks forfeiting essential structural dynamics. To resolve this dilemma, we introduce Regional First-Moment Memory Control ($\mathcal{H}$-short), shifting the intervention from data withholding to optimizer memory. Operating with complete data retention, $\mathcal{H}$-short dampens the accumulated momentum staleness of targeted temporal regions within Adam, absorbing current learning signals without inheriting outdated directional bias. Across diverse forecasters and benchmarks, $\mathcal{H}$-short consistently alleviates trajectory drag in expressive deep architectures while safeguarding critical physical transitions where data withholding fails, establishing source-aware memory control as a safe, non-destructive paradigm for temporal optimization.