Dual-State Predictive Momentum for Online Continual Learning

Online continual learning (OCL) empowers models to learn from non-stationary streams in a single pass while retaining prior knowledge. In this context, exponential moving average (EMA)-based strategies have achieved impressive results and underpin many state-of-the-art approaches. However, their first-order temporal smoothing only tracks the average parameter level, leading to a low-fidelity approximation of the evolving low-loss center. To this end, we propose $\text{D}$ual-$\text{S}$tate $\text{P}$redictive $\text{M}$omentum (DSPM), which jointly estimates the level and trend states of the student trajectory through two momentum estimators and extrapolates them to construct a one-step predictive teacher. From the perspective of linear mode connectivity, we disclose the tracking bias of EMA and show that DSPM yields an unbiased trajectory error and a tighter upper bound on the average empirical loss. We further introduce dual-granularity complementary distillation, which aligns teacher-student output distributions while preserving stable level-state representations, promoting classifier plasticity and representation stability. Extensive experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-100 validate the effectiveness of DSPM. Specifically, DSPM improves model stability by 2.66%, 1.40%, 3.36%, and 5.79% over the latest state-of-the-art method on these benchmarks, respectively, with an average test accuracy gain of 1.08%. Our code is available at https://anonymous.4open.science/r/DSPM.