Distribution Matching Guided Replay Sample Selection for Online Class-Incremental Learning

Online Class-Incremental Learning (OCIL) focuses on learning from continuously evolving data streams that are ubiquitous in dynamic environments. Data replay is an effective OCIL method, which maintains a replay buffer and learns from a mixture of replay samples and new samples to mitigate catastrophic forgetting. Previous studies have proposed various strategies for selecting replay samples; however, they often suffer from limited data availability and error accumulation, and only achieve suboptimal performance in practice. To this end, we propose a new replay sample selection strategy called Distribution Matching Replay (DMR). By continuously aligning the distribution of replay samples with that of all historical data, DMR significantly alleviates the issues of data availability and error accumulation. To make second-order distribution matching memory-efficient in the streaming setting, we further introduce a randomized sketch of the RFF covariance statistics, which approximates Maximum Covariance Discrepancy (MCD) without explicitly storing per-class covariance matrices. Extensive experiments on multiple benchmarks demonstrate that DMR outperforms state-of-the-art selection strategies and can be seamlessly integrated into various OCIL methods to further improve their performance.