Adaptive Distillation via Online Drift Projection for Exemplar-Free Class-Incremental Learning

Exemplar-free class-incremental learning must preserve old classes without storing raw examples. Prototype-based methods address this constraint by caching compact class statistics, but these statistics become stale as the feature extractor evolves. Representation drift can be heterogeneous: some old classes remain relatively stable, whereas others move substantially. This motivates adapting knowledge distillation to class-wise drift instead of weighting all old classes equally. We propose Online Drift Projection (ODP), a co-trained module that estimates class-wise drift during incremental training. ODP is optimized jointly with the evolving backbone and projects stored old-class prototypes into the current feature space. The induced prototype displacement provides an exemplar-free drift signal for each old class, without requiring access to past samples. We use this signal to construct drift-weighted distillation, which increases the contribution of high-drift classes to the distillation objective. Experiments on CIFAR-100, TinyImageNet, and ImageNet-100 show gains over AdaGauss and DPCR, while results on pretrained CUB-200 are mixed. Additional analyses show agreement between surrogate and oracle displacement, and an association between oracle displacement and class-wise forgetting, supporting online drift projection as a useful signal for adaptive distillation.