Availability and Predictivity: Improving Coreset Training at Few Images Per Class with Covariance Jitter

Coreset selection aims to replace a large training set with a small subset while preserving downstream performance. We show that models trained on just a few images per class (IPC) rely on features along the data’s high-variance directions, even when features along the low-variance directions are more predictive. This motivates our method, Coreset Covariance Jitter (CCJ), a simple, low-cost training augmentation applicable to any existing coreset selection method. CCJ adds noise to each selected image at every training step, scaled along each direction by how much images of the same class vary along it in the full training set. We further increase performance by adding a Quadratic Discriminant Analysis (QDA) at test time. On CIFAR-10, CIFAR-100 and ImageNet-1K, CCJ+QDA improves all evaluated coreset selection methods when running at 10 images per class.