Class-incremental continual learning with many tasks

We present an in-depth experimental analysis of replay-based, class-incremental continual learning (CL), emphasizing a large number of tasks and the particular choice of implementing replay (single-loss or weighted sum of two losses). We exclusively focus on class-incremental learning as the most difficult of current CL paradigms, since we believe that recent results on task-incremental CL cannot be generalized to CL in general. Datasets used for the investigation are various splits based on MNIST, EMNIST-Letters, and CIFAR-10, analyzed for an exhaustive set of relevant hyper-parameters. Based on a mathematical analysis, we show that the way of implementing the replay loss has little impact, justifying the use of the more implementation-friendly single-loss strategy. More importantly, a mathematical analysis and experimental findings show that, for a large number of tasks, the size of the replay buffer must be scaled according to the total number of tasks. This renders naive replay-based CL with a fixed replay budget, as favored in recent literature, problematic in practice, since the compute requirements of CL will thus scale with the total number of tasks. We discuss the implications of this finding for CL and outline potential solutions.