Adapter Sharing via Task-Conditioned Feature Transformations for Class Incremental Learning

Exemplar-free class-incremental learning requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while preserving the ability to recognize all seen classes. Recently, pretrained-model-based approaches have become prevalent by adapting a frozen backbone with additional lightweight trainable modules. Existing methods, however, exhibit limitations. Task-specific adapters learn explicit per-task representations but are parameter- and computation-inefficient, while LoRA-based merging methods combine per-task LoRA updates into a single static model. Because the merged update must serve all tasks with the same parameters, the resulting static update may reduce the separability of task-specific feature distributions during inference. To address these problems, we present FACET: task-conditioned FeAture transformation with ConditionEd feature consisTency, achieving excellent parameter efficiency while producing highly discriminative features during inference. When continually trained on a task sequence, FACET shares core adapter weights across tasks and employs task-conditioned feature transformations to shape the overall feature distribution of the adapter into a mixture of task-specific components with reduced overlap. This design supports task-dependent features without allocating a complete adapter to every task. Additionally, we propose an efficient replay-free task-conditioned feature consistency loss to mitigate drift in previously learned task-conditioned features as the shared parameters evolve during continual training. At inference, lightweight auxiliary classifiers estimate the task context without requiring an oracle task identity. We demonstrate scalability to task sequences of up to 200 tasks. Across short and long task sequences, FACET achieves leading final accuracy compared to strong baselines while reducing inference cost. On OmniBenchmark-1K with 200 tasks, FACET achieves 36.64 ms latency and 40.23 GFLOPs per image, corresponding to a 3.85× speedup and a 62.69% reduction in GFLOPs compared with MIN. The code and models will be made open source upon acceptance.