RELORA-CL: FROM REUSABLE UPDATES TO RELI- ABLE REPRESENTATION MAINTENANCE IN CLASS- INCREMENTAL LEARNING
Abstract
Low-Rank Adaptation (LoRA) enables parameter-efficient class-incremental learn- ing, but successive updates shift historical class representations and render stored prototypes outdated. Without historical inputs, correcting this mismatch requires estimating their displacement from current-task samples. We investigate how continual updates can accommodate new classes while keeping historical represen- tation drift amenable to estimation. Our analysis reveals depth-dependent functional reuse: early-task input subspaces remain more sufficient for shallow-block updates, whereas deeper blocks increasingly require additional directions. Motivated by this observation, we propose reuse-aware LoRA (ReLoRA-CL), which preserves learned input subspaces in shallow blocks and expands complementary directions in deeper blocks. Semantic drift compensation and balanced classifier alignment subsequently maintain historical class statistics and decision boundaries. We formu- late prototype migration as finite-sample drift estimation and derive an error bound involving local drift regularity, support mismatch, and residual uncertainty. Under additional sensitivity assumptions, we connect drift regularity to consecutive-task parameter changes, providing a conditional rationale for structured adaptation. ReLoRA-CL improves final accuracy over the strongest baseline by 3.58, 1.64, and 4.53 percentage points on ImageNet-R with 20 tasks, CIFAR-100 with 10 tasks, and ImageNet-A with 10 tasks, respectively. Drift diagnostics further show lower local drift variation and prototype displacement estimation error. The code is available at: https://anonymous.4open.science/r/ReLoRA-8C3A/
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.