Sequential Orthogonal Low-Rank Adaptationwith Feature Distribution Drift Compensation for Continual Learning

Orthogonal low-rank adaptation offers a parameter-efficient approach to class-incremental learning, but data-dependent subspace construction can introduce additional computational overhead.Moreover, orthogonality between parameter updates does not ensure stable feature representations, leaving stored historical class statistics misaligned with the evolving model and limiting the effectiveness of classifier alignment. To address these challenges, we propose Sequential Orthogonal Low-Rank Adaptation with Feature Distribution Drift Compensation. Our method sequentially allocates disjoint blocks of a shared, fixed orthonormal basis to the input-side LoRA factors, constructing mutually orthogonal adaptation subspaces without data-dependent subspace estimation. To account for the remaining feature drift, we fit a regularized residual transition using paired features extracted from current-task data before and after adaptation. This transition updates historical class means and covariances before classifier alignment, without retaining historical exemplars. By combining explicit parameter-space separation with feature-distribution correction, our framework addresses both subspace construction overhead and stale statistical representations. Experiments on standard and fine-grained class-incremental learning benchmarks demonstrate competitive performance while simplifying orthogonal subspace construction