SCULPT-LoRA: Spectral Sculpting for Continual Low-Rank Adaptation

Low-Rank Adaptation (LoRA) enables parameter-efficient continual learning, yet a fixed rank budget neither safeguards pretrained representations nor guarantees effective use of adaptation capacity. We identify two complementary spectral challenges: interference with dominant pretrained directions and concentration of update energy in a few singular modes. Addressing both requires controlling where updates act and how their energy is distributed. To alleviate these problem, We propose SCULPT-LoRA, a framework that couples Elastic Principal Subspace Protection (EPSP) with Polynomial Spectral Balancing (PSB). EPSP constructs a fixed orthonormal adaptation basis by attenuating its overlap with the pretrained principal subspace, allowing controlled modification rather than strict exclusion. PSB reshapes the trainable factor through a compact Gram-matrix polynomial during the forward pass, reducing spectral concentration. The two mechanisms are linked by row orthonormality, which transfers the nonzero singular spectrum of the conditioned factor exactly to the effective LoRA update. We derive a sharp worst-case bound on principal-subspace perturbation that decreases with protection strength under a fixed update-energy budget, and establish conditions for condition-number improvement under polynomial spectral shaping. Experiments across five continual learning benchmarks show improved final and average accuracy, with ablations supporting the complementary benefits of elastic protection and spectral balancing.