BARS-LoRA: Budget-Aware Rank Stabilization for Adaptive Low-Rank Adaptation

Adaptive-rank LoRA decides during training where to spend a limited rank budget, and each decision is only as reliable as the evidence behind it. This evidence degrades in two distinct ways. Hard masks, as in AdaLoRA, commit rank from mini-batch salience estimates that remain noisy under short schedules and can leave entire modules without rank. Soft gates, as in SoRA, read their sparsity pattern from a rank-one dictionary that keeps drifting as training continues. We propose BARS-LoRA (Budget-Aware Rank Stabilization), which applies one principle, stabilize the evidence before committing rank, through two instantiations. SCALP-LoRA scores each hard channel by its first-order deletion effect, which we show is the common signed quantity underlying all three terms of the AdaLoRA triplet score; it discounts this score by its relative dispersion and allocates rank per Linear module with a guaranteed minimum rank. SAGE-SoRA orthogonalizes the gated dictionary and consolidates its basis before the sparse gate pattern is read out. The two branches specialize by training budget. On GLUE with Qwen3-0.6B, SCALP reaches 84.75% six-task accuracy under a four-epoch schedule, 1.15 points above AdaLoRA. At 800 updates, SAGE reaches 82.55% four-task accuracy, 1.64 points above SoRA. On Qwen3-1.7B, SAGE ranks first among seven methods on every task and exceeds DoRA by 1.74 points.