Curvature-Guided Rank Scheduling for LoRA Fine-Tuning of Vision Transformers

Low-rank adaptation (LoRA) makes transfer learning parameter-efficient, but its rank is usually fixed before optimization and shared across layers. We introduce Curvature-Guided Rank Scheduling (CGRS), which uses a diagonal empirical-Fisher signal to change LoRA capacity during training. A global scheduler grows or shrinks one shared rank, while layer-wise schedulers allocate heterogeneous ranks across Vision Transformer (ViT) blocks; within each block, the query and value projections share a common rank. Rank changes preserve the learned update through a singular value decomposition (SVD)-based transfer and are delayed by a grace period to avoid reacting to noisy early curvature. We evaluate ImageNet-pretrained ViT-Base/16 on CIFAR-100, Street View House Numbers (SVHN), and Oxford Flowers-102 against full fine-tuning, a frozen backbone, a 14-rank LoRA sweep, and AdaLoRA. Global CGRS reaches 91.16% on CIFAR-100 and 97.45% on SVHN, improving over fixed LoRA at the same final rank by 0.70 and 0.47 percentage points, respectively. It also exceeds the best tested AdaLoRA accuracy by 1.10 and 1.54 points on these datasets. On Flowers-102, however, AdaLoRA is better (98.78% versus 98.52%), and some layer-wise schedules collapse to the maximum rank. These results support curvature as a useful capacity signal while identifying low-data calibration and one-way rank growth as important failure modes. Given single-seed evaluation, we treat the results as a pilot study establishing feasibility rather than a statistically validated comparison; multi-seed replication is the immediate next step.