Rethinking Layer Selection for Low-Rank Adaptation

Layer selection concentrates a low-rank adaptation budget on layers judged important by a pretrained model. We show that internal coordinates can change these recommendations while leaving the available adaptation functions unchanged. Our capacity-preserving audit preserves both the network function and the set of LoRA functions at every fixed placement. Gradient-based selections change across the tested architectures and calibration tasks, whereas contraction and block-ablation controls retain their predicted invariances. Controlled calibration experiments and norm balancing connect this sensitivity to internal scale. Training every selected set on the original backbone then measures its adaptation value: random-placement controls improve on the canonical gradient selection, and fixed sets exhibit different utility across tasks. Finally, comparisons with static placements, AdaLoRA, and FoRa match final effective adapter parameters, training steps, and learning-rate search. Full-depth, uniform low-rank LoRA achieves the highest accuracy in these comparisons, including against adaptive rank allocation that retains every layer. These results connect score interpretation to practical allocation: test coordinate sensitivity, measure the utility of the selected layers, and compare against distributed adaptation under a common parameter budget.