LoRA in the Last Layers: Score-Constrained Placement Search

Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning of pretrained models, but parameter efficiency does not imply training efficiency. Adaptive methods select modules, parameters, or ranks, often changing adaptation capacity together with layer placement. Yet placement determines how far gradients propa- gate through the frozen backbone: similar parameter budgets can require materially different computation. We analyze this placement-dependent backward cost and introduce LaLoS, a score-constrained search over LoRA placements close to the output. A briefly trained probe and a held-out selection score determine the adapter placement for each task, without prescribing a fixed budget. Across language and vision benchmarks, placement close to the output reduces final fine-tuning compu- tation and elapsed time at matched adapter capacity. LaLoS achieves competitive performance with task-dependent adapter counts. Fixed placements in the last layers remain strong baselines. Together, the analysis and experiments show that layer placement affects fine-tuning efficiency beyond trainable-parameter count.