Not How Many, But Which: Parameter Placement in Low-Rank Adaptation

LoRA introduces low-rank adapter parameters for fine-tuning. We study whether, given a fixed budget of $k$ trainable entries in the $\mathbf{B}$ matrix of a LoRA adapter ($\mathbf{A}$ frozen), it matters which entries are trained. We call this the *parameter placement problem*: which $k$ entries should be trained? Under supervised fine-tuning, random and informed subsets perform comparably in most settings. Under GRPO on base models, random placement fails to improve over the base model, whereas, on mathematical reasoning and on text-to-SQL, gradient-informed placement recovers standard LoRA accuracy. The gradient structure explains this behavior: SFT gradients are low-rank and directionally stable, so any subset accumulates coherent updates; GRPO gradients are higher-rank and less aligned across steps, so only elements with consistently signed gradients retain the learning signal. Our scoring procedure identifies these parameters in under 10 seconds at less than 0.5% of training cost. The selected parameters are concentrated in the attention value projection matrices across model families and scales (1.5B - 7B).