Beyond Adapter Rank: The Shared-Subspace Complexity of LoRA Training

Low-rank adaptation constrains each weight update, but its training checkpoints need not share a small input subspace. This distinction matters whenever one fixed representation must describe a training history or preserve its predictions. We study the minimum shared input dimension of a LoRA path and how the weighting of checkpoints affects its approximation. In aligned linear regression, simultaneous gradient descent produces rank-one checkpoints whose shared hidden input space has dimension $d-1$, even though the initialization space already contains a solution. We also derive the exact minimax error for shared linear measurements when the observer does not know the probe orientation. Spectral approximation, finite-horizon bounds, and a conditional network perturbation bound connect these results to measurable interventions. Across 24 language-model fine-tuning runs, paths move substantially outside the initial input space, yet a few shared directions retain 99% of their energy. Joint projections across adapted matrices approximately preserve aggregate prediction quality in language modeling and arithmetic reasoning, and outperform random directions at the same dimension. Similar aggregate accuracy nevertheless coexists with changes in individual decisions. These results distinguish instantaneous rank, shared path dimension, and predictive fidelity as separate properties of low-rank training.