When Does Learning-Rate Retuning Help? Attainable Output Geometry Under a Fixed Update Budget

Learning-rate schedules are often tuned for one training setting and reused when models are adapted to new data under fixed update budgets. A finite-step optimization trajectory can itself act as a statistical regularizer. We study when retuning the schedule after a task change improves on preserving the nominal finite-step filter. We characterize the leading retuned-minus-preserved risk through the statistical distance between the oracle drift and the one-sided attainable output set. For $K$ distinct interior steps and a simple spectrum in $p$ dimensions, the attainable correction space has dimension $\min (K,p)$. Some spectral corrections are therefore inaccessible when $K<p$, whereas the unrestricted commuting optimum becomes locally attainable when $K\ge p$. At equal steps, $O(\sqrt{|r|})$ step splitting generates first-order output corrections missed by the parameter Jacobian, where $r$ indexes the local task change. For two steps, this singular geometry is the endpoint of a near-equal transition governed by ${\delta }_r^2/|r|$, where ${\delta }_r$ is the nominal half-gap. These results explain why unrestricted spectral retuning can overestimate attainable corrections while local Jacobian sensitivity can underestimate them. Synthetic experiments support the theoretical predictions, while real-data diagnostics distinguish preservation, schedule reuse, and retuning.