Reachability and Control of Directional Steps in Scale-Invariant SGD

In scale-invariant models, the learning rate controls optimization only indirectly. Weight decay changes the parameter norm and thus also influences gradient updates in directions that subsequently affect prediction. We study this coupling through directional forcing $B_t$, which is the input that learning rate and decay jointly supply to the effective-step dynamics. This viewpoint turns schedule design into an inverse path-control problem, requiring a choice of a `forcing path' and a characterization of how bounded weight decay can realize it. We characterize exactly the paths permitted by bounded decay multipliers, derive their optimal approximation, and solve exactly a noisy alignment problem whose optimal policy is shared by every finite horizon. We then establish a sharp stochastic classification of directional steps and prove almost-sure stationarity on the sphere under weak moment assumptions. The classification reveals that weight decay can change the decay exponent and summability class of effective steps even under a fixed harmonic learning-rate shape. The resulting design perspective has practical consequences. Across datasets, we find that a narrow forcing ramp improves mean accuracy in different tested settings, even over popular cosine learning rate setup. In a separate exact-symmetry experiment with signed decay, merely reversing an identical forcing multiset results in a massive accuracy drop of $30.34$ percentage points. Together, the results set foundations for making directional motion a tractable object of schedule design.