From Spectral Measurement to Feedback Collapse: The Limits of Adaptive Orthogonal Optimization

Low-rank orthogonal optimizers reduce computation and communication but require a rank hyperparameter that is costly to tune. We ask whether this rank can be inferred directly from the momentum spectrum. We develop a spectral diagnostic based on random matrix theory, using corrected residual-energy and tail-quantile estimators to identify structured directions above the estimated noise edge. Experiments on GPT-style models from 60M to 350M parameters show that momentum spectra are gapless and approximately power-law, while sampling noise accounts for under $11\%$ of the estimated noise floor. The diagnostic remains stable across scales, selecting a rank fraction of $0.353$–$0.361$ as model size grows sixfold, but validation loss continues to improve beyond the estimated rank, indicating that it provides a lower bound on useful rank rather than an optimization optimum. When used online with error feedback, the criterion induces a positive feedback loop that drives $83$–$88\%$ of weight matrices toward rank one. A pool-wide subtraction modification eliminates this collapse across all tested scales. These results distinguish spectral rank measurement from adaptive rank control in low-rank orthogonal optimization.