What Determines a Local Update Map?
Abstract
What information determines the result of local optimization? Two groupings of the same strictly convex quadratic losses can preserve the objective and complete separate gradient and Hessian marginals, yet produce different two-step updates. Their within-child pairing resolves this ambiguity. For quadratic children, agreement at the first horizons identifies all later endpoints at that input, where counts the actual union spectrum. Agreement at other inputs additionally depends on matrix-valued spectral weights: even an entire shared trajectory and matching eigenvalue distributions can leave the maps different elsewhere. We realize sharp information limits with partitions of the same atomic losses, including noncommuting children, and quantify the observation depth needed with noisy logs. Smooth remainder bounds and controlled logistic and trained-neural evaluations locate the range of leading-pairing prediction beyond quadratics. A ridge comparison illustrates a further distinction: covariance-aware recovery can resolve later-update comparisons that spectrum-only bounds leave undecided with the same early logs. The results separate information that controls the size of an update change from information that identifies the update itself.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.