Silver Steps Beyond Convexity: Accelerated Convergence for Low-Rank Matrix Recovery

Low-rank matrix recovery problems are often solved through a low-rank factorization, which turns the rank constraint into a nonconvex optimization problem. Recent work has shown that carefully designed Silver stepsize schedules can accelerate gradient descent for convex optimization. In this paper, we study whether this acceleration extends to nonconvex low-rank factorization for matrix recovery from linear measurements. We show that gradient descent with a fixed-depth Silver stepsize schedule achieves an accelerated convergence rate when initialized near a constrained minimizer, while all intermediate iterates remain in its neighborhood despite the occasional large steps in the schedule. We also show that, even in a simple rank-one problem, the same fixed-depth Silver schedule diverges with nonvanishing probability under arbitrarily small Gaussian initialization, demonstrating that the local initialization condition cannot in general be removed. We apply our results to Gaussian matrix sensing and low-rank multivariate regression, and provide numerical experiments that support our theoretical findings.