Symmetrized Phase Retrieval: Infinite versus Finite-Size Dynamics in Learning
Abstract
Even in its simplest setting, phase retrieval is a difficult optimization problem with a rough, high-dimensional loss landscape. In the standard problem, glassy dynamics and finite-size effects obscure the thermodynamic learning transition. Here, we introduce a symmetrized variant, analyze its finite-size behavior numerically, and solve its infinite-dimensional gradient-flow dynamics using dynamical mean-field theory (DMFT). We show that sample complexity is linear in the input dimension and identify two thresholds: a dynamical threshold, above which gradient flow optimization typically generalizes under random initialization; and a landscape-trivialization threshold, at which nongeneralizing minima lose stability. At finite dimensions, the former is significantly lower and learning requires much less data than is necessary for landscape trivialization. In the infinite dimensional limit the thresholds coincide at a limiting value of samples per dimension. We understand this gap in terms of the loss landscape geometry: reflection symmetry confines nongeneralizing minima to the zero-overlap manifold, and retrieval occurs when the initial teacher overlap exceeds a critical value , which depends on the sample-to-dimension ratio . The competition between the initial overlap – which scales with the input dimension as – and – which decreases much more strongly – allows retrieval well before trivialization at finite . In addition, we show that, remarkably, determines not only the learning success but also the dynamical timescale. An approximate exponential-growth description captures the observed dynamics and yields a learning time logarithmic in . Thus, one scalar links basin geometry, retrieval probability, and learning timescale. Extensive finite-size simulations verify the predicted thresholds, retrieval and failure trajectories, and learning-time statistics.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.