acceptodds
Under review as a conference paper at ICLR 2027

Population Onset and Finite-Batch Resolution in One-Step Feature Learning

Abstract

A population gradient can point toward a task-relevant direction even when a finite empirical gradient cannot resolve it. We study this gap for one-step feature learning in two-layer networks with spiked Gaussian inputs. We identify two target-weighted covariance quantities with distinct roles. The population amplification sets the learning-rate scale at which task-directed overlap becomes order one, whereas the ratio gives a sufficient independent-batch scale for recovering that movement empirically. These scales can vary independently, so anisotropy can amplify feature movement without making it easier to estimate from finite data. Under a narrower smooth activation class and explicit triangular conditions, we derive at the critical batch scale a joint random law separating coherent task alignment from total representation movement. For rank-one spikes, an exact alignment–anisotropy map characterizes where population amplification, finite-batch difficulty, and the critical law arise; the population and critical results also extend to fixed-rank spikes. Direct-onset, factorial, and geometry-wide experiments support the predicted normalizations and expose finite-dimensional deviations. The results concern one-step representation dynamics, not downstream risk, and the recovery scale is sufficient rather than necessary.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.