On Gating Dynamics: Spectral Condensation and Distinct Paths of Return

Motivated by the possibility that the Edge of Stability (EoS) accompanies a reorganization of neuron participation, we examine how ReLU gating evolves across training regimes. Binary neuron–sample supports let us distinguish how much activity is present, how participation is distributed, and how gates change over time. Across ReLU MLPs on FashionMNIST and CIFAR-10, support density initially rises while raw-support effective rank falls—a spectral condensation. Subsequent rank recovery and density decrease follow distinct paths of return. In shallow FashionMNIST, the higher-rate run recovers rank and renews gate switching near EoS, whereas the lower-rate run remains nearly frozen with little rank recovery. Renewed switching becomes largely balanced exchange, with the net on–off current exhibiting approximate period-two recurrence. Yet deep-layer density–rank return precedes the late increase in gate switching, and returns occur both with a larger low-participation population and, in CIFAR-10, with little such population and weaker class-associated sharing. The SGD examples also exhibit condensation and return, but with gate switching present from the outset. Together, these contrasts motivate a hypothesis that participation structure shapes the available paths of return from spectral condensation, while gate switching influences how those paths are traversed.