acceptodds
Under review as a conference paper at ICLR 2027

Sustained Is Not Stable: A Reference-Free Test for Attractors in Free-Running Sequence Models

Abstract

Long-horizon generative sequence models are trained and scored under teacher forcing, which resets the state to ground truth at every step, and are deployed in closed loop, where nothing does. We show the objective is structurally silent on the closed loop: for a linear recurrence the one-step loss is flat, at a perfect fit, along readout directions that move the closed-loop spectral radius, and its sublevel sets contain both decaying and diverging loops at any residual. Across 64 identically trained single-layer state-space models the held-out loss is unrelated to that radius (Spearman +0.06, CI [-0.21, +0.34]), the radius alone classifies all 64 free-running outcomes, and free-running amplitude spans ten orders of magnitude. We give a reference-free test: scale the burned-in state by , run the model on its own output, and fit settled amplitude against on log-log axes. Slope 0 is an attractor. Slope 1 is a theorem: every positively homogeneous closed loop—LTI recurrences, bias-free ReLU stacks—returns it at any width, depth or training length. Calibrated on twenty known systems (25/26 verdicts), the test agrees with Lyapunov exponents and Floquet multipliers on every trained loop inside its basin, needs no gradients, Jacobians, reference or period, and reports finite basins that the local methods cannot see. Across eight architectures it locates the attractor: not in depth but in whether a non-homogeneous operation lies on the closed loop—a biased readout fed back at depth one supplies it, the same readout outside the loop does not, a two-layer bias-free ReLU stack returns exactly 1.000, depth 2 to 8 changes nothing—and normalisation does not create the attractor but decides whether it is global. On MIT-BIH a two-layer stack collapses on 26 of 32 seeds while beating persistence, and the collapsed seeds include the seven best by loss. The test certifies existence, not correctness; and existence cannot be assumed.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.