Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers

Looped Transformers reuse shared layers to provide a parameter-efficient route to test-time scaling through iterative latent reasoning. Additional loops can lower support for a reference answer, but a harmful endpoint does not reveal whether the update direction is unhelpful or a useful direction was applied too far. We study this distinction by measuring reference utility, a continuous measure of support for the reference answer, along the model's own update path under teacher forcing. We vary the fraction of the proposed displacement supplied to the readout. A finite-step failure occurs when utility initially improves along the update direction but decreases at the full step. Pathwise curvature shows how this mismatch arises, while a local quadratic model predicts full-step utility changes and useful finite scales. Integrated curvature variation bounds the approximation error of these predictions. Across three models and three tasks, finite-step failures occur in all nine model–task conditions. A fixed quarter step recovers positive reference-utility gains for 72.2–83.2% of finite-step failures across four settings, and geometry-based scale selection further improves recovery on a full test split. These results identify update direction and step scale as distinct factors in recurrent progress, revealing recoverable progress within harmful updates and suggesting that step-scale control could help looped Transformers make better use of their existing computation.