acceptodds
Under review as a conference paper at ICLR 2027

Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers

Abstract

Looped Transformers reuse shared layers to provide a parameter-efficient route to test-time scaling through iterative latent reasoning. Additional loops can lower support for a reference answer, but a harmful endpoint does not reveal whether the update direction is unhelpful or a useful direction was applied too far. We study this distinction by measuring reference utility, a continuous measure of support for the reference answer, along the model's own update path under teacher forcing. We vary the fraction of the proposed displacement supplied to the readout. A finite-step failure occurs when utility initially improves along the update direction but decreases at the full step. Pathwise curvature shows how this mismatch arises, while a local quadratic model predicts full-step utility changes and useful finite scales. Integrated curvature variation bounds the approximation error of these predictions. Across three models and three tasks, finite-step failures occur in all nine model–task conditions. A fixed quarter step recovers positive reference-utility gains for 72.2–83.2% of finite-step failures across four settings, and geometry-based scale selection further improves recovery on a full test split. These results identify update direction and step scale as distinct factors in recurrent progress, revealing recoverable progress within harmful updates and suggesting that step-scale control could help looped Transformers make better use of their existing computation.

Primary area:
interpretability and explainable AI
Keywords:
Looped Transformers, Latent Reasoning, Recurrent Dynamics, Mechanistic Analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_udAmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies recurrent updates that initially increase reference utility but reduce it when applied in full. Using teacher-forced interpolation along the model’s proposed displacement, the authors distinguish finite-step failures from locally unhelpful directions. Local derivatives and curvature predict utility changes and useful step scales, while contraction experiments show that intermediate states can outperform both original endpoints. Experiments across Ouro and Huginn provide convincing evidence for this distinction under immediate reference-conditioned readout. The main limitations concern its implications for subsequent recurrence and generation, the reference dependence of adaptive scale selection, and the narrower validation of scale prediction for Huginn.

Critical strengths
  1. Separating update direction from step size gives a clear and useful account of why a harmful endpoint need not imply an unhelpful direction. The curvature analysis connects this distinction to testable predictions about intermediate states.

  2. The experiments cover two model families and multiple tasks, with comparisons against common state- and output-change diagnostics. The full-test-set Ouro/GSM8K intervention is particularly helpful: quadratic selection recovers positive reference-utility gains for 425 of 429 finite-step failures, compared with 302 for a quarter step.

Critical weaknesses
  1. The connection between immediate reference-utility recovery and actual generation remains uncertain. Intervened states are read out before further recurrence, so the experiments do not establish whether the benefit survives later updates or improves final answer accuracy. The authors clearly acknowledge this scope. Their HellaSwag example, where utility decreases while option accuracy increases, also shows why the two outcomes should remain distinct.

  2. Adaptive scale selection requires derivatives of reference utility. Its advantage over fixed scales therefore measures the value of per-example adaptation when reference information is available. The paper proposes using this geometry as offline supervision for inference-time decisions, but the evidence does not yet show how accurately useful scales can be selected without the reference.

  3. The Huginn scale-prediction results in Table 2 use a subset of the same 256 GSM8K training questions used to select the 19→2019\to20 transition. The paper provides full-test-set mechanism analysis for Huginn and additional test-set intervention results for Ouro, so this is not a general absence of held-out evaluation. However, Huginn’s scale-prediction accuracy and recovery gains still need validation beyond the transition-selection subset.

Questions for authors
  1. As a preliminary extension, could the authors test a fixed damping factor during free-running generation and compare final accuracy under matched computation budgets? This would help assess the practical implications of the immediate utility gains.

  2. The paper already reports a held-out GSM8K mechanism analysis with the 19→2019\to20 transition fixed, but the scale-prediction and contraction/recovery results for Huginn use the 256-question training cohort. Could these analyses also be evaluated on the held-out GSM8K test set using the same pre-specified transition and a matched initialization protocol?

5 comments
AuthorsOpenReview ↗

Follow-up on Our Responses and Revised Manuscript

Thank you again for your constructive suggestions. We have added the held-out Huginn evaluation and the continuation and generation experiments discussed in our response. Could you kindly review our responses and revised manuscript?

We hope this additional evidence addresses your concerns and would appreciate your updated assessment.

Thank you for the detailed responses and additional experiments. The results strengthen the paper and address some of my concerns. However, I have two questions:

  1. In the free-running generation experiment, the overall numerical-answer accuracy improvement is quite modest. Could the authors provide a more detailed analysis about this?

  2. Since reference-conditioned scale selection remains an important limitation, could the authors further clarify how the proposed local geometric insights could be translated into a practical reference-free scale prediction strategy?

Thank you for recognising that the additional results strengthen the paper.

1. More detailed analysis of the modest generation improvement.

Thank you for suggesting a closer analysis. The small net gain reflects corrections that are almost offset by new errors. The quarter step corrected six answers and made five previously correct answers wrong, changing the score from 52/128 to 53/128.

We compared the generated solutions to understand these changes. In one GSM8K question, the quarter step corrected the answer by using division instead of multiplication. In another, it introduced an arithmetic error. We used the same factor of 1/4 for the final recurrent update of every generated token. The six corrections show that changing step scale can improve individual generated answers at the same recurrent depth. This gives a practical reason to ask which updates benefit from shortening and by how much. Our geometric analysis addresses this question for reference utility by identifying useful shorter steps within harmful updates and predicting their scales (revised §\S 3.3 and §\S 4.4).

2. Translating the geometry into reference-free scale prediction.

Thank you for raising this question. We found that some harmful updates improve reference utility when shortened, and local geometry helps identify useful scales. A natural way to use this finding is to train a scale predictor with those scales as labels. We would use reference answers to compute the training labels. During generation, the predictor would choose a scale using only the hidden state, the computed update and the loop depth.

For example, Ouro's exit gate decides whether to stop or continue with another complete loop [1]. In its second-stage training, a full loop receives a zero-improvement signal if it increases the next-token loss. Scale labels could indicate whether a shorter step would still lower that loss. Under teacher forcing on the GSM8K test set, quadratic selection recovered reference utility for 100 of 102 finite-step failures in Huginn. A quarter step recovered 71 (revised §\S 4.4).

During Ouro pretraining, the correct next tokens are already available. We could use local geometry to propose a shorter step and compare its next-token loss with that of the full step. If shortening lowers the loss, the shorter scale would be the training target. Otherwise, we would keep the full step as the target. Think-at-Hard already trains a decider from reference-based labels and uses hidden states at inference [2]. LoopFormer trains updates conditioned on step size [3], and LoopUS learns update gates [4]. These methods provide examples of learning when to continue or how to apply an update. Our geometric analysis adds scale labels based on which harmful updates still contain useful shorter steps. Ouro could use these labels during pretraining to learn when and how much to shorten an update, with the aim of lowering next-token loss at the same loop depth.


We hope the added analysis and training plan address your questions. We would be grateful if you could consider raising your rating.


[1] Scaling Latent Reasoning via Looped Language Models. ByteDance, 2025

[2] Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning. PMLR 2026

[3] LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation. ICLR 2026

[4] LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models.

Reviewer_no4nOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates finite-step failures in looped Transformers, a phenomenon where shared-layer iterative latent reasoning proposes a locally useful update direction that ultimately degrades reference utility when fully applied. The authors propose a pathwise geometric framework using Taylor-integral curvature corrections, local quadratic approximations, and integrated curvature variation bounds to predict full-step utility changes and identify optimal step scales. Through empirical evaluations across models (Ouro, Huginn) and benchmarks (MATH-500, GSM8K, HellaSwag), the study demonstrates that simple step contractions or quadratic scale selections can systematically recover reference-utility progress within harmful updates.

Critical strengths
  1. Introduces a pathwise geometric framework that formally distinguishes unhelpful update directions from useful directions applied too far in looped Transformers.

  2. Proves that local quadratic approximations and integrated curvature variation bounds can accurately predict optimal finite step scales and bound approximation errors without requiring architectural retraining.

Critical weaknesses
  1. The main weakness is that the paper is hard to read, and its text seems very much AI written, without proper human proofread. There are several hiccups as I read through the paper, with connections/transitions unclear.

  2. The evaluations and theoretical guarantees are derived under static, single-step teacher-forced conditions, leaving open how these recovery mechanics and approximation errors compound over longer multi-step autoregressive generation horizons (e.g., Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach, Geiping et al., 2025).

  3. Certified numerical curvature bounds and explicit integrated variation envelopes are established primarily for affine-readout architectures (such as Ouro), while nonlinear readouts and joint option utilities (such as Huginn) lack certified upper bounds, limiting the universal applicability of the theoretical guarantees.

  4. The introduction motivates test-time scaling and looped Transformers, introducing the phenomenon where additional recurrent iterations degrade performance. While the text frames the problem around distinguishing unhelpful update directions from useful directions applied too far, readers might initially conflate these static, teacher-forced pathwise analyses with free-running autoregressive generation.

  5. The introduction and background reference various adaptive computation techniques, early-exit mechanisms, and residual-dependent damping methods. However, the conceptual boundary between heuristic stopping/damping proxies and the proposed local quadratic geometry with curvature bounds could be sharpened.

  6. The empirical evaluation measures progress primarily via reference utility (mean log-probability of reference tokens under teacher forcing) and analyzes updates immediately after scaling, without letting the model execute subsequent unconstrained auto-regressive decoding from the intermediate state. A potential concern is that local utility improvements at a specific token position under a constrained single-step intervention might not consistently translate into downstream answer correctness or sustained multi-step trajectory gains during free generation. Discussing or providing empirical context on how these local scalar improvements relate to final generation accuracy would strengthen the practical validity of the intervention results.

Questions for authors

Please see the weaknesses.

2 comments
AuthorsOpenReview ↗

Follow-up on Our Responses and Revised Manuscript

Thank you again for your detailed feedback. The revised manuscript includes additional theoretical derivations, numerical evaluation for Huginn, and experiments beyond the immediate readout, alongside substantial presentation improvements. Our follow-up identifies where each change appears.

We would be grateful if you could review these updates and reconsider your assessment where your concerns have been addressed.

Dear Reviewer no4n,

As the discussion period is drawing to a close, we wanted to briefly follow up on our responses and revised manuscript. In particular, we added the interval-based curvature bounds for nonlinear readouts and verified them across Huginn’s held-out GSM8K test paths (Table 3), included the multi-step continuation and free-running generation evaluations (Table 4, Appendix E.10), and restructured Sections 1–3 to clarify the teacher-forced setting, terminology, and relation to early exit and damping.

Please let us know if anything remains unclear or if there are any remaining questions we can address before the deadline. If our revisions have resolved your concerns, we would be very grateful if you could consider raising your assessment and scores accordingly. Thank you again for your time and helpful feedback.

Reviewer_MHy5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies why additional recurrent computation can reduce reference utility in looped Transformers. It identifies finite-step failures: having the right direction at the beginning but take too big a step thus ends up as a negative recurrence.

The authors analyse this discrepancy through pathwise curvature, use a local quadratic approximation to predict utility changes and step scales, and derive approximation-error bounds. Experiments across three models and three tasks identify finite-step failures in all nine model–task conditions and show that shortening updates either by a quarter, or further choosing step size with quadratic model can recover utility gains.

Critical strengths
  • Interesting phenomenon and a intuitive analytical perspective. The paper identifies harmful updates that contain recoverable progress and analyses this phenomenon by separating update direction from step scale (interpolation).

  • Theory is connected to empirical evaluation. The curvature analysis motivates concrete predictions of full-step utility changes and useful step scales, which are then tested against measured outcomes.

  • Consistent evidence across models and tasks. Finite-step failures appear in all 3 models x 3 tasks conditions, and both fixed step contraction and adaptive scale selection recover positive utility gains.

Critical weaknesses
  • Experimental significance and applicability

    • What about actual generation performance? The experiments show recovery in reference utility under teacher forcing, can this improves generation (NTP) accuracy remains unanswered.

    • Adaptive scale selection requires ground-truth answers. The proposed method based on reference utility, so it cannot directly select step scales when the answer is unknown.

  • Clarity and presentation

    • Unexplained terminology. Many terms in Introduction are not explained or even do not appear twice through out the manuscript: “learned updates”, “reference support”, “this local prediction”, “local quadratic geometry”, “the error of the local predictions”, “pathwise analysis”.

    • Delivery logic needs polishing The Introduction discusses local predictions and their errors before clearly establishing the underlying approximation. It also reports recovery results before explaining the intervention and evaluation protocol (line 78, last sentence of intro before listing out contributions) needed to interpret them.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Follow-up on Our Responses and Revised Manuscript

Thank you again for your helpful feedback and for recognising the value of our findings. We have submitted the revised manuscript with clearer explanations and additional recurrence and generation experiments.

We would appreciate it if you could review our responses and the revision and reconsider your assessment in light of these updates.

Thank you for addressing my concerns and revising the manuscript. I still have substantial reservations about the generation results: the improvement from 52/128 to 53/128, with six corrections offset by five new errors, is not convincing evidence of a generation accuracy benefit.

That said, I find your response to Reviewer udAm’s second question reasonably persuasive regarding the potential applicability of the analysis, although this remains to be validated. Taking this into account, I am raising my rating from 2 to 3.

Thank you for reading our follow-up discussions, reviewing the revised manuscript, and raising your rating.

We agree with your assessment of the free-running generation result: a change from 52/128 to 53/128, where six corrected answers are offset by five new errors, shows that applying a fixed quarter step uniformly to every token is too blunt to yield a consistent generation accuracy gain. In the revised manuscript, we present this comparison in Appendix E.10 specifically to illustrate that trade-off rather than to claim a net generation improvement. We have also incorporated the training-target discussion from our response to Reviewer udAm into §\S 5 and Appendix E.8, clarifying how the update-specific scales identified by our geometric analysis during training—when reference answers are available—can provide supervision for learning when and how much to shorten an update during generation.

Thank you again for your constructive feedback, which helped us sharpen both the experiments and the framing of our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Response and Revisions

We thank the reviewers, the Area Chair, and the Senior Area Chair for their feedback. Looped Transformers reuse shared layers to perform more computation, yet an additional loop can lower the model’s score for a reference answer. When this happens, is the update direction unhelpful, or did the model move too far along a useful direction? If it moved too far, can a shorter step recover the lost progress?

We hold the proposed direction fixed, vary the step size, and measure reference utility, the score for the reference solution or correct option under teacher forcing. A finite-step failure occurs when utility initially rises along the update but falls below its starting value at the full step. The decomposition in §\S 3.2 shows how negative curvature accumulated along the update can outweigh the initial gain. We observe these failures in all nine model and task settings, where they account for 1.4%–49.2% of harmful updates (Table 1).

A positive initial slope guarantees improvement for a sufficiently small step. To predict the gain at larger steps, we use the initial slope and curvature to approximate utility along the update. This quadratic approximation predicts the sign of the full-step change with 97.2%–99.9% accuracy across the nine settings (Table 1). It also predicts the scale that maximises reference utility. When the initial curvature is negative, Theorem 3.6 bounds the error in that scale and the resulting utility loss in terms of how much curvature varies from its initial value.

We tested recovery by applying shorter steps along the same direction. A fixed quarter step raised reference utility above both original endpoints in 72.2%–83.2% of finite-step failures across four settings. On the GSM8K test set, we used the quadratic rule to choose a scale for each update. Compared with the fixed quarter step, recoveries rose from 302 to 425 of 429 failures on Ouro-1.4B (Table 13) and from 71 to 100 of 102 on Huginn (§\S 4.4). In these cases, the model had already computed a useful direction. Shortening the step recovered the lost progress.

In the revision, we made three changes in response to the reviews.

  • Nonlinear readouts and held-out prediction. We derived an interval bound that applies to nonlinear readouts (Proposition A.8). Across 713 eligible updates on Huginn’s held-out GSM8K test set, predicted scales and grid optima had a Spearman correlation of 0.9755 (Table 2). On all 712 paths that met the numerical convergence checks, both the observed scale error and the utility loss fell below their respective bound values computed from numerical estimates of curvature variation ($\S$4.3).
  • Later loops and generated answers. After a single shorter step in Ouro, mean reference utility remained higher than in the unmodified model three loops later under teacher forcing (Table 4). During Huginn generation, a fixed quarter step for the final update of each token corrected six answers and introduced five new errors, for a net gain of one correct answer out of 128 (Appendix E.10). Since the same scale helped on some questions and hurt on others, we would like to learn when and how much to shorten each update. Reference answers are already available during training—for example, when Ouro trains its exit gate from improvements in task loss—so the scales we identify can serve as targets for learning to choose a step size from the model’s own states during generation ($\S$5, Appendix E.8).
  • Clearer presentation. We standardised terminology throughout the paper, introduced the intervention before its predictions, and clarified its relation to early exit. An exit gate chooses whether to run another loop. Our analysis asks how much of the update proposed by that loop is useful ($\S$1–3).

We hope this work helps researchers interpret why an additional loop can hurt and gives them a principled way to study how far each recurrent update should move.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.