acceptodds
Under review as a conference paper at ICLR 2027

Decoding Looped Transformers Better for (Almost) Free

Abstract

Looped Transformers achieve parameter efficiency by repeatedly executing a shared block across recurrent loops. Each loop yields an intermediate representation that is decodable for the same next token, yet standard decoding discards earlier states. Since earlier loops embody less computation, recurrence inherently supplies aligned weak-and-strong prediction pairs without auxiliary models or external training. We introduce , a training-free contrastive decoding method that guides token selection by contrasting the final prediction with an earlier recurrent pass, operating either in logit space with one extra output pass (LoopCD-Logits) or in hidden-state space with zero output overhead (LoopCD-Hidden). Across four looped Transformer families, LoopCD delivers substantial, consistent gains at full recurrent depth: LoopCD-Logits raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%, while LoopCD-Hidden lifts Huginn's HumanEval pass@1 from 22.56% to 31.71%. Crucially, these performance gains enable halving the number of recurrent loops while still matching or exceeding full-depth unguided baselines, reducing forward FLOPs by 22.5% to 48.2%. By transforming intermediate recurrent states into effective guidance signals, LoopCD achieves superior decoding quality while substantially reducing inference compute.

Primary area:
foundation or frontier models, including LLMs
Keywords:
looped transformers, recurrent computation, contrastive decoding, test-time compute

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_QwJ7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a training-free contrastive decoding method for looped Transformers called LoopCD. The author finds that intermediate recurrent states already provide weaker predictions, while the final state provides stronger predictions for looped Transformers. Thus, LoopCD guides token selection by contrasting the final prediction with an earlier recurrent pass, including two variants LoopCD-Logits and LoopCD-Hidden. LoopCD-Logits contrasts logits after the coda and LM head and requires one additional output pass, and LoopCD-Hidden combines hidden states before the coda and outputs without extra output projection. Experiments show that LoopCD matches or exceeds the unguided full-depth baseline while reducing forward FLOPs.

Critical strengths
  1. The idea is motivated and clear to understand.
  2. The paper evaluates the method across several looped Transformer families, which makes the empirical claim more convincing.
  3. The paper provides a mechanistic analysis. The authors analyze the reference state selection, the concentration of gains on low-confidence samples, and the finding that guidance primarily works by re-ranking close decisions.
Critical weaknesses
  1. Computational benefits are based on theoretical counts FLOPs. Since the method influences decoding behavior and may interact with batching, caching, and output-layer cost, the practical latency and benefits remain unclear. It is better to report end-to-end wall-clock latency.
  2. The reduced-depth experiments are limited to the multiple-choice setting. It lacks the performance of LoopCD in more challenging autoregressive generation settings such as code, math reasoning, or long-form generation.
  3. The method depends on choosing the reference recurrent state. The authors can claim more clearly how robust the method is under practical use where tuning data may not be available.
Questions for authors

See Weaknesses.

3 comments

We thank the reviewer for the positive assessment and the concrete suggestions, and we address each point below.

W1. End-to-end latency. We have measured wall-clock time (bf16, HuggingFace implementation with a KV cache, one A100). The measured time of every setting stays within 0.02 of its FLOP ratio at batch size 1 and within 0.05 at batch size 16.

ModelSettingFLOP ratioDecode, ms/token (ratio)
Ouro-1.4B4 loops, unguided1.0086.6 (1.00)
Ouro-1.4B4 loops, LoopCD-Logits1.0287.2 (1.01)
Ouro-1.4B2 loops, LoopCD-Logits0.5344.0 (0.51)
Huginn32 iterations, unguided1.00136.3 (1.00)
Huginn32 iterations, LoopCD-Logits1.02138.2 (1.01)
Huginn32 iterations, LoopCD-Hidden1.00135.6 (0.99)
Huginn16 iterations, LoopCD-Logits0.5472.9 (0.53)
Huginn16 iterations, LoopCD-Hidden0.5270.6 (0.52)

On the other interactions you list, LoopCD-Hidden adds no cache, and LoopCD-Logits caches one extra coda pass, which is empty for Ouro (it has no coda) and 1.5% of the KV cache for Huginn. LoopCD-Logits also runs the coda and head twice per token, which accounts for the measured overhead of at most 2% per decoded token.

W2. Reduced depth on generation and reasoning. The paper already reports Huginn on code at 16 of its 32 iterations (Tables 1b and 3b), and we have added Ouro at three of its four loops on code and AIME. Because Ouro has no prelude or coda, dropping one loop saves 22.5% to 23.75% of the FLOPs. With LoopCD, both models outperform their full-depth unguided baselines on code, measured as mean pass@1 over HumanEval, HumanEval+, MBPP, and MBPP+, and Ouro-Thinking at three loops also outperforms four unguided loops on mean pass@1 over AIME 2024 and 2025. Every row except the LoopCD-Hidden one uses LoopCD-Logits with the adaptive rule, and all runs reuse the full-depth strengths without re-tuning.

Model, metricDepthFull depth, unguidedReduced depth + LoopCD
Huginn, code mean (LoopCD-Logits)32 → 1629.4731.54 (+2.07)
Huginn, code mean (LoopCD-Hidden)32 → 1629.4731.26 (+1.79)
Ouro-1.4B, code mean4 → 367.4470.29 (+2.85)
Ouro-2.6B, code mean4 → 371.8876.42 (+4.54)
Ouro-1.4B-Thinking, AIME 2024/2025 mean pass@14 → 344.9046.77 (+1.87)
Ouro-2.6B-Thinking, AIME 2024/2025 mean pass@14 → 355.7361.25 (+5.52)

W3. Reference choice without tuning data. LoopCD-Logits needs no reference choice: it always uses the first iteration, for every model and task. LoopCD-Hidden uses h1h_1 for Parcae and Looped-Qwen3. For Huginn, every state from h4h_4 to h8h_8 improves the multiple-choice mean (+0.31 to +0.86), so the choice within that range matters little. The strength needs only one value per task type, and each value sits inside a band of strengths that all help (Appendix G).

We thank the reviewer again and are happy to clarify further.

Thank you again for your positive assessment. As the discussion period closes shortly, we would be glad to know whether our response has addressed your concerns.

Thanks for addressing my concerns. I will maintain my rating.

Reviewer_cxrgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LoopCD, a training-free contrastive decoding method for looped Transformers. It uses the first recurrent iteration as a built-in weak reference and extrapolates the final prediction away from it, either in logit space (one extra coda/head pass) or in hidden-state space (no extra pass). Across Ouro, Huginn, Parcae and Looped-Qwen3, it improves math, code and multiple-choice results. At half depth, it matches the full-depth baseline on the multiple-choice suite with 22.5–48.2% fewer FLOPs.

Critical strengths
  1. Reasonable and applicable idea: Looped models already produce a weaker prediction along the way, so there is no need for a separate small model as in standard contrastive decoding [1]. The method works on four quite different model families, and the hidden-state version adds almost no cost.
  2. Informative empirical analysis: The authors show that the gain depends on how much the reference disagrees with the final prediction, not on how accurate the reference is. Unlike in dense models [2], layers inside a loop make poor references. Most of the gain comes from changing the order of close candidates when the model is unsure.

[1] Xiang Lisa Li, Ari Holtzman, Daniel Fried, et al. Contrastive Decoding: Open-ended Text Generation as Optimization. ACL, 2023.

[2] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, et al. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models. ICLR, 2024.

Critical weaknesses
  1. Setting-dependent tuning and potential test-set tuning. Several choices seem to vary across settings. The value of ω\omega changes, e.g., 0.50.5 for multiple-choice but 0.20.2–0.30.3 for generation, 1.01.0 for Parcae-1.3B hidden-state guidance, and ωmax⁡\omega_{\max} ranges from 0.250.25 (Huginn generation) to 1.51.5 (Ouro-2.6B-Thinking) (Table A5). The reference state is also not fixed: Huginn hidden-state guidance uses h6h_6 at R=32R=32 and h7h_7 at R=16R=16 for multiple-choice but the reverse for generation. These values appear to be selected by sweeps on the evaluation benchmarks without another split.
  2. The mechanistic explanation remains incomplete. Ouro [1] finds that running more loops than it was trained with lowers accuracy, while pushing the final prediction further in the same direction improves it here. What makes the two behave differently is not analyzed. Besides, the gains on math reasoning are much larger than on other tasks, but the mechanism analysis covers only multiple-choice tasks. Why guidance helps math so much remains unexplained.

[1] Rui-Jie Zhu, Zixuan Wang, Kai Hua, et al. Scaling Latent Reasoning via Looped Language Models. arXiv:2510.25741, 2026.

Questions for authors
  1. Why are the gains on math reasoning much larger than on other tasks?
  2. Running Ouro beyond its trained depth hurts accuracy [1], yet extrapolating along the same direction helps here. What explains this difference?
  3. How were the strength rule, ω\omega, and the reference state chosen for each setting? Do the gains hold with a single fixed setting or one selected on a held-out validation split?

[1] Rui-Jie Zhu, Zixuan Wang, Kai Hua, et al. Scaling Latent Reasoning via Looped Language Models. arXiv:2510.25741, 2026.

3 comments

We thank the reviewer for the clear and constructive feedback and address each point below.

W1 and Q3. Setting-dependent tuning. The values can differ across settings, but the gains do not depend on choosing them precisely. Multiple choice uses a single ω=0.5\omega = 0.5 for all seven model configurations, and in the strength sweep (Appendix G.1) every swept model also gains at ω=0.25\omega = 0.25, while the mean gain over the six models stays positive for every ω\omega from 0.25 to 1.0. Greedy generation uses a smaller strength because token shifts compound over the generated sequence, and Ouro, Huginn, and Looped-Qwen3 all gain at both ω=0.2\omega = 0.2 and 0.30.3 in the generation sweep (Appendix G.1). For Parcae-1.3B, hidden-state guidance uses ω=1.0\omega = 1.0 because its eight-layer coda damps the hidden-state update, so the same strength moves the prediction less than it does in logit space (Appendix D.3).

On reasoning, the fixed rule alone raises the mean pass@1 of the two Ouro-Thinking models by 5.27 and 6.70 points (Table 1a), so the gains do not come from the adaptive caps. For Huginn's hidden-state reference, every state from h4h_4 to h8h_8 improves the multiple-choice mean (+0.31 to +0.86), so swapping h6h_6 and h7h_7 makes little difference.

The gains also hold under a single fixed setting. With the reference h1h_1 and ω=0.5\omega = 0.5 for every multiple-choice configuration and the fixed rule for reasoning, LoopCD-Logits already achieves the gains reported in Tables 2 and 1a. Every logit-space result reports both strength rules, so the rule is not chosen per setting.

W2 and Q2. Extra loops versus extrapolation. We have run Ouro beyond its trained depth and read out every loop on five multiple-choice benchmarks (ARC-Challenge, ARC-Easy, PIQA, SciQ, WinoGrande). The table reports the change in their mean from four unguided loops, alongside the four-loop gain of LoopCD-Logits (ω=0.5\omega = 0.5) from Table 2, measured against its own baseline.

Model4 loops (mean)5 loops6 loops7 loops8 loops4 loops + LoopCD
Ouro-1.4B77.51−0.63−0.82−1.06−1.36+0.47
Ouro-2.6B81.21+0.17+0.03−0.15−0.45+0.72

Both operations act mostly on close decisions. Extra loops change 10% to 21% of the answers in the least confident fifth of questions and almost none in the most confident two fifths, just as LoopCD's gains concentrate in the least confident fifth (Section 5.2). Their net effects, however, go in opposite directions. In the same runs, eight loops turn 241 correct answers wrong and 148 wrong answers correct on Ouro-1.4B (146 and 114 on Ouro-2.6B), whereas LoopCD at four loops turns 128 correct answers wrong and 155 wrong answers correct (116 and 176).

Their update directions also differ. Within the trained depth, each loop's change in the mean-centered logits aligns with the direction LoopCD extrapolates, z4−z1z_4 - z_1, with cosines of 0.84, 0.63, and 0.27 for loops 2, 3, and 4 on Ouro-1.4B, and 0.83, 0.54, and 0.26 on Ouro-2.6B. Beyond the trained depth this alignment disappears. The fifth loop's cosine drops to −0.17 on Ouro-1.4B and −0.02 on Ouro-2.6B, and later loops stay within 0.06 of zero. Extra loops therefore do not push the prediction further along this direction, whereas LoopCD continues the refinement that the trained loops have built.

W2 and Q1. Larger gains on math. The difference comes from how the two task types are evaluated (Appendix C.2). A multiple-choice question involves a single likelihood comparison, so LoopCD can change at most one decision per question. A sampled reasoning solution, by contrast, involves hundreds of token-level decisions, and LoopCD re-ranks the close ones, which is where its effect concentrates (Section 5.2). These per-decision effects accumulate over a solution, and pass@k credits a problem as solved if any of its k samples succeeds.

We hope these results address the reviewer's concerns, and we are happy to clarify further.

Thanks for addressing my concerns about Ouro and the math gains. The robustness results are helpful, but it remains unclear which data were used to select the parameters. And choosing task-specific and model-specific settings still raises concerns about practical usability. I therefore maintain my original rating.

Thank you for the follow-up. We further clarify which data were used to select the strengths and why the selected settings remain practical to use.

Selection data. The strengths were chosen from sweeps on the evaluation benchmarks. And the sweeps in Appendix G show that the gains do not specifically depend on a single choice. On multiple choice, all six swept models gain at both ω=0.25\omega = 0.25 and 0.50.5, and their mean gain stays positive for every fixed ω\omega from 0.25 to 1.0 and every adaptive cap from 0.25 to 2.

Shared strengths across models. No setting is tuned per benchmark. The fixed rule uses ω=0.5\omega = 0.5 for all seven multiple-choice configurations, and in greedy generation Ouro, Huginn, and Looped-Qwen3 all gain at both 0.2 and 0.3. For sampled reasoning, the fixed rule with the same ω=0.5\omega = 0.5 as multiple choice raises the mean pass@1 of both Ouro-Thinking models by 5.27 and 6.70 points (Table 1a). At full depth, the logit reference is always the first iteration. Ouro at three loops reuses the four-loop strengths without re-tuning and still outperforms four unguided loops on code and AIME.

We hope this resolves the remaining concern, and we would be grateful if you could reconsider your rating.

Reviewer_t5dkOpenReview ↗
Rating
2(−1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LoopCD, a training-free decoding method for improving the predictions of looped language models. It amplifies the change from an early loop’s prediction or hidden state to the final one, which can change uncertain token decisions. This follows the idea of contrastive decoding with a weak and strong model pair, with the looped model itself providing both predictions. The authors introduce a variant in logit space that requires two passes through the output layers (LoopCD-Logits) and one in hidden-state space that adds negligible computation (LoopCD-Hidden). Experiments on Ouro, Huginn, Parcae, and Looped-Qwen3 show improvements on several reasoning, code generation, and multiple-choice benchmarks. The method also allows reducing the number of loops (and thus compute FLOPs) while matching the unguided baseline's performance.

Critical strengths

1. Simple and training-free method. The method uses states already computed by the model and requires no additional training or separate reference model. The hidden-state variant is important to have for models with large codas because it avoids much compute overhead. The practical guidance on choosing reference states and guidance strengths, mainly in the appendix, also helps readers apply the method.

2. Consistent gains across many model families. The experiments show improvements across four model families with different architectures and training methods, including substantial gains on some reasoning and code generation benchmarks. The benefit seems to generalize across looped models. Additionally, the analysis in Sec. 5 gives useful insight into how reference states work and that guidance mainly changes uncertain predictions.

Critical weaknesses

1. Limited discussion of limitations and trade-offs. Many important limitations are not in the main text but scattered throughout the appendix. For example, sensitivity to guidance strength (Apdx. G.1 and G.2), model- and task-dependent reference selection (Apdx. B.4, Table A5, and Apdx. E.1), performance declines on some tasks or settings (Apdx. C.1 and C.2, Tables A6 and A7), and theoretical FLOP savings vs actual runtime (lines 1010–1012). Unmentioned I think are memory overhead from keeping the reference state or logits, sensitivity of selected settings across tasks, and compatibility of LoopCD with important LLM serving features like prefix caching. An explicit limitations section and qualifications in the main text would make the paper more informative.

2. Incomplete evaluation and unclear parameter-selection. The reduced-depth experiments in Sec. 4.3 omit Ouro, without explaining why. The two LoopCD variants are also evaluated on different subsets of models and tasks, sometimes with different prompting settings and separate baselines. This makes their relative benefits difficult to assess. I would like to see a matched comparison at least for the main results or an explanation why not possible in the manuscript. Additionally, the paper does not clearly state how to select reference states and guidance strengths across tasks. E.g., the data used to choose their reference states and guidance strengths are not reported.

Additional comments:

3. Comparison with other looped LM inference methods. Since this paper is an inference paper for looped LMs, it should discuss its relationship and compatibility to other looped inference strategies, including early exit [R1,R2,R4], parallel loop sampling [R3], KV-cache sharing [R1,R2], and serving methods [R4]. At least in the related works section in Appendix H.

4. Comparison with zero init for Huginn. Sec. 5.1 attributes the benefit of a later reference state in Huginn to its random initialization. However, Huginn also supports zero init. Comparing reference-state selection under both settings would help test this explanation.

References

[R1] Geiping et al. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. NeurIPS 2025.

[R2] Zhu et al. Scaling Latent Reasoning via Looped Language Models. arXiv, 2025.

[R3] Geiping et al. Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models. arXiv, 2025.

[R4] Schwethelm et al. Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching. arXiv, 2026.

Questions for authors
  1. Related to weakness 1: What do you see as the main limitations and trade-offs of the method, and when would you advise against using it?
  2. Related to weakness 1: Is LoopCD-Hidden only applied during decode or also during prefill (since guidance during prefill would affect the coda KV cache entries)? Similarly, how does LoopCD-Hidden affect KV caching and prefix-reuse of the coda?
  3. Related to weakness 2: Which data were used to select the reference states and guidance strengths? Do we need to calibrate the parameters to each new task and model?
  4. Related to weakness 2: Can LoopCD at two loops match the performance of unguided Ouro at four loops (the comparison missing from Sec. 4.3)?
  5. Related to weakness 2: Why do the evaluations of LoopCD-Logits and LoopCD-Hidden use different settings and benchmarks? Could you provide matched evaluations for the main results?
  6. Related to weakness 3: Is LoopCD compatible with the mentioned looped inference methods?
  7. Related to weakness 4: Does Huginn still benefit from a later hidden reference state when using zero initialization?
3 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for the detailed and constructive feedback.

W1 and Q1. Limitations and trade-offs. We will add a Limitations paragraph to the main text. Regarding the trade-offs you raise, wall-clock time measured on a single A100 stays within 0.02 of the FLOP ratio, and LoopCD adds at most 2% per decoded token:

ModelSettingFLOP ratioDecode, ms/token (ratio)
Ouro-1.4B4 loops, unguided1.0086.6 (1.00)
Ouro-1.4B4 loops, LoopCD-Logits1.0287.2 (1.01)
Ouro-1.4B2 loops, LoopCD-Logits0.5344.0 (0.51)
Huginn32 iterations, unguided1.00136.3 (1.00)
Huginn32 iterations, LoopCD-Logits1.02138.2 (1.01)
Huginn32 iterations, LoopCD-Hidden1.00135.6 (0.99)
Huginn16 iterations, LoopCD-Logits0.5472.9 (0.53)
Huginn16 iterations, LoopCD-Hidden0.5270.6 (0.52)

LoopCD-Hidden requires no extra cache, whereas LoopCD-Logits caches one additional coda pass, which is empty for Ouro and adds 1.5% to the KV cache for Huginn. Because the prompt is processed exactly as in unguided decoding, prefix caches remain unchanged. On strength, every swept model improves on multiple choice at both ω=0.25\omega = 0.25 and 0.50.5. Greedy generation needs a smaller strength, and its default of 0.2 to 0.3 sits well below about 0.6, beyond which accuracy drops sharply (Appendix G).

As for when to use which form, LoopCD helps most where the final prediction is uncertain and changes little where it is already confident (Section 5.2). LoopCD-Hidden adds no compute, though its reference state depends on the model. LoopCD-Logits uses the first iteration for every model, at up to 2% extra decoding time.

W2 and Q4. Ouro at reduced depth. We have added experiments on Ouro at three and two of its four loops, on code generation and on AIME. Because Ouro has no prelude or coda, each of its four loops accounts for about a quarter of the compute. Running three loops with LoopCD-Logits saves 22.5% (Ouro-1.4B) and 23.75% (Ouro-2.6B) of the forward FLOPs, comparable to halving Parcae-370M (22.5%) or Looped-Qwen3 (23.6%), and still outperforms unguided Ouro at four loops in every generation and reasoning setting under both strength rules. Mean pass@1 rises by 1.75 to 4.54 points on code (HumanEval, HumanEval+, MBPP, and MBPP+) and by 1.87 to 5.73 points on AIME (AIME 2024 and 2025). All runs reuse the four-loop strengths without re-tuning.

Model, metric4 loops, unguided3 loops + LoopCD (fixed / adaptive)2 loops + LoopCD (fixed / adaptive)
Ouro-1.4B, code mean67.4469.69 (+2.25) / 70.29 (+2.85)60.63 / 61.47
Ouro-2.6B, code mean71.8873.63 (+1.75) / 76.42 (+4.54)71.59 / 70.86
Ouro-1.4B-Thinking, AIME mean pass@144.9050.00 (+5.10) / 46.77 (+1.87)28.55 / 28.54
Ouro-2.6B-Thinking, AIME mean pass@155.7361.46 (+5.73) / 61.25 (+5.52)44.58 / 43.44

At two loops, LoopCD falls short of four unguided loops. We believe the two-loop readout itself is too weak: its final state sits only one loop past the reference h1h_1, so the direction LoopCD extrapolates is not yet stable. We will report the full results in the revision.

W2 and Q5. Matched comparison of the two forms. We have aligned the generation settings of LoopCD-Hidden with those of LoopCD-Logits and rerun LoopCD-Hidden on Huginn's code benchmarks:

HuginnHumanEvalHumanEval+MBPPMBPP+Mean
R = 32, unguided23.1720.1242.0632.5429.47
LoopCD-Logits, fixed27.44 (+4.27)23.78 (+3.66)44.97 (+2.91)37.57 (+5.03)33.44 (+3.97)
LoopCD-Logits, adaptive28.66 (+5.49)24.39 (+4.27)41.2733.86 (+1.32)32.05 (+2.58)
LoopCD-Hidden30.49 (+7.32)26.83 (+6.71)42.59 (+0.53)33.86 (+1.32)33.44 (+3.97)
R = 16, unguided20.1217.6840.7432.0127.64
LoopCD-Logits, fixed26.83 (+6.71)20.73 (+3.05)41.01 (+0.27)34.13 (+2.12)30.68 (+3.04)
LoopCD-Logits, adaptive28.05 (+7.93)25.61 (+7.93)39.6832.80 (+0.79)31.54 (+3.90)
LoopCD-Hidden27.44 (+7.32)21.95 (+4.27)42.06 (+1.32)33.60 (+1.59)31.26 (+3.62)

We will replace Table 3b with these results.

(continued in part 2)

AuthorsOpenReview ↗

Rebuttal (2/2)

(part 2)

W2 and Q3. Selection and calibration. The strengths are not tuned per benchmark. We compared a few values and kept one per task type and model family, and the gains do not depend on choosing them precisely, because LoopCD helps over a wide range of strengths (Appendix G). For multiple choice, the single ω=0.5\omega = 0.5 raises the seven-benchmark mean for all seven configurations under both strength rules (+0.29 to +1.59, Table 2). In the sweep, all six swept models gain at both ω=0.25\omega = 0.25 and ω=0.5\omega = 0.5, and their mean gain stays positive for every fixed ω\omega from 0.25 to 1.0 and every adaptive cap from 0.25 to 2. For greedy generation, the selected 0.2 to 0.3 sits well below about 0.6, beyond which accuracy drops sharply. For sampled reasoning, the fixed rule and the adaptive caps of 1.0 and 1.5 all raise the mean pass@1 of both Ouro-Thinking models, by 5.27 to 7.33 points (Table 1a).

Adapting to a new task therefore needs little calibration. The logit reference is always h1h_1, and one strength per task type suffices: 0.5 for multiple choice, 0.2 to 0.3 for greedy generation, and 0.4 to 0.5 for sampled reasoning. The hidden-state reference depends on the model: h1h_1 works directly for Parcae and Looped-Qwen3, while for Huginn any state from h4h_4 to h8h_8 improves the multiple-choice mean (+0.31 to +0.86).

W4 and Q7. Zero initialization for Huginn. Yes, Huginn still benefits from a later reference under zero initialization. We use R=32R = 32 and LoopCD-Hidden with ω=0.5\omega = 0.5, test references h1h_1 to h8h_8, and report the change in mean accuracy over ARC-Challenge, WinoGrande, and SciQ (0-shot) from each initialization's own unguided baseline.

Initializationh1h_1h2h_2h3h_3h4h_4h5h_5h6h_6h7h_7h8h_8
Gaussian (default)−0.28−0.47−0.06+0.41+0.70+0.68+1.01+0.64
Zero−0.51−0.47−0.11+0.21+0.59+0.78+0.72+0.38

Gaussian initialization therefore does not explain the need for a later reference. We believe the reason is a transient burn-in of Huginn's recurrence itself, which occurs regardless of the initialization: its first hidden state lies far from the final one (relative distance 1.16 on ARC-Challenge, compared with 0.63 and 0.40 for the first states of Parcae-1.3B and Looped-Qwen3, which serve as references directly). We will revise the explanation in Section 5.1 accordingly and add this comparison to Appendix E.1.

Q2. Prefill and KV caching. LoopCD-Hidden is applied only during decoding. Because the prompt is processed exactly as in unguided decoding, the prefill caches of the prelude, the loop, and the coda are unchanged and can be reused or shared across decoding settings like any prefix cache. At each decoding step, the guided state of the new token passes through the coda once, as in unguided decoding. LoopCD-Logits changes only the output distribution and leaves every cache entry untouched.

W3 and Q6. Other looped inference methods. Yes. LoopCD only changes how the next-token distribution is formed from two states the loop already computes, so it can be combined with methods that change how many iterations run or how they are scheduled. We have tested it with Huginn's per-token early exit, where a token exits once its latent state changes by less than 3% between iterations, together with KV-cache sharing across iterations [R1]. We apply LoopCD-Hidden at each token's exit step, and tokens exit after about 24 of the 32 iterations with or without LoopCD:

SettingHumanEvalHumanEval+
Early exit + KV sharing [R1]25.0021.95
+ LoopCD-Hidden26.22 (+1.22)21.34

Decode-time cache reuse [R2] changes only which past keys and values an iteration reads. The parallel sampler of [R3] refines the latent states of several tokens at once, and continuous depth batching [R4] regroups tokens into new batches between loop steps. In both, every token still passes through the first iteration and the iteration its prediction is read from, so LoopCD still applies token by token. Integrating LoopCD into these samplers and serving systems is an engineering step we leave to future work. We will add this discussion and the result to the related work in Appendix H.

We thank the reviewer again and are happy to clarify further.

Thank you again for your detailed review. As the discussion period closes shortly, we would be glad to know whether our response, including the new experiments you requested, has addressed your concerns. If any point remains open, we would be happy to clarify it. Otherwise, we would be grateful if you could reconsider your rating.

Reviewer_pKZWOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

At the beginning of the review period, I requested reassignment because I have limited experience with looped Transformers. However, since the review task seemed to remain here, I have provided a review from my perspective and set my confidence to 1.

This paper proposes LoopCD, a training-free contrastive decoding method for looped Transformers. The key idea is to leverage intermediate recurrent states as weaker predictions and extrapolate the difference between early and final predictions to improve decoding. The authors introduce two variants that operate in logit space and hidden-state space, respectively, with the latter introducing negligible computational overhead. Experiments across four looped Transformer families show improvements on reasoning, code generation, and multiple-choice benchmarks. The authors further demonstrate that LoopCD can maintain comparable performance with fewer recurrent iterations, reducing theoretical inference FLOPs.

Critical strengths
  1. Clear presentation and simple method. The paper is well written and easy to follow. The proposed method is simple, intuitive, and requires no additional training.

  2. Promising empirical improvements. The method achieves substantial improvements on certain benchmarks, particularly AIME and HumanEval, demonstrating its potential effectiveness across different tasks and model architectures.

Critical weaknesses
  1. Insufficiently optimized inference-time baselines. LoopCD tunes guidance strength and reference iterations, but it is unclear whether the unguided baselines receive comparable optimization over inference-time parameters, such as recurrent depth, sampling temperature, and top-p. This raises concerns about whether the reported gains, particularly on mathematical reasoning, would persist against equally tuned baselines. The use of held-out data for hyperparameter selection is also unclear.

  2. Limited methodological novelty and mechanistic understanding. The method is essentially an application of existing contrastive decoding to intermediate recurrent states. Although empirically interesting, the explanation of why it works remains incomplete. Improvements vary substantially across tasks and models, occasionally becoming negative, while the analysis largely characterizes prediction changes rather than explaining when and why extrapolation improves correctness.

  3. Limited validation of inference efficiency. The claimed compute savings are demonstrated only on multiple-choice benchmarks, where halving recurrent depth already causes relatively small accuracy degradation. It remains unclear whether similar savings hold for more challenging reasoning and generation tasks, where the largest performance gains are reported. Moreover, efficiency is measured using theoretical FLOPs rather than actual inference latency.

Questions for authors

N / A

2 comments

We thank the reviewer for the valuable feedback and address each point below.

W1. Optimization of the baselines. Guided and unguided runs use identical inference-time settings: checkpoint, recurrent depth, prompt, temperature and top-p, generation length, and answer extraction. Each comparison therefore isolates the decoding rule, and LoopCD receives no depth tuning beyond what the baseline gets. For reasoning, both arms sample at temperature 1.0 and top-p 0.7. Because these parameters act on the distribution that LoopCD produces, tuning them would affect both arms alike and is complementary to the guidance.

Multiple choice uses a single ω=0.5\omega = 0.5 for all seven configurations, and the strength sweep shows that the gains stay positive across a wide band of ω\omega around this value (Appendix G.1).

W2. Novelty and mechanism. LoopCD builds on contrastive decoding. Its contribution is showing that a looped model already contains the weaker model that contrastive decoding needs, so guidance requires neither a second model nor any training. Recurrence also enables two uses that a separate weak model cannot offer: a hidden-state form that needs no extra pass, and trading guidance for iterations (Section 4.3).

Extrapolation improves correctness mainly on decisions where the final prediction is still uncertain. On ARC-Challenge, the lowest-margin fifth of questions gains 6.4 to 13.3 points across models, compared with 0.0 to 0.4 points for the highest-margin fifth (Section 5.2). LoopCD extrapolates in the direction the trained iterations already follow: on Ouro-1.4B, iterations 2, 3, and 4 each move the logits along z4−z1z_4 - z_1 (cosine 0.84, 0.63, and 0.27). It thus continues this refinement on the decisions the model has not yet settled.

Across tasks, the seven-benchmark multiple-choice mean improves for all seven configurations (+0.29 to +1.59 points), and mean reasoning pass@1 rises by 5.27 to 7.33 points for both Ouro-Thinking models (Table 1a).

W3. Efficiency on harder tasks, and wall-clock time. We have measured wall-clock time on a single A100 (bf16, KV cache). Every setting's measured time ratio falls within 0.02 of its FLOP ratio, LoopCD adds at most 2% per decoded token, and half-depth runs take 0.51 to 0.53 of the full-depth time.

ModelSettingFLOP ratioDecode, ms/token (ratio)
Ouro-1.4B4 loops, unguided1.0086.6 (1.00)
Ouro-1.4B4 loops, LoopCD-Logits1.0287.2 (1.01)
Ouro-1.4B2 loops, LoopCD-Logits0.5344.0 (0.51)
Huginn32 iterations, unguided1.00136.3 (1.00)
Huginn32 iterations, LoopCD-Logits1.02138.2 (1.01)
Huginn32 iterations, LoopCD-Hidden1.00135.6 (0.99)
Huginn16 iterations, LoopCD-Logits0.5472.9 (0.53)
Huginn16 iterations, LoopCD-Hidden0.5270.6 (0.52)

These savings also extend to generation and reasoning. The paper already reports Huginn on code at 16 of its 32 iterations (Tables 1b and 3b), and we have added Ouro at three of its four loops, which saves 22.5% to 23.75% of the FLOPs. With LoopCD, both models outperform their full-depth unguided baselines on code (mean pass@1 over HumanEval, HumanEval+, MBPP, and MBPP+), and Ouro-Thinking at three loops does the same on mean pass@1 over AIME 2024 and 2025. Every row except the LoopCD-Hidden one uses LoopCD-Logits with the adaptive rule:

Model, metricDepthFull depth, unguidedReduced depth + LoopCD
Huginn, code mean (LoopCD-Logits)32 → 1629.4731.54 (+2.07)
Huginn, code mean (LoopCD-Hidden)32 → 1629.4731.26 (+1.79)
Ouro-1.4B, code mean4 → 367.4470.29 (+2.85)
Ouro-2.6B, code mean4 → 371.8876.42 (+4.54)
Ouro-1.4B-Thinking, AIME 2024/2025 mean pass@14 → 344.9046.77 (+1.87)
Ouro-2.6B-Thinking, AIME 2024/2025 mean pass@14 → 355.7361.25 (+5.52)

At 16 iterations, Huginn runs at 0.53 (LoopCD-Logits) and 0.52 (LoopCD-Hidden) of the full-depth time per token.

We hope these results address the reviewer's concerns, and we are happy to clarify further.

Thank you again for your review. As the discussion period closes shortly, we would appreciate knowing whether our responses have addressed your concerns, or whether any point remains open.

Comments on the paper

2 comments
~Byeongho_Yu2OpenReview ↗

Request for citation and clarification regarding our LoopCD paper

Dear Authors,

I’m Byeongho Yu, the first author of LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models, published in the proceedings of the EMNLP 2026 Main Conference.

I noticed that this paper uses a similar core method of contrasting earlier and final-loop predictions in looped language models and the same “LoopCD” name. To avoid confusion, it would be helpful if the revised paper cited our work and distinguished the two more clearly, possibly by using a different method name. Thank you for considering this.

Our paper: https://aclanthology.org/2026.emnlp-main.1367/

Best regards,
Byeongho Yu

Thank you for reaching out and for sharing your work!

For context on the timeline, our ICLR submission was first recorded on OpenReview on September 16, 2026, and already included both the LoopCD name and the core method. Since this predates your arXiv preprint of September 21 and the EMNLP proceedings released in October, we view the two papers as independent, concurrent work.

We will be happy to cite your paper as concurrent work and clarify the relationship between the two studies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.