- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a training-free contrastive decoding method for looped Transformers called LoopCD. The author finds that intermediate recurrent states already provide weaker predictions, while the final state provides stronger predictions for looped Transformers. Thus, LoopCD guides token selection by contrasting the final prediction with an earlier recurrent pass, including two variants LoopCD-Logits and LoopCD-Hidden. LoopCD-Logits contrasts logits after the coda and LM head and requires one additional output pass, and LoopCD-Hidden combines hidden states before the coda and outputs without extra output projection. Experiments show that LoopCD matches or exceeds the unguided full-depth baseline while reducing forward FLOPs.
Critical strengths
- The idea is motivated and clear to understand.
- The paper evaluates the method across several looped Transformer families, which makes the empirical claim more convincing.
- The paper provides a mechanistic analysis. The authors analyze the reference state selection, the concentration of gains on low-confidence samples, and the finding that guidance primarily works by re-ranking close decisions.
Critical weaknesses
- Computational benefits are based on theoretical counts FLOPs. Since the method influences decoding behavior and may interact with batching, caching, and output-layer cost, the practical latency and benefits remain unclear. It is better to report end-to-end wall-clock latency.
- The reduced-depth experiments are limited to the multiple-choice setting. It lacks the performance of LoopCD in more challenging autoregressive generation settings such as code, math reasoning, or long-form generation.
- The method depends on choosing the reference recurrent state. The authors can claim more clearly how robust the method is under practical use where tuning data may not be available.
Questions for authors
See Weaknesses.
3 comments
Thank you again for your positive assessment. As the discussion period closes shortly, we would be glad to know whether our response has addressed your concerns.
Thanks for addressing my concerns. I will maintain my rating.
We thank the reviewer for the positive assessment and the concrete suggestions, and we address each point below.
W1. End-to-end latency. We have measured wall-clock time (bf16, HuggingFace implementation with a KV cache, one A100). The measured time of every setting stays within 0.02 of its FLOP ratio at batch size 1 and within 0.05 at batch size 16.
On the other interactions you list, LoopCD-Hidden adds no cache, and LoopCD-Logits caches one extra coda pass, which is empty for Ouro (it has no coda) and 1.5% of the KV cache for Huginn. LoopCD-Logits also runs the coda and head twice per token, which accounts for the measured overhead of at most 2% per decoded token.
W2. Reduced depth on generation and reasoning. The paper already reports Huginn on code at 16 of its 32 iterations (Tables 1b and 3b), and we have added Ouro at three of its four loops on code and AIME. Because Ouro has no prelude or coda, dropping one loop saves 22.5% to 23.75% of the FLOPs. With LoopCD, both models outperform their full-depth unguided baselines on code, measured as mean pass@1 over HumanEval, HumanEval+, MBPP, and MBPP+, and Ouro-Thinking at three loops also outperforms four unguided loops on mean pass@1 over AIME 2024 and 2025. Every row except the LoopCD-Hidden one uses LoopCD-Logits with the adaptive rule, and all runs reuse the full-depth strengths without re-tuning.
W3. Reference choice without tuning data. LoopCD-Logits needs no reference choice: it always uses the first iteration, for every model and task. LoopCD-Hidden uses h1 for Parcae and Looped-Qwen3. For Huginn, every state from h4 to h8 improves the multiple-choice mean (+0.31 to +0.86), so the choice within that range matters little. The strength needs only one value per task type, and each value sits inside a band of strengths that all help (Appendix G).
We thank the reviewer again and are happy to clarify further.