- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper is interested in post-training standard LLM into looped models to improve their test-time scaling capabilities. It proposes TaH2, which adds a "decider" module that predicts, for each token, whether to continue looping over the whole model or to stop. The decider is supervised using the per-token change in validation loss to determine when to stop, and is trained jointly with the post-training of the backbone. Validation compares TaH2 against post-trained versions of Ouro and Huginn, on various sizes of Qwen3, evaluating both absolute performance and scaling trends.
Critical strengths
The main strength of the paper is the analysis of post-training models into looped models. The experiments show that Ouro and Huginn, when adapted to post-train a standard model, do not improve over base performance. The proposed TaH2 show both improvement of performance and better scaling.
Critical weaknesses
Major:
- The comparison is limited to Ouro and Huginn, which were both designed to learn looped LMs from scratch, not from post-training. The relevant comparisons would be to other test-time scaling approaches, and methods that convert pretrained LLMs into Looped LMs. The paper itself discusses at length about such work in the appendix. Without these, the claim of improved scaling is not adequately supported.
- The technical novelty seems limited with respect to TaH. The only apparent difference is jointly post-training the backbone and the decider, and a direct comparison to TaH is missing in the paper.
Minor:
- The description in section 4 is not enough to clearly grasp the method. I had to refer to Appendix D to better understand.
Questions for authors
I would like to see a proper comparison with other test-time training method, not only reimplementation in a different setup of Ouro and Huginn:
- In terms of scaling, does TaH2 outperform state-of-the-art test-time scaling methods?
- Could you add comparisons to methods that were designed to convert pretrained LLMs into looped models (e.g., those mentioned in the appendix)?
- How does TaH perform in this setting?
3 comments
Thank you for the additional experiments and for revising the paper. The new comparisons address my first concern, and I appreciate the effort. However, I still find the novelty with respect to TaH limited, and the rebuttal reinforces this view. I understand that deriving routing labels online from the current model, rather than offline from a reference model as in TaH, is more practical and can improve performance. Beyond this change, however, the central question, the analysis, and the motivation are essentially the same as in TaH.
We sincerely thank you for the thoughtful follow-up. We summarize the core differences between TaH2 and TaH below:
| Aspect | TaH | TaH2 (Ours) |
|---|---|---|
| 1. Central Question | How to improve looped LLMs' accuracy at matched parameters. | How to improve looped LLMs' accuracy and its scaling behavior at matched FLOPs. |
| 2. Finding | Latent overthinking: 2nd pass flips some correct 1st-pass tokens into errors. | Test-time scaling behavior: looped models have steeper scaling slopes than Standard, but lower matched-FLOPs accuracy. |
| 3. Depth Scaling | Mainly . | Scales from to with continuous accuracy gains. |
| 4. Training | 2-stage decoupled SFT under a static oracle (train–inference mismatch). | End-to-end joint training along actual routes; extends to On-Policy Distillation. |
| 5. Routing Principle | Static 1st-pass mismatch: iterates if pass 1 is wrong, ignoring whether iteration actually helps. | Dynamic lookahead gain: continues only where another iteration actually reduces the current model's loss. |
| 6. Infra | Naive Transformers implementation. | Mini-SGLang integration with continuous depth batching. |
In short, TaH2 extends looping from a binary (1-or-2 loop) choice to scalable loop depth, and shifts the focus from fixed-budget accuracy to scaling gains and steeper slopes as test-time compute grows.
We sincerely thank Reviewer bRzn for the constructive suggestions. We appreciate your recognition of our analysis of post-training looped models and TaH2's improvements in performance and test-time scaling. We address your concerns below.
W1, Q1: Comparison with SOTA test-time scaling methods
We added three comparisons using Qwen3-1.7B on AIME24–26 (Appendix C.4):
TaH2, with matched decoding FLOPs, improves accuracy by 3.43, 5.26, and 1.61 points across these scaling setups.
W1, Q2: Compare with other post-trained loop models
We add post-trained loop models from Retrofitted Recurrence [1], which increase Huginn's core loops from 1 to 3 with two curriculum setups. Other training and evaluation settings follow Table 1 in the paper.
Training depth counts loops (core loops for Huginn); inference depth counts equivalent full-backbone passes (3 Huginn core loops = 2 passes).
TaH2 exceeds the best post-trained baseline by 2.26 points, with lower mean inference depth (1.23 vs. 2.00).
[1] Sean McLeish et al. Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence. arXiv preprint arXiv:2511.07384, 2025.
W2, Q3: Comparison with TaH
We add a direct comparison with TaH. All models below use Qwen3-1.7B-Base and the same SFT data. We compare TaH with different routing-label sources.
TaH2 outperforms the stronger TaH adaptation by 1.29 points. Note that TaH2 does not require an additional LLM for routing-label generation, as TaH does. This makes TaH2 more practical if such an LLM is not available, which is typical in SOTA model training. TaH uses offline routing labels from a reference model and trains its decider separately. TaH2 derives labels online from the current model's one-step loss gain and jointly trains the backbone, updater and decider, so routing supervision adapts as the model changes.
Minor weakness: clarify Section 4
We have updated Section 4 to be clearer and more intuitive, with Figure 2 as a clear illustration.