- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies a readout blind spot in looped language models and shows that combining dense supervision with explicit scale control can bound hidden-state norms and improve perplexity under certain training configurations. The experiments and theoretical analysis provide a coherent explanation of the observations and may offer useful guidance for designing and training looped language models.
Critical strengths
- The paper provides a potentially useful principle for training looped language models by treating prediction supervision and recurrent-state scale control as separate design requirements.
- The experimental design appears reasonable, and the empirical results and theoretical analysis provide consistent and coherent support for the proposed mechanism across various setups.
- The authors clearly distinguish norm control from performance improvement and also acknowledge the limitations of the current study.
Critical weaknesses
- Further ablation studies would help clarify which training-recipe factors explain the reversal in the perplexity gains and when scale control is beneficial.
- The theoretical results about attenuation of gradients from later-loop losses have not yet been empirically verified in the trained models.
Questions for authors
- Table 1 and 3 show opposite effects of raw readouts on perplexity; could additional ablations that change one training-recipe component at a time help identify the cause beyond weight decay?
- Table 2 reports gradients from local losses; could the authors also measure gradients from later-loop losses to test the scale-dependent attenuation demonstrated by Proposition 3?
Minor comment: The title on the submission page differs from the title in the submitted pdf; please make them consistent.
2 comments
Thank you for the point-by-point responses. I appreciate the clarifications and will keep my original score.
We thank the reviewer for the positive assessment, and for appreciating that we keep norm control and performance apart and are open about the limitations.
Yes, we ran these experiments. Starting from the 1.4B recipe of Table 3, we moved one factor at a time to its small-model value, with two seeds each. These factors are the learning-rate schedule, the batch size, Adam β2, tied embeddings and dropout. If one of them were the cause, moving it alone would make the raw readout better than the normalized readout, as in Table 1. The raw readout stays worse in every run, so none of these factors decides the result on its own, and neither does weight decay (Table 4). This points to several factors acting together, or to the data itself (sequence length, repeated data, tokenizer and positions), which we plan to test next. In all of these runs the norm behaves as the paper predicts. It grows 1.7 to 3.1 times from loop 1 to loop 4 with the normalized readout and changes by at most 20% with the raw readout. We have added these results to Appendix C.
Yes, we measured them on the 1.4B models, and the gradient shrinks as Proposition 3 predicts. Here the scale is the RMS norm of the hidden state. The later losses do reach the scale through the loops, but the fraction of their gradient that changes the scale shrinks as one over the scale. Across loops and checkpoints, this fraction is 1.2% at a scale of 10.7 and 0.39% at a scale of 34.7. So the later losses push back on a growing scale only weakly. Doubling the scale raises them by at most 0.09 nats per token with the normalized readout, against more than 1.6 nats with the raw readout. We have added these measurements to Appendix A and a summary to Section 6.
Thank you. The revised PDF now has the same title as the submission page, "Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models".