acceptodds
Under review as a conference paper at ICLR 2027

Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models

Abstract

Looped language models iteratively apply the same transformer block, which enables us to vary computational depth at inference time. To ensure intermediate loops produce useful predictions, these models are usually trained with dense supervision applying a cross-entropy loss at every loop. In this work, we study a supervision failure that arises from this practice, which we term the readout blind spot. Because standard output readouts normalize the hidden state before prediction, they effectively mask the state's magnitude from the local loss gradient. Meanwhile, pre-norm residual connections continuously accumulate scale. As a result, dense supervision forces intermediate states to become predictive but entirely fails to regulate their residual norm, allowing it to grow uncontrollably across loops. Through ablations at 44M and 129M, and a matched 1.4B scaling study, we demonstrate that stabilizing looped models requires actively decoupling exit supervision from scale control. Exposing the scale to the loss (via raw readouts or norm penalties) or resetting it within the loop (via inter-loop normalization) successfully bounds the residual norm. Every form of scale control improved perplexity in our small-model recipes and at 1.4B parameters, the blind spot hold in norm readout.

Primary area:
foundation or frontier models, including LLMs
Keywords:
looped language models, recurrent depth, normalization, RMSNorm, hidden-state scale, deep supervision, adaptive computation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EeF1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a readout blind spot in looped language models and shows that combining dense supervision with explicit scale control can bound hidden-state norms and improve perplexity under certain training configurations. The experiments and theoretical analysis provide a coherent explanation of the observations and may offer useful guidance for designing and training looped language models.

Critical strengths
  • The paper provides a potentially useful principle for training looped language models by treating prediction supervision and recurrent-state scale control as separate design requirements.
  • The experimental design appears reasonable, and the empirical results and theoretical analysis provide consistent and coherent support for the proposed mechanism across various setups.
  • The authors clearly distinguish norm control from performance improvement and also acknowledge the limitations of the current study.
Critical weaknesses
  • Further ablation studies would help clarify which training-recipe factors explain the reversal in the perplexity gains and when scale control is beneficial.
  • The theoretical results about attenuation of gradients from later-loop losses have not yet been empirically verified in the trained models.
Questions for authors
  • Table 1 and 3 show opposite effects of raw readouts on perplexity; could additional ablations that change one training-recipe component at a time help identify the cause beyond weight decay?
  • Table 2 reports gradients from local losses; could the authors also measure gradients from later-loop losses to test the scale-dependent attenuation demonstrated by Proposition 3?

Minor comment: The title on the submission page differs from the title in the submitted pdf; please make them consistent.

2 comments

We thank the reviewer for the positive assessment, and for appreciating that we keep norm control and performance apart and are open about the limitations.

Table 1 and 3 show opposite effects of raw readouts on perplexity; could additional ablations that change one training-recipe component at a time help identify the cause beyond weight decay?

Yes, we ran these experiments. Starting from the 1.4B recipe of Table 3, we moved one factor at a time to its small-model value, with two seeds each. These factors are the learning-rate schedule, the batch size, Adam β2, tied embeddings and dropout. If one of them were the cause, moving it alone would make the raw readout better than the normalized readout, as in Table 1. The raw readout stays worse in every run, so none of these factors decides the result on its own, and neither does weight decay (Table 4). This points to several factors acting together, or to the data itself (sequence length, repeated data, tokenizer and positions), which we plan to test next. In all of these runs the norm behaves as the paper predicts. It grows 1.7 to 3.1 times from loop 1 to loop 4 with the normalized readout and changes by at most 20% with the raw readout. We have added these results to Appendix C.

Table 2 reports gradients from local losses; could the authors also measure gradients from later-loop losses to test the scale-dependent attenuation demonstrated by Proposition 3?

Yes, we measured them on the 1.4B models, and the gradient shrinks as Proposition 3 predicts. Here the scale is the RMS norm of the hidden state. The later losses do reach the scale through the loops, but the fraction of their gradient that changes the scale shrinks as one over the scale. Across loops and checkpoints, this fraction is 1.2% at a scale of 10.7 and 0.39% at a scale of 34.7. So the later losses push back on a growing scale only weakly. Doubling the scale raises them by at most 0.09 nats per token with the normalized readout, against more than 1.6 nats with the raw readout. We have added these measurements to Appendix A and a summary to Section 6.

Minor comment: The title on the submission page differs from the title in the submitted pdf; please make them consistent.

Thank you. The revised PDF now has the same title as the submission page, "Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models".

Thank you for the point-by-point responses. I appreciate the clarifications and will keep my original score.

Reviewer_2M2nOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies why applying cross entropy loss at every loop is not enough to control the hidden state in looped language models.

With a normalized output head, the loss is almost insensitive to the magnitude of the hidden state, so intermediate predictions can improve while the hidden state norm keeps growing across loops.

The authors call this the readout blind spot and evaluate three ways to control the scale: raw readouts, norm penalties, and inter loop normalization, on 44M, 129M, and 1.4B models.

Critical strengths

The paper isolates the readout blind spot with several controlled interventions that separately modify scale visibility in the loss and scale propagation in the recurrent path.

The mechanism is supported by multiple diagnostics, including radial gradient measurements, norm trajectories, scale clamping, three seed small model experiments, and a matched 1.4B study.

Critical weaknesses

The paper clearly identifies a mechanism that causes hidden state norms to grow, but fixing this problem does not consistently improve model performance. Scale control improves perplexity under the small model recipe, while the 1.4B and recipe transfer results are mixed or negative.

The main novelty is limited because the lack of radial gradients under normalized readouts is closely related to prior work, and inter loop normalization is already used in Ouro. The contribution is mainly the diagnosis of how these effects interact with dense supervision in looped models.

Questions for authors

Can the authors identify which training recipe factors determine whether scale control improves perplexity? In particular, the small model recipe shows clear gains, while the 1.4B recipe and recipe transfer experiments show mixed or negative results.

2 comments

We thank the reviewer for the comments, and for appreciating the controlled experiments and the range of diagnostics behind the mechanism.

The paper clearly identifies a mechanism that causes hidden state norms to grow, but fixing this problem does not consistently improve model performance.

We agree, and this is why the paper keeps the two questions apart. Our main study is about the norm. With a normalized readout, a loss at every loop does not keep the norm of the hidden state in check, so the scale needs its own control. Whether that control also improves perplexity depends on the training recipe, and our new experiments on this are in the answer to your last question. We have revised the abstract and the introduction to say this plainly.

The main novelty is limited because the lack of radial gradients under normalized readouts is closely related to prior work, and inter loop normalization is already used in Ouro. The contribution is mainly the diagnosis of how these effects interact with dense supervision in looped models.

We agree with this description, and the diagnosis is our main contribution. TaperNorm shows that a normalized readout gives no gradient on the scale, the RMS norm of the hidden state, and Ouro uses inter-loop normalization. What we add is an explanation of when and why looped models need such a control. One might expect a loss at every loop to keep the hidden states in check, and we show that it does not when the readout is normalized. The later losses reach the scale only through a gradient that shrinks as one over the scale (Proposition 3), and we have now measured this on the 1.4B models. We also tested Ouro's own design without its inter-loop normalization. With Ouro's block norms and a loss at every loop, the norm still grows in both seeds, from about 150 at loop 1 to 546 and 529 at loop 4, more than the 306 and 300 of the same model without them. So the inter-loop normalization is what keeps the scale in check there. We have added both experiments to the revision (Appendices A and C).

Can the authors identify which training recipe factors determine whether scale control improves perplexity?

We ran new experiments on this. The two recipes in the paper disagree on perplexity. Under the small-model recipe the raw readout is 8% better than the normalized readout, while under the 1.4B recipe it is about 3 to 4% worse, even with the same 129M stack (Table 3). So we asked which difference between the two recipes causes this. Starting from the 1.4B recipe, we moved one factor at a time to its small-model value, with two seeds each. These factors are the learning-rate schedule, the batch size, Adam β2, tied embeddings and dropout, and Table 4 already covers weight decay. If one of them were the cause, moving it alone would make the raw readout better. The raw readout stays worse in every run, so none of the six factors decides the result on its own. This points to several factors acting together, or to the data itself (sequence length, repeated data, tokenizer and positions), which we plan to test next. In all of these runs the norm behaves as the paper predicts. It grows 1.7 to 3.1 times from loop 1 to loop 4 with the normalized readout and changes by at most 20% with the raw readout. We have added these results to Appendix C and the open question to the Limitations.

Thank you for the rebuttal. My main concerns regarding limited novelty and the unclear practical benefits remain, so I will maintain my original rating.

Reviewer_iXsKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The setting of the paper investigates training pre-norm looped language models with Transformer blocks. The hidden state representation before the readout is used as an input for the next iteration, forming a recurrence in the hidden state across iterations. In this setting, the magnitude of the hidden representation grows uncontrollably. The loss is computed on the signal after the readout layer. The authors present that the common way to apply the readout is to first apply the scale-invariant normalization layer. In this setting, they identify that scale discrepancy between the signal used to compute the loss and train the network and the signal used for the recurrent update of the hidden state. This leads to the blind spot of the loss, as it cannot see the scale of the hidden state update. This applies to both the deep supervision setting, when the normalized readout is performed (and loss computed) after each loop (recurrence) and in case of applying the normalized readout after K loops. They identify that the blind spot in the deep supervision case (per-loop) is more severe. The remedies that they propose are: (1) raw readout without scale-invariant normalization (make the magnitude visible to the loss) or (2) remove the scale from the recurrence by normalizing the state between loops. Authors perform "theoretical" and experimental analysis that in varying degree support the claims.

Critical strengths
  • The paper identifies a useful distinction between controlling the recurrent hidden state in looped models and the signal used in the supervision. It is supported theoretically and experimentally across different model sizes and token budgets.
  • The proposed remedies to the identified problem are experimentally shown to alleviate the problem.
Critical weaknesses
  • Proposition 3 provides a relevant explanation for weakened scale supervision in the observed large-norm regime in the case of normalized readout. Direct measurements of gradients from later losses would strengthen the connection between this theoretical result and the experimental dynamics.
  • The conditions under which scale control improves prediction quality remain incompletely characterized. Although the paper examines multiple training recipes, it does not isolate the factors responsible for the differing outcomes across interventions. This limits its practical guidance on when scale control improves model quality and which intervention should be preferred. This weakness was not decisive for the overall assessment.
  • The 1.4B model is undertrained at 6B tokens. This makes the results less reliable.
  • Some of the results of the paper are presented in a poor manner. For instance, the results of Table 6 would fit better in a plot as opposed to a table.
  • The notation in the paper is not well-defined and confusing. For example, if I'm not mistaken the variable s in line 158 is a scalar. However, it's denoted the same way as a vector is denoted in the paper.
Questions for authors
  • Could the authors give more references to justify the specific architectural setting that they use within the looped Transformers architectures? To the best of my knowledge, the critical choice of having the normalized readout and only pre-norm Transformer blocks is not common. They would either use post-norm or some other recurrence scale control method to stabilize it, so the described discrepancy would not be a problem. See [1, 2, 3, 4]
  • Could authors also address the weaknesses above?

References

[1] Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B., Kailkhura, B., Bhatele, A. and Goldstein, T., 2026. Scaling up test-time compute with latent reasoning: A recurrent depth approach.

[2] Prairie, H., Novack, Z., Berg-Kirkpatrick, T. and Fu, D.Y., 2026. Parcae: Scaling laws for stable looped language models.

[3] Movahedi, S., Milovanović, V., Feigin, S.L., Theus, A., Hofmann, T., Boeva, V., Rusch, T.K. and Orvieto, A., 2026. Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers.

[4] Zhu, R.J., Wang, Z., Hua, K., Zhang, T., Li, Z., Que, H., Wei, B., Wen, Z., Yin, F., Xing, H., Li, L., Shi, J., Ma, K., Li, S., Kergan, T., Smith, A., Qu, X., Hui, M., Wu, B., Min, Q., Huang, H., Zhou, X., Ye, W., Liu, J., Yang, J., Shi, Y., Lin, C., Zhao, E., Cai, T., Zhang, G., Huang, W., Bengio, Y. and Eshraghian, J., 2025. Scaling Latent Reasoning via Looped Language Models.

2 comments

We thank the reviewer for the detailed comments, and for seeing that controlling the hidden state and supervising every loop are two different things, and that the remedies we study help.

To the best of my knowledge, the critical choice of having the normalized readout and only pre-norm Transformer blocks is not common. They would either use post-norm or some other recurrence scale control method to stabilize it, so the described discrepancy would not be a problem.

Thank you for raising this. You are right that the published looped models with losses on intermediate loops also control the size of the hidden state. Our paper explains why they need to. One might expect a loss at every loop to keep the hidden states in check by itself, and we show that it does not.

The layout we study, pre-norm blocks with one final norm, is used in published looped models such as Mixture-of-Recursions and the Llama version of retrofitted recurrence, which put a loss on the last loop only. We chose it because it has no other scale control, so it lets us test whether a loss at every loop keeps the hidden state in check by itself. In this layout, adding a loss at every loop raises the norm of our 129M model from 5,997 to 56,051 (Table 1).

We also checked [1-4], both the papers and the released code, and each of them controls the scale in its own way. Huginn [1] normalizes after every residual add, a form of the post-norm you mention, and Parcae [2] damps what it adds to the state at each loop. Both put a loss on the last loop only. Fixed-Point Reasoners [3] put a loss every few loops, read out the hidden state without a final norm, which is our raw readout, and scale and damp each residual update. Ouro [4] puts a loss on every loop and normalizes the hidden state between loops, which is our inter-loop normalization. Its extra block norms are applied before the residual add, so they do not limit the size of the hidden state.

So we asked whether Ouro still needs its inter-loop normalization when every loop already has a loss. To test this, we trained the 129M stack under our 1.4B recipe with Ouro's block norms, a loss at every loop and a normalized readout, and removed only the inter-loop normalization. The norm then grows at every loop in both seeds, from about 150 at loop 1 to 546 and 529 at loop 4, more than the 306 and 300 of the same model without the block norms. So the inter-loop normalization is the part of Ouro's design that keeps the hidden state in check, and our analysis explains why it is needed. We have added this experiment to Appendix C, the survey to the related work (Section 8), and the motivation to Section 3.

Direct measurements of gradients from later losses would strengthen the connection between this theoretical result and the experimental dynamics.

This is a valid point, and we have now measured these gradients on the 1.4B models. For each loop, we take the model's hidden state, run the remaining loops from it, and backpropagate the later losses to that state. Here the scale is the RMS norm of the hidden state. The loss at the same loop gives no gradient on the scale, as in Table 2. The later losses do, but the fraction of their gradient that changes the scale shrinks as one over the scale, as Proposition 3 predicts. Across loops and checkpoints, this fraction is 1.2% at a scale of 10.7 and 0.39% at a scale of 34.7. So the later losses push back on a growing scale only weakly. Doubling the scale raises them by at most 0.09 nats per token with the normalized readout, against more than 1.6 nats with the raw readout. We have added these measurements to Appendix A, next to the proof of Proposition 3, and a short summary to Section 6.

Although the paper examines multiple training recipes, it does not isolate the factors responsible for the differing outcomes across interventions.

We agree that this is an important question, and we ran new experiments on it. The two recipes in the paper disagree on perplexity. Under the small-model recipe the raw readout is 8% better than the normalized readout, while under the 1.4B recipe it is about 3 to 4% worse, even with the same 129M stack (Table 3). So we asked which difference between the two recipes causes this. Starting from the 1.4B recipe, we moved one factor at a time to its small-model value, with two seeds each. These factors are the learning-rate schedule, the batch size, Adam β2, tied embeddings and dropout, and Table 4 already covers weight decay. If one of them were the cause, moving it alone would make the raw readout better. The raw readout stays worse in every run, so none of the six factors decides the result on its own. This points to several factors acting together, or to the data itself (sequence length, repeated data, tokenizer and positions), which we plan to test next. In all of these runs the norm behaves as the paper predicts. It grows 1.7 to 3.1 times from loop 1 to loop 4 with the normalized readout and changes by at most 20% with the raw readout. We have added these results to Appendix C and the data factors to the Limitations.

The 1.4B model is undertrained at 6B tokens. This makes the results less reliable.

We agree, and we trained the normalized and raw readout models to 14B tokens, continuing from their 5B checkpoints on the original cosine schedule. The mechanism stays the same. With the normalized readout, the norm still grows from 480 at loop 1 to 1,464 at loop 4, while with the raw readout it stays between 100 and 115. The raw readout's gap in perplexity narrows from 11% at 6B tokens to 7-8% from 10B tokens on (one run each). We have added these results to Section 5 and Appendix B.

For instance, the results of Table 6 would fit better in a plot as opposed to a table.

We agree. In the submitted version, Figure 6 in Appendix K already plotted these results for the 129M models at every depth. In the revision it replaces Table 6 in the main text as Figure 4, and the table with both model sizes is now Table 22 in Appendix K.

For example, if I'm not mistaken the variable s in line 158 is a scalar. However, it's denoted the same way as a vector is denoted in the paper.

You are correct that s is a scalar. It is the RMS norm of the hidden state H, and the direction u = H/s is a vector. We now write vectors and matrices in bold and scalars in italic in the text and equations, and we say at the first use that s is a scalar.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.