- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies how boundary normalization affects looped language models when more loops are used at test time. The authors modify pretrained models and compare small models trained from scratch with and without boundary normalization. Removing it from pretrained models hurts performance, while training without it leads to less degradation with extra loops. The authors argue that keeping the loop computation consistent between training and inference is important for maintaining performance.
Critical strengths
1.The paper studies why models lose accuracy when more loops are used at test time and how boundary normalization affects this behavior.
2.The authors combine changes to pretrained models with controlled experiments on models trained from scratch. Multiple random seeds and implementation checks make the results more reliable.
Critical weaknesses
1.The paper mainly shows that training without boundary normalization makes accuracy drop less when more loops are used. However, it does not show how extra loops can improve performance, so the contribution remains limited.
2.In the main experiments, more loops require more computation but do not improve accuracy. The paper does not clearly explain why this extra cost is needed.
3.The authors provide several analysis experiments, but they do not directly test their explanation of why boundary normalization affects performance with more loops.
4.A framework showing the loop structure, normalization placement, and experimental steps is missing, which makes the paper harder to follow.
Questions for authors
Can extra loops improve accuracy on harder tasks? In what practical situations would more loops be useful?