- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Loopy, a post-training quantization framework for looped language models. It selects scaling and rotation configurations based on the final prediction loss at the target recurrent depth. To reduce calibration costs, Loopy adopts a progressive selection strategy based on successive halving. Experiments demonstrate improved quantization performance and calibration efficiency compared with existing baselines.
Critical strengths
The paper is clearly written and well-structured. The proposed method is an efficient PTQ method.
Critical weaknesses
- The proposed method mainly combines existing scaling and rotation techniques with successive halving and applies them to looped LMs. The overall methodological novelty seems limited.
- Could the authors clarify the initial candidate pool size and the number of candidates retained at each stage? Since Loopy relies on configuration search, these details are important for understanding its computational cost and reproducibility.
- Results look surprising. Loopy searches over predefined non-trained scales and rotations but outperforms learnable quantization methods. Could the authors provide some insights into why this is the case? Also, please clarify how SpinQuant and FlatQuant are adapted to looped LMs, particularly whether they use target-depth NLL during optimization.
- Figure 7 shows relatively small improvements from target-depth calibration in some settings, despite substantially higher calibration costs. Could the authors clarify under what conditions full-depth selection is particularly beneficial?
- Loopy selects quantization configurations for a fixed target recurrent depth. Although the paper analyzes configuration rankings across different depths, it remains unclear how well a selected configuration transfers to other deployment depths. Would changing the inference depth require repeating the configuration search?
Questions for authors
Please refer to weakness