acceptodds
Under review as a conference paper at ICLR 2027

Loopy: Low-Bit Quantization Framework for Looped Language Models

Abstract

Looped language models provide a parameter-efficient way to scale iterative test-time computation by repeatedly executing a shared recurrent core. Post-training quantization (PTQ) can reduce the memory footprint and inference cost of looped language models, but errors introduced by a quantized shared core affect subsequent cores. Among PTQ methods, channel scaling and orthogonal rotations preserve the floating-point computation while producing representations with different quantization quality. We find that quantization configuration candidate rankings can change with recurrent depth, motivating configuration selection at the target deployment depth. However, evaluating every candidate over the full calibration set at this depth is costly. We therefore propose LOOPY, a PTQ framework that formulates shared-core quantization through a recurrent-depth-aware objective, selecting shared low-bit representations by their final prediction loss at the target deployment depth. Channel scaling and orthogonal rotations parameterize the candidate representations. To approximately solve this selection problem efficiently, LOOPY progressively allocates calibration windows to promising candidates while preserving complete target-depth execution, using only forward evaluations. Across eight settings, LOOPY achieves the state-of-the-art results among different baselines. On Ouro-1.4B under W4A4, LOOPY reduces LAMBADA perplexity by 36.5% relative to SpinQuant. Our code is available at https://anonymous.4open.science/r/Loopy-ICLR2027/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, quantization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MnsjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Loopy, a post-training quantization framework for looped language models. It selects scaling and rotation configurations based on the final prediction loss at the target recurrent depth. To reduce calibration costs, Loopy adopts a progressive selection strategy based on successive halving. Experiments demonstrate improved quantization performance and calibration efficiency compared with existing baselines.

Critical strengths

The paper is clearly written and well-structured. The proposed method is an efficient PTQ method.

Critical weaknesses
  1. The proposed method mainly combines existing scaling and rotation techniques with successive halving and applies them to looped LMs. The overall methodological novelty seems limited.
  2. Could the authors clarify the initial candidate pool size and the number of candidates retained at each stage? Since Loopy relies on configuration search, these details are important for understanding its computational cost and reproducibility.
  3. Results look surprising. Loopy searches over predefined non-trained scales and rotations but outperforms learnable quantization methods. Could the authors provide some insights into why this is the case? Also, please clarify how SpinQuant and FlatQuant are adapted to looped LMs, particularly whether they use target-depth NLL during optimization.
  4. Figure 7 shows relatively small improvements from target-depth calibration in some settings, despite substantially higher calibration costs. Could the authors clarify under what conditions full-depth selection is particularly beneficial?
  5. Loopy selects quantization configurations for a fixed target recurrent depth. Although the paper analyzes configuration rankings across different depths, it remains unclear how well a selected configuration transfers to other deployment depths. Would changing the inference depth require repeating the configuration search?
Questions for authors

Please refer to weakness

Reviewer_vb5LOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Loopy is a post-training quantization framework for large language models. It constructs quantization candidates using channel scaling and orthogonal rotations shared across recurrent steps, selects configurations by final NLL at the target deployment depth, and reduces search cost by progressively increasing the number of calibration windows while eliminating candidates. Experiments cover Ouro, Huginn, and Recurrent-Llama under W4A4/W4A8, with additional studies of selection cost, K/V quantization quality, and decoding throughput.

Critical strengths

Screening retains the full deployment depth and adjusts cost only through the number of calibration windows, avoiding a mismatch between shallow evaluation and deployment computation. Local paired transforms preserve the function before quantization in exact arithmetic, and quantized weights and transforms are shared across recurrent steps without per-step weight copies. The W4A4 experiment results show Loopy is better than SOTA in most of cases.

Critical weaknesses
  1. The method largely combines established scaling, rotation, and successive halving. Although selection-strategy ablations are provided, comparisons against fixed configurations, random selection, and scaling/rotation ablations within the same implementation are missing. The source of gain from Loopy needs to be further evaluated.
  2. Some results require clarification: Tables 6 reports 55.81% LAMBADA accuracy for Ouro-1.4B W4A4, where Tables 7 reports accuracy from 45.97% to 45.68%. There is no words to explain the differences in these tables.
  3. There is no experiment on larger models, so the scalability of Loopy to large model is in doubt.
Questions for authors
  1. With the base quantization implementation fixed, how much do scaling/rotation, target-depth scoring, and progressive screening each contribute? Please add fixed-configuration and random-selection controls and other variations to clarify each individual contribution.
  2. Please provide a configuration/checkpoint mapping in experiment, explain the cross-table differences, and report full accuracy, throughput, and GPU memory usage for the same actual low-bit deployment configuration.
  3. Please add more experiment results for larger models.
Reviewer_QNPLOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The authors show that rotation/scaling configurations rank differently at different recurrent depths and propose selecting the shared configuration.

Critical strengths

The depth dependence of configuration quality is a real observation for this architecture family and is well documented. Well written paper.

Critical weaknesses

Once the depth-aware objective is stated, the method is a hyperparameter search over (seed, scaling power, block size) with successive halving, so the technical contribution is thin. The contribution is a parameter sweep over existing components: an existing model, an existing dataset, and a small modification of one calibration or inference knob, followed by a careful measurement. Many similar papers have been done

Questions for authors

How much of the gain comes from depth aware selection versus baseline adaptation choices?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.