acceptodds
Under review as a conference paper at ICLR 2027

LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers

20 views

Abstract

Looped Language Models (LoopLMs) improve parameter efficiency by recursively reusing shared Transformer modules, yet their repeated computation over evolving recurrent states poses new challenges for low-bit deployment. Existing post-training quantization (PTQ) methods, largely designed for feed-forward Transformers, fail to account for cross-loop activation dynamics, heterogeneous loop sensitivity, and recursive error propagation, leading to substantial performance degradation at low precision. To address these challenges, we propose LoopQuant, a loop-aware PTQ framework for accurate and efficient low-bit quantization of LoopLMs. LoopQuant is built on three core components: 1) Cross-Loop Distribution Canonicalization (CLDC) learns a shared quantization-friendly representation across recurrent states while preserving parameter sharing; 2) Interaction-Aware Loop-wise Precision Allocation (ILPA) allocates activation precision under a fixed bit budget by modeling task-level loop sensitivity and cross-loop interactions; and 3) Loop-Conditioned Task Calibration (LCTC) further aligns quantized inference with the low-bit recursive trajectory through lightweight task-space calibration. Extensive experiments across LoopLM architectures demonstrate that LoopQuant consistently outperforms existing low-bit PTQ methods. On Ouro-2.6B, LoopQuant under W4A4 surpasses all W4A8 baselines in performance, while reducing model memory by 65.3% with only 3.4% additional storage overhead over naive INT4 and achieving a 1.7 decoding speedup over BF16. These results demonstrate a strong balance among accuracy, compression, and inference efficiency, providing a practical and deployment-friendly solution for low-bit LoopLM inference. The code is available at https://anonymous.4open.science/r/LoopQuant-D211/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Looped Transformers, post-training quantization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_416GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoopQuant, a post-training quantization framework specifically designed for Looped Language Models. The method addresses three properties of recursive inference: cross-loop activation distribution shifts, heterogeneous loop sensitivity, and error accumulation along the recursive trajectory. It combines Cross-Loop Distribution Canonicalization (CLDC), Interaction-Aware Loop-wise Precision Allocation (ILPA), and Loop-Conditioned Task Calibration (LCTC). Experiments cover several LoopLM families and both standard language-modeling/zero-shot benchmarks and reasoning tasks. LoopQuant consistently improves over conventional PTQ baselines, with particularly large gains under W4A4.

Critical strengths
  1. The results are not limited to one LoopLM or one metric. Under W4A4, LoopQuant substantially outperforms the strongest conventional PTQ baselines across Ouro, Nanbeige, and Huginn, while W4A6 can even outperform W4A8 baselines. The reasoning results are also convincing: the gains persist on MMLU, GSM8K, and HumanEval rather than only on perplexity or simple zero-shot tasks.
  2. The component ablations show meaningful incremental improvements from CLDC, ILPA, and LCTC, and the ILPA analysis compares against uniform, random, and sensitivity-based precision allocation. The paper also includes deployment-level measurements, reporting substantial memory reduction and a 1.5–1.7× decoding throughput improvement over BF16.
Critical weaknesses
  1. Most main-table baselines were originally developed for conventional Transformers, so outperforming them on a recursive architecture does not by itself establish the advantage over the strongest loop-aware alternative. The closest related method is LoopQ, but its implementation is unavailable; consequently, the authors compare against numbers taken from the original paper rather than reproducing it under the same calibration data, GPTQ backend, and evaluation pipeline. This does not invalidate the paper, but it weakens the strongest novelty claim: it remains somewhat unclear how much of the gain over prior loop-aware quantization comes from the proposed formulation rather than experimental differences.
  2. LoopQuant is considerably more involved than a standard PTQ pipeline: CLDC optimizes transformations for every reused module, ILPA evaluates recursive precision configurations and estimates Shapley-style sensitivities/interactions, and LCTC performs teacher-guided optimization of loop-conditioned calibration parameters. The paper carefully reports persistent storage and inference throughput, but I could not find an end-to-end wall-clock or GPU-cost comparison for producing the quantized model. This matters because the paper repeatedly motivates LoopQuant as an efficient and deployment-friendly PTQ framework. The deployed model is clearly efficient; the practicality of the quantization procedure itself, especially as the number of loops increases, is less clear.
Questions for authors
  1. Can the authors provide a more controlled comparison with LoopQ, either through a faithful reimplementation or by matching as many calibration, quantization-backend, and evaluation details as possible? This would materially strengthen the claim that the gains come from the proposed loop-aware design rather than cross-paper differences.
  2. What is the end-to-end calibration/quantization cost of LoopQuant for each evaluated model, compared with SmoothQuant/QuaRot/SpinQuant/FlatQuant/DartQuant? In particular, how does ILPA's cost scale with the number of loops, and what is the wall-clock contribution of CLDC and LCTC?
Reviewer_aH2iOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

LoopQuant targets post-training quantization of looped language models. It combines shared cross-loop distribution canonicalization, interaction-aware loop-wise activation precision allocation, and lightweight loop-conditioned task-space calibration. The paper reports substantial improvements over static PTQ methods on several looped backbones, including thinking-oriented variants, together with memory and decoding-efficiency results.

Critical strengths
  1. The method addresses challenges specific to shared modules operating over changing recurrent states. The three components have an understandable connection to distribution mismatch, heterogeneous sensitivity and recursive error propagation.
  2. Table 1 reports substantial improvements at the same W4A4 precision. The task-level results and component ablations provide useful evidence that the framework is effective within the reported settings.
  3. The appendices contain inspectable derivations, allocation comparisons and calibration-sensitivity analyses. The shared-transform design and static deployment parameters are practically motivated.
Critical weaknesses
  1. The storage overhead figures are inconsistent between Table 3 (p. 8) and Table 9/Appendix A.3.8 (pp. 30 and 39). Please verify it.
  2. There are many unresolved cross-references: “Eq. equation ??” and “Sec. ??”.
  3. The stated “1.7× speedup on Ouro-2.6B” is not clearly supported by the main text, which reports only “1.5–1.7× across three models.” The dual-axis plot in Figure 3 does not provide exact values for each model, I think a table of per-model throughput in tokens/s would be helpful for readers.
Questions for authors
  1. Regarding the storage accounting in Tables 3 and 9, does the complete LCTC component add 0.57 MB or 1.46 MB under Nanbeige W4A6? Which model’s footprint is used as the denominator for the percentages in Table 9?
  2. What precision is used for the KV cache? Is it kept at full precision or quantized as well?
  3. Could the method be evaluated on one or two larger models? Larger and smaller models may differ in their tolerance to quantization and their behavior after quantization. Additional results would help establish the method’s robustness and generalizability.
  4. Several methods have identical GSM8K scores in Table 2, which seems unusual for a eval dataset with many questions. Please verify these results. I am also concerned that evaluation variability may be substantial for smaller models. Were evaluations repeated or results averaged across runs to reduce the influence of randomness?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Review Outcomes

Dear Senior Area Chairs and Area Chairs,

We appreciate your time. We summarize the outcomes, evidence, and alignment with the meta-review.

The initial ratings were 3 / 3 / 2 (aH2i / 416G / jo8v). After the rebuttal, aH2i said their primary questions were “adequately addressed,” raised their Presentation score, and maintained a positive overall rating of 3 (weak acceptance) after weighing novelty and generalizability. 416G found the independent LoopQ comparison and calibration-cost analysis useful, confirmed that their main concerns were addressed, and retained 3. jo8v has not replied to our three follow-ups.

ReviewerStrengthsMain concernsKey updatesOutcome
aH2iClear rationale for the loop-specific components, strong matched W4A4 results, practical shared-weight designLCTC storage discrepancy, exact throughput, KV-cache precision, evaluation at larger scale, run-to-run variabilityCorrected Nanbeige4.2-3B LCTC storage from 1.46 to 0.57 MB at W4A6 (Table 9). Specified full-precision KV cache and reported a 1.70× decoding speedup over BF16 on Ouro-2.6B (Table 12). Added evaluation of the SenseNova-Loop-9B-Preview language component (+0.96 points at matched W4A4, Table 13) and explained five-seed averaging.Primary questions addressed, Presentation raised, positive overall rating of 3 maintained
416GGains across model families and reasoning tasks, informative component and allocation ablationsControlled LoopQ comparison, end-to-end calibration cost, how ILPA scales with loop countIndependently reimplemented LoopQ at W4A4. LoopQuant gains 8.52 and 19.22 percentage points in average accuracy on Ouro-1.4B and Ouro-2.6B over that implementation (Table 6). W4A6 calibration takes 0.87 and 2.63 GPU-hours on these models, with a per-stage breakdown (Table 11).Main concerns addressed, positive overall rating of 3 maintained
jo8vPrincipled analysis of loop errors, strong results across backbones, practical storage and inference measurementsMissing W4A8 comparison, novelty relative to prior PTQ, fair baseline adaptation, calibration cost and variabilityAdded matched W4A8 results across four models, with average accuracy gains of 1.08–2.41 percentage points over the strongest baselines and lower perplexity on both corpora (Table 1). Clarified that all baselines preserve weight sharing and use calibration activations from each loop. Added controlled CLDC, ILPA, and LCTC ablations (Table 10).2 (weak rejection), no reply after three follow-ups

We respectfully ask the AC to consider the following.

  1. Both reviewers who responded maintained positive ratings. aH2i credited the clearer reporting and raised Presentation. 416G found the independent LoopQ reimplementation useful, said the calibration-cost analysis clarified LoopQuant’s practical overhead, and appreciated our disclosure of the remaining differences in experimental settings.

  2. The rebuttal added direct evidence. jo8v identified the missing W4A8 comparison as the largest empirical gap. LoopQuant now outperforms the strongest same-precision baselines on all four original models in average zero-shot accuracy and both perplexity benchmarks. The results on the language component of SenseNova-Loop-9B-Preview extend this evidence to a larger model. The independent LoopQ comparison documents the remaining calibration and quantization differences.

  3. The concerns overlap. 416G found the calibration-cost analysis useful. The same per-stage breakdown directly addresses jo8v’s question about calibration cost. aH2i credited the clearer evaluation reporting, and the five-seed protocol addresses the question about run-to-run variability shared by aH2i and jo8v.

  4. Controlled ablations clarify the loop-specific contribution. We credit prior work on rotations and scaling, CoopQ’s Shapley allocation, and Fisher-based task-space calibration. The new ablations test the worst-loop objective and cross-loop synchronization in CLDC, the interaction term in ILPA, and the trajectory term in LCTC. Each improves accuracy in the controlled setting.

For the camera-ready version, we will foreground the equal-precision results, pair each loop-specific contribution with its ablation, and distinguish LoopQ’s published results from our reimplementation. The revised manuscript includes the comparisons, calibration costs, throughput measurements, and storage correction.

In summary, both reviewers who responded to the rebuttal, aH2i and 416G, found their main concerns adequately addressed and maintained positive overall ratings of 3 (weak acceptance). LoopQuant improves accuracy at equal precision, preserves shared weights, and delivers a measured decoding speedup. We respectfully ask the AC to weigh the updated evidence and discussion outcomes in the final assessment.

Thank you for your time and consideration.

Best regards,

Authors of Submission 57939

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.