- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LoopQuant, a post-training quantization framework specifically designed for Looped Language Models. The method addresses three properties of recursive inference: cross-loop activation distribution shifts, heterogeneous loop sensitivity, and error accumulation along the recursive trajectory. It combines Cross-Loop Distribution Canonicalization (CLDC), Interaction-Aware Loop-wise Precision Allocation (ILPA), and Loop-Conditioned Task Calibration (LCTC). Experiments cover several LoopLM families and both standard language-modeling/zero-shot benchmarks and reasoning tasks. LoopQuant consistently improves over conventional PTQ baselines, with particularly large gains under W4A4.
Critical strengths
- The results are not limited to one LoopLM or one metric. Under W4A4, LoopQuant substantially outperforms the strongest conventional PTQ baselines across Ouro, Nanbeige, and Huginn, while W4A6 can even outperform W4A8 baselines. The reasoning results are also convincing: the gains persist on MMLU, GSM8K, and HumanEval rather than only on perplexity or simple zero-shot tasks.
- The component ablations show meaningful incremental improvements from CLDC, ILPA, and LCTC, and the ILPA analysis compares against uniform, random, and sensitivity-based precision allocation. The paper also includes deployment-level measurements, reporting substantial memory reduction and a 1.5–1.7× decoding throughput improvement over BF16.
Critical weaknesses
- Most main-table baselines were originally developed for conventional Transformers, so outperforming them on a recursive architecture does not by itself establish the advantage over the strongest loop-aware alternative. The closest related method is LoopQ, but its implementation is unavailable; consequently, the authors compare against numbers taken from the original paper rather than reproducing it under the same calibration data, GPTQ backend, and evaluation pipeline. This does not invalidate the paper, but it weakens the strongest novelty claim: it remains somewhat unclear how much of the gain over prior loop-aware quantization comes from the proposed formulation rather than experimental differences.
- LoopQuant is considerably more involved than a standard PTQ pipeline: CLDC optimizes transformations for every reused module, ILPA evaluates recursive precision configurations and estimates Shapley-style sensitivities/interactions, and LCTC performs teacher-guided optimization of loop-conditioned calibration parameters. The paper carefully reports persistent storage and inference throughput, but I could not find an end-to-end wall-clock or GPU-cost comparison for producing the quantized model. This matters because the paper repeatedly motivates LoopQuant as an efficient and deployment-friendly PTQ framework. The deployed model is clearly efficient; the practicality of the quantization procedure itself, especially as the number of loops increases, is less clear.
Questions for authors
- Can the authors provide a more controlled comparison with LoopQ, either through a faithful reimplementation or by matching as many calibration, quantization-backend, and evaluation details as possible? This would materially strengthen the claim that the gains come from the proposed loop-aware design rather than cross-paper differences.
- What is the end-to-end calibration/quantization cost of LoopQuant for each evaluated model, compared with SmoothQuant/QuaRot/SpinQuant/FlatQuant/DartQuant? In particular, how does ILPA's cost scale with the number of loops, and what is the wall-clock contribution of CLDC and LCTC?
Summary of Rebuttal and Review Outcomes
Dear Senior Area Chairs and Area Chairs,
We appreciate your time. We summarize the outcomes, evidence, and alignment with the meta-review.
The initial ratings were 3 / 3 / 2 (aH2i / 416G / jo8v). After the rebuttal, aH2i said their primary questions were “adequately addressed,” raised their Presentation score, and maintained a positive overall rating of 3 (weak acceptance) after weighing novelty and generalizability. 416G found the independent LoopQ comparison and calibration-cost analysis useful, confirmed that their main concerns were addressed, and retained 3. jo8v has not replied to our three follow-ups.
We respectfully ask the AC to consider the following.
Both reviewers who responded maintained positive ratings. aH2i credited the clearer reporting and raised Presentation. 416G found the independent LoopQ reimplementation useful, said the calibration-cost analysis clarified LoopQuant’s practical overhead, and appreciated our disclosure of the remaining differences in experimental settings.
The rebuttal added direct evidence. jo8v identified the missing W4A8 comparison as the largest empirical gap. LoopQuant now outperforms the strongest same-precision baselines on all four original models in average zero-shot accuracy and both perplexity benchmarks. The results on the language component of SenseNova-Loop-9B-Preview extend this evidence to a larger model. The independent LoopQ comparison documents the remaining calibration and quantization differences.
The concerns overlap. 416G found the calibration-cost analysis useful. The same per-stage breakdown directly addresses jo8v’s question about calibration cost. aH2i credited the clearer evaluation reporting, and the five-seed protocol addresses the question about run-to-run variability shared by aH2i and jo8v.
Controlled ablations clarify the loop-specific contribution. We credit prior work on rotations and scaling, CoopQ’s Shapley allocation, and Fisher-based task-space calibration. The new ablations test the worst-loop objective and cross-loop synchronization in CLDC, the interaction term in ILPA, and the trajectory term in LCTC. Each improves accuracy in the controlled setting.
For the camera-ready version, we will foreground the equal-precision results, pair each loop-specific contribution with its ablation, and distinguish LoopQ’s published results from our reimplementation. The revised manuscript includes the comparisons, calibration costs, throughput measurements, and storage correction.
In summary, both reviewers who responded to the rebuttal, aH2i and 416G, found their main concerns adequately addressed and maintained positive overall ratings of 3 (weak acceptance). LoopQuant improves accuracy at equal precision, preserves shared weights, and delivers a measured decoding speedup. We respectfully ask the AC to weigh the updated evidence and discussion outcomes in the final assessment.
Thank you for your time and consideration.
Best regards,
Authors of Submission 57939