- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work introduces a joint alternating refinement method for standard integer quantizers, which can serve as a drop-in replacement for conventional integer quantization in a PTQ framework. According to the reported results, the proposed method generally improves the performance of existing PTQ frameworks, although in some cases it yields negligible or even negative gains. However, the integration details with existing PTQ frameworks remain unclear. For instance, OmniQuant already jointly optimizes the quantization scale and offset through block-wise learning. Since the proposed method also refines the quantization scale, it is unclear how the two optimization procedures are combined without conflict. Moreover, several important baselines are missing. In particular, HQQ [1] also adopts an alternating optimization scheme for quantization scales and zero-points, while COMQ [2] directly performs joint optimization of the quantization scale and discrete quantization codes. Besides, this work formulates quantized code optimization as a Babai nearest-plane problem and solves it in a group-wise manner. This is closely related to GPTQ, which has recently been shown to be equivalent to Babai’s nearest-plane algorithm under an appropriate formulation [3]. However, the paper does not discuss or compare against this connection. Therefore, the paper requires further improvement.
[1] Badri, H. and Shaji, A. Half-quadratic quantization of large machine learning models, November 2023. URL https://mobiusml.github.io/hqq_blog/.
[2] Zhang A, Yang Z, Wang N, et al. Comq: A backpropagation-free algorithm for post-training quantization[J]. IEEE Access, 2025, 13: 99879-99891.
[3] Chen, Jiale, et al. "The geometry of llm quantization: Gptq as babai's nearest plane algorithm." International Conference on Learning Representations. Vol. 2026. 2026.
Critical strengths
- The proposed joint refinement method is designed as a generic replacement for standard integer quantization, making it potentially applicable to a wide range of existing PTQ frameworks.
- In most reported cases, the proposed method further improves the performance of the compared PTQ approaches.
Critical weaknesses
- The paper does not clearly explain how the proposed refinement is integrated into existing PTQ frameworks such as OmniQuant, which already optimizes quantization scales and offsets, or GPTQ, which has recently been shown to be equivalent to Babai’s nearest-plane algorithm. This makes it unclear whether the proposed optimization complements, replaces, or potentially overlaps with the original optimization procedures.
- The comparison with closely related prior work is also incomplete. Methods such as HQQ [1] and COMQ [2] are not included, and the relationship between the proposed group-wise Babai code optimization and GPTQ’s interpretation as Babai’s nearest-plane algorithm [3] is not sufficiently discussed, making the methodological novelty difficult to assess.
[1] Badri, H. and Shaji, A. Half-quadratic quantization of large machine learning models, November 2023. URL https://mobiusml.github.io/hqq_blog/.
[2] Zhang A, Yang Z, Wang N, et al. Comq: A backpropagation-free algorithm for post-training quantization[J]. IEEE Access, 2025, 13: 99879-99891.
[3] Chen, Jiale, et al. "The geometry of llm quantization: Gptq as babai's nearest plane algorithm." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors
- The characterization of HQQ in the related work seems somewhat incomplete, since HQQ also performs iterative refinement through half-quadratic alternating optimization rather than simply optimizing the zero point once. Could the authors clarify the conceptual and algorithmic difference between HQQ’s alternating refinement and JARQ’s proposed alternating optimization?
- Could you directly compare GPTQ+HQQ with GPTQ+JARQ under matched settings? Please keep the other experimental conditions as controlled as possible, including the initialization, bit width, group size, and calibration data.
- Although is chosen because it captures most of the perplexity improvement in the Llama-2 experiments, what happens when is further increased for downstream tasks? Does the calibration objective continue to decrease while held-out or downstream performance begins to degrade due to overfitting to the calibration data?
Response to Comments on Numerical Stability and Propagation-Aware Reconstruction
We sincerely thank the reviewer for the positive assessment of JARQ's theoretical contributions, general applicability, and experimental evaluation. We also appreciate the thoughtful suggestions regarding numerical stability, error propagation, and computational efficiency.
Q1. Ridge Stabilization and Monotonic Descent
Thank you for raising this important question. We conducted an additional numerical audit across 8 configurations (Llama-2-7B/Qwen3-8B × RTN/GPTQ × W3/W4).
Among 33,128,444 scale solves, ridge stabilization was triggered only 59 times (0.000178%), all in Qwen3-8B/GPTQ/W3, blocks 2–5. Importantly, none of these updates increased the original reconstruction objective.
All 59 cases involved groups with zero centered integer-code vectors, which provides a structural explanation. The scale-fitting system includes a positive diagonal contribution proportional to ν2∥ci∥2, where ν=0.6 is the weight-reconstruction regularization parameter. When ci=0, the corresponding scale is unidentifiable. In practice, the risk is therefore confined to groups whose centered codes are all zero, which we observed only at three bits.
As discussed in Remark 1, a simple safeguard is to keep a ridge-regularized scale update only when it does not increase the original objective and to retain the previous scales otherwise. With this safeguard, the monotonicity guarantee (Theorem 1(i) and Corollary 1) holds without the exact-solve assumption; only Part (iii) still requires exact scale solves. Alternatively, one can retain the scales of inactive zero-code groups and solve the reduced least-squares system. We will clarify these safeguards and the empirical stability results in the revision.
Q2. Propagation-Aware Reconstruction
We appreciate the reviewer highlighting the distinction between local reconstruction and propagated block-output error. We first clarify that JARQ's standard target is XW rather than XW: each layer is fitted on the inputs it actually receives from the quantized preceding layers. We conducted additional experiments to examine the choice of reconstruction target and its interaction with propagation-aware quantization.
Direct comparison of reconstruction targets. Following Appendix E.5, we compared the standard target (μ=1, used throughout the paper) with the propagation-aware interpolated target (μ=0.5, corresponding to QEP's default correction strength), while keeping the host initialization, calibration data, group size (G128), and iteration budget (K=3) fixed.
The experiments cover three models (Llama-3-8B, Qwen2.5-7B, Qwen3-8B), two hosts (GPTQ and OmniQuant), and both W3A16 and W4A16. The standard target (μ=1) achieves lower WT2 and C4 perplexity in all 24 model–host–bit-width–corpus comparisons, with absolute PPL differences ranging from 0.32 to 10.02.
For example, on Qwen3-8B/GPTQ/W3, WT2 PPL is 10.54 with the standard target (Table 1), compared with 13.24 with μ=0.5. These controlled results provide empirical support for the standard reconstruction objective adopted in JARQ. We will include the full ablation in the revision.
Compatibility with propagation-aware quantization. Importantly, JARQ can directly complement QEP. As demonstrated in Table 3(a), applying JARQ after QEP+GPTQ improves WT2 perplexity in all four settings:
This establishes that propagation-aware correction in the host stage and JARQ refinement are complementary.
Propagated error under the standard target. Our existing experiments further show that JARQ improves local reconstruction in 92.9% of 7,616 block–module comparisons on held-out WT2/C4 inputs. It reduces final block-output error in 23/32 configurations: 8/8 for RTN, 7/8 for GPTQ, 6/8 for AWQ, and 2/8 for OmniQuant. OmniQuant already optimizes block outputs, leaving less room for further improvement.
Together, these results support the standard μ=1 objective while demonstrating that JARQ can be effectively combined with propagation-aware host quantization.