acceptodds
Under review as a conference paper at ICLR 2027

ARSO: Reconstruction-Aware Rounding with Joint Scale Optimization for Weight-Only LLM Quantization

Abstract

Post-training quantization (PTQ) compresses large language models (LLMs) into low-precision representations to reduce memory and accelerate inference without retraining. In weight-only PTQ, existing methods either rely on round-to-nearest (RTN), whose local decisions ignore reconstruction error, or employ costly iterative rounding optimization. Moreover, quantization scales are often reused for dequantization, coupling integer-grid construction with weight reconstruction. We identify reconstruction freedom as an important optimization resource. Scale Optimization (SO) exploits scale freedom by deriving reconstruction-optimal dequantization scales for fixed integer codes; eliminating this scale freedom further reveals a directional criterion over integer weights, enabling Reconstruction-Aware Rounding (AR). In sequential quantization, AR exploits future compensation freedom through a conditional reconstruction metric, while J-SO exploits prefix scale freedom by jointly updating processed-group scales. This stage-wise exploitation naturally yields ARSO, a PTQ method with single-pass reconstruction-aware rounding that achieves the lowest perplexity in 32 of 36 settings and the highest average zero-shot accuracy across all six models at 2 bits among the evaluated methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Post-Training Quantization, Weight-Only Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_PzB4OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ARSO, a weight-only post-training quantization method that combines scale optimization, reconstruction-aware rounding, and sequential compensation to improve quantization accuracy. Experiments on LLaMA and Qwen models across 2-4 bits demonstrate improved perplexity and zero-shot accuracy over some existing baselines.

Critical strengths
  1. This paper reports strong improvements across multiple LLaMA and Qwen models, with particularly substantial gains at 2 bits. The ablations also provide some evidence that the proposed components contribute to the overall result.
  2. This paper present a coherent sequential quantization procedure. ARSO improves quantization accuracy without introducing additional inference-time overhead, while requiring only about 2$\times$ the quantization time of GPTQ. This makes the approach practical for LLM deployment.
Critical weaknesses
  1. The claimed contributions are not new, and the relationship to closely related prior work is insufficiently established. First, the proposed SO is essentially the same optimization problem considered in "Two-Stage Grid Optimization for Group-wise Quantization of LLMs" (ICASSP 2026). The latter solves it through closed-form coordinate-descent updates, whereas ARSO formulates the corresponding normal equations. These are different solution procedures for the same quadratic objective rather than fundamentally different optimization formulations. Second, the proposed AR in Section 3.2.2 relies on the Schur-complement conditional curvature obtained by eliminating the optimal future compensation variables. This has already been explicitly developed in STLA (ICML 2026), and should not be presented as a contribution. Neither work is cited or discussed, raising serious concerns about the originality of the paper's core contributions.
  2. The experiments omit comparisons with these closely related methods. Moreover, the ablation results suggest that most improvements come from SO, while the additional gains from AR and J-SO are relatively modest, making it difficult to establish the value of the remaining novel components.
Questions for authors
  1. How J-SO differs from the group-scale refinement with fixed integer codes in "Two-Stage Grid Optimization for Group-wise Quantization of LLMs" (ICASSP 2026).
  2. Can the authors clarify how ARSO’s reconstruction-aware rounding relates to "STLA: Spatiotemporal Lookahead Alignment for Post-Training Quantization" (ICML 2026), particularly their use of a Schur-complement objective to account for future weight compensation?
  3. Can the authors provide comparisons with these closely related methods under matched experimental settings?
  4. Can the authors explain when repeatedly re-optimizing the processed-prefix scales is expected to help, and whether the reported gains depend on performing these updates during sequential quantization rather than after it?
Reviewer_AvMmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a post-training quantization method that jointly determines how weights are rounded and quantization scales are selected. They derive a closed-form for optimizing the scales of multiple quantization groups at once. The paper extends these ideas to sequential quantization, where unquantized groups can compensate for errors introduced by previously quantized groups. Experiments across Llama and Qwen models seem to support the proposed method, and the baselines compared against are reasonable but can improve with additional experiments.

Critical strengths

1/ The paper is on quantization an important topic in AI efficiency. 2/ The experimental setup is reasonable, and the method seems to show improvements over some state-of-the-art methods such as GPTQ, OmniQuant.

Critical weaknesses

1/ My main concern is the novelty in the paper. It is my impression that ARSO uses the same type of second-order and error feedback ideas as GPTQ [1] and LDLQ [2] (e.g. Eqs. 16 and 18 of this paper), and is primarily remixing these ideas and adding the scale optimization on top. This scale optimization is simply a closed-form least-squares update, which I do not think by itself is novel. For example, COMQ [3] is another method for fitting the scales, and appears to have very similar updates to this paper. Overall, I think the paper lacks much original content.

2/ I find the presentation to be relatively poor. For example, there is repeated use of the vague term ‘reconstruction freedom’. This is clarified later, though I find this term adds more confusion than it’s worth. Similarly, in the appendix there is a supposed equation ‘sequential GPTQ compensation≡recursive conditional minimization’. This seems AI generated, and undermines the presentation of the paper.

3/ Figure 1 is hard to follow and not very illuminating – I don’t think it has enough information in it to appear in the beginning of the paper.

4/ The method is set up as optimization of a layerwise least squares loss under quantization constraints. How does this compare with other weight quantization procedures such as ADMM-Q [4]?

[1] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR, 2023

[2] Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa. QuIP: 2-Bit Quantization of Large Language Models With Guarantees. NeurIPS, 2023.

[3] Aozhong Zhang, Zi Yang, Naigang Wang, Yingyong Qi, Jack Xin, Xin Li, and Penghang Yin. COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization. IEEE Access, 2025.

[4] Lucas, R., Makni, M., Meng, X., Deng, A., & Mazumder, R. (2026). ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models. arXiv preprint arXiv:2605.11222.

Questions for authors

See questions above.

1/ I find the current Appendix B.2 “Connection to GPTQ compensation” to be lacking in clarity with respect to the connection between ARSO and GPTQ. Most of the explanation is focused on GPTQ, and not on how the methods are related. Can the authors please spell this out more carefully? A connection to LDLQ would also be helpful.

2/ COMQ [3] is another method for fitting the scales/integers jointly, and appears to have very similar updates to this paper. Could the authors clarify the novelty with respect to COMQ?

3/ Compare with other PTQ weight optimization method(s) beyond GPTQ as noted above (ADMM-Q).

4/ It seems from Table 3 that most of the gains come from the scale optimization. Could the authors provide a more detailed ablation of the extra components (J-SO and the rest of ARSO) and explain why they are necessary?

Reviewer_rj78OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates weight-only PTQ by re-examining the degrees of freedom available during quantization. The authors observe that standard affine quantization couples integer grid selection with real-valued dequantization magnitude. They introduce Scale Optimization (SO) to derive the optimal dequantization scale in closed form for fixed integer codes. Analytically substituting the optimal scale into the reconstruction loss shows that integer rounding reduces strictly to maximizing directional cosine alignment. Extending this to sequential quantization, the authors introduce Reconstruction-Aware Rounding (AR) under a conditional metric that accounts for future compensation freedom, combined with Joint Scale Optimization (J-SO) over processed prefix groups. Experiments across LLaMA-2/3 and Qwen3 models show strong improvements, especially at 2-bit precision.

Critical strengths
  1. Conceptual Clarity and Analytical Insight: Decomposing quantization into directional code selection and magnitude scaling via closed-form scale elimination is theoretically grounded and provides fresh insight into rounding without heuristic iterative relaxations (like AdaRound).
  2. Single-Pass Efficiency with State-of-the-Art 2-Bit Accuracy: ARSO avoids iterative backpropagation while achieving notable accuracy improvements over OmniQuant and SpinQuant at 2 bits, requiring only ~2× the offline runtime of standard GPTQ.
Critical weaknesses
  1. Narrow Improvement Margins at Higher Bit-Widths: While 2-bit quantization shows clear advantages, the gains at 3-bit and 4-bit over standard GPTQ/AWQ are relatively modest (and in a few sub-benchmarks, baseline methods remain competitive or slightly better).
  2. Computational Overhead of Prefix Scale Inversion: In J-SO, re-optimizing all scales across the processed prefix PP involves repeatedly solving linear systems (Hs,P+ϵI)sP∗=bP(H_{s,P} + \epsilon I) s_P^* = b_P. Although manageable for moderate context/model sizes, scaling behavior to very large layers or context windows needs clearer profiling.
Questions for authors
  1. As the prefix group size PP grows large towards the end of a wide linear layer, how does the cost of solving the linear system for sP∗s_P^* scale, and are any iterative or block-cholesky approximations utilized?
  2. How does ARSO behave when combined with activation transformation techniques (such as QuaRot or SpinQuant rotations)?
Reviewer_S8uVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ARSO, a weight-only PTQ method, which decouples the quantization and dequantization scales. The dequantization scales are derived in closed form, and substituting them back into the objective yields a scale-invariant rounding decision, which is used within GPTQ-style sequential compensation.

Critical strengths
  1. Most of the derivation is clean and intuitive.
  2. The extension of the scale-eliminated rounding criterion to the group-wise setting, using a Schur-complement metric that accounts for future error compensation.
  3. The components are integrated cleanly into the GPTQ framework with modest offline quantization cost.
Critical weaknesses

Many of the proposed ideas are not new, and the authors do not draw a clear line between existing prior work and the proposed novelty. For example, decoupling the quantization and dequantization scales is not new: Eq. (9) is essentially Eq. (7) in decoupleQ (Guo et al., 2024) without the zero point. Additionally, the idea and equations in Section 3.2.1 are almost identical to Section 3 of Beacon (Zhang and Saab, 2025). Neither a qualitative nor a quantitative comparison is provided.

Some details of the method are unclear. For example, in lines 4–5 of Algorithm 1, after the rounding decision for t_i is made, what is the new criterion score for the rest of the weights in the group? I cannot find the implementation details or the derivation in the paper’s write-up.

Experiments are limited to models of 14B or smaller. Given the extra quantization overhead, it is unclear how the method performs on larger models (e.g., at the 70B level). In addition, comparisons with prior works on reasoning models for generation-based evaluation tasks are missing.

Although the W2 results look better than prior works, the 12–23 pp drop on downstream tasks limits the practical significance.

Questions for authors

Please see and address my comments above.

What is the format of the dequantization scale?

Some important ablations are missing. Please report the performance of AR using H_GG (Eq. 15). This would help identify whether it is enough to simply extend Beacon from per-channel to per-group, without the compensation terms for the future groups. Please also report the performance of GPTQ + SO + J-SO, so that it is clear how much AR helps when scale optimization is applied.

It seems fair to compare with weight-only NVFP4/MXFP4 (e.g., GPTQ in those formats). Depending on the precision of the scales chosen by this work (which is not disclosed in the paper), the equivalent bit-width can be very similar. Since this work’s scope is weight-only quantization, the compute advantage of FP4 formats is not relevant anyway; only memory bandwidth matters. Another reason is that recent work (FOCUS, Yan et al., 2026) also explores the same scale-decoupling idea, though it focuses on the freedom of the quantization scale. So it would be interesting to understand the difference here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.