acceptodds
Under review as a conference paper at ICLR 2027

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

Abstract

Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models. Existing PTQ methods typically initialize quantized models with heuristic rules or greedy optimization, and usually treat the resulting integer assignments as final. We introduce ReQuant, a backpropagation-free fixed-grid refinement stage *within* PTQ that optimizes an executable quantized model while preserving its format. Agnostic to the PTQ initializer, ReQuant starts from an existing feasible quantized model and iteratively revisits its discrete weight assignments on the fixed quantization grid. Accepted updates strictly reduce the mean squared reconstruction error and remain on the original grid. ReQuant thus integrates with existing PTQ pipelines as a plug-and-play post-processing stage. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers, with especially large gains on simple initializers and lower bit-widths. Notably, ReQuant can refine a simple round-to-nearest initialization across multiple sweeps until it approaches or surpasses GPTAQ under the same quantization format.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post-training quantization, Large language models, Discrete optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hJLAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors proposed ReQuant to address activation mismatch for PTQ methods. Specifically, ReQuant reformulates the activation-aware reconstruction objective from a row-wise perspective, deriving a closed-form gradient and loss update that enable discrete coordinate search for optimization. ReQuant integrates into existing PTQ approaches in a plug-and-play manner. Experimental results also further validate ReQuant's effectiveness and generality across model families, quantization initializers, and bit-widths.

Critical strengths
  1. ReQuant is consistently effective across various PTQ settings and model families, with significant gains in average accuracy when applied to AWQ and GPTQ.
  2. ReQuant is elegant in its method design, as closed-form updates of the gradient and loss become trackable via a row-wise decomposition.
Critical weaknesses
  1. Incremental novelty against GPTAQ. ReQuant and GPTAQ address the same issue of 'activation mismatch,' and the proposed method is essentially a reformulation of the same objective given in GPTAQ without a solid guarantee of better performance. Results in Table 1 support this: average accuracy improves significantly when combined with AWQ and GPTQ because ReQuant fixes the activation mismatch omitted by the two PTQ methods; however, when applied to GPTAQ, the improvement is quite marginal since GPTAQ already addresses activation mismatch. On the other hand, ReQuant does not achieve better efficiency, as its quantization wall-clock time is significantly longer than GPTAQ and other PTQ methods: Table 7 reports at least a 3.64x increase in quantization runtime on Qwen-3-14B. Therefore, I question ReQuant's contribution, as it achieves only marginal performance at a much longer runtime.
  2. Insufficient discussion on the PTQ family. The PTQ methods involved in the experiments (i.e., GPTQ, AWQ, and GPTAQ) all belong to the scalar-based family. I wonder whether ReQuant is also compatible with codebook-based methods, such as AQLM [1] and NF4 [2], since the coordinate update described in Eq. 8 assumes only a scalar-based formulation. If not, please explicitly state this limitation in the revisions.

References

[1] Extreme Compression of Large Language Models via Additive Quantization (ICML 2024)

[2] QLoRA: Efficient Fine-tuning of Quantized LLMs (NeurIPS 2023)

Questions for authors

Based on the weaknesses identified, I hope the authors can answer the following questions to address my major concerns:

  1. Could you clarify ReQuant's novelty and advantage over GPTAQ, with detailed support such as a theoretical guarantee of better performance/convergence and more persuasive experimental findings?
  2. Is there a solution to mitigate its efficiency issues, i.e., long quantization times?
  3. Could you discuss ReQuant's compatibility with codebook-based methods?
Reviewer_ARFxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a new backpropagation-free post-processing stage for PTQ, which iteratively refines discrete weight assignments.

Critical strengths

The paper is generally well-written and easy to follow.

Critical weaknesses
  1. The most significant concern is that the proposed method seems to be almost the same as CDQuant, which assigns integers based on the principle of coordinate descent. While the authors incorporated input deviations ΔX\Delta \mathbf{X} incurred by the quantization of previous layers, this technique is well-known since GPTAQ, and the integration is trivial. Is there any new thing?

  2. While the authors claimed that their approach can be integrated with existing PTQ pipelines as a plug-and-play post-processing stage, I think that such statement only applies to layer-wise PTQ methods. How can this approach be extended for block-wise methods (e.g., FlexRound, OmniQuant, Block-AP of EfficientQAT), which target the reconstruction of transformer block outputs, without relying on backpropagation?

  3. The comparison is limited. There have been various methods that improve GPTQ / GPTAQ (e.g., BoA, TurboBoA, YAQA).

Questions for authors

see weaknesses above

Reviewer_LJU2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReQuant, which takes quantized weights obtained from existing post-training quantization (PTQ) methods as initialization and iteratively refines their quantized values on a fixed quantization grid. By fixing the bit width, scale, and zero-point, the method minimizes layer-wise error that accounts for the input mismatch between the original and quantized models through discrete updates. For each quantized value, the change in loss is evaluated exactly using a quadratic expression, and the gradient is updated incrementally, eliminating the need for backpropagation. The theoretical analysis shows that, under fixed calibration data and a fixed quantization grid, accepted updates monotonically decrease the loss and the number of accepted updates is finite. The method is evaluated across multiple models, quantization configurations, and initial quantization methods, demonstrating particularly large improvements for RTN, AWQ, and low-bit settings. However, the improvements are smaller when the initial quantization method is already strong, and performance degradation is observed on certain tasks and corpora.

Critical strengths
  1. Practical applicability as an additional optimization step for existing quantization methods and formats. Since the method optimizes only the quantized values without modifying the scale, zero-point, or other quantization parameters, existing inference implementations can be used without modification. Furthermore, it does not require computationally expensive operations such as backpropagation, making it practical as an additional optimization step.

  2. Demonstration of room for improvement in existing quantization methods across diverse settings. The paper demonstrates improvements across multiple PTQ methods, model sizes, and bit widths. In particular, the substantial improvements over RTN and AWQ indicate that there is considerable room for further optimization of quantized weights obtained from existing methods. The method is also applicable to a practical large-scale 235B MoE model.

Critical weaknesses
  1. Insufficient differentiation from and direct comparisons with the most closely related prior work, making it difficult to assess novelty and relative advantages. Although they may not explicitly frame their approaches as re-quantization, existing methods such as QuantEase [1], QuIP [2], and CDQuant [3] already optimize quantized values after initialization. I believe direct comparisons with these methods are essential.

  2. The publicly released implementation contains a component that is not described in the paper. Specifically, in the released implementation, the correction term arising from the activation mismatch is multiplied by a coefficient beta, whose default value is 0.3. If this coefficient affects performance, I believe it should be explicitly described in the paper.

[1] QuantEase: Optimization-based Quantization for Language Models (https://arxiv.org/abs/2309.01885)

[2] QuIP: 2-Bit Quantization of Large Language Models With Guarantees (https://arxiv.org/abs/2307.13304)

[3] CDQuant: Greedy Coordinate Descent for Accurate LLM Quantization (https://arxiv.org/abs/2406.17542)

Questions for authors
  1. What are the technical differences between ReQuant and the post-initialization quantized-value optimization methods mentioned in the Weaknesses section? Does ReQuant demonstrate any empirical advantages over these methods?

  2. Does the beta parameter in the publicly released script affect the monotonic-decrease guarantee? Also, to what extent does it affect performance? The default value of 0.3 is considerably smaller than beta = 1, which appears to be implicitly assumed in the paper's formulation (although beta is not mentioned in the paper at all). Therefore, I would expect this difference to have some impact on performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.