- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors proposed ReQuant to address activation mismatch for PTQ methods. Specifically, ReQuant reformulates the activation-aware reconstruction objective from a row-wise perspective, deriving a closed-form gradient and loss update that enable discrete coordinate search for optimization. ReQuant integrates into existing PTQ approaches in a plug-and-play manner. Experimental results also further validate ReQuant's effectiveness and generality across model families, quantization initializers, and bit-widths.
Critical strengths
- ReQuant is consistently effective across various PTQ settings and model families, with significant gains in average accuracy when applied to AWQ and GPTQ.
- ReQuant is elegant in its method design, as closed-form updates of the gradient and loss become trackable via a row-wise decomposition.
Critical weaknesses
- Incremental novelty against GPTAQ. ReQuant and GPTAQ address the same issue of 'activation mismatch,' and the proposed method is essentially a reformulation of the same objective given in GPTAQ without a solid guarantee of better performance. Results in Table 1 support this: average accuracy improves significantly when combined with AWQ and GPTQ because ReQuant fixes the activation mismatch omitted by the two PTQ methods; however, when applied to GPTAQ, the improvement is quite marginal since GPTAQ already addresses activation mismatch. On the other hand, ReQuant does not achieve better efficiency, as its quantization wall-clock time is significantly longer than GPTAQ and other PTQ methods: Table 7 reports at least a 3.64x increase in quantization runtime on Qwen-3-14B. Therefore, I question ReQuant's contribution, as it achieves only marginal performance at a much longer runtime.
- Insufficient discussion on the PTQ family. The PTQ methods involved in the experiments (i.e., GPTQ, AWQ, and GPTAQ) all belong to the scalar-based family. I wonder whether ReQuant is also compatible with codebook-based methods, such as AQLM [1] and NF4 [2], since the coordinate update described in Eq. 8 assumes only a scalar-based formulation. If not, please explicitly state this limitation in the revisions.
References
[1] Extreme Compression of Large Language Models via Additive Quantization (ICML 2024)
[2] QLoRA: Efficient Fine-tuning of Quantized LLMs (NeurIPS 2023)
Questions for authors
Based on the weaknesses identified, I hope the authors can answer the following questions to address my major concerns:
- Could you clarify ReQuant's novelty and advantage over GPTAQ, with detailed support such as a theoretical guarantee of better performance/convergence and more persuasive experimental findings?
- Is there a solution to mitigate its efficiency issues, i.e., long quantization times?
- Could you discuss ReQuant's compatibility with codebook-based methods?