- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents SignRoundV2, a post-training quantization framework designed to maintain model performance under extremely low-bit quantization. Building on SignRoundV1, it combines gradient information and quantization-induced errors to guide layer-wise mixed-precision allocation, and introduces loss filtering and a pre-tuning scale search to improve tuning effectiveness. Experiments on Llama and Qwen models show improved accuracy in 2-bit weight-only quantization and average task performance close to full precision at an average of 4.5 bits under mixed MXFP4/6 quantization.
Critical strengths
1 The method combines gradient-based sensitivity estimation and layer-wise bit allocation with lightweight scale initialization and loss filtering. These components are simple and use a small calibration dataset.
2 SignRoundV2 improves over SignRoundV1 on all models reported in Table 1 under the same 2-bit weight-only quantization setting, providing clear evidence of its effectiveness in this regime.
3 The reported results show competitive accuracy on large models at substantially lower quantization cost than the compared QAT and vector-quantization methods, while keeping the cost close to that of SignRoundV1.
Critical weaknesses
1 The paper would benefit from a clearer account of its new contributions. The method builds on SignRoundV1, uses an established approach for bit allocation, and draws on the importance matrix in llama.cpp for scale initialization. In particular, please clarify how DeltaLoss differs from prior sensitivity metrics and what advantages these differences bring.
2 Section 3.2 defines the discrete optimization objective and constraints, but states only that the problem can be solved by dynamic programming or integer linear programming. Please specify the solver used in the experiments and provide sufficient implementation detail or pseudocode to reproduce the bit assignments.
3 The ablation results need further discussion. In Table 4, V1+Filtering outperforms the full method on four tasks for Qwen3-8B. The ablation results need further discussion.
Questions for authors
No further questions for the authors.
3 comments
Thank you for your response. The ablation results show that DeltaLoss performs better, but they do not explain why. Its formulation looks very similar to that of LLM-mq. Please explain how the two methods differ in principle and why these differences lead to better performance.
Thank you for the quick reply!
Thanks for the quick reply, especially over the weekend.
The major difference between our method and LLM-MQ lies in the point at which the gradients are computed.
LLM-MQ computes gradients with respect to the full-precision weights. This raises two theoretical concerns. First, when a model is well-trained and close to a local optimum, the gradients with respect to the full-precision weights are expected to be close to zero, making them less informative for sensitivity estimation. Second, the gradients computed using full-precision weights are shared across different bit-widths, and therefore cannot adequately reflect the loss associated with different quantization bit-widths.
In contrast, our method computes gradients with respect to the quantized weights, avoiding the above issues.
We have explained these differences in Lines 213–226. Please let us know if any part of our explanation is unclear or if you have further concerns.
Thank you!
Thank you for taking the time to review our work. We hope you have a great weekend!
1. In particular, please clarify how DeltaLoss differs from prior sensitivity metrics and what advantages these differences bring.
We have added a new ablation study in Table 7 comparing DeltaLoss with LLM-mq, as well as an ablation study on Hessian information in Table 37. Both studies demonstrate the advantages of our method and provide further insights into the importance of first-order information, which has potentially been overlooked by the community.
2. Dynamic programming reproducibility
We have added pseudocode in Appendix J (line 1532) to clarify the dynamic programming procedure. The implementation is relatively straightforward, and we will open-source the complete code to ensure reproducibility.
3. In Table 4, V1+Filtering outperforms the full method on four tasks for Qwen3-8B.
Similar to how a quantized model can sometimes outperform its full-precision counterpart on individual tasks, such performance variations across tasks are expected. This is precisely why we evaluate our algorithm across multiple tasks rather than drawing conclusions from individual results. Therefore, we do not believe this observation requires further discussion.