- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes JacQuant, an alternative to the straight-through estimator for ultra-low-bit quantization-aware training. Instead of using a fixed backward rule, JacQuant learns group-wise gradient scaling factors from the quantizer's responses to Gaussian weight perturbations, while preserving the hard quantizer in the forward pass. These factors are updated periodically and reused across training steps to limit computational overhead.
The paper also provides a theoretical analysis of the learned sensitivity estimates and conditional convergence guarantees for a local surrogate objective. Experiments on 1B–8B LLaMA and Qwen models show improvements in perplexity and zero-shot accuracy over ParetoQ and WinQ baselines, with relatively low reported training overhead and no additional inference cost.
Critical strengths
-
The proposed method is easy to follow. JacQuant estimates quantizer sensitivity using lightweight probes and maintains only one scalar gain per weight group. The periodic update strategy also helps reduce the additional computational cost.
-
Promising experimental results . The method is evaluated on different model families, model sizes, and quantization settings. The results show consistent improvements in perplexity and average zero-shot accuracy over the corresponding baselines.
Critical weaknesses
-
Insufficient ablation studies. The main results in Table 1 combine sensitivity learning with variance reduction, making it difficult to assess the individual contribution of the proposed method. It is also unclear whether the Rotation Trick baseline uses the same variance-reduction setting.
-
Limited discussion of the theoretical analysis. The convergence guarantees rely on a local surrogate with fixed quantization codes and parameters, while actual training uses AdamW with evolving codes and scales. The connection between the theoretical results and practical optimization remains unclear.
-
Insufficient implementation details. Appendix E describes different variance-reduction strategies without specifying which is used in the main experiments. This limits reproducibility and makes it difficult to assess the reported training overhead in the main paper.
Questions for authors
-
Could the authors provide a comparison of STE and JacQuant, both with and without variance reduction, under the same experimental setting, such as ParetoQ on LLaMA3-1B W2A16?
-
Does the proposed method still show improvements compared with group-shuffled gains, where the gain distribution and refresh timing are kept the same at each step?
-
Could the authors clarify which variance-reduction estimator is used for each quality and runtime result?