- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a Transformer quantization method that exploits parameter-space symmetry in attention. It jointly transforms Query and Key weights while preserving attention scores, and uses an analytical estimate of quantization noise to guide optimization without calibration samples.
Critical strengths
-
The joint Query–Key transformation preserves attention scores while changing weight distributions, providing a principled way to improve quantization without altering the full-precision model.
-
The method requires no calibration samples and can be combined with existing quantization methods, including AWQ and bitsandbytes, making it easy to integrate into existing quantization pipelines.
Critical weaknesses
-
The motivation for data-free quantization is not sufficiently established. The paper does not clearly explain when avoiding calibration data is necessary or what practical limitations of existing calibration-based methods it addresses. Since the method analytically estimates quantization noise without calibration samples, “calibration-free” may also better describe its setting and distinguish it from approaches that synthesize calibration data.
-
The experimental comparisons with existing methods are limited. Prior work, including SmoothQuant (Xiao et al., 2023) and OmniQuant (Shao et al., 2024), has explored function-preserving transformations for quantization, including transformations within attention. Comparisons with these methods and prior Q–K reparameterization approaches would help clarify the practical advantages of the proposed method.
-
The analytical noise estimate relies on assumptions about quantization errors and activation distributions, but its agreement with actual quantization errors is not fully clear. Although Fig. 1(b) shows that the estimated noise decreases during optimization, its trend does not fully align with the empirical noise, particularly in later iterations.
-
The experiments mainly cover BERT and relatively small OPT models, and the gains vary across tasks and quantization settings. Evaluating more model scales and bit-widths would help clarify how well the method generalizes, particularly under low-bit quantization.
Questions for authors
-
To what extent does the analytical noise estimate reflect actual quantization errors across different transformations and quantization settings?
-
How does the proposed Q–K transformation compare with existing Q–K reparameterization methods in terms of quantization performance?
We thank the reviewers for their feedback and insightful comments. We are especially grateful for the additional citations and baseline recommendations. Reviewers point out that many of these baselines also explore orthogonal rotations for improving quantization but to our knowledge we are the only fully data-free approach. We will withdraw the paper at this time and work on addressing these concerns and clarifying the unique aspects of our approach in a future revision of the work.