- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PrismQuant, an orthogonal rotation method for group quantizers, aimed at reducing activation errors in W4A4KV4 quantization of large language models. The authors exploit the property that the constant direction within a group does not increase the quantization range, align the dominant activation energy to these directions via spectral decomposition, and prove the optimality of the energy-capture objective based on the Ky Fan principle. In implementation, Householder reflections, the Compact-WY representation, and grouped Hadamard transforms are employed, combined with weight folding and low-rank online correction to reduce computational overhead. Experiments cover Llama, Qwen, Mistral, and MoE models, and the results show that the method effectively reduces quantization error and improves perplexity and task performance on most tested models.
Critical strengths
- This paper designs an orthogonal rotation based on the geometric structure of the quantizer itself, rather than focusing solely on the distribution of activation outliers, and provides a clear optimization objective and optimal solution through theoretical analysis, which offers a certain degree of innovation and theoretical value.
- The experiments cover Dense and MoE models of different scales, and the effectiveness of the method is verified through multiple ablation studies. In addition, the authors consider the computational overhead of structured rotation, demonstrating good potential for engineering applications.
Critical weaknesses
- The paper emphasizes using the affine offset of asymmetric quantizers to absorb activation energy, but experiments show that the method still achieves similar performance advantages under symmetric quantization. This suggests that the main benefits may stem more from activation flattening and energy localization, and the contribution of the core mechanism still needs to be further clarified. Moreover, the theoretical optimality of energy alignment does not directly guarantee optimal practical quantization performance.
- The deployment performance experiments and the main accuracy experiments adopt different quantization configurations, and the performance tests use random weights. Therefore, although the existing results verify the execution efficiency of structured rotation, they cannot yet fully demonstrate that the complete method can simultaneously achieve the reported accuracy and speed benefits in real models.
Questions for authors
- Could you further explain the respective contributions of spectral alignment, activation flattening, and affine offset absorption to the performance improvement, as well as the relationship between theoretical optimality and practical quantization performance?
- Could you clarify what additional deployment overhead the complete method may introduce under the asymmetric group quantization configuration consistent with the main accuracy experiments, and whether such overhead would affect the existing performance conclusions?
2 comments
A brief follow-up before the discussion period closes
Thank you again for your careful review. As today is the last day of the discussion period, we would like to check whether our response has answered your two questions, or whether anything remains unclear.
In brief:
- Mechanism (W1 / Q1). Your observation was right. The new attribution on Llama-3.1-8B (Table 17, p. 41) shows that flattening and localization carry the larger part of the gain and that offset absorption accounts for about 40%. The paper is reframed accordingly, and Section 4.3 now states that optimality holds for the alignment objective.
- Deployment (W2 / Q2). The checkpoint of Table 1 is now deployed with real weights in the group-128 asymmetric configuration, and its perplexity (6.99, identical to Table 1) and latency are read from the same loaded model (Table 20, p. 43).
If any point is still open, we would be glad to address it today. If these two points are now resolved, we would be sincerely grateful if you would consider updating your rating to reflect that. Your questions led to the mechanism analysis in Appendix F.2, which we consider one of the most valuable additions to the paper.
Response to Reviewer NeUz
Thank you for the careful and insightful review. We were glad that you describe the rotation as designed "based on the geometric structure of the quantizer itself". Your two questions probe exactly what that claim must support, and answering them has made the paper more precise. New experiments and analyses are in Appendix F (pp. 39–44, in blue).
W1 / Q1. "... the respective contributions of spectral alignment, activation flattening, and affine offset absorption ... as well as the relationship between theoretical optimality and practical quantization performance?"
1. Contributions of the three mechanisms (Appendix F.2, Table 17, p. 41). You are right: at g=128, flattening and localization carry the larger part of the gain, and offset absorption accounts for 16–40% (now stated in Sec. 1 and App. A.4). Our original framing leaned too much on the offset, and your observation corrected it. The three act in sequence, and ablations on Llama-3.1-8B (g=128, k=max, 3 seeds) isolate each:
2. Theoretical optimality vs. practical performance (Sec. 4.3, p. 9; App. F.2, p. 41). We agree: Proposition 1 guarantees optimality for the alignment objective (Eq. 6), not for perplexity, and your question prompted us to state this explicitly. Maximizing that objective minimizes residual energy, which bounds the sum of squared group ranges (Eq. 26). The mean quantization step also depends on the crest-factor ratio of the residual (Eq. 30, p. 26; 0.87–0.92 at the down-projection input, Table 12), which a second-moment objective does not control. This is why PPL is not strictly monotone across ranks k (Table 3). With the construction fixed, PPL tracks the captured energy f monotonically, across subspace estimators (Table 7) and solver passes (Table 8).
W2 / Q2. "... what additional deployment overhead the complete method may introduce under the asymmetric group quantization configuration ... and whether such overhead would affect the existing performance conclusions?"
It does not affect the conclusions. Your concern was a fair description of the submission, and we have now deployed the group-128 asymmetric Llama-3.1-8B checkpoint of Table 1 with real weights and integer kernels (Appendix F.1, pp. 39–40).
We hope this answers both of your questions. If it does, we would be grateful if you would consider raising your score, and we are glad to clarify anything further.