- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LoPRo, a fine-tuning-free post-training low-bit weight quantization method for LLMs that combines low-rank decomposition with importance-aware partial rotation. It identifies a conflict between low-rank decomposition and global rotation caused by a negative relationship between Hessian-based importance and residual error, and addresses it by protecting important columns while applying block-wise Walsh–Hadamard rotations to the rest. The method also introduces R1SVD for efficient low-rank approximation and supports a vector-quantization variant. Experiments on multiple LLM families show competitive low-bit performance.
Critical strengths
- The paper identifies an interaction between low-rank decomposition and global rotation, motivating a partial-rotation strategy rather than applying global transformations indiscriminately.
- The importance-aware column ordering provides an intuitive strategy for preserving important residual components, while R1SVD reduces the computational cost of low-rank decomposition.
- The experiments cover multiple LLM families across different quantization settings. The results show promising performance compared with existing post-training quantization baselines, alongside fast quantization and modest additional memory overhead.
Critical weaknesses
- The theoretical justification relies on a diagonal Hessian approximation and additional simplifying assumptions, which may limit the generality of the analysis under practical neural network conditions.
- The experiments do not fully isolate the individual contributions of importance-aware column ordering and partial rotation, making it difficult to attribute the observed gains to specific design choices.
- The improvements vary across models and settings. It remains unclear whether the method provides consistent advantages when effective bit-widths and total storage overheads are strictly matched against strong baselines.
Questions for authors
- Could the authors provide additional ablations comparing importance-aware ordering with random ordering and no ordering, while keeping the rotation strategy and quantizer fixed?
- How consistent is the relationship between Hessian-based importance and residual quantization error across different layers and architectures? How sensitive is LoPRo to deviations from this relationship?
- How does LoPRo compare with vector-quantization and low-rank baselines when the total memory footprint is strictly matched, including the overhead introduced by low-rank components and vector-quantization codebooks?
- The paper reports inference latency overhead at batch size 1. Could the authors clarify how this overhead is distributed between the low-rank branch and rotation-related operations, and how it changes with batch size?
Dear Reviewers,
I would like to kindly remind everyone to review the authors' responses and provide your feedback.
If you have any further questions or concerns regarding the authors' responses, please feel free to raise them early, so that the authors have sufficient time to address them before the rebuttal period ends.
Thank you all very much for your time and effort! And, have a nice weekend. :)
Best regards,
AC