- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the limitations of low-bit PTQ by treating codebook optimization, outlier restoration, and pruning as related decisions under a unified projected-distortion metric. Beyond optimizing the codebook with activation-aware weighting and entropy regularization, PDGQ uses a rate-distortion-based criterion to select weights for full-precision restoration and removes weights when zeroing introduces less error than quantization.
Critical strengths
-
The activation-weighted Lloyd refinement explicitly accounts for input-channel importance during codebook optimization, improving low-bit quantization performance over unweighted refinement.
-
The evaluation covers multiple model families and bit-widths, with results on both language modeling perplexity and downstream reasoning tasks.
Critical weaknesses
-
PDOR uses a rate-distortion-based reference to normalize quantization errors and derives a shared restoration threshold based on a chi-square distribution. However, the statistic in Eq. (6) also includes a normalized activation-energy factor, whose effect on the threshold is not fully explained. Could the authors clarify the derivation and provide evidence that the threshold generalizes across layers and bit-widths?
-
The codebook optimization combines activation-weighted Lloyd refinement with entropy regularization, while DCP simply compares the errors of quantization and zeroing. Although these designs are reasonable, it remains unclear how much they improve over straightforward alternatives. More detailed ablations against activation-weighted quantization, magnitude-based outlier restoration, and zero-aware quantization would help clarify their contributions.
-
The projected-distortion objective in Eq. (1) is approximated by a per-weight sum in Eq. (2), ignoring the off-diagonal terms of the input covariance matrix . Since these terms capture correlations between input channels, how much does this approximation affect the distortion estimates and subsequent restoration decisions? A comparison with the full covariance-based objective on representative layers would help clarify this point.
-
PDGQ restores some weights to full precision and introduces additional indexing or masking overhead, so its effective bit-width can be higher than the nominal quantization bit-width. Comparing methods at matched effective bit-width, including these overheads, would provide a clearer picture of its actual compression–accuracy trade-off.
Questions for authors
-
Could the authors clarify how sensitive PDGQ is to the choice of the entropy regularization coefficient, and whether the observed gains remain consistent across different models and bit-widths?
-
How much does the diagonal approximation in Eq. (2) affect the distortion estimates and restoration decisions compared with using the full covariance matrix?
-
Could the authors report comparisons at matched effective bit-widths, including the overhead of storing restored weights and their indices, to clarify whether PDGQ retains its advantage under the same storage budget?