- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes an empirical α-law, , to characterize scale contraction in low-bit post-training quantization (PTQ). The authors show that method-specific coefficients remain relatively stable across models and bit widths for AutoRound, GPTQ, and AWQ. Experiments across seven models and a held-out 70B model validate the fitting accuracy, while additional analyses explore its relationship with quantization accuracy and calibration sensitivity.
Critical strengths
S1. Extensive empirical validation. The paper systematically analyzes multiple PTQ methods across diverse model families and bit widths, including validation on Llama-3.1-70B. The consistently high fitting accuracy provides empirical support for the observed scale-selection regularity.
Critical weaknesses
W1. Limited novelty and unclear non-triviality. Existing PTQ methods already employ specific scale-selection and clipping strategies, often with fixed optimization procedures and hyperparameters. Therefore, observing similar scale-contraction behavior across models and bit widths may largely reflect these existing algorithmic designs. The paper does not sufficiently establish what fundamentally new insight the proposed α-law reveals beyond them.
W2. Limited practical significance. Although the α-law predicts median scale multipliers accurately, its implications for improving quantization remain unclear. The paper demonstrates neither improved quantization accuracy nor reduced optimization costs or effective method selection using this law. Moreover, the large accuracy gap between AutoRound and AR0 despite their similar median α values raises further questions about the practical utility of this statistic.
Questions for authors
Q1. Non-triviality of the α-law. To what extent can the observed stability of (c) be explained directly by the existing clipping-search and scale-optimization strategies of GPTQ, AWQ, and AutoRound? Can the authors provide concrete evidence that this law captures a non-trivial property beyond these predefined algorithmic choices?
Q2. Practical implications. Can the authors demonstrate how the α-law can meaningfully guide quantization, for example by improving scale selection, reducing calibration or optimization costs, or predicting quantization performance? Concrete evidence of such utility would substantially strengthen the contribution.