acceptodds
Under review as a conference paper at ICLR 2027

An -Law for Scale Multipliers in Low-Bit Post-Training Quantization

Abstract

Preserving language-model accuracy below four bits per weight motivates understanding how quantizers allocate their limited representable values. We ask whether post-training quantizers with different optimization procedures choose scales that follow a shared, predictive relationship. We compare their choices through a per-group multiplier that measures scale contraction relative to the group's largest weight magnitude. For AutoRound, GPTQ, and AWQ, its median follows , where is the largest positive integer quantization code and is fitted per method and configuration. A symmetric clipped mean-squared-error approximation interprets this contraction through the trade-off between clipping and rounding error. Across seven models from six families at two, three, and four bits per weight, the law achieves in-sample . Constants fitted on models with at most nine billion parameters predict the sampled median of Llama-3.1-70B with absolute errors below 0.016 for AutoRound and 0.008 for GPTQ at two and three bits per weight. Among the four tested methods, larger fitted corresponds to higher two-bit accuracy averaged across the seven models. For Llama-3.1-70B quantized to two bits per weight, AutoRound and its ablation without rounding iterations have similar sampled median scale multipliers yet retain 91% and 45% of the half-precision reference model's ten-task mean accuracy. The -law reveals shared structure in method-specific scale choices and supports prediction across bit widths and, for AutoRound and GPTQ, to a held-out 70B model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training quantization, calibration data, low-bit quantization, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QThsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an empirical α-law, α=qmax⁡/(qmax⁡+c)\alpha=q_{\max}/(q_{\max}+c), to characterize scale contraction in low-bit post-training quantization (PTQ). The authors show that method-specific coefficients remain relatively stable across models and bit widths for AutoRound, GPTQ, and AWQ. Experiments across seven models and a held-out 70B model validate the fitting accuracy, while additional analyses explore its relationship with quantization accuracy and calibration sensitivity.

Critical strengths

S1. Extensive empirical validation. The paper systematically analyzes multiple PTQ methods across diverse model families and bit widths, including validation on Llama-3.1-70B. The consistently high fitting accuracy provides empirical support for the observed scale-selection regularity.

Critical weaknesses

W1. Limited novelty and unclear non-triviality. Existing PTQ methods already employ specific scale-selection and clipping strategies, often with fixed optimization procedures and hyperparameters. Therefore, observing similar scale-contraction behavior across models and bit widths may largely reflect these existing algorithmic designs. The paper does not sufficiently establish what fundamentally new insight the proposed α-law reveals beyond them.

W2. Limited practical significance. Although the α-law predicts median scale multipliers accurately, its implications for improving quantization remain unclear. The paper demonstrates neither improved quantization accuracy nor reduced optimization costs or effective method selection using this law. Moreover, the large accuracy gap between AutoRound and AR0 despite their similar median α values raises further questions about the practical utility of this statistic.

Questions for authors

Q1. Non-triviality of the α-law. To what extent can the observed stability of (c) be explained directly by the existing clipping-search and scale-optimization strategies of GPTQ, AWQ, and AutoRound? Can the authors provide concrete evidence that this law captures a non-trivial property beyond these predefined algorithmic choices?

Q2. Practical implications. Can the authors demonstrate how the α-law can meaningfully guide quantization, for example by improving scale selection, reducing calibration or optimization costs, or predicting quantization performance? Concrete evidence of such utility would substantially strengthen the contribution.

Reviewer_GuQ9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the shrinkage ratio of each group relative to the absmax scale during quantization after low-bit training. Key findings include that the AutoRound, GPTQ, and AWQ methods all follow the same family of single parameters across seven models, with method-level constants c of 1.76, 0.49, and 0.35, respectively. The authors explain the mechanism using the clipping-vs-rounding MSE tradeoff of symmetrically truncated uniform quantizers.

Critical strengths
  1. The experiments are rigorous. Width-based holdout and 70B model holdout exclude the test unit during fitting; the median α prediction error for GPTQ/AutoRound on 70B is only 0.008/0.015. Three quantization processes with completely different optimization principles converge to the same single-parameter family.

  2. Good reproducibility. The limitations and reproducibility list are comprehensive.

Critical weaknesses
  1. The derivation cannot quantitatively predict c, and the universality of the law is over-packaged. At λ=1, the Gaussian approximation gives c=1.29c=1.29, which does not match the observed values ​​of 0.35–1.76; the width-wise fitting λ=1.96/1.67/2.03 is unstable. The per-group MSE does not produce a constant c, and empirical fitting does not identify its cause.

  2. The experimental results for the calibration content are insufficient and should not be listed as contributions. Controlled paired tests are significant only for GPTQ W3, AutoRound is marginal, and there are no downstream differences between the two calibration sets on 70B W3.

Questions for authors

The questions are the same as the weekness, thanks.

Reviewer_AVpDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how the per-group scales picked by PTQ methods relate to the largest weight in the group. It defines αg=sgqmax/Mg\alpha_g = s_g q_{max}/M_g and claims that for AutoRound, GPTQ and AWQ the median α\alpha follows α∗=qmax/(qmax+c)\alpha^* = q_{max}/(q_{max}+c), with one constant cc per method: about 1.76 for AutoRound, 0.49 for GPTQ, 0.35 for AWQ, and 1.38 for AR0 (AutoRound's scale tuning without the rounding iterations). RTN and OmniQuant stay near α=1\alpha = 1. A clipped-MSE model is offered as interpretation. Validation is in-sample fits on seven models up to 9B at W2/W3/W4 (R2>0.97R^2 > 0.97), a W3/W4-to-W2 holdout, and prediction of the median α\alpha on Llama-3.1-70B. The paper also reports that larger cc goes with better W2 accuracy, that AutoRound and AR0 have nearly the same α\alpha at 70B W2 but keep 91% vs 45% of FP16 accuracy, and an activation-patching study of calibration data.

Critical strengths

The measurements are careful and the paper is honest about where its claims fail (AR0 at 70B, the cc ordering breaking at 70B, AutoRound's group-size dependence). Scales are read straight from checkpoints across seven models, three bit widths and a 70B model, with seeds, solver settings, group sizes and calibration sets reported. The AutoRound vs AR0 result at 70B is the best thing in the paper: median α\alpha of 0.377 vs 0.372, per-group scales differing by ~3.8%, but 91% vs 45% retention. That shows clearly that the scale summary doesn't determine low-bit accuracy and the rounding optimization does.

Critical weaknesses
  1. The law looks like it comes from quantizer conventions, not optimization. GPTQ's standard symmetric quantizer (sym=true, Sec. K.1) sets s=2M/(2b−1)s = 2M/(2^b-1), which in the paper's coordinates is α=qmax/(qmax+1/2)\alpha = q_{max}/(q_{max}+1/2), i.e. exactly the law with c=0.5c = 0.5. That gives 0.667 / 0.857 / 0.933 at W2/W3/W4. The paper observes 0.681 / 0.860 / 0.934 with fitted cc of 0.48-0.49. If that's right, it explains why GPTQ's cc barely moves across corpora, solver settings, group sizes, Mixtral and 70B: nothing is optimizing it. The 70B "transfer" just shows the default formula is the same. AutoRound has a similar issue. Its export uses s=M/2b−1s = M/2^{b-1} (Footnote 1, Sec. K.5), so plain RTN under that convention gives α=qmax/(qmax+1)\alpha = q_{max}/(q_{max}+1), which fits the law exactly with c=1c = 1. So whether RTN is in the "absmax family" or the "α-law family" depends on the convention. AWQ's W2 value is exactly 0.750 on all seven models and two AR0 W2 cells are exactly 0.500, which look like search-grid points or boundaries. One cc fit to three points per method leaves two degrees of freedom, and the pooled R2R^2 is high mostly because variance across bit widths dominates. The clipped-MSE model doesn't fix this: T=M−cΔT = M - c\Delta is a reparameterization, the MSE optimum gives non-constant cc (1.29 / 1.34 / 1.00), and λ\lambda has to be fit to AutoRound alone.

  2. The c-index doesn't predict anything useful. The link to accuracy rests on four method-level points, and those cc values mix up grid convention, scale search and rounding. At 70B the ordering breaks (AR0 at c=1.38c = 1.38 matches AWQ at c=0.35c = 0.35), and the paper's own AutoRound vs AR0 result shows similar α\alpha with very different accuracy. The "practical use" in Sec. 5 is predicting a W2 median α\alpha that you can read off the checkpoint anyway, and that by the paper's own evidence says little about accuracy. The activation-patching study doesn't connect to the law.

Questions for authors
  1. [Clarification] Can you confirm the scale formula in your GPTQ setup (symmetric, s=2max⁡∣w∣/(2b−1)s = 2\max|w|/(2^b-1), mse=False)? If so, please show the law after dividing each method's α\alpha by its own default RTN α\alpha under the same export convention. Does GPTQ's cc go to about zero, and does the law still hold for AutoRound and AWQ? This determines whether the main claim survives.
  2. [Clarification] What are AWQ's clip-search grid and AutoRound/AR0's scale search range? Are the 0.750 and 0.500 values grid points or boundaries?
  3. [Clarification] Under auto-round's export convention RTN fits the law exactly with c=1c = 1. Why put RTN in a separate family?
  4. [Future work] Is there any setting where knowing cc helps pick a better method or config, e.g. forcing GPTQ or AR0 scales toward AutoRound's cc and measuring accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.