- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
By progressively replacing full-precision weights with quantized weights, the paper observes that activation error can jump at a small number of layers and propagate downstream, a pattern it calls jump degradation. The authors identify mismatch between uniform quantization and non-uniform weight distributions as one controllable source of error, and propose WSQ: transform the weight space, sample uniformly or design a Lloyd–Max codebook there, then map back to obtain distribution-matched non-uniform levels. Experiments report results across models, bit widths, language modeling, understanding, and agent tasks, with larger gains at lower precision. However, WSQ and GPTQ do not provide additive gains.
Critical strengths
- Layerwise activation-error tracking gives a direct view of error propagation that complements static reconstruction metrics.
- WSQ's warp–sample–inverse-warp construction is simple, requires no input-dependent runtime transform, and is evaluated across several models and bit widths.
Critical weaknesses
- The “jump degradation” pattern may depend on calibration samples, the error norm, layer boundaries, and replacement order. The paper should systematically specify the metric and quantify the stability of jump locations and magnitudes across data samples and models. The current argument from correlation to causal mechanism is limited.
- The relationship between WSQ and standard PTQ/rounding optimization remains unclear. The paper reports only a small additional gain from WSQ after GPTQ and suggests GPTQ may undermine its codebook assumptions. More matched-budget comparisons, repeated runs, and quantization/calibration cost analysis are needed.
- The method's novelty is limited: applying Lloyd–Max or distribution-matched levels after a space transform is rooted in classical scalar quantization. The paper needs to establish a clearer contribution over NF4, prior non-uniform codebooks, and data-aware quantization. The ablation showing nearly no gain after GPTQ further weakens the case for a distinct practical contribution.
Questions for authors
- How robust are the jump metric and identified layers to the error norm, input samples, and replacement order? Can you report confidence intervals across samples?
- Does the WSQ–GPTQ interaction persist across models, bit widths, and level constructions? What is the cost/benefit relative to jointly optimizing the codebook and compensation?