acceptodds
Withdrawn from ICLR 2027

From Jump Degradation to Warped-Space Quantization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The growing parameter scale of large language models imposes substantial memory and computational costs, making weight quantization an important approach to efficient deployment. Quantization represents continuous weights with a limited set of discrete levels, and its effectiveness depends increasingly on how this limited precision is allocated as the bit width decreases. Existing methods improve this allocation through optimized rounding, scaling, codebooks, or transforms, but they are commonly designed using static statistics collected from weights or calibration data. During model execution, however, quantization error evolves dynamically: fixed weight residuals interact with changing activations that may already contain perturbations propagated from upstream layers. To characterize this process, we progressively replace full-precision weights with their quantized counterparts and compare the resulting intermediate activations with those of the original model. We observe jump degradation, in which activation error rises abruptly at a small number of layers and continues to amplify downstream. Further analysis identifies the mismatch between equally spaced quantization levels and non-uniform weight distributions as one controllable source of the residual involved in this process. We therefore propose Warped-Space Quantization (WSQ), which maps weights into a flatter space for uniform sampling and applies the inverse map to construct distribution-matched non-uniform levels. By allocating more of the limited precision to high-density regions, WSQ reduces the inputindependent weight residual before it interacts with changing activations. Experiments across multiple models and benchmarks show that WSQ consistently improves quantized-model quality and outperforms the compared baselines under directly comparable low-bit quantization settings.

Primary area:
optimization
Keywords:
ML: Efficient, Edge, Green & Hardware-aware ML

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.60 over 5 reviews, above 10% of the others.
Reviewer_279BOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

By progressively replacing full-precision weights with quantized weights, the paper observes that activation error can jump at a small number of layers and propagate downstream, a pattern it calls jump degradation. The authors identify mismatch between uniform quantization and non-uniform weight distributions as one controllable source of error, and propose WSQ: transform the weight space, sample uniformly or design a Lloyd–Max codebook there, then map back to obtain distribution-matched non-uniform levels. Experiments report results across models, bit widths, language modeling, understanding, and agent tasks, with larger gains at lower precision. However, WSQ and GPTQ do not provide additive gains.

Critical strengths
  1. Layerwise activation-error tracking gives a direct view of error propagation that complements static reconstruction metrics.
  2. WSQ's warp–sample–inverse-warp construction is simple, requires no input-dependent runtime transform, and is evaluated across several models and bit widths.
Critical weaknesses
  1. The “jump degradation” pattern may depend on calibration samples, the error norm, layer boundaries, and replacement order. The paper should systematically specify the metric and quantify the stability of jump locations and magnitudes across data samples and models. The current argument from correlation to causal mechanism is limited.
  2. The relationship between WSQ and standard PTQ/rounding optimization remains unclear. The paper reports only a small additional gain from WSQ after GPTQ and suggests GPTQ may undermine its codebook assumptions. More matched-budget comparisons, repeated runs, and quantization/calibration cost analysis are needed.
  3. The method's novelty is limited: applying Lloyd–Max or distribution-matched levels after a space transform is rooted in classical scalar quantization. The paper needs to establish a clearer contribution over NF4, prior non-uniform codebooks, and data-aware quantization. The ablation showing nearly no gain after GPTQ further weakens the case for a distinct practical contribution.
Questions for authors
  1. How robust are the jump metric and identified layers to the error norm, input samples, and replacement order? Can you report confidence intervals across samples?
  2. Does the WSQ–GPTQ interaction persist across models, bit widths, and level constructions? What is the cost/benefit relative to jointly optimizing the codebook and compensation?
Reviewer_M7ZsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper Introduces Warped Space Quantization (WSQ) a novel method for non-uniform quantization aimed to address "jump degradation" a phenomenon observed where the residual quantization error increases drastically in certain MLP layers. WSQ makes use of a simple strategy to adaptively define a non uniform quantization quantization scheme that is better suited for quantizing heavy-tailed weight distributions, which are observed in recent LLMs. An experimental section demonstrates the effectiveness of WSQ against other methods in literature.

Critical strengths
  • The paper has a clear structure presenting a clear problem definition, method development and experimental analysis.
  • The paper identifies a relevant problem involving quantization error propagation and narrows down some possible explainations.
Critical weaknesses

Main concerns

  • The authors detail the analysis behind the "jump error degradation" providing intuition and illustrations. However, the claims in the text are backed up by little empirical evidence and experimental analysis (Only table 3)
  • The connection between the identified error pattern and the solution is not clear as it seems to indicate that the solution is to decrease the quantization error magnitude, which is essentially the same target as any other PTQ method in the literature. Furthermore, the claim that the proposed method indeed solves the identified problem is also very limited (Only table 3) since the method seems to also produce new jumps at times.
  • AWQ, GPTQ and AQLM are included in the comparison for Figure 6 but not in Table 1. This makes the trade-off between the algorithms less clear
  • The Efficiency analysis does not include any estimate of the de-quantization time that would be required to perform the matmul.

Other issues

  • Figure 6 is not easily readable because of the small components
  • The claim that "Attention projections follow the gaussian reference, whereas MLP down_proj deviates in the tails" in Figure 4 is not visible from the picture as the two plot seem almost identical.
  • The claim that WSQ and GPTQ provide complementary gain is a bit misleading as applying GPTQ to WSQ degrades its performance.
  • Although a reference for the Lloyd-Max optimization is provided, not all terms in equation 7 are defined or explained. Overall the definition of the warping function is quite unclear, especially on how vectors are assigned to codewords.
  • Table 2 does not provide a useful comparison since different methods are reported at different bit-widths
Questions for authors
  • Figure 3a shows that most of the excess kurtosis is accumulated in the initial layers. However, Table 3 seems to suggest that the biggest jumps occur in layers 16,17 and 31-35. The proposed WSQ method is motivated by the mismatch between the weight distribution and the uniform quantization schemes. Shouldn't we expect the observed jump correlating with the higher kurtosis?
  • Table 3 shows that WSQ removed the jumps in layer 16 and 17 but also creates a new one in layer 34. Can the author clarify the connection between the observed problem and the proposed solution considering these observations?
  • Can the authors clarify how the parameters for the warping function are stored and provide an explicit definition of the forward (or inverse) warp c(w)c(w)?
  • Does the proposed WSQ solve the Jump Degradation problem? How is the proposed method associated to the problem in the first case is not very clear.
Reviewer_h9Q7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how quantization errors propagate across layers in large language models. The authors identify jump degradation, where activation errors increase abruptly at certain layers and propagate downstream, and associate this behavior in part with the mismatch between uniform quantization levels and non-uniform weight distributions. Based on this analysis, they propose Warped-Space Quantization (WSQ), which uses distribution-matched non-uniform quantization levels and optionally incorporates geometric transformations and vector quantization to reduce weight reconstruction errors. Experiments across Qwen3, Gemma 2, and Llama 3.1 models evaluate reconstruction error, activation fidelity, perplexity, and downstream performance. The reported results show improvements over several quantization baselines, particularly under low-bit settings.

Critical strengths

The paper provides a useful empirical characterization of layerwise quantization error, including abrupt increases in activation error and downstream amplification. This analysis provides additional evidence about error propagation beyond conventional weight reconstruction metrics. The 4-bit experiments show improvements in perplexity over several established baselines across multiple model architectures. The accompanying reductions in weight reconstruction and activation errors provide empirical evidence for the effectiveness of the proposed quantization approach.

Critical weaknesses

The paper does not clearly explain what its scalar quantizer adds to existing Gaussian-based non-uniform quantization. Section 3.3 constructs a shared level table using Lloyd–Max optimization for a standard Gaussian, while Q-Palette [1] constructs scalar levels using 1D k-means on Gaussian samples. Both optimize squared reconstruction error for a Gaussian reference, although their implementations and scaling differ. WSQ cites Q-Palette, but the distinction between these constructions remains unclear, making it difficult to assess the scalar quantizer's contribution separately from the useful jump analysis.

Section 2.2 emphasizes earlier sensitivity and reconstruction studies, but would benefit from a more direct discussion of layer-wise and block-wise PTQ for LLMs. The relationship between WSQ's analysis and these approaches also needs clarification. For example, QEP [2], which examines downstream error growth in LLMs, is not discussed. HIGGS [3], which combines Hadamard preprocessing with Gaussian MSE-optimized vector quantization, is also not discussed or evaluated. Although the experiments include established LLM baselines, comparisons with closely related Gaussian-based quantizers would help establish whether WSQ provides additional benefits over methods that already use this design principle.

[1] Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment. NeurIPS 2025.

[2] Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization. NeurIPS 2025.

[3] HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem. NAACL 2025.

Questions for authors

Could the authors clarify how WSQ's Gaussian-based Lloyd–Max scalar quantizer differs from Q-Palette's scalar quantizer, and how its geometric and vector quantization components relate to HIGGS? What does the warped-space formulation add to these existing procedures, and is there evidence of an additional benefit under comparable quantization settings, including the bit budget?

Could the authors clarify how the jump analysis extends prior work on error propagation in LLMs, particularly QEP? For researchers developing or evaluating LLM quantization methods, what additional understanding of quantization failures or quantizer effectiveness does identifying abrupt jumps provide beyond the downstream error growth examined in QEP?

The title in the PDF differs from the OpenReview title. It would be appropriate to confirm that the uploaded PDF is the intended version and align the titles.

Reviewer_MbCuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • The paper addresses low-bit weight quantization of language models, where quantization error jump sharply at a few layers.
  • It proposes Warped-Space Quantization, which places quantization levels more densely where weights are common, optionally with a rotation and learned vector codebooks.
  • The method achieves lower perplexity than the compared baselines.
Critical strengths

None

Critical weaknesses
  1. The low-bit results do not come from the method in the main text and are compared unfairly. The appendix states that results use learned vector codebooks with LoRA-based training, while all baselines are post-training methods.
  2. The scalar method is close to known work, and the closest baselines are missing.
Questions for authors
  1. How does scalar WSQ compare with NF4 and SqueezeLLM at the same group size?
Reviewer_sFjyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tracks activation error as weights are progressively quantized and reports "jump degradation," where error rises sharply at a few layers and then amplifies. It attributes part of the weight residual to a mismatch between uniform levels and non-uniform weight distributions. WSQ warps weights (companding plus optional Hadamard rotation), samples uniformly or with Lloyd–Max/vector codebooks, and inverts the warp. It reports lower perplexity than AWQ, OmniQuant, and LeanQuant on Qwen3-8B at 2–4 bits and results on three more models.

Critical strengths

Layerwise tracking of quantization error is a good idea and the authors provide intuition of how this relates to quantized model performance. Negative results are published in the appendix and are discussed in the analysis of the MSE-PPL gap.

Critical weaknesses

Jump-degradation is only shown in the context of uniform RTN quantization without examining whether the degradations are still present in other methods (GPTQ, AWQ, etc.). The jump degradation may also be impacted by the quality of calibration data among many other factors. Limited experimentation showing that WSQ reduces the quantization MSE (Table 3). Results are run on a single model once and do not consider other setups like calibration data, etc. The method does not seem very complimentary with existing PTQ approaches. The gain shrinks from 0.106 to 0.016 PPL under GPTQ and WSQ is slightly worse than Uniform+GPTQ, which limits practical value (the authors acknowledge repeated runs are needed).

Questions for authors

Can you show a plot of the quantization error trajectories that are summarized in Table 3? Is there a difference between cascading patterns and the simple propagation of error? Does jump degradation appear in other models, and do relative-error metrics show the same pattern?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.