acceptodds
Under review as a conference paper at ICLR 2027

Who Pays for Compression? The Answer Depends on the Instrument

Abstract

Post-training quantization (PTQ) degrades some languages far more than others, and the standard explanation blames the calibration set: calibration-based methods such as GPTQ are said to push quantization error out of the subspace the calibration data spans, so languages far from that subspace should suffer most. We show that the mechanism is real, that it is protective rather than harmful, and that the evidence for the attribution is far more fragile than the literature assumes. We confirm the mechanism directly: GPTQ drives the normalized in-subspace residual energy γ̄ to 0.213–0.277 while round-to-nearest and norm-matched isotropic noise sit at 1.00, across 3 multilingual encoders and an autoregressive decoder. Evacuation is beneficial: at identical weight-error norm, GPTQ does 1.51×–11.61× less damage than isotropic noise. We then ask whether a pre-computable misalignment statistic R_ell predicts which languages pay, and whether that prediction is specific to calibration-based quantizers. Using quantized weights that are bit-identical across conditions, we show the answer depends on the measurement instrument. Under non-parallel evaluation the apparent calibration-specificity is inflated; under content-controlled parallel evaluation it disappears on encoders scored by a masked-language-model proxy, yet remains large on a decoder scored by bits-per-byte. We argue that per-language quantization damage must be measured on parallel corpora with the deployment objective, and we quantify how many languages such claims actually require.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
post-training quantization, calibration, multilingual, GPTQ, low-resource languages, model compression, subspace alignment, fairness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2D3zOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the mechanisms underlying language-specific performance degradation after post-training quantization (PTQ). The authors examine whether quantization errors are pushed away from the calibration activation subspace and whether the resulting activation-subspace misalignment predicts quantization damage across languages. The paper introduces two quantities. Experiments are conducted on three multilingual encoders and one autoregressive decoder, using GPTQ, AWQ-style quantization, RTN, and isotropic noise controls.

Critical strengths

The paper addresses an important problem in multilingual model compression and attempts to move beyond simple correlations between language characteristics and quantization degradation. The comparison between calibration-based quantization methods and calibration-free controls is conceptually useful. The observation that evaluation protocols can substantially affect reported cross-lingual correlations is potentially valuable.

Critical weaknesses
  1. The paper is poorly written and presented.
  2. The experimental coverage is relatively limited given the breadth of the conclusions. In particular, the key decoder result is supported by only one model and 15 languages. The lack of AWQ results for the decoder also limits the generality of claims concerning calibration-based quantizers.
  3. The central instrument-dependent conclusion is confounded by model architecture and evaluation objective. The paper argues that calibration-specificity disappears under encoder-based MLM evaluation but remains significant under decoder-based bits-per-byte evaluation. However, the comparison simultaneously changes several factors. Consequently, the reported difference cannot be uniquely attributed to the measurement instrument.
Questions for authors

none

Reviewer_riZPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates dataset (esp. language) dependency on the effect of calibration data on quantized model performance. It shows that GPTQ suppresses quantization residual energy in the principal subspace of calibration activations. It uses R_l, the fraction of a language’s activation energy outside this subspace, as a predictor of quantization-induced loss. Experiments with GPTQ and RTN demonstrate the utility of R_l to compare the amount of performance degradation due to the given quantization method between datasets (e.g., languages in multi-lingual models).

Critical strengths

R_l provides an interpretable diagnostic of calibration-subspace misalignment and shows predictive value for language-wise quantization damage in some settings. Table 5 is useful in distinguishing how the evaluation corpus affects the estimation of R_l versus the measurement of quantization damage, while keeping the quantized weights fixed.

Critical weaknesses

It is not clear whether R_l is a purely pre-quantization metric. According to Appendix A, R_l^avg needs quantization to calculate evacuation (gamma). If so, it will be a post-quantization metric. Similarly, it would be nice to clarify whether R_l is calculated from the evacuation of GPTQ-En.

Evaluation metrics and model configurations are not clearly separated. It would be nice to obtain metrics, e.g., token CE, bits-per-byte under the same model configuration, e.g., the same Qwen model with the same weights and input texts.

Statistical results and analyses need to be carefully examined. Tables 3-4, and 8 show a few numerical mismatches. The p-value results (up to ~0.9) look difficult to draw strong conclusions.

Table 7 evaluates balanced multilingual calibration, but the additional benefit of using R_l to guide the construction of the calibration dataset remains unclear. It would be interesting to try R_l aware language selection (e.g., favoring high R_l languages) in building the new multi-lingual calibration dataset.

Questions for authors

See Weaknesses.

Reviewer_rdJsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper challenges the effect of Post Training Quantization of multilingual models on languages that are far from the subspace that the calibration set spans. They argue that the effect is actually protective rather than damaging.

Across 3 encoders and a decoder setup GPTQ drives the normalized in-subspace residual energy down while RTN and norm matched isotropic noise sit at 1.00 confirming the mechanism, however at identical weight error norm, GPTQ does 1.51$\times$ - 11.61$\times$ less damage when compared to isotropic noise.

Further they investigate the possibility of a pre-computable misalignment statistic RlR_l predicting which languages suffer most and if that prediction is specific to calibration-based quantizers.

Critical strengths
  1. Results from Table 1 are quite informative, GPTQ regularly shows reduced residual energy in calibration subspaces and at identical weight error norm in Table 2 show really strong performance of GPTQ as compared to Isotropic Noise model for XLM-R-base
  2. The cross corpus evaluation is useful and shows different patterns for different models. EuroBERT-210m gap tracks the outcome corpus and XLM-R base gap tracks the predictor corpus
Critical weaknesses
  1. Equation (2) that defines the per layer statistic RlR_l based on activations, and appendix A describes the layer aggregation for R−lR-l which uses the quantization residual E(n)E^(n). This then follows that the model-level statistic RlR_l contracts the ability to predict damage "pre-quanitization"
  2. There are a bunch of inconsistencies in the reported tables. Reported gaps for the Qwen models in Table 4 do not match the compute gaps from Table 3. Table 8 mentions that the required nln_l for a power of 0.80 is > 150 for gap 0.34 but Power at 92 reads 0.96.
Questions for authors
  1. Please clarify the errors in the tables mentioned in the weaknesses sections
  2. Also, Table 3 seems incomplete / missing rows, EuroBERT 610 on Wikipedia is missing and Qwen perf on AWQ-En is also missing or not specified
  3. Can you provide specifics for the calibration pool, token budgets, language lists and which decoder blocks were used. Section 7 says decoder blocks were declared but it was not identified.
  4. Can you explain the discrepancies between Tables 3, 4, and 6 a bit more so we can understand the reported results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.