- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the mechanisms underlying language-specific performance degradation after post-training quantization (PTQ). The authors examine whether quantization errors are pushed away from the calibration activation subspace and whether the resulting activation-subspace misalignment predicts quantization damage across languages. The paper introduces two quantities. Experiments are conducted on three multilingual encoders and one autoregressive decoder, using GPTQ, AWQ-style quantization, RTN, and isotropic noise controls.
Critical strengths
The paper addresses an important problem in multilingual model compression and attempts to move beyond simple correlations between language characteristics and quantization degradation. The comparison between calibration-based quantization methods and calibration-free controls is conceptually useful. The observation that evaluation protocols can substantially affect reported cross-lingual correlations is potentially valuable.
Critical weaknesses
- The paper is poorly written and presented.
- The experimental coverage is relatively limited given the breadth of the conclusions. In particular, the key decoder result is supported by only one model and 15 languages. The lack of AWQ results for the decoder also limits the generality of claims concerning calibration-based quantizers.
- The central instrument-dependent conclusion is confounded by model architecture and evaluation objective. The paper argues that calibration-specificity disappears under encoder-based MLM evaluation but remains significant under decoder-based bits-per-byte evaluation. However, the comparison simultaneously changes several factors. Consequently, the reported difference cannot be uniquely attributed to the measurement instrument.
Questions for authors
none