- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the degradation of downstream capabilities when instruction-tuned language models undergo extremely low-bit quantization. Building on EfficientQAT, RoQAT introduces sensitivity-adaptive progressive quantization (SAPQ), which adjusts each weight group’s transition toward the target bit-width. It also applies quantization-aware sharpness minimization (QSAM) to weights during block-wise training and to quantization scales during end-to-end refinement. Experiments cover three Qwen-family models at 2 and 3 bits, reporting improved average performance on mathematical reasoning, coding, and instruction following. Validation includes comparisons with progressive methods, component ablations, and matched-data experiments.
Critical strengths
- The paper addresses the practically relevant gap between recovering general benchmark performance and preserving downstream capabilities. Evaluating mathematical reasoning, coding, and instruction following alongside general benchmarks provides a useful assessment of quantized models.
- The paper provides controlled evidence for SAPQ. Table 2 uses the same training data and block-wise setting while excluding QSAM, and reports improvements over other progressive methods. The matched-data comparison in Table 5 also favors RoQAT over EfficientQAT on both models, supporting a benefit from the optimization strategy.
Critical weaknesses
-
The attribution of the headline improvements is insufficiently explained. On Qwen3-4B, the downstream-average gain over EfficientQAT decreases from 16.15 percentage points in Table 1 to 1.47 points under matched training data in Table 5. The latter remains a positive result, but the configuration differences between these experiments are unclear, making it difficult to attribute the larger headline improvement to the proposed algorithm.
-
Evidence for the complementary benefits of the two QSAM spaces remains limited. The component ablation in Table 3 covers one model and two tasks, and adding scale-space QSAM reduces HumanEval performance. Although the complete method performs best, the corresponding weight-space-only configuration and variability across independent runs are not reported. The necessity of using both spaces and the stability of their additional gains therefore remain uncertain.
Questions for authors
-
What changes between Tables 1 and 5 in training data, data mixtures, and training budgets for each method? Please explain the substantial change in EfficientQAT’s performance to help distinguish the contributions of training configurations and the proposed algorithm.
-
Under the same settings as Table 3, how does SAPQ combined with only weight-space QSAM perform? Can the authors provide repeated-run results or uncertainty estimates for the key configurations to assess the additional benefit and stability of the two-space design?
-
The Qwen3-4B downstream average is 37.33 in Table 2 but 34.01 under the default settings in Tables 6–7. Do these results use the same data and training configurations? Please clarify any differences or explain the source of the discrepancy.
-
How are the interpolated weights used during progressive training converted into the final target-bit representation? Please specify when and how this conversion occurs and confirm that all reported evaluations use the final target-bit weights.