- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents a novel quantization technique, RCDQ. It is a two-stage group-wise post-training quantization method that uses one full-precision reference objective for both continuous weight compensation, and discrete per-channel scale-multiplier selection.
Critical strengths
The study works on the continuous-to-discrete gap, in quantization, and backs it with mechanism analysis, and a proxy-fidelity study. The method also plugs into GPTQ with cached statistics and needs no extra Transformer passes.
Critical weaknesses
Quantization helps for larger models, The largest model is Mistral-Nemo (12B). The paper needs results on 70B models (Llama-2-70B, Llama-3-70B), other families (Qwen2.5/3, Gemma), MoE models (Mixtral, DeepSeek). Generative evaluations such as GSM8K, HumanEval, and long-context tasks would also help, since perplexity and multiple-choice accuracy can hide degradation. Weight-only results compare only against GPTQ-family methods. They omit AWQ, OmniQuant, LeanQuant, and SchurQuant, even though the paper cites several of these as closely related. Results use a single calibration seed (seed 0), with no variance reported.
Questions for authors
See weakness section
3 comments
Thank you for the detailed response and the additional experiments. The multi-seed results, generative evaluations (GSM8K/HumanEval), and the OmniQuant comparison address several of my concerns. However, the 70B evaluation (perplexity only on Llama-2-70B) and the baseline coverage (weight-only AWQ, LeanQuant, SchurQuant) remain only partially resolved, so I will maintain my score.
Thank you for acknowledging the additional experiments. We hope you will reconsider your score for the following reasons.
1. 70B evaluation
The original review questioned whether RCDQ’s gains extend beyond the models of up to 12B parameters evaluated in the submission. Our added Llama-2-70B experiment directly tests this question, comparing RCDQ against GPTQ, GPTAQ, QEP, and Qronos under matched calibration and evaluation settings. RCDQ achieves the lowest WikiText-2 perplexity, 3.7378 versus 3.8400 for the strongest baseline.
This representative large-model experiment demonstrates RCDQ’s effectiveness at 70B scale, where it outperforms all four baselines. We therefore do not think that testing another 70B model is necessary to support this conclusion.
2. AWQ comparisons
Our submission already included AWQ at W4A8, and our response added W4A4 comparisons. RCDQ achieves lower perplexity on both Llama-2-7B and Llama-3-8B under both settings. The relative reductions are approximately 2.9% and 6.9% at W4A8, and 15.1% and 17.5% at W4A4.
AWQ has therefore received direct experimental coverage across two models and two quantization settings. Given these consistent improvements, we do not think an additional weight-only AWQ comparison is necessary.
3. LeanQuant, SchurQuant, and baseline coverage
Although we cite LeanQuant and SchurQuant in related work, we consider QEP, LoaQ, and weight-MSE scale search more directly relevant to testing RCDQ’s specific contributions. The revised Appendix H therefore adds detailed comparisons and ablations examining group-wise compensation, the residual-output reference, and reference-guided scale selection.
To broaden the baseline comparisons, we also added OmniQuant and a KronQ-based weight-only baseline. Together with the baselines already evaluated in the main paper, these additions provide substantial evidence of RCDQ’s competitive performance. We do not think that establishing this requires experiments against every method cited in related work.
We would appreciate it if you could consider these new results alongside the original experiments when reassessing the paper.
Thank you for your review. Unless specified, following added experiments use W3A16/G128 and 128×2048 calibration tokens. Paired methods share calibration/evaluation data and task protocols. Downstream tasks use C4-calibrated checkpoints; WikiText-2/C4 PPL uses separately calibrated checkpoints. Accuracies are percentages; L=Llama.
1. 70B. Reloaded Llama-2-70B WikiText-2 PPL is 3.8712/3.8515/3.8409/3.8400/3.7378 for GPTQ/GPTAQ/QEP/Qronos/RCDQ: a 2.66% reduction over the best baseline. Quantization on one A100-80GB with CPU offloading takes 208.36 min, with peak GPU allocated memory 25.85 GiB and host RSS 254.03 GiB.
2. Qwen. On Qwen3-8B-Base, RCDQ achieves WikiText-2/C4 PPL of 7.5996/11.3984 and six-task/MMLU/GSM8K accuracy of 67.55/70.92/54.36. It ranks first among five methods on both PPL metrics, MMLU and GSM8K; MMLU/GSM8K improve by 0.72/0.68 points over the respective best baselines.
3. MoE. On DeepSeek-V2-Lite, RCDQ achieves WikiText-2/C4 PPL of 6.7953/9.8786 and six-task/MMLU/GSM8K accuracy of 65.91/50.43/22.14. It ranks first among five methods on both PPL metrics, six-task mean and GSM8K, and ranks second on MMLU. Routers retain FP precision; FP/quantized trajectories may select different experts. Expert down projections use token-aligned aggregate-output residuals as references; gate/up use GPTQ.
4. Generative evaluation. All seven original models are evaluated on 1,319 GSM8K questions (8-shot CoT, greedy, strict-match, 2048-token limit) and 164 HumanEval problems (0-shot, greedy pass@1, one sample/problem, 1024-token limit, official tests).
Each cell below is GSM8K/HumanEval. Best/second among the five listed methods are bold/underlined; ties share ranks.
RCDQ ranks first on 4/7 GSM8K models and second on one; on HumanEval it ranks first on 6/7 and tied second on the remaining model. On Llama-3.2-3B, HumanEval rises from the best baseline's 6.71% to 14.63% (11 to 24 solved problems).
5. Other PTQ baselines. Added OmniQuant follows its official weight-only training procedure and native asymmetric quantization; RCDQ uses symmetric quantization. The paired W3 results below show RCDQ/OmniQuant, with the better value bold.
RCDQ wins all seven C4 PPL and MMLU comparisons, and five GSM8K comparisons. MMLU uses 5-shot evaluation on all 57 subjects.
Original Table 5 already compares AWQ at W4A8: WikiText-2 PPL is 5.8309/7.0264 for AWQ versus 5.6638/6.5386 for RCDQ on L2-7B/L3-8B. Added W4A4 uses matched full Hadamard rotation and A4 settings: rotation-adapted AWQ gives 7.1145/9.7606 versus RCDQ's 6.0384/8.0511.
6. Calibration seeds. We evaluate five methods on two models with seeds 0–4 (50 configurations), sharing calibration samples per model/seed. Below are RCDQ's mean±sample SD (n−1 denominator); accuracy SD is in percentage points. Bold/underline mark best/second mean among five methods.
Qronos C4 PPL is 7.8227±0.0072/11.8076±0.3044, respectively. RCDQ has the best mean and smallest SD on C4 PPL, six-task mean and MMLU for both models, and the lowest PPL in all 10 model×seed comparisons. The six-task mean averages 0-shot PIQA, ARC-Easy, ARC-Challenge, HellaSwag, WinoGrande and BoolQ; MMLU is 5-shot and GSM8K 8-shot greedy strict-match.
We will add these results and full protocols to the revised paper.