- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the output deviations caused by changes in expert routing during post-training quantization of Mixture-of-Experts models. The authors argue that existing metrics cannot fully measure the output deviations caused by routing changes. To address this issue, the paper proposes a routing deviation metric based on two inference modes: Free Run and Frozen Run. Experiments on multiple MoE models evaluate the effectiveness of this metric and analyze how errors change under different quantization bit widths.
Critical strengths
This paper focuses on changes in expert routing caused by weight errors during post-training quantization of MoE models, which is a meaningful research problem. The authors conduct systematic experiments to analyze routing deviations under different quantization bit widths and provide theoretical analysis to explain how different errors change.
Critical weaknesses
The proposed routing deviation metric (R) can distinguish output deviations caused by changes in expert routing from other quantization errors, which has some research value. However, the paper does not fully explain what meaningful new findings this more detailed error analysis can provide or how it can guide the evaluation and selection of quantization methods. In particular, since existing metrics such as KL Divergence and answer disagreement rate can already detect changes in model outputs, the additional value of R has not been clearly demonstrated. Therefore, the overall contribution and practical significance of this paper are not sufficiently convincing.
Questions for authors
- The goal of model quantization is to improve storage and inference efficiency while maintaining accuracy as much as possible. If two quantization methods have similar efficiency and storage costs, but one has lower Accuracy and a larger R, it would generally not be preferred. Conversely, if one method achieves higher Accuracy or better inference efficiency but also has a larger R, how should we choose between them? In other words, beyond measuring routing deviations, what additional value can R provide for practical quantization decisions?
- The paper finds that R and D_frozen follow different scaling patterns at higher bit widths. However, as the bit width increases, the absolute values of both deviations decrease. R simply decreases more slowly, so its relative contribution to the total deviation increases. Beyond showing that reconstruction error cannot fully capture routing deviations, what new insights does this finding provide? For example, can it help guide the selection of quantization bit widths or the comparison of different quantization strategies?
- In Section 4.4, restoring the Router weights reduces R by approximately 68%, but Accuracy improves by only 0.65 percentage points, with a very small gain. Meanwhile, KL Divergence and answer disagreement rate can also detect changes in model outputs. Although R can further distinguish the contribution of routing changes, what is the practical value of this additional information? For example, when different quantization methods have similar overall output deviations, can R help determine whether the Router or other modules should be optimized first, rather than simply explaining the observed experimental results?
3 comments
I thank the authors for their detailed rebuttal, which has partially addressed my concerns. However, after considering the other reviewers’ comments, I remain concerned about the novelty of the work. Therefore, I have decided to maintain my original recommendation.
Reply to Reviewer RHcp
Thank you for reading the rebuttal and for the reply. We are glad that it addressed part of your concerns. Your questions on what R adds beyond KL and answer disagreement, and on how to choose when accuracy and R disagree, led us to compare more released checkpoints and to state in Section 5 that accuracy remains the criterion. We will keep your remaining concern about novelty in mind as we continue this work. Thank you for the careful reading.
Response to Reviewer RHcp
Thank you for the review. Your questions ask what R adds beyond KL and answer disagreement. We answer them with concrete decisions.
W1. Added value over KL and answer disagreement. R tells which change can reduce the deviation, which the total deviation cannot. At 8 bits, methods that reduce the reconstruction error lower R by at most 12% and router calibration by at most 8%, while correcting the router weights of the AWQ checkpoints removes 83% and 90% of R_gate. KL ranked the Qwen3 AWQ checkpoint last of four 4-bit checkpoints without telling why. R_gate showed that 72% of its deviation passes through the router, and after the correction its KL is below that of the official Int4 release (0.049 against 0.063).
Q1. Higher accuracy but larger R. We would choose by accuracy and efficiency, and use R to see which change can reduce the remaining deviation. Section 5 now states that accuracy remains the criterion. The nearest case in our data is the AutoRound checkpoint. It has 1.5 times the R of the official Int4 release with the same benchmark accuracy (+0.24 pp [−0.44, +0.91]). Its larger R comes with quantized router weights, and restoring them removes 60% of R_gate and 34% of KL at no cost, with accuracy unchanged. The opposite case also occurs. The Qwen3 AWQ checkpoint matched the official Int4 release on MMLU-Pro but had 5 times its R_gate, and the correction recovered 2.35 pp on five other benchmarks.
Q2. What the different scaling adds. It tells how much one more bit reduces the deviation. One more bit reduces it by a factor of 3.1 to 5.4 when m < 0.2 but only by 1.4 to 2.0 when m > 0.5. On held-out models the per-bit factor estimated from m has a median log2 error of 0.21 (ε² rule 0.63), and it picks the bit width for a target deviation within 0.13 bits (Section 4.2, Appendix M.2). For comparing methods, please see W1.
Q3. Router or other modules first. R_gate answers this before any change, because a change to the router alone leaves the frozen run that fixes the whole router output unchanged. The official Int4 release and the AutoRound checkpoint have similar KL (0.063 and 0.055), but 28% and 56% of their deviation passes through the router. Restoring the router weights of the AutoRound checkpoint removes 34% of its KL, while router calibration of the official Int4 release removes nothing (−16% of R_gate), so the router comes first only for the AutoRound checkpoint. Over five changes to the router, the deviation removed followed the order of R_gate (Spearman 1.0), not that of R or f (0.7). As for Section 4.4 of the submission, its +0.65 pp came from answers parsed from generated text. With option-letter scoring it is +0.85 pp [−0.70, +2.30], and on five benchmarks +0.15 pp, a gain in fidelity but not in accuracy.
We are glad to discuss any remaining points.