acceptodds
Under review as a conference paper at ICLR 2027

The Blind Spot of Post-Training Quantization in Mixture-of-Experts Models

Abstract

In mixture-of-experts (MoE) models, quantization changes which experts each token is routed to. Post-training quantization (PTQ) methods minimize the reconstruction error of each layer, and prior work evaluates the quantized models with benchmark accuracy or the expert-selection change rate. However, we argue that none of these captures how much deviation the routing changes cause. In this paper, we propose a metric for this routing-induced deviation, which compares the quantized model's deviation from the original model with and without fixing its expert selection to the original one. We further derive a law for this deviation, which shows that the layer-wise reconstruction error misses a growing fraction of the deviation as the bit width increases. Lastly, we apply our metric to a released quantized checkpoint and show that it captures a change that accuracy cannot.

Primary area:
foundation or frontier models, including LLMs
Keywords:
mixture-of-experts, post-training quantization, expert routing, model compression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RHcpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the output deviations caused by changes in expert routing during post-training quantization of Mixture-of-Experts models. The authors argue that existing metrics cannot fully measure the output deviations caused by routing changes. To address this issue, the paper proposes a routing deviation metric based on two inference modes: Free Run and Frozen Run. Experiments on multiple MoE models evaluate the effectiveness of this metric and analyze how errors change under different quantization bit widths.

Critical strengths

This paper focuses on changes in expert routing caused by weight errors during post-training quantization of MoE models, which is a meaningful research problem. The authors conduct systematic experiments to analyze routing deviations under different quantization bit widths and provide theoretical analysis to explain how different errors change.

Critical weaknesses

The proposed routing deviation metric (R) can distinguish output deviations caused by changes in expert routing from other quantization errors, which has some research value. However, the paper does not fully explain what meaningful new findings this more detailed error analysis can provide or how it can guide the evaluation and selection of quantization methods. In particular, since existing metrics such as KL Divergence and answer disagreement rate can already detect changes in model outputs, the additional value of R has not been clearly demonstrated. Therefore, the overall contribution and practical significance of this paper are not sufficiently convincing.

Questions for authors
  1. The goal of model quantization is to improve storage and inference efficiency while maintaining accuracy as much as possible. If two quantization methods have similar efficiency and storage costs, but one has lower Accuracy and a larger R, it would generally not be preferred. Conversely, if one method achieves higher Accuracy or better inference efficiency but also has a larger R, how should we choose between them? In other words, beyond measuring routing deviations, what additional value can R provide for practical quantization decisions?
  2. The paper finds that R and D_frozen follow different scaling patterns at higher bit widths. However, as the bit width increases, the absolute values of both deviations decrease. R simply decreases more slowly, so its relative contribution to the total deviation increases. Beyond showing that reconstruction error cannot fully capture routing deviations, what new insights does this finding provide? For example, can it help guide the selection of quantization bit widths or the comparison of different quantization strategies?
  3. In Section 4.4, restoring the Router weights reduces R by approximately 68%, but Accuracy improves by only 0.65 percentage points, with a very small gain. Meanwhile, KL Divergence and answer disagreement rate can also detect changes in model outputs. Although R can further distinguish the contribution of routing changes, what is the practical value of this additional information? For example, when different quantization methods have similar overall output deviations, can R help determine whether the Router or other modules should be optimized first, rather than simply explaining the observed experimental results?
3 comments
AuthorsOpenReview ↗

Response to Reviewer RHcp

Thank you for the review. Your questions ask what R adds beyond KL and answer disagreement. We answer them with concrete decisions.

W1. Added value over KL and answer disagreement. R tells which change can reduce the deviation, which the total deviation cannot. At 8 bits, methods that reduce the reconstruction error lower R by at most 12% and router calibration by at most 8%, while correcting the router weights of the AWQ checkpoints removes 83% and 90% of R_gate. KL ranked the Qwen3 AWQ checkpoint last of four 4-bit checkpoints without telling why. R_gate showed that 72% of its deviation passes through the router, and after the correction its KL is below that of the official Int4 release (0.049 against 0.063).

Q1. Higher accuracy but larger R. We would choose by accuracy and efficiency, and use R to see which change can reduce the remaining deviation. Section 5 now states that accuracy remains the criterion. The nearest case in our data is the AutoRound checkpoint. It has 1.5 times the R of the official Int4 release with the same benchmark accuracy (+0.24 pp [−0.44, +0.91]). Its larger R comes with quantized router weights, and restoring them removes 60% of R_gate and 34% of KL at no cost, with accuracy unchanged. The opposite case also occurs. The Qwen3 AWQ checkpoint matched the official Int4 release on MMLU-Pro but had 5 times its R_gate, and the correction recovered 2.35 pp on five other benchmarks.

Q2. What the different scaling adds. It tells how much one more bit reduces the deviation. One more bit reduces it by a factor of 3.1 to 5.4 when m < 0.2 but only by 1.4 to 2.0 when m > 0.5. On held-out models the per-bit factor estimated from m has a median log2 error of 0.21 (ε² rule 0.63), and it picks the bit width for a target deviation within 0.13 bits (Section 4.2, Appendix M.2). For comparing methods, please see W1.

Q3. Router or other modules first. R_gate answers this before any change, because a change to the router alone leaves the frozen run that fixes the whole router output unchanged. The official Int4 release and the AutoRound checkpoint have similar KL (0.063 and 0.055), but 28% and 56% of their deviation passes through the router. Restoring the router weights of the AutoRound checkpoint removes 34% of its KL, while router calibration of the official Int4 release removes nothing (−16% of R_gate), so the router comes first only for the AutoRound checkpoint. Over five changes to the router, the deviation removed followed the order of R_gate (Spearman 1.0), not that of R or f (0.7). As for Section 4.4 of the submission, its +0.65 pp came from answers parsed from generated text. With option-letter scoring it is +0.85 pp [−0.70, +2.30], and on five benchmarks +0.15 pp, a gain in fidelity but not in accuracy.

We are glad to discuss any remaining points.

I thank the authors for their detailed rebuttal, which has partially addressed my concerns. However, after considering the other reviewers’ comments, I remain concerned about the novelty of the work. Therefore, I have decided to maintain my original recommendation.

AuthorsOpenReview ↗

Reply to Reviewer RHcp

Thank you for reading the rebuttal and for the reply. We are glad that it addressed part of your concerns. Your questions on what R adds beyond KL and answer disagreement, and on how to choose when accuracy and R disagree, led us to compare more released checkpoints and to state in Section 5 that accuracy remains the criterion. We will keep your remaining concern about novelty in mind as we continue this work. Thank you for the careful reading.

Reviewer_JfR8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates how post-training quantization affects expert routing in mixture-of-experts models, arguing that conventional reconstruction errors and evaluation metrics do not adequately capture the resulting deviations. The authors propose a metric based on comparing quantized model outputs with and without restoring the original expert selections, derive a scaling law for the routing-induced deviation, and evaluate their findings across several MoE architectures and quantization settings.

Critical strengths
  • The paper addresses a relevant problem concerning the reliability of post-training quantization in MoE models, particularly the effects of quantization on expert routing.
  • The proposed metric is simple and intuitive, providing a seemingly promising way to measure how much of the output deviation is associated with changes in expert selection.
  • The experimental evaluation is relatively extensive and convincing.
  • The scaling analysis provides an interesting explanation for the different behavior of reconstruction error and routing-induced deviation. I also appreciate the experiments on an existing released quantized checkpoint.
Critical weaknesses
  • I think the novelty of the proposed approach is relatively limited. The fact that quantization changes expert routing, and that these changes can affect the model outputs, is not new. Specifically, prior work investigates restoring the original routing decisions to measure their effects, e.g. RouteQuant (Fang & Huang, 2026), SRA-MoE (Lee et al., 2026), and Gu (2026). The authors acknowledge these works in Appendix E.5, but I am not convinced that changing the precise intervention and evaluating it across more bit widths constitutes a substantial enough contribution. The observation that benchmark accuracy can hide differences between model outputs is also well established, as discussed by Dutta et al. (NeurIPS 2024).

  • I am not convinced that the proposed metric provides a sufficiently useful advantage over existing measurements. In Section 4.3, the authors themselves show that, at high bit widths, changes in the expert-selection change rate explain 97-100% of the changes in routing-induced deviation within each model. Furthermore, Appendix K.2 investigates quantization variants with similar reconstruction errors, but even when deliberately choosing roundings to increase or decrease the proposed metric, the difference is at most 1.9%. If the existing measurements already track the routing-induced deviation so closely in these settings, how much additional information does the new metric provide? I would have liked to see a more convincing example where it leads to substantially different conclusions or better quantization decisions.

  • The theoretical contribution is also limited. The main scaling result essentially follows from the observation that small perturbations change the routing decisions of a fraction of tokens proportional to the perturbation magnitude, while the squared error under fixed routing decreases quadratically. I find this observation useful, but relatively intuitive given the discontinuity of top-k selection. Also, the derivation is primarily established for individual quantized routers with fixed inputs and layer-local squared errors. It is unclear how broadly the same law applies to full-model quantization and output-level metrics.

  • The experiments rely heavily on simplified quantization settings. Most of the scaling analysis quantizes only the router weights of one layer at a time, while the model-output experiments leave attention layers, shared experts, and other components in full precision. The authors provide some additional experiments involving attention quantization in Appendix K.1, which is useful, but these also show that the measured blind fraction changes when additional components are quantized. Moreover, most experiments use RTN, with GPTQ evaluated in the router-only setting. I think more experiments with realistic full-model quantization methods would be needed to establish the generality of the findings.

  • The practical implications of the findings are still unclear to me. The released-checkpoint experiment shows that restoring a small fraction of the model weights substantially improves agreement with the original model, but produces no statistically significant improvement in accuracy. This supports the proposed metric as a measure of output fidelity, but does not show that it is useful for preserving downstream model performance. The paper also does not demonstrate how the metric could be used to improve existing quantization methods. Given that the work is primarily diagnostic, I think a more convincing demonstration of its practical usefulness would substantially strengthen the contribution.

Questions for authors
  1. I would like the authors to clarify the novelty of the proposed metric relative to existing routing-restoration experiments, particularly RouteQuant, SRA-MoE, and Gu (2026). What do you consider the main methodological contribution beyond these approaches?

  2. Can the authors provide more convincing evidence that the proposed metric offers a practical advantage over existing reconstruction-error and routing-change measurements? In particular, I would be interested in an example where it meaningfully improves quantization decisions or predicts downstream performance differences that the existing metrics miss.

3 comments
AuthorsOpenReview ↗

Response to Reviewer JfR8

Thank you for the review. We take your two main concerns first, novelty and practical advantage.

W1 and Q1. Novelty relative to RouteQuant, SRA-MoE and Gu (2026). The intervention is not new, and Section 2.3 and Table 1 now credit these studies in the main text. The contribution is what the measurement shows. First, the blind fraction m increases with the bit width, and at 8 bits, where it exceeds 0.5, GPTAQ and Qronos lower R by only 12% and 10% relative to GPTQ. Second, earlier single numbers need not conflict. In one OLMoE setting the share attributed to routing is 0.02 or 0.12 with the true-token loss (indices only, or indices and gate values as in Gu, who reports 0.115 to 0.123) and 0.11 or 0.20 with the output KL (Appendix M.4). Third, R_gate tells how much a change to the router can remove before the change is made, which led to the correction (general response C1). Dutta et al. showed that accuracy masks changed answers. We add how much of that deviation is routing-induced and what reduces it.

W2. Advantage over the change rate and Appendix K.2. Within one model at high bit widths f tracks R closely, as you note. Across models it does not. At 8 bits OLMoE, ERNIE and Moonlight have the same f (0.046 to 0.047) but their R differs by a factor of 2.7, and the deviation per selection change (R/f) differs by a factor of about 7 across models and of up to 1.9 among released 4-bit checkpoints of one model. The relation of f to the reduction from one more bit changes sign between comparisons within a model (+1.0) and across models (−0.69), so a regression on f errs more on held-out models (0.30 against 0.21 for m) and on held-out methods (0.66 against 0.11). In the AWQ checkpoints f was the highest (0.79) but did not tell how much deviation these changes produce. R_gate did. Appendix K.2 is a control showing that R does not depend on one particular rounding.

W3. Theory and full-model quantization. We agree that the mechanism is intuitive. Its consequence, that the routing-induced deviation decreases slowly and becomes most of the deviation at high bit widths, is what the reconstruction error misses, and it holds at the model output with all attention and expert weights quantized (α_R 0.67 to 0.92, α_frozen 2.00 to 2.11, Section 4.2). The sum of the two power laws fitted on 3 to 5 bits gives 0.80 to 1.03 times the measured 8-bit deviation, against 0.38 to 0.49 times for a single power law (Appendix M.2).

W4. Simplified settings. We added GPTQ, GPTAQ, Qronos and AutoRound on all attention and expert weights and GPTQ on OLMoE (Table 2), and measured seven released checkpoints as shipped (Appendix L.1). Quantizing more modules changes the value of m, as you note, but wherever the bit width varies m still increases with it and exceeds 0.5 at 8 bits.

W5 and Q2. Practical implications. The submission's checkpoint experiment showed fidelity and not accuracy (+0.15 pp on five benchmarks), and we now say so. The AWQ checkpoints give the example you ask for. MMLU-Pro and the reconstruction error of the router weights show no problem with the released Qwen3 AWQ checkpoint (45.0% against 44.5% for the official Int4 release, router weights identical to those of the original model). Yet R_gate shows that 72% of its deviation passes through the router, and the checkpoint is 2.72 pp below the official Int4 release on five other benchmarks. The correction closes this gap (−0.37 pp [−1.08, +0.31]) and leaves the checkpoint 1.99 pp [1.24, 2.74] above the unquantized model with only the wrong input scale.

We are happy to clarify anything further.

I appreciate the additional experiments - these additions address several of my original concerns regarding experimental generality and practical relevance. I still have reservations about the novelty and significance of the contribution. The underlying routing-restoration approach has been explored in prior work, the scaling behavior follows from relatively intuitive properties of top-k selection, and the AWQ correction demonstrated in the new experiments is also well-known. While the revised paper provides a more systematic characterization of these phenomena, I am not fully convinced that the resulting insights constitute a sufficiently substantial contribution for ICLR. I therefore appreciate the authors' efforts and the improvements made, but will maintain my original score.

AuthorsOpenReview ↗

Reply to Reviewer JfR8

Thank you for reading the revision and for the reply. We are glad that the additions address several of your concerns about experimental generality and practical relevance.

You are right that the AWQ correction is known, and our response should have said so. The manuscript states that the problem is known in one quantization toolkit (llm-compressor PR #2451, Section 4.4) and that we "do not claim to have found the problem" (Appendix L.2). We include the case for what the measurements show. The problem remains in four of the five released AWQ MoE checkpoints we examined. In the two we measured, the router weights equal those of the original model, so their reconstruction error is zero, and the Qwen3 checkpoint matches the official Int4 release on MMLU-Pro (45.0% against 44.5%). R_gate, measured on the checkpoints as shipped, shows that 72% and 83% of their deviation passes through the router.

We also agree that the intervention is prior work and that the mechanism is intuitive. Our claim is the measured consequence. At 8 bits the routing-induced deviation is more than half of the deviation in every setting of Table 2 (m from 0.53 to 0.91), and GPTAQ and Qronos lower R by only 12% and 10% compared with GPTQ.

We understand that significance is a matter of judgment, and we thank you for the careful reading. If a specific analysis would help, we would be glad to add it before the discussion closes.

Reviewer_AeRwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose a metric for measuring routing-induced deviation between quantized and floating-point mixture-of-experts models. The metric compares the quantized model’s deviation with and without forcing its expert selections to match the original model. Across seven models, the authors find that routing changes can account for substantial deviation even when benchmark accuracy changes little. They demonstrate the metric on a released checkpoint.

Critical strengths
  • The paper focuses on a relevant topic, namely evaluating the impact of quantization on MoE models
  • The metric measures the effect of routing changes, which expert-selection counts alone cannot show.
  • The experiments cover seven MoE architectures and a broad range of bit widths.
Critical weaknesses
  • The practical significance remains unclear. The experiments establish changes in fidelity to the original model, but do not establish when (or even if) these changes matter for task performance or inference cost. A different expert selection is not necessarily worse if it produces an equally good or better answer.
  • The scope is narrower than the general PTQ framing suggests. Most experiments use RTN, and the GPTQ comparison quantizes individual routers. It remains unclear whether the conclusions extend to gradient-based PTQ, learned rotations, QAT, or quantization-aware distillation, where useful adaptation may change the original routing.
  • Building from the above, the experiments do not establish how the findings change with methods that directly account for accumulated activation error, such as Qronos [1] or GPTAQ [2]. This limits the generality of the conclusions about PTQ.
  • The paper should discuss Expected Acceptance Rate (EAR) [3], which similarly separates distribution fidelity from task accuracy. How would applying EAR to the router relate to the proposed metric? Could EAR serve as the underlying distance measure in the free-versus-frozen comparison? Please clarify what additional insight the proposed measurement provides beyond existing metrics.
  • The paper does not demonstrate how the metric can guide a useful quantization decision beyond retaining higher-precision routers. The official Qwen checkpoint already retains them, and the paper acknowledges that using the metric during training remains untested.
  • The computational requirements are not quantified. Runtime and memory measurements are needed to assess its usefulness during calibration or training.
  • The abstract promises a “law,” but its purpose and practical significance remain unclear. Equation (6) proposes a model for how the routing-induced fraction changes with bit width, but the paper does not establish what useful prediction or quantization decision this relation enables.
  • Differences among toolkit defaults and uploaded checkpoints establish variation in implementation practice (e.g., lines 049-52). They do not establish whether these differences reflect measured performance differences.
  • The presentation makes the paper difficult to follow. As two of many examples, plot fonts are too small and key distinctions from prior routing-restoration work are left to the appendix.

[1] Zhang, Shihao, et al. "Qronos: Correcting the past by shaping the future... in post-training quantization." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Li, Yuhang, et al. "GPTAQ: Efficient finetuning-free quantization for asymmetric calibration." arXiv preprint arXiv:2504.02692 (2025).

[3] Helcig, Michael, Eldar Kurtic, and Dan Alistarh. "Statistically-Lossless Quantization of Large Language Models." arXiv preprint arXiv:2605.02404 (2026).

Questions for authors
  • How can a practitioner use RR to improve a quantized model? Can the authors demonstrate a decision or optimization that improves task performance or efficiency, and explain its advantage over output distillation or quantization-aware distillation?
  • How should RR be interpreted for methods that adapt the model through optimization? Could encouraging the original routing prevent a quantized model from finding a different expert combination that produces equally good or better answers?
3 comments
AuthorsOpenReview ↗

Response to Reviewer AeRw

Thank you for the review. We ran the experiments you asked for, and general response C1 answers your main concern, practical significance.

W1. Does the deviation matter for task performance or inference cost? Yes in the two AWQ checkpoints. Correcting their router weights raises benchmark accuracy by 2.35 and 2.70 pp and the tokens per verification step in speculative decoding (Table 3). It does not in the community Int4 and AutoRound checkpoints, where restoring the router weights changes benchmark accuracy by only +0.15 and −0.01 pp. A different expert selection is not necessarily worse, and R does not assume it is (Q2 below).

Q1. How a practitioner uses it, and the advantage over distillation. R_gate tells how much a change to the router can remove before the change is made. Over five such changes, the deviation removed followed the order of R_gate (Spearman 1.0) but not that of R or f (0.7). In the AWQ checkpoints it pointed to the correction, which we compared with distilling the router on the output KL of the original model (DeepSeek, Appendix L.2). Distillation needs 112 calibration sequences and 553 seconds on two GPUs, removes 73% of the KL that the correction removes, and ends 1.41 pp [0.82, 2.03] below the correction on five benchmarks. Started from the corrected router weights, it only overfits. Light quantization-aware distillation (QAD) mostly reduced D_frozen (94 to 99% of its gain) and left R positive (Appendix M.1).

W2 and W3. Gradient-based PTQ, rotations, QAT, QAD, Qronos and GPTAQ. All are in general response C2 and Appendix M.1. The QAT and QAD runs are light, on one model. With AutoRound and with a Hadamard or learned rotation, m still increases with the bit width. Qronos and GPTAQ lower R by 21 to 33% and 13 to 19% at 3 to 6 bits, but only by 10% and 12% at 8 bits.

W4. EAR. Yes, 1 − EAR (total variation, TV) can serve as the distance D. With TV, R_TV,gate separates the checkpoints as R_gate does with KL (DeepSeek 0.143 before the correction and 0.018 after, Qwen3 0.056 and 0.012). The smaller index-only R_TV needs fp32. In fp16 (Qwen3) it stays within twice the floor that batch composition alone produces (0.0078). In fp32 (DeepSeek) that floor is 0.00003, and R_TV (0.0095 to 0.0102) is more than 300 times it. Applied to the router, EAR ranks checkpoints exactly as the change rate does (Spearman −1.0), so it counts changes in expert selection, while R measures how much these changes alter the output (Section 2.3, Appendix M.5).

W5. A decision beyond keeping routers in full precision. The AWQ checkpoints already kept the router weights in full precision. The router input was at the wrong scale, and only R_gate (0.72, against m = 0.17) captured it. Otherwise, router calibration lowers R by at most 8% at 8 bits, so bits or other modules come first.

W6. Cost. Each frozen run is one extra forward pass and takes 1.012 and 1.014 times as long as the free run, with the same peak memory (Appendix M.6).

W7. The law. Please see general response C4.

W8. Toolkit differences. Section 1 now gives measurements. Restoring the router weights of the community Int4 checkpoint (lines 049 to 052) lowers KL by 41% with benchmark accuracy unchanged, while the wrong input scale of the AWQ checkpoints costs accuracy. It alone accounts for 2.84 of the 3.21 pp lost by the released Qwen3 checkpoint.

W9. Presentation. The comparison with prior routing-restoration work is now in Section 2.3 (Table 1). All figures are redrawn at print size, with 8 to 10 pt text.

Q2. Could the original routing block a better combination? Yes. OLMoE trained at 4 bits on the next-token loss moves its expert selection away from that of the original model (f from 0.388 to 0.551). Its own selection gives a lower WikiText-2 loss (R_NLL = −0.003 [−0.005, −0.001]) but a higher loss on held-out C4 (R_NLL = +0.0022 [+0.0012, +0.0031]). A KL-based R counts such a change as a deviation, so the revision also reports the loss-based R_NLL (Section 3.1). R is a diagnostic and not a training target.

If the revision addresses your concerns, we would appreciate it if you could take it into account in your assessment.

Thank you for the additional results, particularly with additional quantization methods used for recovering model accuracy.

The practical significance of the metric remains unclear as the effect is not consistent or predictable, as evidenced by the AWQ vs. AutoRound checkpoint comparison. Furthermore, I agree with the point from Reviewer RHcp, who points out that in the event downstream accuracy disagrees with the proposed metric, a practitioner is inclined to optimize for accuracy. This point aligns with my second question in my initial review. Thus, it seems the metric is neither consistently predictive nor actionable.

While I appreciate the authors' efforts in addressing the concerns, I remain unconvinced that the proposed metric and the resulting insights represent a sufficiently substantial contribution to warrant a change in my original score.

AuthorsOpenReview ↗

Reply to Reviewer AeRw

Thank you for reading the revision and for the reply. Your requests for Qronos, GPTAQ and the other quantization methods, for the comparison with EAR, and for the cost measurements made the revision more complete. Your remaining point, that the metric should be consistently predictive and actionable, is the one we will work on next. Thank you for the time you spent on our paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

Dear Area Chair and Reviewers,

Thank you for the reviews. We answer the four shared concerns below. The revised PDF marks all new content in yellow.

C1. Does the metric support a decision that matters for accuracy? (AeRw W1, W5, Q1. JfR8 W5, Q2. RHcp W1, Q1, Q3)

Yes. Two released AWQ checkpoints (Qwen3-30B-A3B and DeepSeek-V2-Lite-Chat) keep the router weights in full precision, but AWQ rescales the router's input without rescaling the router weights, so the router receives its input at the wrong scale. The router weights equal those of the original model, so their reconstruction error is zero, and index-only R captures little of the problem (m = 0.17). Our new quantity R_gate (Eq. 5), which fixes the gate values as well, captures it. It shows that 72% and 83% of the deviation passes through the router. Multiplying the router weights by the AWQ scales, with no data or training, raises benchmark accuracy from 69.57% to 71.92% in Qwen3 (+2.35 pp [1.63, 3.02]) and from 64.84% to 67.53% in DeepSeek (+2.70 pp [2.09, 3.30]), and DeepSeek GSM8K from 22.1% to 35.8% (Section 4.4, Table 3). The corrected Qwen3 checkpoint matches the official Int4 release (−0.37 pp [−1.08, +0.31]). Before the correction it was 2.72 pp below the official Int4 release on these benchmarks while matching it on MMLU-Pro. Distilling the router instead takes 553 seconds on two GPUs and ends 1.41 pp [0.82, 2.03] below the correction on five benchmarks (DeepSeek, Appendix L.2).

We also report a case without an accuracy gain. Restoring the router weights of the community Int4 and AutoRound checkpoints lowers their KL by 41% and 34% but changes benchmark accuracy by only +0.15 and −0.01 pp, both within noise. Accuracy remains the criterion for decisions about task performance. R and R_gate tell which change can reduce the remaining deviation.

C2. Do the findings hold beyond RTN? (AeRw W2, W3. JfR8 W3, W4)

Yes. We now quantize all attention and expert weights with GPTQ, GPTAQ, Qronos and AutoRound (Qwen1.5-MoE, 3 to 8 bits) and with GPTQ (OLMoE). m still increases with the bit width and lies between 0.53 and 0.91 at 8 bits in every setting of Table 2. At 8 bits GPTAQ and Qronos lower R by only 12% and 10% compared with GPTQ, router calibration lowers it by at most 8%, and light quantization-aware distillation (QAD) leaves R positive (Appendix M.1).

C3. What is new relative to prior routing-restoration work? (JfR8 W1, Q1. AeRw W9)

The intervention is not new, and Section 2.3 and Table 1 now credit RouteQuant, SRA-MoE, Gu (2026) and EAC-MoE. The measurement adds three results. The blind fraction m increases with the bit width and exceeds 0.5 at 8 bits (Eq. 6, Table 2). Earlier numbers, from 2.7% to "the principal source", differ because they fix different things and use different metrics (0.02 to 0.20 in one OLMoE setting, Appendix M.4). And R_gate tells how much a change to the router can remove before the change is made.

C4. What is the law for? (AeRw W7. RHcp Q2)

It tells how much one more bit reduces the deviation. On six held-out models, the per-bit factor estimated from m has a median log2 error of 0.21, against 0.30 for a regression on the change rate and 0.63 for the ε² rule. Fitted on RTN alone, it errs by 0.11 on GPTQ, GPTAQ and Qronos and picks the bit width for a target deviation within 0.13 bits (Section 4.2, Appendix M.2).

A correction. The +0.65 pp gain in the submission's released-checkpoint experiment came from answers parsed from generated text, and a quarter to a third of those answers could not be parsed. With option-letter scoring the gain is +0.85 pp [−0.70, +2.30], and on five benchmarks it is +0.15 pp [−0.52, +0.81]. We no longer use that experiment as evidence of an accuracy change.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.