acceptodds
Under review as a conference paper at ICLR 2027

Owen-Value-Guided Mixed-Precision Quantization for Mixture-of-Experts Models

Abstract

Mixture-of-experts (MoE) models activate only a few experts per token, yet low-latency inference can require the entire expert pool in memory. At ultra-low average precision, mixed-precision quantization must decide which experts retain scarce higher-bit slots. Individual sensitivity scores can miss the joint effects of co-routed experts. We propose Owen-MoE, which values experts in an availability game while accounting for an output-similarity partition. Two-level Monte Carlo sampling estimates Owen values in O(TK) evaluations per layer for T trials over K experts. A scale-invariant Influence Ratio converts these values into a static 2,4-bit map that can protect locally influential experts within lower-value coalitions. We compare rankings under identical per-layer 4-bit counts and AutoRound settings. Across eight zero-shot benchmarks on Qwen3-30B-A3B, three-seed means at 2.75–2.125 average expert bits, Owen-MoE exceeds Router-norm by 3.57–7.02 points and PMQ by 4.86–10.56 points on average. It also exceeds measured fractional Uniform by 2.78–13.31 points at matched expert-weight payload. Equal-budget controls show an additional gain over flat Shapley of 1.78 points at 2.50 bits and 2.58 points at 2.25 bits. In 96 selected high-coactivation expert pairs where the rankings disagree, Owen's choice reduces round-to-nearest quantization MSE in 78.1% of cases, with a median reduction of 17.6%. Mixtral and Qwen1.5-MoE extend the evaluation to different expert counts and routing configurations. We will release the complete code upon acceptance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, mixed-precision quantization, model compression, post-training quantization, Owen value, cooperative game theory, expert importance

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_oRArOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Owen-MoE to decide which experts should retain higher precision when quantizing MoE models. It groups experts with similar outputs, estimates their contributions using Owen values, and assigns them either 2-bit or 4-bit precision. It also protects experts that make substantial contributions within otherwise low-value coalitions. The authors evaluate the method on several MoE model checkpoints.

Critical strengths

This paper considers how experts work together when deciding which ones should retain higher precision. The comparison with flat Shapley attribution is particularly useful: Owen-MoE achieves higher accuracy with the same sampling and memory budgets, providing evidence that the coalition structure adds value.

Critical weaknesses
  1. The paper estimates expert contributions through masking in an expert availability game, but this does not directly measure the loss caused by quantizing an expert’s weights. An expert may be robust to low-bit quantization yet highly sensitive to being masked. The paper does not sufficiently establish how well availability-game contributions reflect quantization sensitivity.
  2. Comparisons with recent work are limited. Moreover, the matched-budget ranking comparisons require competing methods to use Owen-MoE’s per-layer 4-bit expert counts rather than their original bit allocators. These comparisons therefore do not establish whether Owen-MoE outperforms the complete methods at the same total expert-weight payload.
Questions for authors

Q1: Relationship between availability-game contributions and quantization sensitivity. Beyond the selected expert-pair interventions, is there broader evidence that contributions in the expert availability game reflect actual quantization sensitivity? Are there experts with large availability-game contributions that benefit little from retaining 4-bit rather than 2-bit precision? How common are such cases? Q2: Practical inference efficiency. The paper reports approximately 14 tokens/s on Jetson. Are there throughput and inference latency comparisons against uniform quantization or other mixed-precision methods on the same hardware under the same workload? Do these results show an actual speedup from the proposed bit allocation? Q3: Effects of calibration data and feature choice. Coalition construction and contribution estimates may depend on the calibration data distribution. Are accuracy results available using different calibration datasets? For coalition construction, have you compared expert input features, expert output features, and their combination? Feature choice and calibration data source are separate questions here. Q4: Comparisons with related methods. The MoPEQ comparisons are limited to the multimodal experiments in the appendix. Are further results available at matched expert-weight payloads? Could you explain why MODE was not included in the multimodal comparisons? For PreMoE, have you considered using its PEU scores in a ranking comparison with matched per-layer 4-bit expert counts to assess the additional benefit of Owen values? Q5: Performance on larger models. Are results available on larger MoE models, including accuracy gains and offline calibration costs? Do the observed advantages hold as model size increases?

7 comments
AuthorsOpenReview ↗

Response to Reviewer oRAr Part1

Q1. How often does a high availability score coincide with a small precision benefit?

Among 32 sampled high-score Qwen3 experts, 13 (40.6%) reduced layer reconstruction MSE by at most 1% when individually upgraded from two to four bits. Your question prompted us to test individual experts, which lets us estimate how often this case occurs beyond the selected disagreement pairs. Masking an expert allows the router to substitute another expert, whereas increasing its precision preserves its availability.

Across eight sampled layers, seven within-layer Spearman correlations between Owen score and individual upgrade gain were positive, with a median of 0.29. Median MSE reductions were 0.38%, 0.71% and 1.24% in the low-, middle- and high-score groups. Higher scores therefore were associated with larger median gains across these strata, with substantial variation among high-score experts. This complements the submitted intervention, in which the Owen-preferred upgrade won 75 of 96 selected expert-pair comparisons on all-token MSE and 80 of 96 on coactivated tokens.

Q2. Does the Jetson measurement establish a speedup?

Your question highlights the distinction between deployment feasibility and inference efficiency. DeepSeek-VL2-Tiny decoded at approximately 14 tokens/s on an 8 GB Jetson Orin Nano, where the 16-bit configuration exceeded available memory. The precision map is computed offline, and inference performance depends on the routing workload and low-bit kernels.

The rebuttal period was too short to complete further matched-workload measurements on Jetson. We regard these comparisons as an important direction for future work and will evaluate decode throughput and prefill latency against uniform and competing mixed-precision allocations on the same device under the same workload.

Q3. How sensitive is accuracy to the calibration corpus and the clustering features?

Following your suggestion, we varied the calibration corpus and the clustering features separately. Holding output features fixed, changing the corpus shifted eight-task accuracy by +0.96 and −0.19 points at 2.50 and 2.25 bits, respectively. We then compared the three feature choices on WikiText. All rows use Qwen3 top-quota selection and report eight-task mean accuracy (%).

Calibration corpus and features2.50 bits2.25 bits
WikiText, output66.8365.36
Second calibration corpus, output67.7965.17
WikiText, input67.6565.32
WikiText, normalized input and output67.1065.10

On WikiText, input features give the highest score at 2.50 bits (+0.82 points over output). At 2.25 bits, input and output differ by just 0.04 points, with output slightly ahead. The feature ordering thus changes across the two budgets.

AuthorsOpenReview ↗

Response to Reviewer oRAr Part2

Q4. What changes with cross-layer allocation, and how do MoPEQ, MODE and PreMoE compare?

Cross-layer allocation. Giving Router-norm+MaxVar freedom to allocate precision across Qwen3 layers raises its accuracy by 2.69 and 3.42 points over inheriting Owen's layer quotas. Owen full retains a lead of 2.77 and 3.54 points at the corresponding budget targets. The first row gives the target expert-weight payload in bytes. The remaining rows give eight-task accuracy (%).

Budget or allocation2.50 bits2.25 bits
Target expert-weight bytes9,059,696,6408,153,726,976
Router-norm+MaxVar, inherited quotas62.1158.98
Router-norm+MaxVar, cross-layer allocation64.8062.40
Owen full, submitted67.5765.94

MoPEQ at a common reported model size. Following your suggestion, we re-evaluated Owen on DeepSeek-VL2-Tiny at the 2.367 GB model size reported for the MoPEQ Hessian-sensitivity (a) configuration. The matching quantity here is total model size as reported to three decimal places. The table gives each difference on the original benchmark scale.

TaskOwenMoPEQOwen minus MoPEQ
AI2D83.31283.970−0.658
DocVQA87.36387.334+0.029
InfoVQA60.40361.645−1.242
MME-Reasoning380.522371.786+8.736
MME-Perception1547.0481561.112−14.064
MMMU36.65638.556−1.900
RealWorldQA64.67963.791+0.888
ScienceQA87.55087.172+0.378
BLINK41.68442.609−0.925

Owen scores higher on four of the nine tasks. For example, Owen gains 8.736 points on MME-Reasoning, while MoPEQ leads by 14.064 points on MME-Perception. DocVQA differs by just 0.029 points at this operating point.

Choice of multimodal baseline. Our main study concerns text-only MoE LLMs. For the multimodal extension, we chose MoPEQ because expert sensitivity can guide precision allocation using calibration data from either setting. MODE addresses a complementary question by estimating expert importance separately for visual and text tokens and filtering redundant visual tokens. Its visual calibration steps require visual inputs, so a text-only adaptation would omit part of the published method. We agree that MODE is relevant to our DeepSeek-VL2 evaluation. Its separate treatment of visual and text tokens suggests examining whether the same experts should be grouped together for both modalities.

PreMoE. Thank you for suggesting PreMoE's Predicted Expert Utility (PEU). It gave us a way to test how well router traces alone identify experts to protect. PEU fits directly into our allocation procedure because its ranking can fill the same four-bit slots in each layer. At 2.50 and 2.25 bits on Qwen3, PEU gives eight-task accuracies of 65.38% and 63.80%. Using Owen scores for the same slots gives 66.83% and 65.36%, gains of 1.45 and 1.56 points. For a deployment with recurring workloads, PEU could also provide an initial precision map from router traces collected during calibration.

If accepted, we will include a dedicated paragraph in the paper discussing both MODE and PreMoE. We will explain how MODE's modality-specific statistics could inform coalition construction and how PreMoE's workload-specific utility scores can be used within a fixed precision budget.

Q5. Do the gains persist on larger models, and what does calibration cost?

We agree that evaluating a larger checkpoint from the same model family would provide a more convincing test of how accuracy and calibration cost change with scale. Our current checkpoints differ in routing width, training and expert design as well as model size. At 2.75 expert bits, the eight-task gain over matched Router-norm is 0.39 points on Mixtral, 0.93 on Qwen1.5-MoE and 3.57 on Qwen3.

Mixtral calibration wall time (seconds) is shown below for both calibration pool sizes. MC denotes Monte Carlo trials.

Calibration samplesMC = 8MC = 16MC = 32MC = 64
1630461012162394
12832162712272463

For Qwen3, counting the empty prefix and all successive prefixes gives 48×16×(128+1)=99,07248\times16\times(128+1)=99{,}072 game evaluations per seed.

The rebuttal period was too short to complete a larger-checkpoint evaluation. In future work, we will compare checkpoints within one family and report accuracy, calibration wall time and peak memory at matched expert-bit budgets.

Q1: Thank you for the additional analysis. It answers my question about high-score experts with small precision benefits, but does not fully resolve my concern. The median correlation of 0.29 and the small upgrade gains for 13 of the 32 sampled high-score experts suggest only a limited correspondence between availability contributions and individual quantization benefits. I understand that the method uses these contributions as a proxy and accounts for expert interactions, so these results do not invalidate the approach. However, I remain unconvinced that the evidence sufficiently supports the proxy as a reliable basis for precision allocation.

Q2: Thank you for the clarification. I agree that the Jetson result demonstrates deployment feasibility rather than a measured speedup. I also note that the DeepSeek-VL2-Tiny and Small results do not show a consistent accuracy advantage over the competing allocations at similar model sizes: Owen-MoE performs better on some tasks but falls behind on several others. Taken together, these results support feasibility, but leave the practical advantage of the proposed allocation on these multimodal models unclear.

Q3: Thank you for testing calibration data and feature choice separately. However, the second calibration corpus is not identified, and only averages are reported. This makes it difficult to judge whether the effects are consistent across tasks or whether gains on some tasks offset losses on others. The WikiText-output scores also differ from the main-table results, and it is unclear whether “top-quota selection” follows the original allocation procedure. These points leave me uncertain about how directly the additional results support the submitted method.

Reviewer_b896OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work uses an output-similarity partition and coalition-aware expert attribution to guide mixed-precision quantization. Expert values are estimated through masked forward passes and converted into a static 2/4-bit allocation with local-core protection. Experiments show strong low-bit accuracy, including gains over flat Shapley attribution at matched budgets.

Critical strengths

Matching per-layer bit budgets and quantization settings makes the ranking comparisons informative. The gains over flat Shapley also support a benefit beyond simply using joint attribution.

The expert-pair interventions provide a useful link between protection choices and actual quantization error. The analysis of failure cases involving small availability prefixes helps explain where the proxy breaks down.

Critical weaknesses

The availability game changes routing and allows expert substitution, whereas quantization retains the expert pool. The interventions partly bridge this gap, but the small-prefix diagnostics suggest that some scores are driven by states quite different from the deployed model.

The Shapley comparison does not show why output-similarity grouping is preferable to other partitions, and Table 28 shows mixed benefits from the Influence Ratio/local-core protection components at a budget above the main sub-3-bit range.

Questions for authors
  1. Does the output-similarity partition outperform random partitions with matched coalition sizes, sampling budgets, and memory? This would help distinguish meaningful grouping from the more general effect of restricting expert orderings.

  2. At the 2.25- and 2.50-bit budgets, what additional benefit does the full Influence Ratio/local-core protection policy provide over simply protecting the highest-ranked Owen experts under identical per-layer quotas?

  3. Section 4.4 links failed protection choices to contributions from prefixes smaller than the routing width. What happens to downstream accuracy when those contributions are excluded? The reported protection-set overlap is helpful, but does not show whether the remaining differences matter.

1 comment
AuthorsOpenReview ↗

Response to Reviewer b896

Thank you for suggesting three controls that distinguish the choice of partition, the protection policy and the treatment of short prefixes. We report eight-task mean accuracy on Qwen3-30B-A3B at average expert precisions of 2.50 and 2.25 bits.

Q1. Does output grouping outperform a size-matched random partition?

Output grouping outperforms size-matched random grouping at both budgets. Eight-task mean accuracy is 66.83% versus 66.50% at 2.50 bits (+0.33 points) and 65.36% versus 64.55% at 2.25 bits (+0.81 points). We randomized expert membership while preserving coalition sizes, the sampling budget and per-layer four-bit counts. Under these controls, the observed benefit of output-based grouping is larger at 2.25 bits.

Q2. What does full IR/local-core protection add over direct Owen ranking?

The submitted full-policy means are 0.74 and 0.58 points above the new Owen top-quota results at 2.50 and 2.25 bits, respectively. With direct ranking, each layer protects its highest-scoring experts up to its quota. The full Influence Ratio/local-core policy can also protect an expert whose contribution is large within its coalition despite a modest layer-wide score.

Protection policy2.50 bits2.25 bits
Full IR/local-core, submitted mean ± SD67.57 ± 0.3165.94 ± 0.29
Direct Owen top-quota66.8365.36
Direct Shapley top-quota65.3663.16

The two direct-ranking rows use identical per-layer counts and quantization settings. Under this rule, changing the attribution from Shapley to Owen improves accuracy by 1.47 and 2.20 points. This comparison isolates the choice of ranking, while the full-policy row gives the submitted reference for the protection rule.

Q3. What happens when contributions from short prefixes are excluded?

Excluding contributions from prefixes with fewer than eight predecessors raises accuracy by 0.76 points at 2.50 bits and lowers it by 0.66 points at 2.25 bits. Both maps use the same top-quota rule and per-layer budget.

Contributions used for ranking2.50 bits2.25 bits
All prefixes66.8365.36
Short prefixes excluded67.5964.70

In the submitted eight-layer diagnostic, short prefixes accounted for a median 57.2% of absolute marginal mass in failed protection choices and 14.9% in successful ones. Your proposed test establishes what this pattern means downstream. Short prefixes can flag vulnerable choices, but removing their contributions from every ranking has opposite effects at the two budgets.

Reviewer_AFniOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Owen-MoE, an expert-wise mixed-precision quantization framework for MoE models. Motivated by the interactions among co-routed experts, the authors employ output-similarity clustering and Owen-value attribution to estimate expert importance, followed by an Influence Ratio policy for 2/4-bit allocation. Experiments on several MoE models demonstrate clear accuracy improvements under ultra-low-bit budgets.

Critical strengths
  1. The paper demonstrates strong empirical performance, particularly on Qwen3-30B-A3B, with substantial accuracy improvements over existing expert-ranking baselines under aggressive quantization budgets.

  2. The experiments are comprehensive, covering multiple MoE architectures, matched-budget comparisons, Shapley-based ablations, and expert-pair quantization interventions.

Critical weaknesses
  1. The methodological and scientific novelty is somewhat limited. Expert interactions and cooperative attribution have been explored in prior work, while the proposed approach primarily adapts existing Owen-value attribution and clustering techniques to mixed-precision quantization.

  2. The proposed coalition structure lacks sufficient justification. This is my primary concern regarding scientific novelty. While Figure 5 demonstrates improvements over flat Shapley, it remains unclear why output-similarity-based grouping is particularly suitable for capturing quantization-relevant expert interactions, compared with alternative coalition structures.

Questions for authors
  1. Figure 5 provides encouraging evidence that Owen attribution outperforms flat Shapley. Could the authors further explain how the higher-order cross-coalition interactions characterized in Proposition 2 contribute to this improvement?

  2. Could the authors compare output-similarity-based coalitions with random and coactivation-based partitions under identical settings, and explain why the proposed grouping better captures quantization sensitivity?

2 comments
AuthorsOpenReview ↗

Response to Reviewer AFni

Thank you for asking what the coalition structure contributes. In Owen-MoE, the partition determines how an expert receives credit for interactions and, through its rank, whether it receives four bits. Your proposed comparisons let us test how much the choice of group membership matters for this decision.

Q1. How can higher-order interactions affect the result in Figure 5?

A higher-order interaction can change which expert crosses the four-bit cutoff. Let dv(S)d_v(S) be the signed Harsanyi dividend of an expert set SS, let C(i)C(i) be the block containing expert ii, and let rSr_S be the number of blocks intersecting SS. Flat Shapley divides the dividend equally among the experts in SS. Owen divides it equally among the participating blocks, then equally among the participating experts within each block. Proposition 2 gives the resulting difference in expert ii's score,

Di=Owi−ϕi=∑S∋idv(S)(1rS,∣S∩C(i)∣−1∣S∣).D_i = \mathrm{Ow}_i - \phi_i = \sum_{S \ni i} d_v(S) \left(\frac{1}{r_S\\,|S \cap C(i)|} - \frac{1}{|S|}\right).

The coefficient vanishes for every one- or two-expert set. For a unit three-expert dividend with blocks 1,2\\{1,2\\} and 3\\{3\\}, Owen assigns (1/4,1/4,1/2)(1/4,1/4,1/2) and Shapley assigns (1/3,1/3,1/3)(1/3,1/3,1/3). Expert 3 therefore receives more credit under Owen. Across dividends, these signed changes can reinforce or cancel. If Shapley ranks ii above jj, their order reverses when Dj−Di>ϕi−ϕjD_j-D_i > \phi_i-\phi_j. Such a reversal changes a direct top-quota allocation when it moves an expert across the selection boundary.

Figure 5 measures the downstream effect of the resulting allocations. At 2.50 and 2.25 bits, Owen exceeds flat Shapley by 1.78 and 2.58 accuracy points on Qwen3 and by 1.10 and 1.84 on Mixtral, with matched per-layer quotas and sampling budgets. On Qwen3, fixing the selection rule to direct top-quota ranking also yields gains of 1.47 and 2.20 points. The proposition explains a route from group membership to the precision map. The accuracy measurements aggregate the effects of all interactions and subsequent quantization.

Q2. Why group by output similarity, and how do other partitions compare?

Coactivation identifies experts whose errors can affect the same token. For fixed routing weights gig_i, the squared error of the combined output contains the cross term 2gigj⟨ei,ej⟩2g_i g_j\langle e_i,e_j\rangle, where eie_i is the change in expert ii's output under quantization. Coactivation determines when this term can arise. Its sign and magnitude also depend on the directions of the errors. Pooled full-precision outputs describe what experts produce on calibration tokens and provide a feature for constructing groups before quantization.

We randomized membership while matching coalition sizes, sampling budgets and per-layer four-bit counts. We also compared output and coactivation features under the same constrained clustering algorithm. All rows use top-quota selection and report Qwen3 eight-task mean accuracy (%).

Partition2.50 bits2.25 bits
Output similarity66.8365.36
Random membership66.5064.55
Output similarity, constrained clustering66.9665.49
Coactivation, same constrained clustering66.7265.05

Output similarity leads random grouping by 0.33 points at 2.50 bits and 0.81 points at 2.25 bits. With the clustering algorithm held fixed, output features lead coactivation by 0.24 and 0.44 points, respectively. Both comparisons show a larger accuracy difference at the tighter budget.

Thanks, I maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.