acceptodds
Under review as a conference paper at ICLR 2027

ConMoE: MoE compression as expert selection and functional assignment

Abstract

Sparse Mixture-of-Experts (MoE) models scale model capacity through sparse activation, but storing their full parameters remains costly in GPU memory. Existing expert compression methods address this problem through pruning, merging, or substitution, typically coupling expert pool construction with the assignment of routed workloads. In this work, we formulate expert compression as two decisions—constructing an expert pool and assigning the original experts’ workloads to that pool—and show that assignment can be optimized after the pool is fixed. Based on this formulation, we introduce **ConMoE**, which constructs a compact pool using Anchor Coverage and optimizes assignment through a router-conditioned functional objective derived from a layer reconstruction bound, treating parameter proximity only as a proxy. ConMoE uses unlabeled calibration inputs and keeps the router and expert weights frozen. With the pool fixed, functional assignment improves macro accuracy by 1.82 and 0.35 percentage points at 50% and 75% expert retention, respectively, and also improves pools constructed by other selectors. Across three MoE architectures and six zero-shot benchmarks, ConMoE achieves the highest macro average among the compared methods at both ratios. At 50% expert retention, ConMoE reduces peak GPU memory by 29.6–48.4% relative to the uncompressed models on our generation workloads. These results establish expert assignment as an independent optimization axis for MoE compression.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, model compression, expert assignment, functional substitutability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CDg3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ConMoE splits MoE expert compression into two decoupled decisions: constructing a retained pool, and then, with pool, router and all weights frozen, assigning each removed expert's routed workload to a pool member by minimizing a router-conditioned functional reconstruction objective estimated on routed calibration states, accepted only when two disjoint calibration folds agree; fingerprint retrieval keeps candidate search cheap, and compact execution stores each retained expert once with integer maps while preserving route weights. Validation: three MoE architectures (Qwen3-30B-A3B, DeepSeek-MoE-16B, OLMoE-1B) at 50%/25% retention on six zero-shot benchmarks against pruning (Frequency, REAP), merging (M-SMoE, HC-SMoE) and substitution (MoNE) baselines, plus fixed-pool controlled comparisons with paired bootstrap CIs, held-out reconstruction diagnostics, assignment-rule and calibration-source controls, and peak-memory/throughput measurements on A100s.

Critical strengths
  • The fixed-pool decoupling is a genuinely clean causal isolation, and it is selector-agnostic. Every nearest-vs-functional pair shares pool, weights, router and calibration data, so the only free variable is the integer identity→expert map; assignments are frozen on 64 construction prompts before evaluation on 64 disjoint prompts, paired bootstrap CIs exclude zero on all three models (+0.55 / +0.82 / +4.12, macro +1.83), and gains replicate across three calibration seeds.
  • All three backbones are real pretrained MoE checkpoints rather than MoEs synthesized from dense models; every score is anchored to the uncompressed checkpoint reported in the same table, so no normalization hop is hidden; evaluation uses full splits via lm-eval-harness at two retention ratios rather than one favorable setting.
Critical weaknesses
  • ConMoE removes 45.6–47.5% of unique parameters for a 29.6–48.4% peak-memory reduction, but activated FLOPs per token are unchanged (top-k fixed), so throughput improves only 1.11–1.24×. Weight-only INT8 on the full model reaches the same ~2× with all experts intact, and INT4 reaches ~4×; A memory-reduction method that never faces the dominant equal-bytes alternative has not established that dropping experts is the right way to spend the budget.
Questions for authors
  1. Why does random substitution beat nearest by 1.90 pp? Please report assignment load concentration (mean removed identities per retained expert, or assignment entropy) for nearest / random / functional.
  2. Full-model weight-only INT8 (≈ your 45–47% reduction) and INT4 on the same three models and six benchmarks, plus ConMoE ⊕ quantization. What evidence is there that expert dropping is the right way to spend a 2× memory budget?
Reviewer_MFBcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper present decomposing the process of compressing the number of experts in an MoE model into two parts. first creating a set of "retained" experts, then determining how to allocate all of the work from a "removed" expert to one or more of these retained experts.

Critical strengths
  • The separation of "which experts to retain" from "who will be responsible for taking on additional work" is based upon an accurate and simple reconstruction bound. Therefore, this approach does not require any backward pass or modification of either the weight or router.

  • Functional assignment has produced positive results across all three pools (Macro +1.83, 95% CI [1.49, 2.17], Frequency +4.73, REAP +1.75).

  • An honest report was provided which included potential negative control groups such as random assignment, token-matched calibration and anchor thresholds.

Critical weaknesses
  • The difference between ConMoE's score and that of MoNE was small (0.20/0.14), and ConMoE had a lower score than MoNE when using OLMoE.

  • Per-task calibration was used in all of the primary experiments; each task had its own model. When using general-purpose WikiText calibration, accuracy dropped significantly (OLMoE 58.03 to 42.54). A complete table comparing the results of using general-purpose calibration versus no calibration can be found at the end of this report for Qwen3. All evaluations were done using multiple choice questions and did not include any generation tasks or measures of perplexity.

  • The authors' own control group demonstrated that the method they developed is less effective than either random assignment or functional assignment. Specifically, random assignment produced a higher result than nearest by 1.9 points; functional assignment produced a higher result than random by 0.74 points; and the candidate pool included the best substitute identified through measurement only 7-15 percent of the time.

  • Pool creation essentially becomes selecting the top K items based upon a threshold value (all thresholds from 0 to 0.8 produce the same pool) and thus limits how novel a new item can be.

Questions for authors
  1. What are the mean and confidence interval of the full comparison with MoNE across calibration samples?
  2. What does the full three model comparison look like under generic calibration? Can you add generation tasks such as GSM8K and HumanEval?
  3. What is the in-degree distribution of retained experts under each assignment rule? How do a load-balanced nearest baseline and a functional argmin over all retained experts perform?
Reviewer_PvQZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ConMoE splits MoE expert compression into two decisions: which experts to keep (the pool) and where each removed expert’s routed workload goes (the assignment). Experts are protected by “demand” (routing-weighted output energy), and the remaining slots are filled greedily to minimize a demand-weighted parameter-distance coverage cost. Each removed expert is remapped to a retained expert. The choice minimizes router-weighted output MSE on the tokens routed to the removed expert. Authors present results on Qwen3-30B-A3B, DeepSeek-MoE-16B and OLMoE, over six zero-shot multiple-choice benchmarks

Critical strengths
  • Clear motivation - Separating pool from assignment is a nice lens, and Eq. 2 covers pruning, merging and substitution in one view.
  • It needs forward passes only, with no gradients and no new parameters. Deployment is just an integer map.
  • Authors present good set of results. Assignments are frozen on 64 prompts and evaluated on a disjoint 64 (Figure 2, Table 3). The paper also reports paired bootstrap intervals (Table 20), three calibration seeds (Table 15), and generic-calibration ablations on WikiText, C4 and Zyda-2.
Critical weaknesses
  • The paper claims that assigning removed experts' workloads to retained experts improves compression, but it never compares against simply dropping them (π=⊥) on the same pool. The Anchor Coverage + pruning baseline is missing, so the claim that assignment helps is not established for ConMoE itself. Where both numbers exist, plain pruning beats functional substitution on the same pools: Frequency 59.03 vs 58.79, REAP 60.10 vs 59.89 (Table 1 vs Table 2). The reported +4.73 and +1.75 gains are therefore measured against a nearest baseline that is worse than doing nothing.

  • The claim of highest macro accuracy is within noise. ConMoE leads MoNE by 0.20 pp at 50% retention and 0.14 pp at 75%, loses on OLMoE at both ratios, and no cross-method confidence intervals are reported. The paper's own paired bootstrap half-width for the macro average is about 0.34 pp (Table 20).

  • The main results use task-specific calibration: a separate compressed model per evaluation task, built from that task's own prompts. This is not a realistic deployment setting. Under shared generic calibration, accuracy drops sharply (OLMoE 58.03 → 42.54, Table 4), and the only baseline comparison in that setting is Qwen3 at 50% (Table 5).-

Questions for authors

Please see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.