acceptodds
Withdrawn from ICLR 2027

Recoup the Experts: Coupling-Aware MoE Pruning under Deployed Routing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Expert-pruning criteria for sparse Mixture-of-Experts (MoE) models are computed under the routing of the original model, but after pruning each token is routed to its top-k experts among the remaining ones and their gates are renormalized. We propose , a criterion that selects the set of experts whose outputs best reconstruct the layer outputs of the removed experts on the tokens they share. The objective has a closed form in the Gram matrix of gated expert outputs, and when expert outputs are orthogonal it reduces to a per-expert ranking that matches REAP up to the moment order. We also propose , which computes the same statistics under the routing of the pruned model. On four MoE models, the form that performs better at low budgets follows the off-diagonal mass of the Gram matrix, which can be computed before pruning. On Gemma-4-26B, the model with the highest off-diagonal mass, scores 59.2 on GSM8K when 25% of the experts are kept, against 15.6 for REAP. On Qwen3.6-35B, the model with the lowest, is 19.5 points above REAP on HumanEval+ when 12.5% are kept. These values are means over three disjoint calibration samples.

Primary area:
foundation or frontier models, including LLMs
Keywords:
MoE, Pruning, Compression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Be9SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies expert pruning in MoE models by jointly evaluating expert subsets to account for correlations between expert outputs. RECOUP selects experts based on their optimal linear reconstruction capability, while RECOUPdeploy uses statistics collected under routing within a candidate pool to approximate deployment error. Both variants use greedy selection with swap refinement, require no recovery training, and demonstrate effectiveness across multiple models and pruning settings.

Critical strengths
  1. The empirical results support the value of joint expert selection. In some experimental settings, the methods substantially outperform independent expert importance ranking and surpass several existing approaches. These results suggest that jointly considering relationships between experts has practical value for expert pruning.
  2. The methods are computationally practical. The methods rely on calibration statistics and require no recovery training. Greedy selection with swap refinement avoids enumerating all expert combinations, reducing the computational cost of subset search and making the approach potentially useful in practice.
Critical weaknesses
  1. The distinction from prior work is insufficiently clear, and the novelty of the core design is not firmly established. SCHUR-A* already performs expert subset selection through unconstrained linear reconstruction of expert outputs, using an objective closely related to the base RECOUP objective. The paper should clarify its substantive contributions beyond SCHUR-A* and provide direct comparisons demonstrating the additional benefits of its proposed designs.
  2. Both variants have insufficiently validated gaps between their surrogate objectives and actual deployment behavior. The base method evaluates expert subsets through their optimal linear combination, but the compensation coefficients are not constrained to be nonnegative or normalized, and it is unclear whether they are used at deployment. If these coefficients are used only for selection, a better reconstruction score does not necessarily imply lower actual deployment error. The theoretical analysis mainly explains the least-squares formulation and the orthogonal case, without sufficiently establishing this connection in the general setting. RECOUPdeploy removes the free compensation coefficients, but its search uses routing statistics from a fixed candidate pool rather than the routing induced by each subset. Consequently, its surrogate error may differ from the actual deployment error defined in Eq. (2). The paper should validate whether the two objectives rank candidate subsets consistently.
  3. The benefits and applicability of RECOUPdeploy relative to the base method are unclear. RECOUPdeploy aims to reduce the mismatch between the scoring objective and actual deployment computation. However, the paper should more clearly summarize which models, retention ratios, and tasks benefit from this variant, and whether these advantages are consistent across calibration samples. If RECOUPdeploy does not consistently outperform the base method, the paper should explain why and provide guidance on choosing between the two variants.
  4. The solution quality of the subset search is insufficiently evaluated. Both variants use greedy selection with swap refinement without a global optimality guarantee. The comparison with random-restart local search in Appendix H is informative, but it does not quantify the gap to the global optimum. The absence of a global guarantee is not itself a decisive weakness; however, further evaluation is needed to characterize the search error and determine whether better optimization of the surrogate objective improves downstream performance.

[1] SCHUR-A*: Layer-wise Optimal Expert Pruning for MoEs via Schur-Complement Guided A* Search. ICML 2026.

Questions for authors
  1. What are RECOUP’s substantive contributions beyond the linear reconstruction framework of SCHUR-A*? Which experiments demonstrate the additional benefits of these contributions?
  2. Are the compensation coefficients of the base method used at deployment? Can you validate the relationship between each variant’s surrogate score and actual deployment error?
  3. Under which settings does RECOUPdeploy consistently outperform the base method? If its advantages are inconsistent, what explains this behavior, and how should practitioners choose between the two variants?
  4. On small-scale instances that can be solved exactly, what is the optimality gap of greedy selection with swap refinement? Do better surrogate objective values translate into better downstream performance?
Reviewer_abDFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the mismatch in the behavior of MoE models before and after expert pruning. Existing methods determine the pruning strategy from statistics of the original model, and the top-k selection and the gate values may shift in the pruned model, leading to unexpected behavior. To address this problem, the proposed method considers how well the layer output can be reconstructed after pruning, and formulates the objective as the error between a linear combination of expert outputs under the original parameters and the bare output of the original model (expressed in terms of the Gram matrix of expert outputs). The proposed method RECOUP selects experts by minimizing this objective with a greedy method. Because this method still depends on the original routing, the authors also propose RECOUP_deploy, which applies several procedures to better approximate the computation after pruning. The experiments cover mid-sized models from about 7B to 35B parameters under various expert budgets. The authors report that RECOUP is particularly effective on models with a high coupling ratio (the ratio of the absolute sum of the off-diagonal/diagonal entries of the Gram matrix, reflecting how well removed experts can be reconstructed), whereas RECOUP_deploy performs well on models with a low coupling ratio, although it is affected by the pool size.

Critical strengths
  • The formulation explicitly handles (approximate) statistics of the pruned model while requiring no network computation during the optimization, which I consider a significant advantage in terms of computational efficiency.
  • In particular, on Gemma 4, the method maintains accuracy as the pruning ratio increases while existing methods collapse rapidly, showing that it remains applicable in cases where the existing methods compared here are not effective.
Critical weaknesses
  • The effect of coupling is unclear. Table 5 shows almost the same scores when only the diagonal entries of the Gram matrix are considered, which reads as if the paper itself refutes the "coupling-aware" behavior that should be its main contribution. In addition, the same expression already exists as a variant of the score definitions in Liu et al., 2026 (Squared-gate activation energy, (0,2,2)(0, 2, 2)). Since this specific experimental setting itself has little novelty, I think it would be better to refer to it by the citation and the differences from it, rather than as "Diagonal only".
  • The choice between RECOUP and RECOUP_deploy based on the coupling ratio is an interpretation over all models used in the experiments, and whether it is a practically useful criterion has not been verified as an open test.
Questions for authors
  • The model google/gemma-4-26b-a4b appears to be a pre-trained model rather than an instruction-tuned one. Is it really appropriate to interpret the experimental results as they are...?
  • During expert selection, the coefficients γi\gamma_i take values other than 1, but the pruned model seems to have no coefficients corresponding to it. Is it acceptable to leave them ignored?
  • The paper encounters a phenomenon in which a specific expert causes problems on Qwen3.6-35B. Is it possible to address such specialized experts within the scope of the proposed method? The paper treats the coupling ratio, which depends more on the model than on the calibration data, as an important indicator, but in this case it seems that the problem could only be detected if the calibration data adequately represented the evaluation targets.
  • This is an opinion based on my personal experience: GSM8K shows quite peaky performance in practice, and I have also observed in other models the behavior mentioned in the paper, in which performance depends extremely on a specific expert or layer. I believe this can distort otherwise valid conclusions, and personally I think it might be better to exclude this dataset altogether.
  • The claim that the method "matches REAP up to the moment order in the orthogonal case" looks like over-claiming, given that the actual impact of the second-order moment and other differences in the formulation that must be taken into account.
Reviewer_GTgXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper analyzes a method of MoE expert pruning that re-weights the unpruned experts in a way that maximizes their reconstruction closeness to original expert selection by focusing on co-routed experts.

The paper shows pruning results on 4 different MoE models where their method outperforms existing baselines at very high pruning ratios.

Critical strengths

The paper shows strong performance particularly in the very high pruning regime.

Critical weaknesses

The paper is not easy to follow, and it would be helped considerably by some clear diagrams or equations that present the main algorithm in a clearer way. More time should be spent on setup & formulation, and less time on text describing results that could be captured in a table and a few sentences.

Questions for authors

What was the original insight or motivation that led to RECOUP? Describing this in introduction may help set it up better & drive clarity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.