acceptodds
Under review as a conference paper at ICLR 2027

MAPLE: MoE Adaptive Plug-and-play Layer-wise Expert allocation

Abstract

Sparsely-activated Mixture-of-Experts (MoE) Transformers universally fix the same number of routed experts across all layers, a convention that ignores the well-documented heterogeneity in layer-wise redundancy. We demonstrate that this uniformity is suboptimal and propose MAPLE, a plug-and-play framework that reallocates the routed-expert budget heterogeneously across layers of any pretrained MoE LLM, pruning computation rather than parameters: no expert weight is modified, removed, or retrained. Our core contribution is a closed-form sensitivity-guided allocation: we probe each layer's response to variation in expert count, quantify sensitivity, and derive a budget assignment that directs capacity towards sensitive layers and absorbs reductions in redundant layers. This closed-form solution is further refined by a sensitivity-constrained genetic search that uses layer-wise sensitivity as a prior guide for exploration, yielding faster convergence and superior allocation quality. The allocation also creates an execution regime that uniform routing never encounters, which we exploit with a dedicated top-1 routing GPU kernel integrated into SGLang and vLLM. On four MoE models spanning different scales and architectures, MAPLE outperforms uniform and pruning-based baselines under a 75% total expert-activation budget. Notably, on DeepSeek-MoE-16B, MAPLE uses only 75% of the experts yet surpasses the original 100% expert-uniform baseline on ARC-E, ARC-C, and BoolQ, improving accuracy from 65.09 to 71.40, 48.49 to 51.50, and 80.03 to 82.38, respectively. These accuracy gains translate into measured deployment efficiency: implementing MAPLE in SGLang reduces single-GPU end-to-end serving latency by 32.2% and improves throughput by 47.4%. These results show that well-designed heterogeneous allocation can be more effective than simply activating more experts, making it a practical axis for improving MoE efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Efficient Artificial Intelligence, Large Language Model, Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4HYEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a plug-and-play method, called MAPLE, to allocate different numbers of active experts to different layers in MoE models. MAPLE first measures how sensitive each layer is to the number of active experts. It then uses a closed-form method to allocate a fixed expert budget across layers. The authors evaluate MAPLE on four MoE models, including DeepSeek-MoE-16B, DeepSeek-V2-Lite, Moonlight-16B-A3B, and OLMoE-1B-7B. Under a 75% expert budget, MAPLE consistently outperforms existing pruning methods. Experiments on GSM8K and BBH also show that MAPLE can reduce computation time while maintaining, or even improving, model accuracy.

Critical strengths
  • Comprehensive experiments. I appreciate the authors’ careful experimental design. They apply the expert allocation found on one task directly to four other tasks, without performing a new search for each task. The results show that the learned allocation can transfer across tasks, though with some performance loss. This provides useful evidence for the generality of MAPLE and makes the practical value of the proposed allocation strategy more convincing.

  • Good algorithm–system integration. Another major strength is that the paper goes beyond the algorithm and also addresses the system-level cost of heterogeneous expert allocation. The authors develop a dedicated Top-1 CUDA kernel for MAPLE and integrate it into both vLLM and SGLang. By optimizing both the expert allocation and the inference system, the paper achieves clear improvements in latency and throughput. This makes MAPLE more practical for real-world MoE deployment than methods that only reduce theoretical FLOPs.

Critical weaknesses
  • Limited evaluation of knowledge capabilities. I have some concerns about the benchmarks used in the paper. The current evaluation mostly uses simple classification tasks and basic reasoning datasets. However, MoE experts often store factual knowledge. Reducing active experts might hit a model's factual knowledge harder than its reasoning ability. To make the evaluation more convincing, the authors should test knowledge-heavy benchmarks like MMLU, GPQA, or MMMLU. It would also help to test different shot counts (e.g., zero-shot vs. few-shot), as prompting may change how expert allocation works. Without these tests, it is hard to tell if MAPLE works well on knowledge-intensive tasks.

  • Need evaluation on stronger and newer models. I also have concerns about model selection. The tested models are relatively small or older. The surprising finding that MAPLE beats the full model with fewer experts might just happen because these models are under-trained or have high redundancy. The authors should test MAPLE on stronger and newer models, such as Qwen3.5-Next-Flash or other state-of-the-art MoE models. This will show if MAPLE provides a general benefit or just takes advantage of redundancy in smaller models.

Questions for authors
  • Why can reducing the average number of active experts improve overall performance? In the experiments, MAPLE achieves higher average accuracy while using a smaller expert activation budget than the full-budget baseline. This result is somewhat counter-intuitive, since reducing the number of active experts would normally be expected to reduce model capacity and hurt performance. How do the authors explain this phenomenon?

  • How should a reasonable search set be determined? Table 8 shows that the expert allocation obtained by searching on different evaluation sets can lead to nearly a 4-point difference in average performance. This suggests that the final allocation may be sensitive to the choice of the search set. How should users choose a suitable search set in practice? More importantly, if the search set is not known in advance, how can we determine an allocation that is robust across different tasks and datasets?

Reviewer_2uuJOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAPLE, a training-free framework for heterogeneous expert allocation in pretrained MoE models. It uses layer-wise sensitivity analysis to derive a closed-form expert allocation, followed by genetic search for refinement. The method preserves pretrained model weights while reducing expert activation. The authors also develop optimized routing kernels for SGLang and vLLM, demonstrating improvements in accuracy and inference efficiency across four MoE architectures.

Critical strengths

S1. The paper presents a clear algorithm–system co-design, combining weight-preserving expert allocation with optimized GPU kernels to achieve practical inference speedups.

S2. The experiments cover four MoE architectures, multiple benchmarks, and real-world serving settings. The ablation and cross-task evaluations also provide useful insights into the method's effectiveness and limitations.

Critical weaknesses

W1. MAPLE's main accuracy gains rely heavily on task-specific calibration and offline optimization. As shown in Table 8, the average accuracy drops from 74.34% to 69.14% under cross-task transfer, below the original model's 71.71%. This raises concerns about its generalizability and practical applicability to general-purpose LLM serving, where target tasks may not be known in advance.

W2. The contribution of the proposed sensitivity-guided allocation is not sufficiently isolated. The paper lacks a same-budget uniform Top-k baseline, making it difficult to determine the actual benefit of heterogeneous allocation. Moreover, Table 7 shows that the closed-form solution achieves only 64.39% on ARC-E, while genetic refinement improves it to 71.40%. It remains unclear how much of the improvement comes from the proposed sensitivity modeling rather than the subsequent search procedure.

Questions for authors

Q1. Please report a same-budget uniform Top-k baseline for all models and tasks. How much improvement does MAPLE achieve over this baseline?

Q2. MAPLE allows up to 20 experts per layer, exceeding the pretrained Top-k (6 or 8). Please report the final layer-wise allocations and an ablation restricting Top-k to its original maximum.

Q3. How stable are the allocations across different calibration subsets, rather than only different genetic-search seeds? How does this affect cross-task generalization?

Q4. How much does the sensitivity-based closed-form allocation contribute independently of genetic search? Please compare different initialization strategies under the same search budget.

Reviewer_VyHBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MAPLE reallocates the number of active routed experts across the layers of a pretrained MoE under a global activation budget, without changing weights. For each layer it varies klk_l from 1 to the original k0k_0 with all other layers fixed and records task accuracy on a calibration set; the accuracy range defines a sensitivity, a weighted least-squares problem with a sum constraint gives a closed-form allocation, and a small genetic search (population 10, 10 generations) with calibration accuracy as fitness refines it, allowing klk_l between 1 and 20. A CUDA top-1 routing kernel in SGLang and vLLM accelerates layers reduced to one expert. With 128 training examples per task for calibration and a 75% activation budget, MAPLE raises the five-task average above the full-budget model on DeepSeek-MoE-16B (71.71 to 74.34), DeepSeek-V2-Lite and OLMoE-1B-7B, outperforms six pruning baselines on all four models, and reduces single-GPU SGLang latency on ARC-Easy by 32.2%.

Critical strengths
  • The method is training-free, keeps all weights and is cheap to run. Allocation needs only forward passes (about 1.5 GPU-hours per task for DeepSeek-MoE-16B on one A6000, Table 14), keeps the full expert pool so it is reversible, and the top-1 kernel is integrated into two serving engines. If its benefit holds, per-layer top-kk reallocation is easy to adopt.
  • The robustness analysis reports results that bound the claims. Table 8 evaluates a jointly calibrated allocation, cross-task transfer and task-agnostic calibration with C4 and FineWeb-Edu NLL, and Table 6 shows that some sensitivity signals perform worse than the uniform baseline. These are the right experiments for judging whether the allocations capture model properties rather than task-specific fits.
  • The evaluation covers four MoE families and both short and long-form tasks. DeepSeek-MoE-16B, DeepSeek-V2-Lite, Moonlight-16B-A3B and OLMoE-1B-7B, plus GSM8K and BBH with chain-of-thought, show that the procedure applies to architectures with and without shared experts.
Critical weaknesses
  1. The accuracy gains come from task-specific, label-supervised search that the baselines do not receive. Each task's allocation is searched over up to 100 candidates with accuracy on 128 labeled training examples of that task as the objective, and a layer may use up to 20 experts (original k0=6k_0 = 6). With task-agnostic calibration (C4 or FineWeb-Edu NLL), the DeepSeek-MoE-16B average drops to 69.59 or 68.88, below the full model (71.71) and within the pruning baselines' range of 69.12–70.82 (Table 8). The same search at 100% budget adds about 3.3 points on ARC-C and RTE (Table 9). The baselines' calibration data are not stated. Giving the baselines the same labeled examples, for example a per-task search over uniform kk, and reporting the searched allocations are needed.
  2. The efficiency headline is the best case among heterogeneous measurements. The abstract and Figure 1 report 32.2% lower latency and 47.4% higher throughput for SGLang on one GPU, while the same allocation and kernel give 4.6% on vLLM on one GPU and 3.9–14.2% with pipeline or expert parallelism (Table 5). On BBH the end-to-end reduction is 5.9% (Table 3), consistent with routed-expert compute taking 37% of wall-clock time (Table 13). The workload is ARC-Easy scoring, which is prefill-dominated, and batch size and concurrency are not stated. An explanation of the SGLang-specific gain and measurements on decode-heavy serving traces at several batch sizes are needed.
  3. Adaptive expert counts have substantial published prior work that is not discussed. Ada-K routing (ICLR 2025) learns per-token expert counts with lightweight allocators and reports over 25% FLOP savings with accuracy gains, AdapMoE (ICCAD 2024) sets the number of activated experts from sensitivity, and top-pp routing ("Harder Task Needs More Experts", ACL 2024) varies the count per token; only LExI is compared. The closed-form step is a weighted least-squares projection onto a sum constraint, and the genetic refinement with task-accuracy fitness resembles EEP's search. Positioning against these methods and comparing with at least one of them at a matched average expert count would clarify the contribution. The small evaluation sets add uncertainty: one example is worth 0.33–0.36 points on ARC-C and RTE.
Questions for authors
  1. Clarification. What calibration data, and how many examples, did each baseline use, and did any baseline see the 128 labeled examples per task?
  2. Clarification. What are the searched per-layer allocations for each model and task, and how many layers exceed k0k_0?
  3. Clarification. Why does the top-1 kernel reduce latency by 32.2% on SGLang but 4.6% on vLLM for the same allocation, and which batch size and concurrency does Table 5 use?
  4. Clarification. How does a single task-agnostic allocation compare with Ada-K or AdapMoE at a matched average number of activated experts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.