- Rating
- 3
- Confidence
- 2
- Informativeness
- 3(+1)
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
The paper thinks the topk can be shrunk to lower number and can use specific methods to select back the k+1th important experts if needed.
Critical strengths
A simple idea with consistent results on 3 MoE families against 8 reimplemented baselines on the same data, plus e2e serving numbers with fused kernels for the baselines too. The 2 expert regime is the most convincing evidence that keeping the router's ranking matters.
Stage II is very cheap and is tested against a random restoration control at matched expert budget。
Critical weaknesses
Every table is a single run with no seeds or confidence intervals. The Stage II gains over Stage I (+0.09 to +0.94 points) and over random restoration (+0.15 to +0.34 points) are within plausible run to run noise, so the second contribution is not established.
Questions for authors
how is the threshold ρ set per model, and how does accuracy move for ρ between 5% and 30%? does the vLLM kernel apply the learned rank 1 to 4 mask values and the paper's renormalization convention, and does the Native Top-8 baseline run on the same sparse execution backend?
2 comments
Thx, this answered my question, increased my score in a few subsessions.
Dear Reviewer ypCR,
We appreciate the reviewer’s thoughtful feedback and valuable suggestions. We address the raised concerns in detail below.
Response to Concern 1: Clarification on Recovery Threshold and Implementation Details
We further clarify how the threshold qρ in Eq. (20) is determined. Here, ρ denotes the target proportion of events in which the Rank-5 expert is additionally activated, while qρ is the predicted utility threshold controlling expert recovery. Both models use ρ=10% in the main experiments. Before evaluation, we compute utility scores on a held-out calibration set for each model, sort them in descending order, and select the cutoff corresponding to the top 10% as qρ. This yields thresholds of 0.18 and 0.28 for Qwen3-30B-A3B and Qwen3.5-35B-A3B, respectively. The thresholds are then fixed during downstream evaluation, and the Rank-5 expert is recovered only when u^l,t≥qρ, following Eq. (20). We investigate the effect of the recovery budget ρ by evaluating four settings: 5%, 10%, 20%, and 30%. All experiments use identical model weights, learned masks, and utility predictor parameters, with only the recovery threshold adjusted.
Response to Concern 2: Clarification on Mask Application and Normalization in vLLM
Our vLLM implementation retains and applies the learned Rank 1–4 mask coefficients from Stage I for each layer. For rank r in layer ℓ, the effective mixing weight can be represented as:
$ w_{\ell,t}^{r}=p_{\ell,t}^{r,(8)}\tilde m_\ell^{r}, \qquad r=1,\ldots,4, $
where pℓ,tr,(8) denotes the weight normalized under the original Top-8 routing rule. Thus, the learned mask coefficients directly affect the weighted aggregation of expert outputs. Regarding normalization, we first apply the original Top-8 normalization and then multiply by the learned masks, without renormalizing the retained expert weights. When Stage II restores Rank-5, its contribution uses the same Top-8-normalized weight pℓ,t5,(8), with no additional renormalization.
Response to Concern 3: Consistency of Sparse Execution Backend
Both the Native Top-8 baseline and our method are evaluated using the Triton sparse MoE backend in vLLM 0.21.0, with TP=2 and BF16 precision. Native Top-8 executes the eight experts selected by the original routing mechanism, while our method uses learned-mask-weighted Prefix-4 with conditional restoration of Rank-5. Both share the same sparse execution backend, while retaining the routing and dispatch logic.