- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigates whether natural MoE routing variation encodes information about answer correctness and aids candidate selection. Using paired trajectories, it quantifies routing gaps between correct and incorrect rollouts, trains linear probes on routing occupancy features, and evaluates two selection strategies: learned scoring and routing-distance diversity.
Critical strengths
The research question is meaningful, and the experimental coverage is relatively broad. This paper investigates whether naturally occurring routing variation in Mixture-of-Experts models is associated with answer correctness and whether routing information can improve candidate selection. Using a paired-trajectory design, the authors compare correct and incorrect candidates for the same question across five MoE models and five reasoning datasets. The evaluation connects routing difference (RD), a matched permutation baseline, linear occupancy probes and nonlinear sensitivity analyses, learned candidate scoring, and routing-distance selection.
Critical weaknesses
-
The candidate selection baselines are relatively limited. The base scorer uses rollout length and four simple text statistics, while the routing-augmented scorer adds the routing feature vector. Comparisons with stronger baselines, such as majority voting where applicable, would make the practical benefits of routing information more convincing.
-
The paper preserves phase-specific routing features but mainly reports aggregate routing differences and overall prediction and selection metrics, without corresponding phase-level analyses. Additional results for the early, middle, and late phases would help readers understand how correctness-related information is distributed across generation progress.
Questions for authors
See above
3 comments
Thanks for the response. Most of my concerns are addressed; I decide to increase my rating.
Comment by Author
Thank you for your time and the constructive feedback throughout the review process. We are glad the clarifications addressed your concerns, and we will ensure all discussed revisions are fully incorporated into the final version.
Response to MC3o
Thank you for recognizing the significance of the research question and the breadth of the evaluation. We address both concerns with additional selection and phase-specific readout experiments.
We compare random selection, majority voting (MV), and routing-augmented voting (MV+R) across five models on DAPO-Math under matched supervision, using the same evaluation questions and candidate pools for the three methods within each model. The method, numerical results, and paired intervals are provided in #82tE Q1.
We added separate linear probes for Early, Middle, and Late routing occupancy, alongside All, which concatenates the three phases, and the original Length baseline. We retain the paper's phase definitions, original candidates and labels, question-level folds, and correct–incorrect pairs. Each probe is trained independently, with standardization and regularization selection restricted to the training folds.
We report pair-order AUC (%) below, with 95% question-level bootstrap intervals.
On DAPO-Math, Qwen3-235B supports above-chance readout in all three phases, Qwen3-30B has its highest AUC in Late, and GLM has similar Middle and Late scores. For Qwen3-30B on HotpotQA and HumanEval, All yields the highest AUC.
These findings extend the cross-question readout results in Table 2 by showing where correctness information is readable across generation progress. In some models, early and middle expert-use patterns already distinguish correct from incorrect candidates. The differing phase profiles reveal structure not visible in aggregate metrics and motivate retaining phase information so that a learned readout can use features from different stages according to the model and task.