- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies budgeted low-rank adaptation of MoE models. It proposes NRG, which uses native routing statistics to select experts for static DoRA adaptation, and evaluates how expert coverage, rank, and identity affect performance. Experiments on Qwen3.5-35B-A3B-Base with VCR and NLVR2 show that routing-selected rank-32 adapters achieve the best observed means, but follow-up controls suggest that the gains cannot be attributed solely to routing-based expert selection.
Critical strengths
- The paper separates expert identity, layer allocation, coverage, and local rank, showing that better performance of TOP32 does not directly imply that routing selects intrinsically better experts.
- The authors clearly distinguish observed-model inference from broader generalization and explicitly report limitations, post-hoc analyses, and reproduction constraints.
Critical weaknesses
- The core expert-selection strategy is closely related to ESFT and MoE-Sieve. The main contribution is primarily the experimental analysis rather than a substantially new adaptation method.
- Evaluation is restricted to one MoE backbone, two datasets, and three main training seeds. Moreover, several controls remain inconclusive, so the generality of the findings is unclear.
Questions for authors
Q1. Can the authors provide a direct comparison with ESFT or MoE-Sieve under the same backbone, data, and parameter budget? Q2. Can the authors better isolate whether the TOP32 gain comes from routing-based expert selection or simply from concentrating more rank on fewer experts? Q3. For the historical-versus-new follow-up runs, which execution differences remain unverified, and could they plausibly affect effects of the reported magnitude?