- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper aims to enable multiple GPUs to complete MoE reinforcement learning training more efficiently. The authors utilized the routing information recorded during the generation process to arrange the training samples and the positions of the experts simultaneously, considering attention, expert computation, and communication overhead. During execution, they overlapped expert transfers with attention computation to reduce waiting time between GPUs. Experiments compared the training speed and load distribution under different arrangements.
Critical strengths
S1. Arranging the samples and experts together is a relatively straightforward approach. The positions on both sides would affect communication, and optimizing only one side would indeed lead to trade-offs.
S2. The experiments not only report the acceleration but also examine the changes in attention, expert load, and cross-machine communication. These results can clearly illustrate where the speed improvement comes from.
S3. The effect of overlapping expert transfers and attention computation has been measured in practice. Compared to only estimating the transfer time, this provides a more reliable indication that the solution can actually run in the test environment.
Critical weaknesses
W1. The communication cost is mainly estimated based on the total amount of cross-machine transmission, but even with the same total amount, speed can be affected by which GPUs have the concentrated traffic. It is not clear yet whether this estimation can reliably select the arrangement with faster communication.
W2. The baseline comparison did not compare the effects of sequential arrangement and alternating adjustment under the same consideration of the three types of costs. Therefore, it is not easy to distinguish how much of the benefit comes from the joint adjustment itself and how much from the more complete consideration of costs.
Questions for authors
Q1. Can the predicted values and actual measured values of communication times for several arrangements be compared to confirm whether the cost model can correctly judge their speed?
Q2. Can the sequential arrangement and alternating adjustment be compared under the same cost model, initial arrangement, and search budget, separately looking at the benefits brought by the joint adjustment?
3 comments
Thank you for the response. It addresses most of my concerns, and I will keep my current positive rating.
Thank you for your thoughtful review and feedback. We’re glad that our response helped address your concerns and appreciate your valuable insights.
Official Comment by Authors [1/1]
Thank you for your encouraging comments and insightful suggestions. We address each point below.
W1 & Q1. Communication Cost Model Accuracy
We have added Appendix D and Fig. 8 in the revised paper to compare predicted and measured All-to-All communication times and identify a suitable model for our configuration:
Fig. 8(c)–(e) compare three All-to-All communication models fitted using total inter-node traffic, maximum per-node communication load, and maximum per-GPU communication load, respectively. On our cluster, the total-volume model achieves the lowest prediction error (5.81%), closely matching the maximum-node model (5.93%) and outperforming the more variable maximum-GPU model (7.49%). Given this comparable (and slightly better) accuracy, we adopt the total inter-node traffic model in Eq. 4 for two reasons: (1) smoother optimization guidance: the maximum-node model yields gain only when reducing traffic on the currently busiest node and is insensitive to traffic reductions among other nodes, whereas the additive total-volume model directly rewards any reduction in inter-node traffic and more effectively guides the search toward communication locality; and (2) lighter incremental evaluation: each candidate swap simply updates a local inter-node traffic delta without maintaining and recomputing per-node send/receive maxima at every trial.
W2 & Q2. Ablations on Solver and Objective Components
We have expanded Fig. 6 (§4.2) in the revised paper with additional ablations to demonstrate the benefit of each component (all speedups are relative to Megatron-Balanced):
These ablations quantify the sources of J-MoE's improvement along three dimensions:
w/o Comm) reduces the speedup from 1.25× to 1.22× on Codeforces and from 1.30× to 1.17× on Mixed, confirming the substantial contribution of communication awareness. Similarly, removing the attention term (w/o Attention: 1.15× / 1.22×) worsens attention-computation stragglers and degrades performance. The expert-computation term is retained across all variants, as expert placement relies on it for guidance.E→SandS→E, which execute Expert-KL and Sample-KL in a single pass under either order. As shown in Fig. 6(a), starting from the exact same initial placement (Megatron-Balanced), three-component objective, and search budget, alternating Sample-KL and Expert-KL (w/o Stage 1: 1.21× / 1.26×) significantly outperforms both one-pass sequential schemes,E→S(1.15× / 1.21×) andS→E(1.19× / 1.19×), because each placement can re-adapt after the other changes traffic sources or destinations. Combining this alternating refinement with our coupled Stage 1 initialization (w/o Stage 2: 1.15× / 1.22×) yields the highest speedup in full J-MoE (1.25× / 1.30×).w/o Cross-MB: 1.19× / 1.27×) likewise degrades performance, because fixing microbatch membership limits the solver's ability to rebalance aggregate expert demand across microbatches.