acceptodds
Under review as a conference paper at ICLR 2027

RailWave: Adaptive Spatial and Temporal Scheduling for Expert-Parallel Communication

Abstract

Irregular All-to-All communication is a major bottleneck in expert-parallel Mixture-of-Experts (MoE) models. Even with fixed expert routing and placement, uneven utilization of parallel network Rails and incast can limit communication performance. We present RailWave, a phase-adaptive communication layer built on DeepEP that addresses these bottlenecks below the routing layer through spatial and temporal traffic shaping. RailBalance redistributes source traffic across eligible Rails using source-local information, while a reusable, topology-derived permutation schedule limits concurrent senders per receiver without rebuilding demand-dependent schedules for each communication phase. A lightweight calibrated selector chooses an execution path according to each phase's traffic characteristics and offline profiling results. On training-derived communication workloads from the 106B GLM-4.5-Air model, RailWave delivers up to speedup on H800 and on H20 over Native.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Mixture-of-Experts (MoE), Expert Parallelism, Distributed Training, Communication Optimization, All-to-All Communication, Rail-Optimized Networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pJRbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

RailWave optimizes dispatch–combine communication for expert-parallel Mixture-of-Experts models after routing and expert placement have been fixed. It combines three components: source-local RailBalance, which redistributes transfer records across eligible communication Rails; reusable cyclic node-pair schedules, which limit each node to at most one incoming and one outgoing peer per wave; and an offline-calibrated selector that chooses among Native, Rail-only, Permutation-only, and Joint execution. Joint applies RailBalance within each scheduled wave (§3).

The evaluation primarily consists of communication replay constructed from GLM-4.5-Air training statistics, executed on two physical 32-GPU clusters using H800 and H20 GPUs. The paper reports up to 5.84× and 4.36× speedups over its Native replay baseline, respectively. The headline metrics sum per-layer communication-latency quantiles across 45 MoE layers (§4.1–4.2). Additional evidence includes same-backend ablations, physical NIC traffic measurements, selector validation, controlled UltraEP integration, and packing measurements (§4.3; Appendices A.2–A.4). The reported improvements concern communication replay; full training-step throughput is not evaluated.

Critical strengths
  1. A clear execution-layer decomposition with a concrete implementation. RailWave separates source-Rail allocation from the ordering of node-pair transfers while preserving routing decisions and logical communication demand. Applying RailBalance within each scheduled wave is a meaningful design choice: balanced traffic over an entire phase does not ensure that an individual destination slice is balanced.
  2. Substantive hardware evidence supports the intended mechanisms under the evaluated conditions. The paper evaluates two physical GPU platforms and compares four execution paths using the same DeepEP backend. On H800, Joint outperforms the better individual mechanism by 1.44× and 1.97× under Moderate and Extreme, respectively.
Critical weaknesses
  1. The evaluation does not sufficiently establish application-level benefits or generality across MoE workloads.

The paper reports communication speedups on real H800 and H20 clusters, but does not provide end-to-end training-step latency or throughput. The main results aggregate per-layer dispatch–combine measurements from a communication replay (§4.1). The UltraEP integration in Appendix A.2 also reports communication latency, while the “end-to-end” measurement in Appendix A.4 refers to the communication path with packing. These experiments provide useful component-level evidence, but leave the impact on actual model execution unresolved. In particular, the evaluation does not establish how much communication is exposed on the training critical path, or how wave execution and RailBalance interact with expert computation, GPU resource usage, and communication–computation overlap. A representative integrated training experiment, reporting step time or throughput together with the exposed communication time, is needed to demonstrate that the measured communication improvements translate into application-level gains.

The workload coverage also limits the generality of the conclusions. All training-derived routing statistics come from GLM-4.5-Air on DAPO-Math-17k (§4.1). Covering 45 MoE layers, multiple sending patterns, and two GPU platforms provides useful variation within this setting, but does not establish robustness across different model routing structures. This matters because the trade-off between redistribution, wave serialization, and communication efficiency depends on payload size, destination multiplicity, and the distribution of demand across node pairs and Rails (§3.2–3.3). The synthetic UltraEP workloads and the DeepSeek-V3-sized static memory example do not constitute additional model performance evaluations. An additional MoE model, or an independently collected routing trace with materially different communication characteristics, would help establish the operating range of the method and the calibration required outside the evaluated workload family.

  1. The incremental benefit of wave scheduling over standard DeepEP execution, including its complete online cost, remains insufficiently demonstrated.

DeepEP already supports GPU-initiated transfers through its gin.put data path. These primitives do not guarantee optimal cross-source concurrency, so an additional scheduling layer is technically plausible. However, the relevant empirical question is whether restricting execution to selected node pairs in successive waves improves performance over a standard DeepEP invocation receiving the complete layer demand. The main evaluation does not resolve this question: in Moderate and Extreme, Native itself follows a prescribed three-wave replay schedule, and Permutation-only and Joint reorganize the same demand into three one-to-one waves (Appendix A.6). This comparison demonstrates that the imposed overlap pattern affects performance, but does not establish the advantage of introducing waves into ordinary full-demand execution. A matched comparison against the complete-demand Native invocation, accompanied by measurements explaining the realized concurrency and receive throughput, would provide substantially stronger justification for the scheduling mechanism.

The paper correctly recognizes that reducing incast must compensate for the cost of additional communication rounds (§3.3), but the complete break-even analysis must also include the incremental work required to construct and execute the schedule. The main timing interval includes RailBalance GPU operations but excludes CPU planning, index construction, and prepacking for all methods (§4.1). Appendix A.4 usefully evaluates packing and its overlap with communication, yet the cost of the full adaptive execution path remains unclear. This includes obtaining the traffic descriptors, selecting a consistent path across ranks, preparing the current phase’s transfers, and coordinating wave progression. Reusing topology-derived peer relationships avoids rebuilding one part of the schedule, but does not establish that the remaining online work is negligible or sufficiently amortized. The decisive quantity is the additional latency exposed on the critical path relative to the baseline, with overlap accounted for. A complete timing breakdown under changing demands is needed to show that the communication time saved exceeds these incremental costs and to identify the workload regimes in which scheduling provides a net benefit.

Questions for authors
  1. End-to-end benefit.

Can you provide a representative integrated training experiment comparing Native and RailWave under matched model, workload, and resource configurations? Please report training-step latency or throughput, together with the communication time remaining on the critical path. If a full training experiment is unavailable, measurements of an integrated MoE layer containing the actual expert computation and its communication overlap would still substantially strengthen the evidence.

  1. Benefit of introducing waves.

For the same complete layer demand, how do Permutation-only and Joint compare against standard DeepEP and Rail-only invocations that receive the full demand, without the prescribed replay waves? What measurements show why restricting concurrency improves completion time in this setting? In particular, do the gains persist when the baseline is allowed to use its existing execution mechanisms on the complete demand?

  1. Scheduling cost and break-even conditions.

What is the complete online overhead of obtaining the traffic descriptors, making rank-consistent execution decisions, constructing indices, packing transfers, and coordinating waves? Please distinguish one-time, cached, amortized, and per-phase work, and identify which costs are already included in the reported interval. Under a matched full-path timing boundary, how much communication time is saved after accounting for the additional work exposed on the critical path, and where does the net benefit disappear?

  1. Generality beyond the evaluated model.

Can you evaluate an additional MoE model or independently collected routing trace with substantially different communication characteristics, such as a different top-k, activation size, or node-pair demand distribution? Please characterize these differences and explain what recalibration is required. This would help determine whether the measured gains and execution-path choices extend beyond the GLM-4.5-Air workload family.

Reviewer_ywE1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RailWave improves MoE expert-parallel communication by combining source-local traffic redistribution across network rails, reusable cyclic schedules to limit incast, and a calibrated execution selector. It reports communication speedups up to 5.84× on H800 and 4.36× on H20.

Critical strengths
  • Addresses a practical bottleneck while preserving routing and expert assignments
  • Present understandable design with spatial and temporal mechanisms
  • Results on two GPU platforms and explicit reporting of several limitations are positives
Critical weaknesses
  • main experiments does not establish gains under ordinary training execution.
  • limited scale: contain only 4 nodes and small number of gpus. large scale behavior remain unclear.
  • Main timings exclude CPU planning, index construction, and prepacking, so missing CPU time
  • missing full training throughput
Questions for authors

What speedups remain with original routing traces and a standard full-demand DeepEP baseline, including all preparation costs?

Reviewer_JmMZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims at improving communication speed for Expert Parallel training of Mixture-of-Experts (MoE) models. The authors study multi-GPU multi-node distributed training settings, which is common in medium-to-large-scale machine learning practice. In particular, the authors study rail-optimized clusters, where only the GPUs of the same index on nodes are directly connected together through fabric. For example, GPU 0 on node 0 is connected with GPU 0 on node 1, and so forth.

In rail-optimized clusters, the authors identify two difficulties through experiments: (1) rail utilization can be imbalanced (having hot rails), and (2) multiple nodes may send to a single node (incast or having hot nodes). For the former, the authors propose leveraging high-speed intra-node NVLink by temporarily moving extra payloads onto the GPUs with less utilized rails. The authors call this RailBalance. For the latter, the authors leverage a permutation schedule and split the payload into waves, such that each node is receiving data from at most one node. The two approaches can happen jointly. The actual execution path is determined by a light-weight selector. As a whole, the authors call this communication layer RailWave.

The authors implement RailWave on top of DeepEP, and use replay to evaluate the speed increase. The replay is based on expert-routing statistics of a GLM-4.5-Air training run. The authors conduct an ablation study to test RailBalance and their permutation schedule. Under the most advantageous setting, dispatch-combine latency is reduced by 5.84 times over Native on H800.

Critical strengths
  • The authors identified real engineering problems through experiments. Specifically, (1) rail utilization can be imbalanced (having hot rails), and (2) multiple nodes may send to a single node (incast or having hot nodes). This contributed positively to the reviewer's overall assessment because machine learning systems research should at least clearly identify a real engineering problem.
  • The solution proposed by the authors is reasonable and is able to balance rail utilization and remove incast. This contributed positively to the reviewer's overall assessment because this is effective problem solving.
Critical weaknesses
  • Limited Informativeness. The identification of the problem with rail utilization and incast is real but not new. Both spatial shaping and temporal shaping already existed.
  • Limited Soundness.
    • The main plots only measured dispatch-combine latency. It does not contain a full picture profiling (at least for the MoE module forward+backward, show the percentage of dispatch-combine latency).
    • It is unclear why FAST's communication latency is significantly higher than RailWave and even higher than Native in Figure 5 (b). The main downside of FAST is "synchronizing that information and rebuilding schedules can incur substantial overhead as routing changes". Is "synchronizing that information and rebuilding schedules" part of FAST's communication latency? If not, is the discrepancy due to the released FAST implementation is implemented differently? If so, a side-by-side comparison of RailWave's and FAST's procedure is needed to ensure this is a fair comparison. Especially since CPU latency is not counted for RailWave.
    • There is not enough information about the GLM-4.5-Air training run. Notably, expert load could be especially imbalanced at the beginning of a training run, and rail imbalance can increase with expert load imbalance (Figure 2 (b)). More information about the GLM-4.5-Air training run is needed to understand if the reported gains (of up to 5.84x) are representative of typical training.
Questions for authors
  • More information about the GLM-4.5-Air training run is needed to understand if the reported gains (of up to 5.84x) are representative of typical training. For the replay in the current paper, what is the number of iterations? Did the load balance loss start to take effect? Did the expert load enter steady-state?
  • A full picture profiling is needed, at least for the MoE module forward+backward, show the percentage of dispatch-combine latency. What is the percentage of dispatch-combine latency compared to the MoE module forward+backward?
    • Clarification: In abstract, the authors mentioned irregular all-to-all communication is a major bottleneck, but is the extra latency caused by rail utilization and incast a major bottleneck?
  • (Same as Weaknesses) It is unclear why FAST's communication latency is significantly higher than RailWave and even higher than Native in Figure 5 (b). The main downside of FAST is "synchronizing that information and rebuilding schedules can incur substantial overhead as routing changes". Is "synchronizing that information and rebuilding schedules" part of FAST's communication latency? If not, is the discrepancy due to the released FAST implementation is implemented differently? If so, a side-by-side comparison of RailWave's and FAST's procedure is needed to ensure this is a fair comparison. Especially since CPU latency is not counted for RailWave.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.