- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies load balancing for group-limited MoE routing. It characterizes the optimality of the released DeepSeek's selection rule and develops HYPERSPACE, which is an LP-based router with a batch-size-independent load allowance and a routing-score guarantee. Experiments confirm tight load control without dropping assignments, but show quality losses and substantial routing overhead.
Critical strengths
-
The theoretical contribution is important and interesting. The paper connects route patterns, group prices and balanced routing through an LP formulation. The optimality characterization, rounding guarantee and impossibility results clarify both the capabilities and limitations of group-limited routing.
-
The load guarantees are supported empirically. HYPERSPACE stays within four assignments of the target across the evaluated batches, including recorded user-traffic replay. It requires no retraining and achieves lower NLL than group dropping in all tested model-domain pairs.
Critical weaknesses
- The practical benefit remains unproven. Tab 5 reports 26.9 ms per layer for a 64-token routing step, vs. 0.16ms for the released router, which is approx. 170× slower. This measurement excludes the placement flow. However, the LP runs on CPU, so the result establishes the cost of the current implementation rather than an inherent limitation of the method. The paper provides neither evidence of that improvement nor end-to-end expert-parallel measurements showing that better balance offsets routing overhead. I do believe that an optimized implementation could substantially improve the paper.
The evaluation also lacks LPLB, which the paper cites. LPLB uses per-batch LP optimization to redistribute assignments among replicas while preserving logical expert choices. Its replication requirements make it a different trade-off, but comparison under a stated memory budget would help establish when changing routes is preferable to redistributing work.
- The quality-balance trade-off needs stronger justification. HYPERSPACE improves load balance at a noticeable quality cost relative to the released router, and its advantage over group dropping is not consistent across downstream tasks (Tab 4, 6). Without end-to-end timing, it remains unclear whether these losses buy a worthwhile latency improvement. I believe that quality evaluation on mixed-domain traffic would strengthen the evidence beyond single-domain NLL batches.
Questions for authors
-
Can an optimized GPU implementation deliver an latency/throughput improvement under expert parallelism, including placement and communication costs?
-
How does routing time scale from decoding to large prefill batches?
-
How does HYPERSPACE compare with LPLB at comparable replication-memory budgets?