- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies load balancing in mixture-of-experts (MoE) models. MoE models use a learned router to send tokens to different expert MLPs. This learned routing suffers from a rich-get-richer problem where high performing experts receive more gradient updates causing performance to become even more uneven. As a result, researchers have investigated several techniques for encouraging even routing across all the experts, or load balancing.
One set of load balancing techniques manipulates bias terms in the router's softmax to evenly distribute the load. These bias terms are not learned via gradient descent but rather set using other rules. The paper frames this bias tuning as a feedback control problem and proposes to solve it with a proportional-integral-derivative (PID) controller. This perspective both unifies and sheds new light on prior work, and leads to a new load balancing method which achieves more even distributions at similar language modeling performance as prior techniques.
Critical strengths
The paper's strengths are:
- The PID controller framing provides a new and interesting perspective on the load-balancing problem.
- The unification of prior methods under PID control helps compare them and better understand their different behaviors.
- The method empirically achieves the authors' stated goal of better balancing tokens across experts.
Critical weaknesses
The paper really only has one major weakness: they evaluate methods based on how well they balance tokens across the experts, but from a practical perspective these metrics are secondary. For the same language modeling performance, "dead" experts could actually be a good thing because you can prune them from the model to reduce its memory and storage requirements.
The original motivation for load balancing was to ensure good learning, not as an end in itself. Thus, how evenly a technique distributes tokens across experts seems more like an intrinsic evaluation or an analysis of the method rather than a primary evaluation.
Questions for authors
One thing I was curious about, since the PID (I) controller modifies biases in the softmax, and the softmax is invariant to shifts in the logits' mean, does it grow the biases without bound in practice and is there a benefit to periodically recentering the biases (i.e., for numerical stability)?
Since the motivation behind load-balancing is to improve the learning early on but it also seems to hurt expert specialization, one interesting direction for future work could be to remove the load-balancing biases partway through training or at inference time and to see if that does lead to a performance improvement.
2 comments
We thank the reviewer for the careful reading and the thoughtful questions. We address the questions raised by the reviewer below.
Q1. Bias growth and recentering: The bias is added to the post-softmax scores and is used only for Top- selection (Eq. 2). Top- selection is unchanged when the same constant is added to all biases, which is the shift the reviewer refers to. For PID (I) the mean of the biases is exactly zero at every step. With dropless routing , which gives and hence . The sign update of ALF-LB lacks this property, and its mean bias drifts during training. The table reports the maximum over training of the mean bias and of , with pretraining values from the 1→1 scenario averaged over 6 seeds.
| Setting | ALF-LB | PID (I) | ALF-LB | PID (I) |
|---|---|---|---|---|
| Pretraining, | 5.91 | 3.8e-7 | 6.02 | 0.86 |
| Pretraining, | 7.22 | 1.7e-7 | 7.28 | 0.86 |
| Pretraining, | 9.17 | 7.9e-8 | 9.20 | 0.81 |
| CPT, cold start | 1.63 | 1.4e-8 | 1.64 | 0.14 |
| CPT, warm start | 2.76 | 2.5e-8 | 2.78 | 0.12 |
The biases of PID (I) stay below 1 in magnitude throughout training. The drift of ALF-LB shifts all biases by the same amount, which leaves the Top- selection unchanged in exact arithmetic. All biases are stored and applied in fp32, where the rounding error at the drifted values is about 1e-6 and negligible next to the gaps between scores. Periodic recentering would therefore leave the biases of PID (I) unchanged. We will add this analysis to the appendix.
Q2. Removing the bias: We ran the inference-time version of this experiment in Appendix D.1. Setting the bias to zero at the end of continual pretraining lowers PPL-CPT by 0.9–1.8% and PPL-Annealing by 2.2–2.6% for both bias-based balancers (Table 15). We measure domain specialization by the mutual information between the input domain and the selected expert. Removing the bias recovers more than half of the specialization that the bias costs relative to Vanilla (Table 14). This agrees with the reviewer's intuition. The bias reduces specialization mainly at selection time, while the gate weights keep most of their domain preference. Without the bias, the load returns most of the way to the imbalance of Vanilla, which matters for expert-parallel deployment. For removal partway through training, our continual pretraining offers a related data point. OLMoE was pretrained with an auxiliary loss, and our Vanilla runs drop it at the start of continual pretraining. Compared with keeping it (Aux-loss), dropping it leaves both perplexities and the downstream scores essentially unchanged, while the fraction of under-trained experts rises from 1.6% to 29.1% in the cold start and from 6.9% to 27.7% in the warm start (Table 2). Appendix D.1 already notes that whether this trade holds at larger scale and in downstream evaluation remains open, and we will extend that discussion to removal partway through training.
We thank the reviewer for the careful summary and for recognizing the PID view of load balancing, the unification of existing balancers, and the improved balance of PID (I). We address the weaknesses raised by the reviewer below.
W1. Balance as a primary metric. We agree that balance is not an end in itself. Its value, however, extends beyond good learning and concerns both overloaded and underloaded experts. An overloaded expert slows down MoE computation. Under expert parallelism the device holding the most loaded expert sets the step time, and under capacity-limited routing the overflow tokens are dropped. MaxVio measures this side and is the primary metric of ALF-LB, which we follow. It is also an affine function of the maximum routing imbalance that Thérien et al. (2025) use as a latency proxy (Section 5 and Appendix D.3). An underloaded expert receives too little training and leaves part of the model capacity unused, which DER and UTR measure. This matters more as MoEs scale by adding experts at a fixed number of active ones. In our pretraining, the dead-expert ratio of Vanilla grows from 7.4% to 12.3% and 16.3% as E grows from 16 to 32 and 64 (Table 1, 1→1).
W2. Expert pruning: We agree that pruning persistently idle experts reduces memory, but we see it as a separate direction with a different objective. Load balancing concerns how tokens are distributed over a given set of experts, which determines throughput and whether every expert is trained. Expert pruning concerns how many experts are kept, which determines the memory footprint. Our setting is distribution shift, where the experts that sit idle change with the data. Under Vanilla and Aux-loss, some experts that are under-loaded before the shift become heavily loaded after it (Figure 3). A pruning decision based on usage therefore holds only for the distribution on which it was made. Removing experts ahead of a shift, as in continual pretraining, would discard capacity that the new distribution may need.
On the soundness score. We read the main concern as being about which metric should be primary, rather than about an error in the analysis or the experiments. If this reading is correct, we hope the clarifications above address it, and we would be grateful if the reviewer would reconsider the soundness score.