- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies expert-load imbalance in extremely sparse Mixture-of-Experts LLMs. It reinterprets two auxiliary-loss-free balancing methods through a generalized PID-control lens. Building on this view, the authors propose ID Balancing, which needs only O(E) token counts per layer and adds no gradient to the LM objective. Experiments on 18.9B models show lower worst-case and final-stage backbone MaxVio and lower MinVio than auxiliary loss, DeepSeek loss-free, and Quantile Balancing.
Critical strengths
- The method is simple and cheap to implement.
Critical weaknesses
-
Load balance by itself shouldn't be the goal. What really matters is whether it leads to more efficient training or a better model. On model quality, the LM loss gain over the best baseline is below 0.002 and the average downstream gap is below 0.1 points, which looks like noise. On efficiency, the paper doesn't report any clear speedup. Most of the baselines don't show loss spikes or other instability in the experiments, so the "stable training" part of the title isn't well supported.
-
The core idea overlaps heavily with prior public work. The gradient descent section of Jianlin Su's blog post from February 2026 (https://spaces.ac.cn/archives/11619) already shows that the gradient of the dual objective w.r.t. the bias is the load error, and that running SignSGD on it is exactly DeepSeek's loss-free update. Most of the improvement in this paper comes from the magnitude-aware integral term, and that term is just plain gradient descent on the same dual objective instead of SignSGD. This change was explicitly suggested in the first comment under that post (Kevin Yin, Feb 22, 2026: "A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory.").
Questions for authors
- Are there results showing that the improved load metrics translate into gains beyond noise, e.g. faster training or better final model quality?
- Could the authors clarify how this work relates to the blog post above, and how ID Balancing differs from the method described there?
8 comments
Responses to the Reviewer 5gBD [2/3]
We further fine-tune the continued-pretraining checkpoints on 1.32M supervised examples and evaluate them using a multi-domain perplexity benchmark containing 600 examples across six domains. Lower PPL indicates better predictive performance under teacher forcing. As shown below, ID Balancing achieves the best PPL after SFT.
| Method | Model Size | PPL↓ |
|---|---|---|
| DeepSeek loss-free | 3.2B / 69.9B | 3.83 |
| Quantile Balancing | 3.2B / 69.9B | 3.84 |
| ID Balancing (Ours) | 3.2B / 69.9B | 3.81 |
(3) ID Balancing provides a slight improvement in training stability:
Current mainstream models rarely exhibit spikes in LM loss during training. Therefore, we focus on spikes in grad-norm instead. During the continued pretraining stage, we define a spike as a training step with grad-norm > τ, where a single shared threshold is used:
τ = 10 × (mean grad-norm of DeepSeek loss-free + mean grad-norm of Quantile Balancing + mean grad-norm of ID Balancing) / 3.
As shown in the table below, the stability of ID Balancing is comparable to that of Quantile Balancing and superior to that of the DeepSeek loss-free.
| Stage | Model Size | Shared threshold τ | DeepSeek loss-free | Quantile Balancing | ID Balancing |
|---|---|---|---|---|---|
| Continued-pretraining | 3.0B / 24.8B | 0.259054 | 2 | 3 | 2 |
| Continued-pretraining | 3.2B / 69.9B | 0.281506 | 4 | 1 | 2 |
Q2: Could the authors clarify how this work relates to Jianlin Su's blog post from February 2026 (https://spaces.ac.cn/archives/11619) ?
A2: We sincerely appreciate your reminder regarding the comment related to our work in the blog discussion section. We address this issue from three perspectives. Firstly, we provide a detailed analysis of the approach discussed in the blog and further derive a new variant inspired by the comment, showing that this approach cannot scale. Secondly, we clarify the broader significance of our work beyond simply introducing a formal method. Finally, we would like to emphasize that a comment in a discussion thread does not invalidate the contributions of our work.
(1) Following the blog, we present a detailed derivation using our manuscript's notation. Substituting and fixing the current score matrix and thresholds , the objective becomes
\underbrace{ \sum_{i=1}^{m}\sum_{j=1}^{E} \max\left(0,S_{i,j}^{(t)}+b_j-\alpha_i^{(t)}\right) -\frac{mK}{E}\sum_{j=1}^{E}b_j }_{\ell_t(\mathbf b;\boldsymbol\alpha^{(t)})}.$$ The objective is convex and piecewise linear. At differentiable points, an ordinary gradient exists. More generally, choosing a zero hinge contribution at equality gives the following subgradient: $$\xi_j^{(t)} :=\sum_{i=1}^{m} \mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] -\frac{mK}{E} \;\in\; \partial_{b_j}\ell_t\left(\mathbf b^{(t)};\boldsymbol\alpha^{(t)}\right). \tag{16E}$$ Here $\mathbf 1[\cdot]$ denotes the indicator function. **If each training step performs one internal bias update ($T_{{inner}}=1$), and the $K$-th and $(K+1)$-th largest entries differ in every row of $\mathbf S^{(t)}+\mathbf b^{(t)}$, the indicator sum in Eq. (16E) equals the actual assignment count for the current expert.** Specifically, let $u_{i,j}^{(t)}=S_{i,j}^{(t)}+b_j^{(t)}$, with $u_{i,(r)}^{(t)}$ denoting the $r$-th largest entry in row $i$. Both routing and threshold computation use the same pre-update bias $\mathbf b^{(t)}$, with $$\alpha_i^{(t)}=u_{i,(K+1)}^{(t)}, \qquad u_{i,(K)}^{(t)}>u_{i,(K+1)}^{(t)}, \qquad 1\le K<E.$$ The experts strictly above the threshold are then exactly the token's Top-$K$ experts: $$\mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] =\mathbf 1\left[j\in\mathcal T(\mathbf x_i)\right].$$ Summing over all tokens in the current batch gives $$\sum_{i=1}^{m} \mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] =\sum_{i=1}^{m}\mathbf 1\left[j\in\mathcal T(\mathbf x_i)\right].$$ Substituting this identity into Eq. (16E), we obtain $$\xi_j^{(t)} =\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}.$$ To recover the DeepSeek loss-free update, apply signSGD: $$b_j^{(t+1)} =b_j^{(t)}-\eta\operatorname{sign}\left(\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}\right).$$ **Following the first comment (Kevin Yin, Feb 22, 2026: "A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory.") under that blog.** We obtain the SGD-based gradient descent version as follows: $$b_j^{(t+1)} =b_j^{(t)}-\eta\left(\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}\right).$$Responses to the Reviewer 5gBD [3/3]
However, the SGD-based gradient descent version is not scalable, since the magnitude of its updates depends on the batch size, TopK selection, and the number of experts. In contrast, the signSGD version is scalable because it removes the dependence on batch size, TopK, and the number of experts. Similarly, in our method, denotes the normalized load error, whose scale remains invariant to changes in batch size, TopK, or the number of experts E:
We tuned the SGD-based gradient descent version under the configuration of global batch size 1024, TopK = 10, and E = 768, and then attempted to reuse the same across other settings. As shown in the table below, the SGD-based gradient descent version is highly sensitive to model configurations, while ID Balancing demonstrates strong parameter generalization across different settings.
| Method | GBS | Top-k | E | LM loss | MaxVio Avg. | MinVio Avg. | ||
|---|---|---|---|---|---|---|---|---|
| SGD-based gradient descent version | 1024 | 10 | 768 | 1.099e-7 | - | 1.7128 | 0.5125 | 0.3733 |
| ID Balancing (Ours) | 1024 | 10 | 768 | - | 6e-3 | 1.7118 | 0.5110 | 0.3800 |
| SGD-based gradient descent version | 1024 | 3 | 768 | 1.099e-7 | - | 1.7433 | 1.1498 | 0.6110 |
| ID Balancing (Ours) | 1024 | 3 | 768 | - | 6e-3 | 1.7426 | 0.7931 | 0.5480 |
| SGD-based gradient descent version | 512 | 3 | 768 | 1.099e-7 | - | 1.7485 | 1.6357 | 0.7166 |
| ID Balancing (Ours) | 512 | 3 | 768 | - | 6e-3 | 1.7479 | 0.8873 | 0.6082 |
(2) We would like to clarify that the contribution of our work extends beyond simply proposing an effective method. More importantly, our work provides a unified control-theoretic perspective on existing load-balancing approaches and systematically studies their behavior. Specifically, our contributions include:
- A unified interpretation of both Quantile Balancing and the DeepSeek loss-free method through the lens of PID control theory, which naturally motivates the design of ID Balancing.
- A generalized PID formulation for MoE load-balancing feedback mechanisms, providing a new understanding of existing methods.
- The introduction of a concrete worsening-gated derivative controller that improves balancing behavior while maintaining stability.
- A detailed comparative analysis of different balancing strategies across sparse MoE settings.
- A comprehensive empirical evaluation covering balancing quality, model quality, and stability across multiple configurations.
Therefore, the key distinction from the SGD-based suggestion is not simply the removal of the sign function, but the development of a complete temporal control framework together with extensive theoretical and empirical analysis.
(3) Finally, we would like to point out that comparing our work to an idea proposed in a blog comment section is not an appropriate comparison, as there remains a substantial gap between an intuitive idea and a practical, scalable method.
We reproduce the comment below in full:
"Statistics has the trick 'Leave One Out'. For element i in a batch, you can use all-but-i to update the bias β, which will be independent of i (no leakage). A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory. Your algorithm maintains only one step of state, so it relies on having a large and diverse batch. It will be more accurate with an EMA. To prevent the EMA state from growing infinitely, only the terms in a window around the expected quantile are preserved. The extreme terms are merged into 'high' and 'low' buckets. Both EMA + LOO are needed in my experience."
In our view, the central idea of the comment is to incorporate EMA into the Quantile Balancing process.
Responses to the Reviewer 5gBD [1/3]
We sincerely appreciate your thoughtful comments and your recognition that "the method is simple and cheap to implement." Below, we provide detailed responses to your questions.
Q1: Are there results showing that the improved load metrics translate into gains?
A1: First, we explain why load balancing is an important objective. In large-scale MoE pretraining, better load balancing improves both training efficiency and parameter utilization:
We further demonstrate the advantages of ID Balancing in terms of training efficiency, model quality, and training stability.
(1) ID Balancing improves training efficiency:
Firstly, ID Balancing introduces substantially lower computational and communication overhead than Quantile Balancing. Let M be the global token count per update, Mr the local token count on rank r, E the number of experts, K the number of selected experts per token, B the histogram bins per expert, and μ the local microbatch token count. Tokens are sharded across ranks, each with access to all expert scores for its local tokens.
Secondly, improved load balancing reduces computational bubbles and improves end-to-end training efficiency. We measure the actual training time under different settings using BF16 with PP=1 on 128 GPUs. The table below reports the mean seconds per iteration. Percentages in the ID Balancing column indicate time reductions relative to DeepSeek loss-free / Quantile Balancing.
The efficiency gains become more significant for larger models and larger EP configurations. For the 3.2B/69.9B model with sequence length 32k, ID Balancing achieves a 7.22% end-to-end speedup over Quantile Balancing. Since MoE computation typically accounts for only about half of the total training time, this improvement is substantial in practice.
(2) ID Balancing improves model quality while maintaining a more balanced load distribution:
Table 2 compares ID Balancing with other methods under intentionally unfavorable settings. Following DeepSeek-V3, we freeze the bias updates of the loss-free balancing baseline during continued pretraining, and apply the same setting to Quantile Balancing. As shown in Figure 6, ID Balancing still effectively reduces MaxVio with Ki=Kd=6e−6 while introducing only a negligible increase in LM loss. We therefore adopt this configuration in our experiments.
As shown below, ID Balancing achieves substantially lower MaxVio than the other methods while maintaining comparable or slightly better downstream performance.