acceptodds
Under review as a conference paper at ICLR 2027

ID Balancing: Stable Training of Extremely Sparse MoE via PID-Based Load Control

Abstract

Scaling Large Language Models (LLMs) via Mixture-of-Experts (MoE) enables massive parameter growth with nearly constant per-token computation. However, further scaling the parameter count requires increasingly sparse routing, where expert load imbalance becomes more severe. This imbalance reduces parameter utilization and training efficiency, and can undermine training stability, becoming a bottleneck to reliable scaling. In this work, we unify two representative auxiliary-loss-free methods as incomplete Proportional-Integral-Derivative (PID) controllers: DeepSeek's loss-free method acts as a fixed-step integral controller, while Kimi K3's Quantile Balancing functions as a generalized proportional controller. Building on this control perspective, we propose ID Balancing, an Integral-Derivative controller. It scales its integral term with load error and activates its derivative term only when imbalance worsens, enabling stronger corrections for large or worsening errors and smaller updates near balance. Evaluated across Top-, Top-, and Top- routing over experts, ID Balancing reduces worst-case backbone MaxVio and training-average backbone MinVio by over % and %, respectively, relative to the best baselines in the Top- setting. When scaling parameters from B to B (Top--of-), its worst-case backbone MaxVio remains nearly unchanged and is approximately % lower than that of the auxiliary-loss baseline. ID Balancing also maintains competitive language-modeling and downstream performance. The advantages of ID Balancing grow as sparsity increases, making it a promising solution for scaling larger, sparser MoE models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts Models, Large Language Models, Training Stability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5gBDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies expert-load imbalance in extremely sparse Mixture-of-Experts LLMs. It reinterprets two auxiliary-loss-free balancing methods through a generalized PID-control lens. Building on this view, the authors propose ID Balancing, which needs only O(E) token counts per layer and adds no gradient to the LM objective. Experiments on 18.9B models show lower worst-case and final-stage backbone MaxVio and lower MinVio than auxiliary loss, DeepSeek loss-free, and Quantile Balancing.

Critical strengths
  1. The method is simple and cheap to implement.
Critical weaknesses
  1. Load balance by itself shouldn't be the goal. What really matters is whether it leads to more efficient training or a better model. On model quality, the LM loss gain over the best baseline is below 0.002 and the average downstream gap is below 0.1 points, which looks like noise. On efficiency, the paper doesn't report any clear speedup. Most of the baselines don't show loss spikes or other instability in the experiments, so the "stable training" part of the title isn't well supported.

  2. The core idea overlaps heavily with prior public work. The gradient descent section of Jianlin Su's blog post from February 2026 (https://spaces.ac.cn/archives/11619) already shows that the gradient of the dual objective w.r.t. the bias is the load error, and that running SignSGD on it is exactly DeepSeek's loss-free update. Most of the improvement in this paper comes from the magnitude-aware integral term, and that term is just plain gradient descent on the same dual objective instead of SignSGD. This change was explicitly suggested in the first comment under that post (Kevin Yin, Feb 22, 2026: "A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory.").

Questions for authors
  1. Are there results showing that the improved load metrics translate into gains beyond noise, e.g. faster training or better final model quality?
  2. Could the authors clarify how this work relates to the blog post above, and how ID Balancing differs from the method described there?
8 comments
AuthorsOpenReview ↗

Responses to the Reviewer 5gBD [1/3]

We sincerely appreciate your thoughtful comments and your recognition that "the method is simple and cheap to implement." Below, we provide detailed responses to your questions.

Q1: Are there results showing that the improved load metrics translate into gains?

A1: First, we explain why load balancing is an important objective. In large-scale MoE pretraining, better load balancing improves both training efficiency and parameter utilization:

  • For MaxVio, when expert parallelism (EP) is enabled, the training bottleneck is determined by the GPU with the highest workload. Better load balancing distributes computation more evenly across GPUs, reducing computational bubbles and improving overall training efficiency.
  • For MinVio, poor load balancing may lead to dead or undertrained experts that receive very few training tokens. This reduces the effective model capacity and can hurt final model quality.

We further demonstrate the advantages of ID Balancing in terms of training efficiency, model quality, and training stability.

(1) ID Balancing improves training efficiency:

Firstly, ID Balancing introduces substantially lower computational and communication overhead than Quantile Balancing. Let MM be the global token count per update, MrM_r the local token count on rank rr, EE the number of experts, KK the number of selected experts per token, BB the histogram bins per expert, and μ\mu the local microbatch token count. Tokens are sharded across ranks, each with access to all expert scores for its local tokens.

MethodAdditional computationAdditional memoryBalancing communication
ID Balancing (Ours)O(MrK+E)O(M_rK+E) per rankO(E)O(E)All-reduce of EE load counts
DeepSeek loss-freeO(MrK+E)O(M_rK+E) per rankO(E)O(E)All-reduce of EE load counts
Quantile Balancing, direct gather, one iterationO(ME)O(ME) global selection work; O(MElog⁡M)O(ME\log M) if expert-side values are fully sortedO(ME)O(ME) at the gathering rankGather O(ME)O(ME) margins in total, then broadcast EE biases
Histogram Quantile Balancing (used in Kimi K3)O(MrE+EB)O(M_rE+EB) per rankO(EB+μ+E)O(EB+\mu+E) with streaming margin processingAll-reduce of EBEB histogram counts

Secondly, improved load balancing reduces computational bubbles and improves end-to-end training efficiency. We measure the actual training time under different settings using BF16 with PP=1 on 128 GPUs. The table below reports the mean seconds per iteration. Percentages in the ID Balancing column indicate time reductions relative to DeepSeek loss-free / Quantile Balancing.

Model SizeTPDPEPGBSMBSSequence lengthDeepSeek loss-free (s)Histogram Quantile Balancing (s)ID Balancing (s)
3.0B / 24.8B112882048440964.9264.9644.890 (↓0.73% / ↓1.49%)
3.0B / 24.8B26482561327687.1787.3037.142 (↓0.51% / ↓2.21%)
3.2B / 69.9B112882048240968.3598.5268.236 (↓1.47% / ↓3.40%)
3.2B / 69.9B4326425613276818.68919.59018.176 (↓2.74% / ↓7.22%)

The efficiency gains become more significant for larger models and larger EP configurations. For the 3.2B/69.9B model with sequence length 32k, ID Balancing achieves a 7.22% end-to-end speedup over Quantile Balancing. Since MoE computation typically accounts for only about half of the total training time, this improvement is substantial in practice.

(2) ID Balancing improves model quality while maintaining a more balanced load distribution:

Table 2 compares ID Balancing with other methods under intentionally unfavorable settings. Following DeepSeek-V3, we freeze the bias updates of the loss-free balancing baseline during continued pretraining, and apply the same setting to Quantile Balancing. As shown in Figure 6, ID Balancing still effectively reduces MaxVio with Ki=Kd=6e−6K_i= K_d=6e-6 while introducing only a negligible increase in LM loss. We therefore adopt this configuration in our experiments.

As shown below, ID Balancing achieves substantially lower MaxVio than the other methods while maintaining comparable or slightly better downstream performance.

MethodModel SizeMaxVio↓MinVio↓Avg. performance
DeepSeek loss-free3.0B / 24.8B2.3275280.79720055.55
Quantile Balancing3.0B / 24.8B2.2584030.81057554.90
ID Balancing (Ours)3.0B / 24.8B0.9813930.73286455.61
DeepSeek loss-free3.2B / 69.9B7.1690720.95781158.53
Quantile Balancing3.2B / 69.9B6.9968230.95418858.48
ID Balancing (Ours)3.2B / 69.9B1.4942740.92072458.61
AuthorsOpenReview ↗

Responses to the Reviewer 5gBD [2/3]

We further fine-tune the continued-pretraining checkpoints on 1.32M supervised examples and evaluate them using a multi-domain perplexity benchmark containing 600 examples across six domains. Lower PPL indicates better predictive performance under teacher forcing. As shown below, ID Balancing achieves the best PPL after SFT.

MethodModel SizePPL↓
DeepSeek loss-free3.2B / 69.9B3.83
Quantile Balancing3.2B / 69.9B3.84
ID Balancing (Ours)3.2B / 69.9B3.81

(3) ID Balancing provides a slight improvement in training stability:

Current mainstream models rarely exhibit spikes in LM loss during training. Therefore, we focus on spikes in grad-norm instead. During the continued pretraining stage, we define a spike as a training step with grad-norm > τ, where a single shared threshold is used:

τ = 10 × (mean grad-norm of DeepSeek loss-free + mean grad-norm of Quantile Balancing + mean grad-norm of ID Balancing) / 3.

As shown in the table below, the stability of ID Balancing is comparable to that of Quantile Balancing and superior to that of the DeepSeek loss-free.

StageModel SizeShared threshold τDeepSeek loss-freeQuantile BalancingID Balancing
Continued-pretraining3.0B / 24.8B0.259054232
Continued-pretraining3.2B / 69.9B0.281506412

Q2: Could the authors clarify how this work relates to Jianlin Su's blog post from February 2026 (https://spaces.ac.cn/archives/11619) ?

A2: We sincerely appreciate your reminder regarding the comment related to our work in the blog discussion section. We address this issue from three perspectives. Firstly, we provide a detailed analysis of the approach discussed in the blog and further derive a new variant inspired by the comment, showing that this approach cannot scale. Secondly, we clarify the broader significance of our work beyond simply introducing a formal method. Finally, we would like to emphasize that a comment in a discussion thread does not invalidate the contributions of our work.

(1) Following the blog, we present a detailed derivation using our manuscript's notation. Substituting βj=−bj\beta_j=-b_j and fixing the current score matrix S(t)\mathbf S^{(t)} and thresholds α(t)\boldsymbol\alpha^{(t)}, the objective becomes

\underbrace{ \sum_{i=1}^{m}\sum_{j=1}^{E} \max\left(0,S_{i,j}^{(t)}+b_j-\alpha_i^{(t)}\right) -\frac{mK}{E}\sum_{j=1}^{E}b_j }_{\ell_t(\mathbf b;\boldsymbol\alpha^{(t)})}.$$ The objective is convex and piecewise linear. At differentiable points, an ordinary gradient exists. More generally, choosing a zero hinge contribution at equality gives the following subgradient: $$\xi_j^{(t)} :=\sum_{i=1}^{m} \mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] -\frac{mK}{E} \;\in\; \partial_{b_j}\ell_t\left(\mathbf b^{(t)};\boldsymbol\alpha^{(t)}\right). \tag{16E}$$ Here $\mathbf 1[\cdot]$ denotes the indicator function. **If each training step performs one internal bias update ($T_{{inner}}=1$), and the $K$-th and $(K+1)$-th largest entries differ in every row of $\mathbf S^{(t)}+\mathbf b^{(t)}$, the indicator sum in Eq. (16E) equals the actual assignment count for the current expert.** Specifically, let $u_{i,j}^{(t)}=S_{i,j}^{(t)}+b_j^{(t)}$, with $u_{i,(r)}^{(t)}$ denoting the $r$-th largest entry in row $i$. Both routing and threshold computation use the same pre-update bias $\mathbf b^{(t)}$, with $$\alpha_i^{(t)}=u_{i,(K+1)}^{(t)}, \qquad u_{i,(K)}^{(t)}>u_{i,(K+1)}^{(t)}, \qquad 1\le K<E.$$ The experts strictly above the threshold are then exactly the token's Top-$K$ experts: $$\mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] =\mathbf 1\left[j\in\mathcal T(\mathbf x_i)\right].$$ Summing over all tokens in the current batch gives $$\sum_{i=1}^{m} \mathbf 1\left[S_{i,j}^{(t)}+b_j^{(t)}-\alpha_i^{(t)}>0\right] =\sum_{i=1}^{m}\mathbf 1\left[j\in\mathcal T(\mathbf x_i)\right].$$ Substituting this identity into Eq. (16E), we obtain $$\xi_j^{(t)} =\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}.$$ To recover the DeepSeek loss-free update, apply signSGD: $$b_j^{(t+1)} =b_j^{(t)}-\eta\operatorname{sign}\left(\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}\right).$$ **Following the first comment (Kevin Yin, Feb 22, 2026: "A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory.") under that blog.** We obtain the SGD-based gradient descent version as follows: $$b_j^{(t+1)} =b_j^{(t)}-\eta\left(\sum_{i=1}^{m}\mathbf 1 \left[j\in\mathcal T(\mathbf x_i)\right] -\frac{mK}{E}\right).$$
AuthorsOpenReview ↗

Responses to the Reviewer 5gBD [3/3]

However, the SGD-based gradient descent version is not scalable, since the magnitude of its updates depends on the batch size, TopK selection, and the number of experts. In contrast, the signSGD version is scalable because it removes the dependence on batch size, TopK, and the number of experts. Similarly, in our method, ee denotes the normalized load error, whose scale remains invariant to changes in batch size, TopK, or the number of experts E:

b~j(t+1)=bj(t)+Kiej(t)+Kdgj(t)Δej(t),\widetilde b_j^{(t+1)}=b_j^{(t)}+{K_i e_j^{(t)}}+{K_d g_j^{(t)}\Delta e_j^{(t)}},

bj(t+1)=b~j(t+1)−1E∑r=1Eb~r(t+1).b_j^{(t+1)}=\widetilde b_j^{(t+1)}-\frac1E\sum_{r=1}^{E}\widetilde b_r^{(t+1)}.

We tuned the SGD-based gradient descent version under the configuration of global batch size 1024, TopK = 10, and E = 768, and then attempted to reuse the same η\eta across other settings. As shown in the table below, the SGD-based gradient descent version is highly sensitive to model configurations, while ID Balancing demonstrates strong parameter generalization across different settings.

MethodGBSTop-kEη\etaKi,KdK_i,K_dLM lossMaxVio Avg.MinVio Avg.
SGD-based gradient descent version1024107681.099e-7-1.71280.51250.3733
ID Balancing (Ours)102410768-6e-31.71180.51100.3800
SGD-based gradient descent version102437681.099e-7-1.74331.14980.6110
ID Balancing (Ours)10243768-6e-31.74260.79310.5480
SGD-based gradient descent version51237681.099e-7-1.74851.63570.7166
ID Balancing (Ours)5123768-6e-31.74790.88730.6082

(2) We would like to clarify that the contribution of our work extends beyond simply proposing an effective method. More importantly, our work provides a unified control-theoretic perspective on existing load-balancing approaches and systematically studies their behavior. Specifically, our contributions include:

  • A unified interpretation of both Quantile Balancing and the DeepSeek loss-free method through the lens of PID control theory, which naturally motivates the design of ID Balancing.
  • A generalized PID formulation for MoE load-balancing feedback mechanisms, providing a new understanding of existing methods.
  • The introduction of a concrete worsening-gated derivative controller that improves balancing behavior while maintaining stability.
  • A detailed comparative analysis of different balancing strategies across sparse MoE settings.
  • A comprehensive empirical evaluation covering balancing quality, model quality, and stability across multiple configurations.

Therefore, the key distinction from the SGD-based suggestion is not simply the removal of the sign function, but the development of a complete temporal control framework together with extensive theoretical and empirical analysis.

(3) Finally, we would like to point out that comparing our work to an idea proposed in a blog comment section is not an appropriate comparison, as there remains a substantial gap between an intuitive idea and a practical, scalable method.

We reproduce the comment below in full:

"Statistics has the trick 'Leave One Out'. For element i in a batch, you can use all-but-i to update the bias β, which will be independent of i (no leakage). A properly-tuned Loss-Free algorithm would avoid the sign function, and instead use SGD, a meaningful improvement. This is standard in control theory. Your algorithm maintains only one step of state, so it relies on having a large and diverse batch. It will be more accurate with an EMA. To prevent the EMA state from growing infinitely, only the terms in a window around the expected quantile are preserved. The extreme terms are merged into 'high' and 'low' buckets. Both EMA + LOO are needed in my experience."

In our view, the central idea of the comment is to incorporate EMA into the Quantile Balancing process.

Reviewer_qCFdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses expert load imbalance in extremely sparse MoE training. It interprets existing auxiliary-loss-free balancing methods through a generalized PID framework and proposes ID Balancing, combining magnitude-aware integral updates with derivative corrections activated only when imbalance worsens. The method adjusts expert-selection biases without introducing balancing gradients into the language-modeling objective. Experiments cover multiple routing sparsities, models up to 69.9B total parameters, and downstream evaluation after 560B training tokens. The results demonstrate improved backbone load control while maintaining competitive language-modeling and downstream performance.

Critical strengths
  1. Clear motivation and lightweight design. The method addresses a practical obstacle to sparse MoE scaling through a simple bias update requiring only expert token-count feedback.

  2. Substantial empirical validation. Experiments span multiple sparsities and model configurations, with large-scale training and nine downstream benchmarks covering knowledge, mathematics, reasoning, multilingual understanding, and code generation.

  3. Strong load-control results. Under Top-3-of-768 routing, ID Balancing substantially reduces worst-case backbone overload and sustained underload. The component analyses and training trajectories provide useful insights into the controller’s behavior.

Critical weaknesses
  1. Gain transferability needs further characterization. The shared default gains work across several configurations, which is encouraging. However, the detailed sensitivity study is limited to Top-3-of-768 at 18.9B. It remains unclear how sensitivity changes with expert count, expert FFN width, and model scale, and when retuning becomes necessary.

  2. Applicability at higher activation ratios remains unclear. The experiments focus on very low expert activation ratios. Evaluation at higher ratios, such as 6.25%–25%, would help determine whether the benefits extend beyond extremely sparse regimes. Reporting both load balance and model quality would clarify the method’s broader applicability.

  3. The individual contributions of I and D could be better isolated. I-only already achieves similar LM loss and full-training backbone balance to I+D, while exhibiting lower MTP overload; D-only results are not reported. Although adding D improves early overload control, comparing I-only, D-only, and I+D on training stability and downstream performance would clarify when their combination is beneficial.

Questions for authors
  1. Recovery from existing imbalance. Can ID Balancing recover an existing MoE checkpoint with persistently underused experts through short continued training? Tracking both load recovery and downstream performance would clarify whether restored balance translates into improved model quality.

  2. Training–inference consistency. How are the balancing biases handled at inference, and could training-induced balancing adversely affect inference-time routing or model quality? Evaluating expert utilization and generation quality under different input distributions would help establish whether the training benefits carry over to inference.

4 comments
AuthorsOpenReview ↗

Responses to the Reviewer qCFd [1/2]

We sincerely appreciate your thoughtful comments and your recognition of the lightweight design and strong empirical performance of our work. We are also encouraged that you found the generalized PID perspective informative and that ID Balancing achieves effective load control under extremely sparse MoE settings while maintaining competitive model performance. Below, we provide detailed responses to your questions and concerns.

Q1: Gain transferability needs further characterization.

A1: We conduct additional experiments and find that ID Balancing is highly robust across different model configurations. Since the control target is a normalized load-balancing metric, the method automatically adjusts the bias update magnitude according to the severity of the imbalance. As a result, the same hyperparameter setting generalizes well across a wide range of model settings.

As shown in the table below, for ID Balancing, both KiK_i and KdK_d are fixed at 6e-3. Under different configurations, this setting consistently achieves strong load-balancing performance.

MethodMoE FFN Intermediate SizeTop-KExpertsAct. / TotalLM lossMaxVio Last1kMaxVio Avg.MaxVio WorstMinVio Last1kMinVio Avg.
Auxiliary loss7684641.06B / 3.89B2.01360.23430.25061.020.22320.2343
ID Balancing (Ours)7684641.06B / 3.89B2.01280.17960.19651.770.16330.1729
Auxiliary loss51282563.0B / 24.8B1.67020.42300.529929.330.35350.3666
DeepSeek loss-free51282563.0B / 24.8B1.66960.29430.389930.940.24140.2631
Quantile Balancing51282563.0B / 24.8B1.66970.26640.297213.430.23690.2480
ID Balancing (Ours)51282563.0B / 24.8B1.66930.22010.23468.480.21360.2109
Auxiliary loss512107683.2B / 69.9B1.62361.50611.573275.030.55780.5876
DeepSeek loss-free512107683.2B / 69.9B1.62020.95701.123375.040.40860.4466
Quantile Balancing512107683.2B / 69.9B1.62160.45950.493312.860.35690.3759
ID Balancing (Ours)512107683.2B / 69.9B1.62120.42730.42497.790.35170.3436

It is worth noting that under the Top-10-of-768 setting, the DeepSeek loss-free method achieves the lowest LM loss, but at the cost of significantly worse load balancing. Our interpretation is that weaker load balancing sometimes encourages the model to overfit the training data. However, load balancing is analogous to a minimum learning-rate resource. If the MoE load distribution is not sufficiently balanced in advance, it may exhaust the model’s potential for post-training adaptation. Therefore, during the pre-training stage, we still prefer to maintain stronger load balancing in order to achieve better generalization capability. We discuss this point in more detail in A4.

Q2: Applicability at higher activation ratios remains unclear.

A2: We conduct additional experiments to evaluate higher activation ratios. As shown in the table below, ID Balancing also applies effectively to models with higher activation ratios and continues to maintain its advantages, demonstrating that the method generalizes well beyond extremely sparse settings.

MethodActivation RatiosTop-KExpertsAct. / TotalLM lossMaxVio Last1kMaxVio Avg.MaxVio WorstMinVio Last1kMinVio Avg.
Auxiliary loss6.25%81281.06B / 3.89B2.00890.31470.33801.150.28980.3085
ID Balancing (Ours)6.25%81281.06B / 3.89B2.00890.23140.24621.880.19640.2127
Auxiliary loss25%321281.62B / 3.89B1.99610.28630.31491.090.23340.2483
ID Balancing (Ours)25%321281.62B / 3.89B1.99260.14730.16281.070.13990.1530

Q3: The individual contributions of I and D could be better isolated.

A3: We do not report the experiment involving the derivative term alone because the derivative component in ID Balancing is designed to reinforce correction only when an existing imbalance worsens, rather than continuously correcting the imbalance itself. Using the normalized load error defined in the manuscript, its update is

\qquad g_j^{(t)}=\mathbf 1\!\left[e_j^{(t-1)}\Delta e_j^{(t)}>0\right], \qquad D_j^{(t)}=K_d g_j^{(t)}\Delta e_j^{(t)}.$$ With $K_i=0$, the bias receives only this derivative correction. An expert can remain overloaded or underloaded while its error stops changing.
AuthorsOpenReview ↗

Responses to the Reviewer qCFd [2/2]

We also conduct experiments based on the suggestions, as shown in the table below. Among them, only the derivative term fails to achieve effective load balancing.

MethodKiKdTop-KExpertsAct. / TotalLM lossMaxVio Last1kMaxVio Avg.MaxVio WorstMinVio Last1kMinVio Avg.
D only00.00637680.87B / 18.9B1.74800.79401.517529.130.62330.8295
D only00.01237680.87B / 18.9B1.74350.82941.288755.240.55190.7451
I only0.006037680.87B / 18.9B1.74310.69240.800314.960.52330.5527

As discussed in Table 5 of the paper, the primary role of the derivative term is to enable faster and more stable training during the early stages.

Q4: Recovery from existing imbalance.

A4: We find that ID Balancing can restore overloaded and underloaded experts in existing MoE checkpoints through short-term training. However, restoring underloaded experts actually harms model performance.

In the experiments below, the baseline model is trained for 10k steps. The ID Balancing runs resume from the baseline’s 8k-step checkpoint and apply ID Balancing for the remaining 2k steps. As shown in the table, although load balancing improves substantially, both LM loss and downstream performance slightly degrade.

Top-K/ExpertsMethodIteration time (s, Last1k mean)Final LM lossFinal MinVioFinal MaxVioMMLU
8/256Frozen baseline7.0581.0453750.7972002.32752870.02
8/256ID Balancing from step 80006.8501.0493040.3411390.46270169.75
10/768Frozen baseline18.6401.0118580.9578117.16907272.29
10/768ID Balancing from step 800017.6551.0156740.5422501.10679971.94

This behavior suggests that long-term underloaded experts receive significantly fewer training tokens and therefore remain undertrained compared with other experts. Although ID Balancing can quickly increase their activation probability by adjusting routing biases, these experts still lack sufficient capability, which temporarily hurts overall model quality.

We therefore view MinVio as a limited training resource. If severe underloading continues during pre-training, part of the model capacity may remain insufficiently trained. Recovering these experts later may require substantial continued pre-training with more data, rather than short-term routing correction alone.

In contrast, restoring only overloaded experts has almost no negative effect on model quality. Since ID Balancing includes a zero-center operation, reducing the bias of highly overloaded experts does not significantly disturb the overall bias distribution. As shown below, correcting only MaxVio substantially improves load balance while leaving LM loss nearly unchanged.

Top-K/ExpertsMethodIteration time (s)Final LM lossFinal MinVioFinal MaxVio
5/384Frozen baseline60.7320.576240.96623.4093
5/384Max-overload ID Balancing Recovery59.6630.576430.95971.2130

Q5: Training–inference consistency.

A5: As discussed in A4, constraining MaxVio does not negatively affect model performance. During inference, the EP-rank-level MaxVio directly impacts system efficiency. We evaluate the EP-rank-level MaxVio using real online data. As shown in the table below, under the setting of EP = 8, batch size = 1, and a prefill chunk size of 32K, max-overload ID Balancing reduces the EP-rank-level MaxVio by 11.94%.

Top-K/ExpertsMethodFinal LM lossReal data EP rank level Maxvio
5/384Frozen baseline0.576240.2812
5/384Max-overload ID Balancing Recovery0.576430.2476(-11.94%)

In conclusion, we suggest that, when possible, ID Balancing should be applied from the beginning of pre-training to achieve both better model performance and higher inference efficiency. If training from scratch is not feasible, Max-overload ID Balancing can still be applied during CPT or SFT. This approach improves training and inference efficiency while maintaining model performance.

Thank you for the detailed responses and additional experiments. The results across different model configurations and higher activation ratios strengthen the evidence for the transferability and broader applicability of ID Balancing. The additional D-only experiments and explanation also help clarify the complementary roles of the integral and derivative terms. The observation that improved load balance does not necessarily translate into immediate model-quality gains provides a useful qualification of the method’s practical benefits. These additions address most of my concerns and strengthen the paper’s empirical support. I remain positive about the paper and maintain my rating of 3

AuthorsOpenReview ↗

Sincere Thanks for Your Positive Assessment

We sincerely appreciate your earlier comment: “I maintain my official rating of 3 (Weak acceptance), although my overall assessment is closer to 3.5. I hope the AC will consider this more positive assessment in the final decision.” Your recognition is deeply encouraging to us and represents a meaningful affirmation of the value of our work.

We are continuing to refine and strengthen this work, and we are currently applying the proposed method to the training of a much larger-scale model. If conditions allow, we will do our best to open-source the resulting model and release a technical report, with the hope of contributing back to the community.

Reviewer_HCerOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses a challenge in MoE model architecture, where when we increase the sparsity of the model (decreasing the ratio of active param to the total params), it becomes increasingly challenging to maintain land balancing among all experts. The paper proposes a method, ID balancing, to improve the load balancing among experts.

  1. The method adds a small bias to the expert's score before the top-k choice. And after each step, the bias goes up for experts that got too few tokens and down for those that got too many. So, ID Balancing makes the correction proportional to how bad the imbalance is, and adds an extra push only when the imbalance is getting worse.
  2. The method creates a load error metric per expert using the ideal load and the actual token count received. This update moves the expert’s bias term in proportion to the error metric, so the badly overloaded expert gets pushed down harder. But the paper only uses this bias term per expert (along with the expert’s score) for the topk selection for that token and uses the expert's original score to weight the outputs.
  3. ID balancing also monitors the errors over steps. So when the error is growing (moving farther form zero in overloaded/underloaded case), it adds an additional term to the expert’s bias for faster response.
Critical strengths
  1. The proposed method is a simple addition to any MoE model architecture to address the load imbalance issue during the pretrain run based on the exact load measures per expert. And unlike prior works (like adding constant boost in the case of deepseek paper), this paper adds the boost based on the severity of the imbalance, hence creating faster correction measure.
  2. Computationally, it adds minimal overhead as it just computes the token counts per layer for calculating the error term.
  3. Comprehensive results: table 1 includes the training metrics including lm loss and the load related metrics, table2 includes benchmark results from different domains.
  4. Table 2 shows the evaluation of the proposed method along with the baselines on different domains. ID Balancing is on par with or slightly above the baselines, with gains of only +0.06 and +0.08 over the best loss-free baseline. So quality does not drop.
Critical weaknesses
  1. The results are from a single run. The small gaps in average load and downstream scores (+0.06 to +0.08) is difficult to distinguish apart from the noise.
  2. The method reduces the worst spikes, but it does not balance the load better on an average. The average overload from the result table 1 is not consistent. backbone MaxVio Avg):
  • Top-3: ID Balancing 0.7931 vs Quantile Balancing 0.7985, a tiny win.
  • Top-5: ID Balancing 0.6498 vs Quantile Balancing 0.6394, a loss.
  • Top-10: ID Balancing 0.5110 vs Quantile Balancing 0.4832, a loss. The average overload flips from a win to a loss as sparsity decreases (top3 to top10). Hence, the consistent win is worst-case overload only. Average overload is inconsistent.
Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Responses to the Reviewer HCer [1/2]

We sincerely appreciate your thoughtful comments and your recognition of the simplicity, low overhead, and strong empirical performance of our method. We are also encouraged that you found the imbalance-aware correction mechanism meaningful and that ID Balancing achieves effective load control under extremely sparse MoE settings while maintaining competitive model quality. Below, we provide detailed responses to your questions and concerns.

Q1: The results are from a single run.

A1: To demonstrate the scalability of the method, we evaluated models under multiple settings, covering all compared methods across scales from 18.9B to 69.9B parameters. Due to limited computing resources, we conducted only one run for each setup. However, the consistent results across different settings still demonstrate the effectiveness and scalability of our method.

Q2: The small gaps in average load and downstream scores (+0.06 to +0.08) is difficult to distinguish apart from the noise.

A2: To better demonstrate the practical value of our method, we provide both the complexity analysis of different methods and the actual training time measured under multiple experimental settings.

Firstly, ID Balancing has significantly lower computational and communication overhead than Quantile Balancing. Let MM be the global token count per update, MrM_r the local token count on rank rr, EE the number of experts, KK the number of selected experts per token, BB the histogram bins per expert, and μ\mu the local microbatch token count. Tokens are sharded across ranks, each with access to all expert scores for its local tokens.

MethodAdditional computationAdditional memoryBalancing communication
ID Balancing (Ours)O(MrK+E)O(M_rK+E) per rankO(E)O(E)All-reduce of EE load counts
DeepSeek loss-freeO(MrK+E)O(M_rK+E) per rankO(E)O(E)All-reduce of EE load counts
Histogram Quantile Balancing (used in Kimi K3)O(MrE+EB)O(M_rE+EB) per rankO(EB+μ+E)O(EB+\mu+E) with streaming margin processingAll-reduce of EBEB histogram counts

Secondly, improved load balancing directly translates into higher end-to-end training efficiency by reducing computational bubbles across EP ranks. We measure actual training throughput under different settings using BF16 with PP=1 on 128 GPUs. Percentages in the ID Balancing column indicate time reductions relative to DeepSeek loss-free / Quantile Balancing.

Model SizeTPDPEPGBSMBSSequence lengthDeepSeek loss-free (s)Histogram Quantile Balancing (s)ID Balancing (s)
3.0B / 24.8B112882048440964.9264.9644.890 (↓0.73% / ↓1.49%)
3.0B / 24.8B26482561327687.1787.3037.142 (↓0.51% / ↓2.21%)
3.2B / 69.9B112882048240968.3598.5268.236 (↓1.47% / ↓3.40%)
3.2B / 69.9B4326425613276818.68919.59018.176 (↓2.74% / ↓7.22%)

The efficiency gains become more significant for larger models and larger EP configurations. For the 3.2B/69.9B model with sequence length 32k, ID Balancing achieves a 7.22% end-to-end speedup over Quantile Balancing. Since MoE computation typically accounts for roughly half of the total training time, this improvement is practically meaningful.

In addition, Table 2 in the paper compares all methods under intentionally unfavorable settings. Following DeepSeek-V3, we freeze the bias updates for the loss-free baseline during continued pretraining, and apply the same setting to Quantile Balancing. Even under this setting, ID Balancing substantially reduces MaxVio while maintaining comparable or slightly improved downstream performance.

MethodModel SizeMaxVio↓MinVio↓Avg. performance
DeepSeek loss-free3.0B / 24.8B2.3275280.79720055.55
Quantile Balancing3.0B / 24.8B2.2584030.81057554.90
ID Balancing (Ours)3.0B / 24.8B0.9813930.73286455.61
DeepSeek loss-free3.2B / 69.9B7.1690720.95781158.53
Quantile Balancing3.2B / 69.9B6.9968230.95418858.48
ID Balancing (Ours)3.2B / 69.9B1.4942740.92072458.61

We also fine-tuned the continued-pretraining checkpoints on 1.32M supervised examples and evaluated them using a multi-domain perplexity benchmark. As shown below, ID Balancing achieves the best PPL among all compared methods after SFT.

MethodModel SizePPL↓
DeepSeek loss-free3.2B / 69.9B3.83
Quantile Balancing3.2B / 69.9B3.84
ID Balancing (Ours)3.2B / 69.9B3.81
AuthorsOpenReview ↗

Responses to the Reviewer HCer [2/2]

Therefore, although the downstream gains are modest, the main advantage of ID Balancing is that it simultaneously achieves:

  • substantially better load balancing,
  • measurable end-to-end efficiency improvements,
  • lower balancing overhead,
  • and comparable or slightly improved model quality.

Q3: The method reduces the worst spikes, but it does not balance the load better on an average.

A3: In our experiments, Quantile Balancing achieves slightly better average MaxVio in some settings, while ID Balancing consistently achieves lower final MaxVio. This suggests that Quantile Balancing can rapidly reduce imbalance under higher learning rates, but its improvement tends to plateau later in training.

Improving the performance of ID Balancing under higher learning rates is an interesting direction for future work.

Nevertheless, considering the system-efficiency benefits discussed in A2, we believe the current version of ID Balancing still provides a better overall trade-off between balancing quality, efficiency, and model performance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We sincerely thank all PCs, SACs, ACs, and Reviewers for their time and effort in evaluating our paper and for their constructive feedback.

We appreciate the reviewers' recognition of the following strengths of our work:

  • All reviewers recognized the simplicity and low computational overhead of ID Balancing.
  • Reviewers HCer and qCFd appreciated the clear presentation and comprehensive experimental evaluation, including training metrics and downstream benchmarks across multiple domains.
  • Reviewer qCFd highlighted the practical motivation and strong reductions in worst-case overload and sustained underload. Reviewer HCer also noted that ID Balancing maintains model quality relative to the baselines.

Following the reviewers' suggestions, we have supplemented the manuscript with the following analyses and experiments:

  1. Computational and communication overhead: A comparison of the additional computation, memory, and balancing communication required by ID Balancing, DeepSeek loss-free, direct-gather Quantile Balancing, and Histogram Quantile Balancing (Table B on Page 21)).
  2. End-to-end training efficiency: Measured mean iteration times on 128 GPUs using BF16, covering different model sizes, sequence lengths, and parallel configurations (Table C on Page 21).
  3. Recovery from persistent underloading: A discussion of the distinction between restoring expert activation and recovering expert capability, explaining why short-term routing correction may not fully compensate for insufficient training exposure (Page 22).
  4. Targeted overload recovery: SFT experiments with Max-overload ID Balancing, showing reduced overload and iteration time while keeping LM loss nearly unchanged (Table D on Page 22).
  5. Training–inference consistency: Evaluation on real online data, showing that the recovered checkpoint also reduces EP-rank-level load imbalance during inference (Table E on Page 23).

All newly added manuscript content is highlighted in blue. Detailed responses to each reviewer's comments are provided in our individual replies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.