acceptodds
Under review as a conference paper at ICLR 2027

Rethinking MoE Load Balancing as Feedback Control

Abstract

Mixture-of-Experts (MoE) training is often not stationary: the data mixture changes at curriculum boundaries, when a model is continually pretrained into a new domain, or when the corpus is refreshed. Routing is learned against the current mixture, so each such change disrupts the per-expert load the router had converged to, and existing balancing methods have only been evaluated on stationary data. We rethink load balancing as a feedback control problem, in which the deviation of each expert's load from its fair share is the control error and the per-expert routing bias is the control input, and define a family of controllers, the PID-Balancer, from the proportional, integral, and derivative terms of classical PID control. Two widely used methods belong to this family: the auxiliary balancing loss behaves like a proportional controller, and the loss-free bias update is an integral controller acting on the sign of the error. Reading the router as a static map with a one-step delay, we show that the integral term is the only one that both removes persistent imbalance and averages out the step-to-step fluctuation of the measured load, whereas the proportional and derivative terms pass or amplify that fluctuation into the routing bias. We therefore adopt PID (I), the pure-integral member of the family with a single gain. Two settings are evaluated: (i) training an OLMoE backbone from scratch with 16 to 64 experts per layer, through a mid-run mixture switch, and (ii) continual pretraining of the released OLMoE-1B-7B on a new corpus. PID (I) holds balance before the shift and restores it more tightly afterwards than either the auxiliary loss or loss-free balancing, at comparable perplexity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, Routing, Load Balancing, Feedback Control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HfqPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies load balancing in mixture-of-experts (MoE) models. MoE models use a learned router to send tokens to different expert MLPs. This learned routing suffers from a rich-get-richer problem where high performing experts receive more gradient updates causing performance to become even more uneven. As a result, researchers have investigated several techniques for encouraging even routing across all the experts, or load balancing.

One set of load balancing techniques manipulates bias terms in the router's softmax to evenly distribute the load. These bias terms are not learned via gradient descent but rather set using other rules. The paper frames this bias tuning as a feedback control problem and proposes to solve it with a proportional-integral-derivative (PID) controller. This perspective both unifies and sheds new light on prior work, and leads to a new load balancing method which achieves more even distributions at similar language modeling performance as prior techniques.

Critical strengths

The paper's strengths are:

  • The PID controller framing provides a new and interesting perspective on the load-balancing problem.
  • The unification of prior methods under PID control helps compare them and better understand their different behaviors.
  • The method empirically achieves the authors' stated goal of better balancing tokens across experts.
Critical weaknesses

The paper really only has one major weakness: they evaluate methods based on how well they balance tokens across the experts, but from a practical perspective these metrics are secondary. For the same language modeling performance, "dead" experts could actually be a good thing because you can prune them from the model to reduce its memory and storage requirements.

The original motivation for load balancing was to ensure good learning, not as an end in itself. Thus, how evenly a technique distributes tokens across experts seems more like an intrinsic evaluation or an analysis of the method rather than a primary evaluation.

Questions for authors

One thing I was curious about, since the PID (I) controller modifies biases in the softmax, and the softmax is invariant to shifts in the logits' mean, does it grow the biases without bound in practice and is there a benefit to periodically recentering the biases (i.e., for numerical stability)?

Since the motivation behind load-balancing is to improve the learning early on but it also seems to hurt expert specialization, one interesting direction for future work could be to remove the load-balancing biases partway through training or at inference time and to see if that does lead to a performance improvement.

2 comments

We thank the reviewer for the careful summary and for recognizing the PID view of load balancing, the unification of existing balancers, and the improved balance of PID (I). We address the weaknesses raised by the reviewer below.

W1. Balance as a primary metric. We agree that balance is not an end in itself. Its value, however, extends beyond good learning and concerns both overloaded and underloaded experts. An overloaded expert slows down MoE computation. Under expert parallelism the device holding the most loaded expert sets the step time, and under capacity-limited routing the overflow tokens are dropped. MaxVio measures this side and is the primary metric of ALF-LB, which we follow. It is also an affine function of the maximum routing imbalance that Thérien et al. (2025) use as a latency proxy (Section 5 and Appendix D.3). An underloaded expert receives too little training and leaves part of the model capacity unused, which DER and UTR measure. This matters more as MoEs scale by adding experts at a fixed number of active ones. In our pretraining, the dead-expert ratio of Vanilla grows from 7.4% to 12.3% and 16.3% as EE grows from 16 to 32 and 64 (Table 1, 1→1).

W2. Expert pruning: We agree that pruning persistently idle experts reduces memory, but we see it as a separate direction with a different objective. Load balancing concerns how tokens are distributed over a given set of experts, which determines throughput and whether every expert is trained. Expert pruning concerns how many experts are kept, which determines the memory footprint. Our setting is distribution shift, where the experts that sit idle change with the data. Under Vanilla and Aux-loss, some experts that are under-loaded before the shift become heavily loaded after it (Figure 3). A pruning decision based on usage therefore holds only for the distribution on which it was made. Removing experts ahead of a shift, as in continual pretraining, would discard capacity that the new distribution may need.

On the soundness score. We read the main concern as being about which metric should be primary, rather than about an error in the analysis or the experiments. If this reading is correct, we hope the clarifications above address it, and we would be grateful if the reviewer would reconsider the soundness score.

We thank the reviewer for the careful reading and the thoughtful questions. We address the questions raised by the reviewer below.

Q1. Bias growth and recentering: The bias is added to the post-softmax scores and is used only for Top-KK selection (Eq. 2). Top-KK selection is unchanged when the same constant is added to all biases, which is the shift the reviewer refers to. For PID (I) the mean of the biases is exactly zero at every step. With dropless routing ∑iLi(t)=TK\sum_i L_i(t) = TK, which gives ∑iei(t)=0\sum_i e_i(t) = 0 and hence ∑iui(t)=0\sum_i u_i(t) = 0. The sign update of ALF-LB lacks this property, and its mean bias drifts during training. The table reports the maximum over training of the mean bias uˉ=1E∑iui\bar u = \frac{1}{E}\sum_i u_i and of max⁡i∣ui∣\max_i \lvert u_i\rvert, with pretraining values from the 1→1 scenario averaged over 6 seeds.

Setting∣uˉ∣\lvert\bar u\rvert ALF-LB∣uˉ∣\lvert\bar u\rvert PID (I)max⁡i∣ui∣\max_i\lvert u_i\rvert ALF-LBmax⁡i∣ui∣\max_i\lvert u_i\rvert PID (I)
Pretraining, E=16E=165.913.8e-76.020.86
Pretraining, E=32E=327.221.7e-77.280.86
Pretraining, E=64E=649.177.9e-89.200.81
CPT, cold start1.631.4e-81.640.14
CPT, warm start2.762.5e-82.780.12

The biases of PID (I) stay below 1 in magnitude throughout training. The drift of ALF-LB shifts all biases by the same amount, which leaves the Top-KK selection unchanged in exact arithmetic. All biases are stored and applied in fp32, where the rounding error at the drifted values is about 1e-6 and negligible next to the gaps between scores. Periodic recentering would therefore leave the biases of PID (I) unchanged. We will add this analysis to the appendix.

Q2. Removing the bias: We ran the inference-time version of this experiment in Appendix D.1. Setting the bias to zero at the end of continual pretraining lowers PPL-CPT by 0.9–1.8% and PPL-Annealing by 2.2–2.6% for both bias-based balancers (Table 15). We measure domain specialization by the mutual information between the input domain and the selected expert. Removing the bias recovers more than half of the specialization that the bias costs relative to Vanilla (Table 14). This agrees with the reviewer's intuition. The bias reduces specialization mainly at selection time, while the gate weights keep most of their domain preference. Without the bias, the load returns most of the way to the imbalance of Vanilla, which matters for expert-parallel deployment. For removal partway through training, our continual pretraining offers a related data point. OLMoE was pretrained with an auxiliary loss, and our Vanilla runs drop it at the start of continual pretraining. Compared with keeping it (Aux-loss), dropping it leaves both perplexities and the downstream scores essentially unchanged, while the fraction of under-trained experts rises from 1.6% to 29.1% in the cold start and from 6.9% to 27.7% in the warm start (Table 2). Appendix D.1 already notes that whether this trade holds at larger scale and in downstream evaluation remains open, and we will extend that discussion to removal partway through training.

Reviewer_5kC4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates MoE load balancing as feedback control and interprets existing balancing methods through proportional and integral mechanisms. It advocates a pure-integral controller using normalized load errors. Experiments on OLMoE-based pretraining and continual pretraining show improved load balance under abrupt distribution shifts, with similar target-domain perplexity but some additional forgetting.

Critical strengths

The feedback-control perspective provides a clear and useful interpretation of load balancing under distribution shifts.

The empirical study includes six-seed pretraining experiments, gain and routing-sparsity ablations, longer continual-pretraining budgets, and informative analyses of specialization and forgetting.

Critical weaknesses
  1. The practical benefit of better balance remains insufficiently established. No measured throughput or step-time improvement is reported. Meanwhile, Table 2 and Appendix C.6 show greater forgetting and lower scores on several downstream tasks for the bias-based balancers, despite similar target-domain perplexity. Continual pretraining uses a single seed and code-only adaptation on one OLMoE checkpoint, limiting confidence in the overall benefit and generality.

  2. Limited novelty relative to the closest prior update. With a fixed batch size and routing configuration, PID(I) is equivalent to the linear bias-update variant of Wang et al. (2024) after rescaling the gain. This variant was already evaluated with softmax routing and is acknowledged in Section 3.2, whereas the main ALF-LB baseline uses the sign-based update. The contribution beyond reinterpreting and evaluating an existing rule under distribution shifts therefore remains limited.

[1] Lean Wang et al. Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts. 2024.

Questions for authors
  1. Given the equivalence between PID(I) and the previously studied linear ALF-LB update, what substantive algorithmic or analytical contribution distinguishes this work? Why is the main comparison restricted to the sign-based ALF-LB variant?

  2. At matched target-domain quality and retention of prior capabilities, does PID(I) provide measurable end-to-end efficiency gains? Please quantify this trade-off and the variability of the continual-pretraining results.

1 comment

We thank the reviewer for finding the feedback-control view clear and useful, and for recognizing the breadth of our ablations and analyses. We address the weaknesses and questions raised by the reviewer below.

W1. Practical benefit. Our response to Q2 estimates expert time under expert parallelism, where PID (I) is the lowest of all methods, with a small margin over ALF-LB and a substantial margin over the auxiliary loss and Vanilla at matched target-domain perplexity. We agree that both bias-based balancers forget more than Vanilla and the auxiliary loss (Tables 2 and 13). Setting the bias to zero at evaluation brings PPL-Annealing back close to Vanilla (Table 15), which locates most of this cost at expert selection rather than in the weights. We acknowledge that the CPT study uses a single OLMoE checkpoint, and in the final version we will extend it to another open-source MoE model.

W2. Novelty. We agree that PID (I) coincides with the linear rule of Wang et al. for a fixed setpoint, and Section 3.2 discusses this rule alongside the sign rule. As detailed in our response to Q1, our contribution is to derive this rule from the control formulation rather than choose it empirically, and to compare the sign and linear rules on per-batch balance under softmax routing, which Wang et al. did not report.

Q1. Contribution beyond the linear rule, and the choice of baseline: We agree that PID (I) coincides with the linear rule of Wang et al. under a fixed setpoint, as discussed in Section 3.2. It is the member of the PID-Balancer family that we adopt after ruling out the proportional and derivative terms, not a new update rule. We will make this explicit in the abstract and the introduction and revise any wording that could suggest otherwise.

The sign rule is ALF-LB as defined in Algorithm 1 of Wang et al., who retained it over the linear rule after comparing the two (their Section 4.3). Their preference for the sign rule under the sigmoid gate and their switch to the linear rule under the softmax gate (their Appendix C.2) were both empirical choices. That comparison used only MaxVio$_\text{global}$, and the per-batch MaxVio was not compared between the two rules. Our main comparison therefore pits the sign rule against the linear rule on the per-batch MaxVio. On this metric ALF-LB stays above PID (I), the gap widens with the number of experts (Table 1 and Figure 1).

Our contribution is to reach the linear rule through the control formulation rather than empirically. Equation (4) is linear in the error, and Sections 3.3 and C.4 rule out its proportional and derivative terms, which leaves the linear integral term. The relay of ALF-LB is a nonlinear version of this term that balances the median load of each expert rather than the mean, and we will state this distinction explicitly in Section 3.2.

Q2. Efficiency at matched quality, the trade-off, and variability: Our runs do not use expert parallelism, and routing imbalance therefore does not set their step time. To estimate its effect under expert parallelism, we regroup the logged per-step expert loads of the runs in Table 2 onto 8, 16 and 64 devices and compute the expert time of the busiest device relative to perfect balance, ignoring communication. The table below reports the post-switch window, with experts assigned to devices in index order.

MethodCold, 8 devicesCold, 16 devicesCold, 64 devicesWarm, 8 devicesWarm, 16 devicesWarm, 64 devices
Vanilla1.792.576.831.782.556.76
Aux-loss1.562.135.431.582.175.51
ALF-LB1.161.292.011.161.302.03
PID (I)1.141.261.901.141.251.87

Only ALF-LB matches PID (I) in both target-domain perplexity and retention (Tables 2, 11 and 13), and against it PID (I) gives a small but consistent reduction in expert time. The auxiliary loss and Vanilla reach the same PPL-CPT as PID (I) and retain prior capabilities better (Table 2), but they need substantially more expert time, and this gap grows as fewer experts share a device. Matching the balance of PID (I) with the auxiliary loss requires a larger coefficient, which raises both PPL-CPT and PPL-Annealing above those of PID (I) (Table 10). We additionally assign experts to devices at random and evaluate the pre-switch window of the warm start, and the ordering of the four methods is unchanged in both cases. We will add the full expert-parallel analysis to the appendix.

Reviewer_ahg6OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper reformulates the Mixture-of-Experts (MoE) load balancing challenge as a classical feedback control problem, introducing the "PID-Balancer" framework. By analyzing the loop dynamics of MoE routing, the authors show that Proportional (P) and Derivative (D) control loops amplify step-level micro-batch fluctuations (noise), while an Integral (I) controller effectively eliminates steady-state load imbalance under non-stationary shifts (e.g., domain shifts or curriculum shifts). Based on this, they propose a pure linear Integral controller, PID (I), which adjusts per-expert routing biases based on past load error history. When evaluated on OLMoE backbones under domain shift benchmarks, PID (I) achieves tighter load balancing (lower MaxVio) without degrading perplexity or downstream task metrics.

Critical strengths

When evaluated on OLMoE backbones under domain shift benchmarks, PID (I) achieves tighter load balancing (lower MaxVio) without degrading perplexity or downstream task metrics.

Critical weaknesses
  1. what happens when a domain shift is super sudden or huge? since a pure integral controller is naturally slow to react (it has to integrate error over time), doesn't it lag and cause massive load spikes right after a domain shift before the bias finally catches up?
  2. how did you tune K_I and does it break if you change the batch size? since the load error is calculated per step, a larger batch size or different sequence length should completely change the noise scale, so does K_I need to be manually retuned every time the training setup changes?
  3. how do you handle dead experts? if an expert's routing score drops so low that it gets zero tokens, its error eᵢ(t) stays positive, causing the integral bias uᵢ(t) to pump up forever (classic integrator windup). did you implement any anti-windup clamping to stop the bias from exploding on inactive experts?
Questions for authors
  1. exp scale is way too small. you only tested on tiny olmoe with like 1b tokens, does this feedback loop actually scale to huge production models without crashing or lagging?
  2. you missed key moe structures. how does this pure-i controller handle shared experts, super fine-grained expert setups (like deepseek), or changing top-k activation ratios?
  3. this looks super similar to loss-free moe (like j. ansel et al., 2026) which already updates routing biases on the fly. what does the pid control theory actually add that's new?
1 comment

Thank you for the review and we address the weaknesses and questions below.

W1: Our shift is already the most abrupt input a per-step balancer can receive, a single instantaneous switch at step 16,500 from prose-dominated Books to code-dominated GitHub in the 1→11\to1 scenario. Figures 2 and 3 show the response at E=64E=64. Without a balancer, Vanilla moves to a worse steady state after the switch. Every balancer shows a burst at the switch, as each meets the new corpus with a bias fitted to the previous one. As noted in Section 4.1 (line 322), PID (I) enters the switch from the most balanced state, and its burst is the most pronounced relative to its pre-shift level. The methods differ in how they recover. Aux-loss decays most slowly, and ALF-LB falls back fastest but settles at a worse steady state. PID (I) overtakes ALF-LB within a few hundred steps and reaches the lowest postMV in every setting (Table 1).

W2: KIK_I was chosen once by the sweep in Section C.2 and then held fixed for all experiments. The error in Equation (1) is normalized by the fair share rr, and its scale therefore does not depend on batch size or sequence length. Batch size changes only the step-to-step fluctuation of the error. We measured this fluctuation by splitting the sequences of each step into disjoint groups, and it scales as the inverse square root of the batch size. A change in batch size by any factor therefore changes the noise only by the square root of that factor. At the same number of tokens per step, pretraining and continual pretraining show nearly the same noise level despite their different sequence lengths (Table 4). An integral balancer removes persistent imbalance at any noise level, and noise only adds fluctuation to the bias. The value chosen on 1→11\to1 was reused unchanged for the other mixtures (Table 1), the Top-K ablation (Section C.3) and continual pretraining (Table 2). The Top-K settings already differ in noise more than a doubled batch would. Continual pretraining also changes the model, corpus and sequence length, and a KIK_I sweep there gives nearly the same postMV for values spanning two orders of magnitude. Only the expert count required a new KIK_I, set by the rule in Section C.2.

W3: The growth of the bias on a starved expert is the intended recovery mechanism of Section 3.3, and it stops once the expert regains its share of tokens. Windup requires a saturated actuator, which PID (I) does not have, and we therefore use no clamp. The bias is added unclipped to the softmax output sij∈[0,1]s_{ij}\in[0,1] (Equation (2)), and a bias exceeding the KK-th largest of the others by more than one already places the expert in every token's selection. Prolonged starvation occurs only in the first steps of training, the bias then rises by exactly KIK_I per step, and the expert returns to selection within a few hundred steps. At the gains we use, no expert stays dead (Tables 1 and 2).

Q1: We have disscused that the scale is the main limitation (Section D.4). Besides pretraining, we apply PID (I) to the fully pretrained OLMoE-1B-7B without retuning its gain (Sections 4.2 and C.6). PID (I) shares the state, cost and injection point of ALF-LB (Section 3.2), the balancer used in DeepSeek-V3, and changes only the update rule from the sign of the error to the error itself. It adds no gradient and its bias is bounded by construction, and it therefore adds no instability beyond ALF-LB. Its advantage over ALF-LB grows with the number of experts (Table 1) and does not shrink with longer continual pretraining (Table 11).

Q2: The activation ratio is ablated in Section C.3, where PID (I) keeps the lowest MaxVio at every Top-K with its gain unchanged (Figure 6). Granularity is the axis of our main experiment, which splits a fixed parameter budget into progressively more and smaller experts (Table 3). The advantage of PID (I) grows as experts become finer (Table 1, Figure 1), and Section C.2 gives a rule for setting its gain at each granularity. A shared expert processes every token without routing and therefore has no load to balance.

Q3: We could not locate J. Ansel et al. (2026) and would appreciate the full reference. The best-known loss-free bias update is that of [1], our main baseline ALF-LB, which Section 3.2 shows to be a relay integrator within our family. The control view explains why only the integral term both removes persistent imbalance and averages out step-to-step fluctuation (Section 3.3), and Section C.4 confirms that adding the proportional or derivative term only degrades balance. It identifies the relay as the limitation of ALF-LB and gives a principled basis for the linear update, which [1] examined only empirically. Under distribution shift, the advantage of PID (I) over ALF-LB grows with the number of experts (Table 1, Figure 1).

[1] Wang, Lean, et al. "Auxiliary-loss-free load balancing strategy for mixture-of-experts." arXiv:2408.15664 (2024).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.