acceptodds
Under review as a conference paper at ICLR 2027

Stuck in Collapse: Understanding Routing Collapse in Sparse Mixture-of-Experts

Abstract

Mixture-of-Experts is a key architecture for scaling model capacity with limited computation. However, routing collapse, where token assignments concentrate on a small subset of experts, remains a long-standing failure mode that degrades both model performance and computational efficiency. To understand how routing collapse emerges and persists, we conduct a controlled pretraining study across two model scales and eight model variants. We find that routing collapse is not a gradual accumulation of imbalance, but a failure to correct routing perturbations, with correction dynamics that differ across routing operators. Tracing the underlying optimization mechanism, we observe that these perturbations coincide with rapid early reorganization of hidden representations, which can act as a common input-side source. Under sparse routing, selection-dependent asymmetry in router gradients can fail to correct the resulting bias. For unselected experts, these gradients either vanish entirely or lose expert-specific information and share a common scalar component. To validate this mechanism, we introduce a targeted router gradient intervention that restores expert-specific information for unselected experts while preserving sparse forward computation. It substantially alleviates routing imbalance without an auxiliary loss, providing evidence that router gradient dynamics for unselected experts play a key role in sustaining routing collapse.

Primary area:
generative models
Keywords:
Mixture-of-Experts, Routing Collapse, Routing Imbalance, Router, Routing Dynamics, Training Dynamics, Optimization Dynamics, Gradient Analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wtpDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies why routing collapse emerges and persists in sparse MoE pretraining. The authors pretrain Dense, Softmax (dense routing), Softmax-TopK, TopK-Softmax and their auxiliary-loss variants at two scales on 100B tokens, measuring utilization with a probability-based effective number of experts N_eff. N_eff drops or fluctuates sharply within the first few hundred steps, then follows one of three trajectories (stuck, rebound, or slow recovery). The collapse severity, its layer patterns, and its response to the auxiliary loss depend on the routing function.

The paper show that hidden representations reorganize early in training for every variant, including Dense, and argue that this reorganization perturbs routing logits. They then derive that, for unselected experts, the logit gradient is zero under TopK-Softmax and a shared scalar times p_i under Softmax-TopK, so the router cannot apply an expert-specific correction. To test this, they replace the router gradient with one computed from counterfactual expert outputs on per-cluster mean hidden states. This intervention raises N_eff substantially without an auxiliary loss, and average zero-shot accuracy rises from 50.62 to 53.21 at 0.68B and from 54.35 to 55.45 at 1.92B.

Critical strengths

The controlled pretraining study isolates the routing operator. The authors hold data, architecture and optimizer fixed and vary only the operator and the auxiliary loss, at two scales on 100B tokens each. This gives a clean comparison where dense Softmax stays balanced while both sparse operators collapse in different ways.

The paper proposes concrete, testable mechanism with a cheap intervention. The gradient decomposition gives a precise explanation for why TopK-Softmax responds poorly to the auxiliary loss. The intervention adds negligible FLOPs and recovers N_eff without any balancing objective.

Critical weaknesses

The causal claim is understudied. The intervention does three things at once. It gives unselected experts a nonzero gradient, it makes that gradient expert-specific, and it changes the selected experts' gradients by replacing y with y_hat as the reference. The paper attributes the effect to "expert-specific information", but no control isolates that factor. For example, a surrogate with shuffled or random counterfactual outputs (nonzero but not expert-specific) would test it directly.

With one seed per variant, the three trajectory types in Sec. 4.1 could also reflect seed variance rather than routing dynamics.

The models use 8 and 16 experts with K = 2, whereas current MoEs use 64–256 fine-grained experts, often with shared experts. Whether the mechanism and intervention carry over to practical scales is unclear. Notably, the intervention is close in spirit to prior methods that give the router a dense gradient by substituting estimated outputs for unselected experts. Examples are the default-output approach of Panda et al. (cited in the paper) and SparseMixer-style gradient estimators. These are neither discussed nor compared. The auxiliary-loss-free bias method (Wang et al., 2024) is cited but not run.

The paper reports no validation loss on pretraining data. Downstream differences are also small: at 1.92B, "Ours" (55.45) is below TopK-Softmax_aux (55.71), and Ours_aux does not improve on Softmax-TopK_aux at 0.68B (52.85 versus 53.08).

Questions for authors

How much do the N_eff trajectories and final values vary across seeds? Do the three trajectory types in Sec. 4.1 persist for a given operator and layer across seeds?

Can you add a control that gives unselected experts a nonzero but non-expert-specific gradient? This would show whether expert-specific information, causes the recovery.

4 comments

We sincerely thank you for your thoughtful and constructive feedback. In the following responses, we carefully address the points you have raised and have updated the manuscript accordingly (*changes highlighted in magenta).

To make the additional experimental results easier to access, we also provide anonymized links.


Given the rebuttal timeline and available compute, we ran the additional experiments for 5K steps, with each run taking roughly 1.3–3.5 days for the 0.68B sparse MoE model, depending on the GPU.

Selected-Expert Gradients in Eq. 10

We agree that Eq. 10 does not fully isolate the unselected-expert gradient effect. We initially modified only unselected-expert gradients to test the limitation of selection-dependent sparse gradients, but this led to unstable training with non-finite activations. We attribute this to violating the zero-sum property of Softmax logit gradients (Appendix D.3), motivating Eq. 10 to preserve this structure and training stability.

We agree that this prevents complete isolation of the unselected-expert gradient effect. We have added Sec. 8 (Limitations) and explicitly noted this limitation of our intervention.

Relation to Prior Router-Gradient Methods

We agree that our intervention is related to prior approaches that provide gradient pathways to unselected experts. We have expanded Appendix B Related Work to discuss DefaultMoE [1] and SparseMixer [2] and clarify their relation to our study.

Given the rebuttal timeline, we prioritized evaluating DefaultMoE, which was also requested by Reviewer Qf4K. DefaultMoE and ours differ in three aspects:

  • Motivation: DefaultMoE approximates dense MoE gradients to improve sparse MoE training, whereas ours investigates the role of selection-dependent router gradients in routing collapse.
  • Intervention: DefaultMoE modifies the MoE forward output to be dense-MoE-like by adding EMA outputs for unselected experts, whereas ours keeps the sparse forward output unchanged and directly modifies only the router gradient.
  • Output estimation: DefaultMoE uses a historical EMA of each expert’s outputs on previously selected tokens, independent of the current unselected token, whereas ours evaluates each expert on the current cluster-mean representation.

Additional Baselines and Gradient Controls

Thank you for suggesting these important controls. Following your suggestion, we implemented RandomGrad by providing nonzero Gaussian gradients to unselected experts while preserving zero-mean router gradients. We also evaluated DefaultMoE [1] and the auxiliary-loss-free bias method (Aux-Free) [3]. RandomGrad and DefaultMoE were evaluated both w/ and w/o auxiliary loss. Results are provided in Appendix E.6 ([Fig. 14]).

  • RandomGrad: Nonzero random gradients alter routing vectors and expert utilization, but provide no consistent corrective direction. Accordingly, NeffN_{\mathrm{eff}} exhibits highly unstable trajectories and fails to recover from collapse across many layers. With auxiliary loss, balancing pressure restores expert utilization, but the random-gradient interference remains visible as substantial fluctuations in NeffN_{\mathrm{eff}}.
  • DefaultMoE: Without auxiliary loss, its EMA-based gradients enable gradual recovery, but recovery is generally weaker than Ours and unstable in some layers (e.g., Layer 2). DefaultMoE estimates the missing response for an unselected token from outputs of tokens that selected the expert, whereas ours uses the current token's cluster, which may provide a more relevant corrective signal.
  • Aux-Free: The auxiliary-loss-free bias method effectively maintains high expert utilization across most layers, confirming its effectiveness as an explicit routing-balancing mechanism.

Reference

[1] Panda et al., Dense backpropagation improves training for sparse mixture-of-experts, NeurIPS, 2026.

[2] Liu et. al., Sparse backpropagation for moe training, arXiv, 2023.

[3] Wang et al., Auxiliary-loss-free load balancing strategy for mixture-of-experts, arXiv, 2024.

Robustness Across Seeds

We repeated the experiments with a different initialization and data order (seed 5), with results provided in Appendix E.5 ([Fig 11] and [Fig 12-13]). While the exact NeffN_{\mathrm{eff}} trajectories vary across seeds, the key qualitative dynamics are reproduced: early routing perturbation followed by persistent collapse, rapid rebound, or gradual recovery. The final layer-wise values also reproduce the operator-dependent collapse patterns.

Purpose of the Gradient Intervention

Our study aims to understand the role of sparse router-gradient limitations in sustaining routing collapse. Sec. 4.3 shows that severe routing collapse is associated with lower downstream performance, while alleviating collapse is accompanied by improved performance. Our intervention further shows that modifying the router gradient alone can alleviate collapse without an explicit balancing objective, while achieving downstream performance comparable to sparse MoEs trained with auxiliary loss.

Validation Loss

We additionally evaluated the validation loss across all model variants and both scales. The results are reported below:

Method0.68B Total0.68B LM0.68B Aux1.92B Total1.92B LM1.92B Aux
Dense3.1773.1770.0003.0263.0260.000
Soft-TopK3.0123.0120.0002.8312.8310.000
TopK-Soft3.0003.0000.0002.8222.8220.000
Softmax2.9562.9560.0002.7312.7310.000
Ours2.9892.9890.0002.8252.8250.000
Soft-TopK aux3.0022.9822.0262.8252.8052.025
TopK-Soft aux3.0142.9912.2672.8182.7942.367
Ours aux3.0072.9862.0482.8212.8012.040

Validation at Larger MoE Scales

We agree that validation at larger MoE scales is important. Our experiments systematically compare eight model variants across two scales under a limited compute budget, following MoE pretraining settings feasible at academic scale [4,5]. We have added Sec. 8 (Limitations) to explicitly acknowledge the limited scale and the lack of evaluation with larger expert counts and shared-expert architectures.


Reference

[4] Jin et al.,Moe++: Accelerating mixture-of-experts methods with zero-computation experts, ICLR, 2025.

[5] Lv et al., Autonomy-of-experts models, ICML, 2025.

Dear Reviewer wtpD,

Thank you again for your thoughtful and constructive feedback.

As the discussion period draws to a close, we wanted to briefly follow up on our rebuttal. We believe that the additional experiments, analyses, and revisions have addressed the concerns raised in your review.

We would be grateful to hear if there is anything further we could clarify. If you find that the new evidence addresses your concerns, we would sincerely appreciate your consideration of an updated rating.

Thank you again for your time and consideration.

Best regards, The Authors

Dear Reviewer wtpD,

We understand this is a busy time. As the discussion period is closing very soon, we would like to kindly follow up one last time.

If any concern remains after our response, please let us know and we will reply right away. If your concerns have been addressed, we would be grateful if you could consider updating your rating.

Thank you again for your time.

Best regards, The Authors

Reviewer_jazrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the phenomenon of router collapse in MoE models. The authors conduct an empirical study of this phenomenon during training, and claim that it happens because of rapid learning in the initial training phase. The experts that receive low-signal updates at these early training collapse. The paper conducted pretraining studies at two different scales to support the claims.

Critical strengths

The paper is well written and easy to follow. It has some strengths wortg noting:

  • Empirical study: the authors conducted the experiment at two model scales, with routing variants, etc. Their experiments show a consistent early disruption in the routing statistics, followed by different recovery patterns. This is in itself an interesting information, which cannot be observed by just looking at final pretrained models.
  • The comparison between different routing operators (Sm-Topk, TopK-Sm, etc) is interesting. The paper suggests that Sm-TopK leads unselected experts with a shared gradient component, while TopK-Sm gives them zero task-loss gradient for a given token.
Critical weaknesses

The paper has some weaknesses that need to be addressed before it becomes publishable

  • The explanation is incomplete and mostly correlation-based: the derivations show that unselected experts receive either zero gradients or gradients lacking their own output information. However, these identities do not explain when imbalance should persist or recover (as shown in some figures). Also, hidden representations and routing both change early in training, but the paper does not separate changes in logits caused by changing representations from those caused by e.g. changing router weights.
  • The auxiliary loss comparison in Section 5.3.2: the topk-sm aux loss is computed using probabilities normalized within the selected set, removing direct gradients to unselected experts. By contrast, the Fedus et al formulation of aux loss uses full softmax probabilities. So the reported weaker response to balancing cannot be attributed to the outing operator without testing the full softmax aux loss.
  • The scale is too small for MoEs: MoE models are usually trained at a much larger scale. It is understandable that not all researchers have access to the compute to train e.g. a 10B/1B(active) MoE model, but it should at least be mentioned as a limitation in the paper.


-- Other comments: the author mention that they don't an explanation for the depth collapse. There is recent work (Wang et al. “The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise”) showing that depth-related router collapse is due to increasing correlation in hidden states.

Questions for authors

See weaknesses.

3 comments

We sincerely thank you for your thoughtful and constructive feedback. In the following responses, we carefully address the points you have raised and have updated the manuscript accordingly (*changes highlighted in magenta).

To make the additional experimental results easier to access, we also provide anonymized links.


Given the rebuttal timeline and available compute, we ran the additional experiments for 5K steps, with each run taking roughly 1.3–3.5 days for the 0.68B sparse MoE model, depending on the GPU.

Layer-wise Collapse and Hidden Representation Concentration

Thank you for pointing us to the recent work by Wang et al. [1]. Following their analysis, we examined whether increasing hh correlation explains the operator-dependent layer-wise collapse patterns. Detailed results are added to Appendix E.7 ([Fig 15], [Fig 16] and [Tab 7]):

  • Opposite layer-wise trend: Both top-1 energy and average cosine similarity of hh decrease with depth, opposite to the increasing concentration reported in [1] (Fig 15).
  • Different NeffN_{\mathrm{eff}} patterns: Despite the clear monotonic trends in both hh metrics, the layer-wise behavior of NeffN_{\mathrm{eff}} differs substantially across routing operators (Fig 15).
  • Operator-dependent correlation: The correlation between hh concentration and NeffN_{\mathrm{eff}} varies substantially across operators, with Softmax-TopK showing little linear correlation (r=-0.18/-0.03) and no statistically significant association (all p$\geq$0.30) (Fig 16 and Tab. 7).

These results suggest that hh concentration alone does not explain the layer-wise collapse patterns.

Hidden Representation and Router Dynamics

Routing depends jointly on hh and router geometry through zi=wi⊤hz_i=w_i^\top h. Eq. 6 further shows that wiw_i updates depend on hh and on the operator-dependent logit gradient. Our analysis therefore focuses on how representation changes interact with operator-dependent router adaptation, rather than treating hh and router-weight changes as independent causes.

We agree that our analysis does not fully predict which layers ultimately persist in collapse or recover. Instead, it characterizes how selection-dependent sparse gradients can limit corrective router adaptation after routing is perturbed. We have added Sec. 8 (Limitations) and explicitly acknowledge this limitation there.

Full-Softmax Auxiliary Loss for TopK-Softmax

Fedus et al. [2] use Softmax-TopK and compute the auxiliary loss from its routing probabilities. We applied the same loss definition to TopK-Softmax using its routing probabilities, a formulation also used in prior work [3].

We conducted an additional experiment using full-softmax probabilities for the auxiliary loss while keeping the TopK-Softmax forward pass unchanged. By providing direct auxiliary gradients to unselected experts, this formulation eliminates the residual imbalance observed with the selected-set TopK-Softmax aux. We have revised Sec. 5.3.2 to clarify this distinction and added the result to Appendix E.8 ([Table 8]).

Validation at Larger MoE Scales

We agree that validation at larger MoE scales is important. Our experiments were designed to systematically compare eight model variants across two scales under a limited compute budget, following MoE pretraining settings feasible at academic scale [4,5]. We have added Sec. 8 (Limitations) to explicitly acknowledge the limited scale of our experiments.


Reference

[1] Wang et al., The myth of expert specialization in moes: Why routing reflects geometry, not necessarily domain expertise, arXiv, 2026.

[2] Fedus et al., Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity, JMLR, 2022.

[3] Cai et al., Long-tailed distribution-aware router for mixture-of-experts in large vision-language model, arXiv, 2025.

[4] Jin et al.,Moe++: Accelerating mixture-of-experts methods with zero-computation experts, ICLR, 2025.

[5] Lv et al., Autonomy-of-experts models, ICML, 2025.

Reviewer_jazrOpenReview ↗

Response

Thank you for the comments and the additional experiments.

My main concern remains unaddressed about the fact that most explanations are correlation-based and not causal. The gradient identities do not explain when and how collapse happens, which is contrary to what the title suggests ("understanding routing collapse"). To make this kind of studies rigorous, a causal test is needed where one carefully designs a controlled setup that allows causal testing.

Thank you for the time and effort you have devoted to reviewing our work throughout the rebuttal process.

We appreciate you further clarifying your remaining concern and would like to address it carefully below.


Scope of Our Analysis

We do not claim that the gradient identities alone determine exactly when or in which layer routing collapse occurs. The realized training trajectory depends on the evolving interaction between hidden representations and the router, which is further shaped by multiple factors such as model parameter initialization, mini-batch data composition, and training data ordering. Rather, our gradient analysis characterizes a structural limitation of selection-dependent router gradients in sparse Mixture-of-Experts, particularly in providing corrective signals to underselected experts once routing becomes imbalanced.

Analysis and Experimental Evidence

To examine this structural limitation, we provide complementary evidence through analytical derivation, controlled pretraining experiments, robustness tests, and direct gradient intervention.

  1. We analytically derive the operator-dependent router gradient structures, highlighting the selection-dependent gradient asymmetry introduced by sparse expert selection.
  2. With the model architecture, parameter initialization, data ordering, optimizer, and training configuration held fixed, we train each routing variant from scratch. Under this controlled setting, we observe distinct routing dynamics across operators, with dense Softmax showing recovery from early perturbations while sparse routing can exhibit persistent imbalance.
  3. These patterns are consistent across both model scales and are reproduced with a new random seed that changes model initialization and data ordering ([Fig. 11]), indicating that they are not specific to a particular model scale, parameter initialization, or data ordering.
  4. Most importantly, in Sec. 6, we directly test the identified limitation of selection-dependent gradients by intervening specifically on the router gradients while leaving the sparse forward computation unchanged. This targeted modification of the router gradients substantially alleviates routing imbalance and produces sustained recovery after the early perturbation without an explicit balancing constraint.
Reviewer_Qf4KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how routing collapse begins and why some layers fail to recover. The authors pretrain 0.68B and 1.92B models with Softmax-TopK and TopK-Softmax routing, with and without a load-balancing loss, and compare them with a dense Softmax router. The results show that the effective number of experts often drops sharply early in training. Whether it recovers depends on the routing operator and the layer.

Their analysis shows that these drops coincide with rapid changes in the hidden states. Dense Softmax undergoes similar changes but recovers. The authors argue that sparse routing may lack the gradient signals needed to correct the imbalance: unselected experts receive no router gradient under TopK-Softmax, and only a shared, probability-scaled term under Softmax-TopK. To address this, they modify Softmax-TopK’s backward pass by grouping tokens that share a top-1 expert and evaluating every expert on each group’s mean hidden state. These outputs provide expert-specific router gradients for unselected experts. Without a balancing loss, the method improves average and worst-layer expert utilization over the sparse baselines, with zero-shot accuracy close to models trained with a balancing loss.

Critical strengths
  1. The comparison between routing operators is valuable. The two sparse operators show different collapse patterns and respond differently to the balancing loss.
  2. The proposed intervention is reasonable and improves worst-layer expert utilization without an auxiliary loss or additional computation.
Critical weaknesses
  1. I do not see a clear practical advantage over a standard balancing loss. Softmax-TopK with aux loss achieves better load balance at both scales, while downstream accuracy is similar (Tables 3 and 6). Although the intervention is useful for studying routing dynamics, it is unclear when it would be preferable in practice.
  2. I am not convinced that restoring expert-specific information explains the improvement. Eq. 10 also changes the selected experts’ gradients, and the experiments do not separate these effects. The connection to Default MoE (2504.12463) also needs clarification. Its router gradient has the same form, using an EMA of expert outputs instead of the cluster-based estimates. Please explain how the proposed method differs.
Questions for authors
  1. How much do the N_eff curves vary across seeds? Given the divergent trajectories you describe, I'd expect them to be sensitive to initialization and data order.
  2. How does the method compare with Default MoE trained without an aux loss? Its EMA of expert outputs plays the role of Ei(h^c)E_i(\hat{h}_c), so this would show whether the cluster-mean counterfactual matters.
1 comment

We sincerely thank you for your thoughtful and constructive feedback. In the following responses, we carefully address the points you have raised and have updated the manuscript accordingly (*changes highlighted in magenta).

To make the additional experimental results easier to access, we also provide anonymized links.


Given the rebuttal timeline and available compute, we ran the additional experiments for 5K steps, with each run taking roughly 1.3–3.5 days for the 0.68B sparse MoE model, depending on the GPU.

Selected-Expert Gradients in Eq. 10

Thank you for this important point. We initially modified only unselected-expert gradients to test the limitation of selection-dependent sparse gradients, but this formulation led to unstable training with non-finite activations. We attribute this to violating the zero-sum property of Softmax logit gradients (Appendix D.3), motivating Eq. 10 to preserve this structure and training stability.

We agree that this prevents complete isolation of the unselected-expert gradient effect. We have added Sec. 8 (Limitations) and explicitly noted this limitation of our intervention.

Difference from DefaultMoE

DefaultMoE [1] and ours differ in three aspects:

  • Motivation: DefaultMoE approximates dense MoE gradients to improve sparse MoE training, whereas ours investigates the role of selection-dependent router gradients in routing collapse.
  • Intervention: DefaultMoE modifies the MoE forward output to be dense-MoE-like by adding EMA outputs for unselected experts, whereas ours keeps the sparse forward output unchanged and directly modifies only the router gradient.
  • Output estimation: DefaultMoE uses a historical EMA of each expert’s outputs on previously selected tokens, independent of the current unselected token, whereas ours evaluates each expert on the current cluster-mean representation.

Comparison with DefaultMoE

We evaluated DefaultMoE w/ and w/o auxiliary loss (Appendix E.6, [Fig. 14]). Without auxiliary loss, DefaultMoE provides router gradients to unselected experts through EMA expert outputs and shows gradual recovery in expert utilization. However, its recovery is generally weaker than Ours and unstable in some layers (e.g., Layer 2).

As analyzed in Sec. 5.2, the router logit gradient reflects whether increasing an expert's contribution would locally improve the loss for the current token. From this perspective, DefaultMoE estimates the missing response for an unselected token from outputs of tokens that selected the expert, whereas ours uses the current token's cluster. The former may therefore provide a less relevant corrective signal for the current token.

Role of Expert-Specific Information

Thank you for this important point. In Sec. 5.2, we use “expert-specific” to describe that the logit gradient reflects whether increasing a particular expert's contribution would locally improve the loss for the current token. We agree that attributing the intervention's improvement directly to “expert-specific information” was too broad. To clarify this distinction, we revised the intervention description to “provides corrective signals reflecting how unselected experts would respond if selected.”

Robustness Across Seeds

We repeated the experiments with a different initialization and data order (seed 5), with results provided in Appendix E.5 ([Fig. 11]). While the exact NeffN_{\mathrm{eff}} trajectories vary across seeds, the key qualitative dynamics are reproduced: early routing perturbation followed by persistent collapse, rapid rebound, or gradual recovery.

Purpose of the Gradient Intervention

Our study aims to understand the role of sparse router-gradient limitations in sustaining routing collapse. Sec. 4.3 shows that severe routing collapse is associated with lower downstream performance, while alleviating collapse is accompanied by improved performance. Our intervention further shows that modifying the router gradient alone can alleviate collapse without an explicit balancing objective, while achieving downstream performance comparable to sparse MoEs trained with auxiliary loss.


Reference

[1] Panda et al., Dense backpropagation improves training for sparse mixture-of-experts, NeurIPS, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.