acceptodds
Under review as a conference paper at ICLR 2027

RIDER-MoE: Reliability-Conditioned Interaction Routing for Robust Multimodal Sentiment Analysis

Abstract

Multimodal sentiment analysis needs to retain strong performance with complete inputs while remaining robust to noisy, missing, or conflicting modalities. We address this goal within a unified framework through reliability-conditioned interaction routing, adapting the mixture of interaction pathways rather than relying solely on modality reweighting. We present RIDER-MoE, a reliability-aware mixture-of-experts architecture that routes each example among modality-uniqueness, redundancy, and synergy experts based on estimated modality reliability and cross-modal agreement. The expert design is motivated by Partial Information Decomposition (PID). We introduce masked-view Disentangled Interaction Regularization (DIR) to encourage distinct U/R/S behaviors during training while retaining single-pass inference. Lightweight unimodal sentiment probes provide uncertainty and consensus cues that guide expert-level priors, softly biasing a router driven by the fused representation. Across CMU-MOSI, CMU-MOSEI, and CH-SIMS, RIDER-MoE remains competitive with strong baselines on clean full-modality test sets and achieves the highest aggregate normalized AUC for Acc-2 and Corr among the evaluated methods under controlled noise, missing-modality, and cross-modal conflict stress tests. Ablations and routing diagnostics support the complementary roles of reliability-aware routing and masked-view expert regularization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Sentiment Analysis, Mixture of Experts, Reliability-Aware Routing, Interaction Disentanglement, Partial Information Decomposition, Robust Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WmeaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RIDER-MoE, a reliability-aware mixture-of-experts framework for robust multimodal sentiment analysis. It employs five PID-inspired experts corresponding to modality uniqueness, redundancy, and synergy, and introduces masked-view Disentangled Interaction Regularization (DIR) to encourage distinct expert behaviors. A reliability-aware router combines unimodal predictive uncertainty, modality importance, and cross-modal consensus to adapt expert weights to individual samples. Experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS report competitive clean performance and improved aggregate robustness under noise, missing modalities, and cross-modal conflict, supported by component ablations and routing analyses.

Critical strengths
  1. The paper addresses a relevant robustness problem by extending adaptive fusion from modality-level weighting to interaction-level routing.

  2. The experimental evaluation is relatively comprehensive.

Critical weaknesses
  1. The interaction-specific expert roles are not sufficiently established by the proposed objectives and analyses. In Section 3.4, the uniqueness and redundancy objectives primarily enforce output invariance under selected modality removals, while the synergy objective trains the corresponding expert to predict the residual unexplained by the other experts. However, these behaviors do not necessarily imply that the experts capture unique, redundant, and synergistic task-relevant information. In particular, the residual assigned to the Synergy Expert may also reflect prediction errors or insufficient capacity of the other experts rather than genuine joint-modality information. Moreover, all five experts operate on the same fused multimodal representation, without explicit restrictions on the information each expert can access. Although Figures 4–5 and the supplementary analyses demonstrate distinct routing patterns and masked-view behaviors, these diagnostics largely reflect the properties encouraged by the training objectives. I would like to see more direct evidence of the experts' predictive roles, such as evaluating their contributions under controlled modality interactions or comparing their behavior against independently constructed interaction-specific targets.

  2. I²MoE [1] already introduces interaction-specific experts with weakly supervised objectives and sample-level reweighting to model unique, redundant, and synergistic multimodal information. Although this work is cited, its methodological relationship with RIDER-MoE is not sufficiently clarified, and it is absent from the experimental comparisons. More recently, QA-MoE [2] explicitly estimates modality reliability to guide expert routing under missing and degraded observations, making it particularly relevant to the proposed reliability-conditioned framework. While the paper includes several strong robustness baselines, including TMDC, the current comparisons do not sufficiently isolate the advantages of combining PID-inspired experts with reliability-aware routing over these closely related alternatives.

Refs:

[1] Jiayi Xin et al. I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts. ICML 2025.

[2] Yitong Zhu et al. QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis. ACL 2026.

Questions for authors
  1. How do the authors distinguish genuinely synergistic task-relevant information from the residual prediction errors of the non-synergy experts? In particular, what evidence supports interpreting the residual-fitting objective in Eq. (7) as learning cross-modal synergy rather than compensating for limitations of the other experts?
  2. Since all five experts receive the same fused multimodal representation, how do the proposed masked-view constraints ensure meaningful interaction-specific specialization? Beyond the output invariance and residual correlation analyses, is there evidence that the experts rely on the intended modality interactions when making predictions?
  3. How does RIDER-MoE differ methodologically from I²MoE [1], particularly regarding the interaction-specific expert design, weak supervision, and sample-level expert weighting? Which improvements can be specifically attributed to reliability-conditioned routing rather than the interaction-aware MoE architecture itself?
  4. Please consider extending the experimental comparison to include closely related methods such as I²MoE [1] and QA-MoE [2], where comparable evaluation protocols are available. Although QA-MoE is relatively recent, its close relevance to the proposed reliability-aware routing mechanism makes a comparison or a more detailed discussion particularly valuable.
  5. I would like to see more targeted experiments that directly evaluate the predictive contributions of the different expert types, such as controlled interaction-specific datasets, comparisons against simpler expert-role assignments, or analyses of expert-specific performance under independently characterized modality interactions.

I would be willing to raise my score if the authors can adequately address these concerns, particularly through clearer methodological differentiation, more comprehensive baseline comparisons, and stronger validation of the expert specialization mechanisms.

Reviewer_jejtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RIDER-MoE combines five interaction-inspired experts with a router biased by unimodal uncertainty and cross-modal agreement. Masked-view regularization encourages uniqueness and redundancy behavior, while a residual objective trains the synergy expert. Evaluation on MOSI, MOSEI, and CH-SIMS includes clean prediction, noise, missing modalities, and synthetic cross-modal conflict. The paper reports the highest aggregate robustness nAUC among the evaluated methods and provides component and training-cost ablations. The expert roles are behavioral constraints, not estimates of PID quantities.

Critical strengths
  1. The evaluation is substantially more informative than clean-test accuracy alone. It includes three datasets, three stressor families, full severity curves, and capacity and augmentation controls. The reported capacity-matched single-head and no-DIR augmentation controls make a simple capacity or basic-augmentation explanation less persuasive (Tables 4–5).

  2. The paper reports normalized training costs for masked-view regularization. Full DIR costs 2.80 times as much as the one-view no-DIR model, while stochastic DIR retains much of the reported robustness at 1.60 times that training cost (Table 4). This is a useful practical comparison, and the paper appropriately avoids treating its behavioral expert constraints as a formal PID decomposition.

Critical weaknesses
  1. A particularly close cited predecessor, I²MoE (Xin et al., 2025, Sections 3.2–3.4; arXiv:2505.19190), already combines modality-indexed uniqueness experts with redundancy and synergy experts, sample-dependent mixture weights, and output-level constraints under modality masking, with single-pass inference. RIDER changes the routing signals, fusion arrangement, and role losses, including its residual-synergy objective. The prior and consensus ablations support the value of those signals within RIDER, but no comparison with I²MoE measures the additional benefit over that interaction-aware design. The no-DIR control does not resolve this question because it removes the interaction losses altogether.

  2. The conflict stress test always leaves text unchanged. Appendix C.2, pp. 22–23, replaces audio and/or vision while retaining the original text and label, preferentially using opposite-polarity donors. A fixed text-only predictor would therefore be unaffected by this corruption, although it need not be accurate enough to match RIDER. The paper reports its clearest gains in this regime but provides no text-only or simple text-fallback comparison using the same text representation and supervision. Such controls would test whether adaptive interaction selection improves on relying on the unchanged text anchor. This concern is specific to the conflict protocol; the separate noise and missingness tests can also corrupt or remove text.

Questions for authors
  1. Can the authors compare with a documented I²MoE adaptation under the same feature, data-split, and stress-test protocol, explaining the fusion and capacity differences? A controlled replacement of the role losses would also help separate their value from that of the reliability prior.

  2. How does RIDER compare with a text-only predictor using the same text features and supervision on the existing conflict examples? Please give variability across available training seeds or corruption draws; a simple validation-chosen text fallback would further test whether interaction-level routing is needed.

Reviewer_8BU9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RIDER-MoE, a reliability-aware mixture-of-experts framework for robust multimodal sentiment analysis. The method introduces PID-inspired uniqueness, redundancy, and synergy experts, along with masked-view regularization to encourage expert specialization. A reliability-aware router dynamically adjusts expert contributions based on unimodal uncertainty and cross-modal agreement. Experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate competitive clean performance and improved robustness against noise, missing modalities, and cross-modal conflicts. Ablation studies and routing analyses further support the effectiveness of the proposed components.

Critical strengths

1. Clear and meaningful problem motivation. The paper distinguishes between Modality Degradation and Cross-modal Conflict, highlighting the limitations of conventional Modality Weighting methods in handling different types of modality failures. Extending robustness modeling from modality selection to interaction strategy selection offers a meaningful research perspective.

2. Well-motivated and structured methodology. The authors construct a MoE framework with PID-inspired Uniqueness, Redundancy, and Synergy Experts, and introduce expert-specific Masked-view Regularization (DIR). This design ensures that expert specialization is not solely driven by the router's implicit learning but is also guided by explicit training constraints.

3. Comprehensive experimental evaluation. The experiments cover three datasets (CMU-MOSI, CMU-MOSEI, and CH-SIMS) and three types of perturbations (Noise, Missingness, and Cross-modal Conflict). The paper also provides component ablations, routing behavior analysis, and a cost–robustness trade-off study, offering relatively thorough empirical validation.

Critical weaknesses

1. Insufficient evidence of genuine interaction disentanglement. All experts receive the same fused representation, while DIR primarily enforces prediction consistency under masked views. These constraints do not guarantee that the Uniqueness, Redundancy, and Synergy Experts actually learn their corresponding information-theoretic components. The existing diagnostics mainly demonstrate that the predefined behavioral constraints are satisfied, rather than providing evidence of genuine information disentanglement.

2. Limited methodological novelty. The proposed framework primarily integrates several existing techniques, including PID-inspired interaction decomposition, Mixture-of-Experts (MoE)-based fusion, uncertainty-aware routing, and masked-view regularization. Although this integration is reasonable, the overall methodological contribution appears relatively incremental.

3. Relatively outdated baselines. Many recent methods for robust multimodal sentiment analysis are not included in the comparisons. As a result, it remains unclear whether the proposed method achieves state-of-the-art performance.

Questions for authors

1. On methodological novelty: Could the authors further clarify the key methodological contributions beyond integrating existing techniques? In particular, how does the proposed PID-inspired expert design compare with a standard MoE equipped with the same routing and regularization mechanisms? Such a comparison would help establish whether the proposed interaction decomposition provides benefits beyond conventional expert specialization.

2. On baseline comparisons: Several recent methods for robust multimodal sentiment analysis appear to be missing from the experimental comparisons. Could the authors clarify the baseline selection criteria and provide comparisons with more recent competitive methods under consistent evaluation settings? This would help assess whether the reported improvements remain competitive with the current state of the art.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.