- Rating
- 1(−1)
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Cross-view Evidence Learning and Alignment (CELA) for multimodal sentiment analysis with incomplete observations. The framework combines Common Evidence Slot Fusion (CESF), full-view task supervision and representation alignment (FSA), and consistency between incomplete views (CMC). Inference requires only a single incomplete observation. Experiments on MOSI, MOSEI, and SIMS report improvements over nine evaluated baselines when averaged across matched training and testing missing rates. Additional experiments investigate alternative training protocols, structured missingness, component ablations, and the number of corrupted views.
Critical strengths
- The training–inference separation is practical and clearly defined. CELA exploits complete observations and additional corrupted views during training while requiring only one incomplete view at inference. This avoids the need for multiple observations or explicit reconstruction at deployment.
- The main improvements extend beyond the proposed aggregate metric. CELA improves rate-averaged BGSS over the strongest evaluated baseline by 0.0255 on MOSI, 0.0149 on MOSEI, and 0.0206 on SIMS.
Critical weaknesses
- The related-work discussion does not sufficiently address relevant studies on multimodal understanding with uncertain or missing modalities, including Yang et al. [1] and Chi et al. [2]. The authors should discuss these works and clarify how CELA differs from them in problem setting and training methodology.
- The distinct contribution of the proposed mechanisms remains insufficiently isolated. The framework combines shared projections, learned-slot cross-attention, supervised complete-view training, and representation consistency. The paper’s own Proposition 2 shows that FSA alignment already includes a dispersion term, while CMC increases its relative weight. These observations do not invalidate the method, but they make it important to establish what CELA contributes beyond a well-controlled combination of conventional fusion and consistency regularization.
- The deployment-oriented evaluation is narrower than the main benchmark coverage. The primary results average over separately trained models whose training missing rate matches the testing missing rate. The paper appropriately adds single-checkpoint fixed-rate and mixed-rate evaluations, but these are restricted to MOSI and four selected baselines. In particular, Self-MM (the strongest rate-averaged MOSI baseline in Table 1) is not included in these comparisons.
[1] Qu Yang, Qinghongya Shi, Tongxin Wang, and Mang Ye. Uncertain Multimodal Intention and Emotion Understanding in the Wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24700–24709, 2025. [2] Haozhe Chi, Minghua Yang, Junhao Zhu, Guanhong Wang, and Gaoang Wang. Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 121–130, 2022.
Questions for authors
- Can you disentangle architecture and objective contributions?
- What explains the discrepancies between Tables 4/18 and Tables 16/17?
- How much improvement remains over a comparably trained text-only model?
- Can you clarify the hardware measurement and robustness protocol?
- Do the single-checkpoint robustness gains generalize beyond MOSI?
2 comments
We noticed that you lowered your rating to 1 (confidence 4) without providing any explanation, despite our detailed rebuttal. Could you please clarify which concerns remain unresolved and what specifically justifies this substantial change in your assessment?
Without such feedback, we have no meaningful opportunity to address your concerns.
Related Work, Controlled Experiments, and Robustness Evaluation
W1: We sincerely thank the reviewer for highlighting Yang et al. (CVPR 2025) and Chi et al. (AACL 2022). We agree they clarify CELA's positioning. Yang et al. study uncertain modality availability and joint emotion-intention understanding in social media; CELA targets fine-grained missingness robustness via evidence fusion and cross-view learning. M3S combines random missingness augmentation with meta-learning. CELA instead coordinates same-sample complete and independently corrupted views via FSA, CMC, and CESF. We appreciate this guidance and will add both to Related Work.
W2: We sincerely thank the reviewer for examining each mechanism's role. CESF's common-evidence branch constructs common/modality-specific evidence for co-designed slot-based fusion. Removing either alters their interaction; w/o CESF tests the joint design. We acknowledge uncertainty in Table 3 (10 MRs × 3 seeds). Appendix Figure 7 shows CESF mainly helps at low-to-moderate MRs, while CMC has larger gains at some moderate-to-high MRs. Proposition 2 shows FSA-align already constrains incomplete-view dispersion; separately weighted CMC adjusts this without stronger full-view anchoring. Table 3, Figure 7, and missing-view tests support overall robustness.
W3: Please see Q4 and Q5.
Q1: We sincerely thank the reviewer for requesting clearer architecture/objective separation. Unlike Concat+MLP, CESF uses Learned-slot Cross-attention to flexibly integrate retained common and modality-specific evidence. Appendix A.6 Tables 16–17 compare CELA with conventional fusion under identical full-view supervision, two corrupted views, optimization, and Test masks; Fusion+Prediction parameters differ by just 0.0502%. Results: Anonymous repository – Anonymous GitHub. CELA gains 0.0238/0.0250 BGSS at MR=0.5/0.9, beyond extra views, supervision, or parameter count. FSA offers sentiment guidance through full-view supervision/alignment; CMC constrains corrupted-view differences. Ablations and view-count tests provide further evidence.
Q2: We sincerely thank the reviewer for identifying discrepancies between Tables 4/18 and 16/17. We agree that clarification is essential for reproducibility. Different training runs, Validation-selected checkpoints, and Test-mask protocols explain the numerical differences. In the revision, we will re-evaluate shared configurations using checkpoints from the same runs and identical Test masks, then update the results accordingly.
Q3: We sincerely thank the reviewer for asking about gains over a strong Text-only model. Under Matched-MR on MOSI: Anonymous repository – Anonymous GitHub. CELA gains slightly at moderate-to-high MRs but trails Text-only at low MR; three-rate averages nearly match. As missingness rises, text's relative advantage may weaken, allowing retained audio/visual cues to help. Under Mixed-MR, one model spans MRs: Anonymous repository – Anonymous GitHub. CELA gains on average, especially at MR=0.5, but nearly ties at high MR. Thus, gains depend on MR and protocol and may be clearer with one model across MRs.
Q4: We sincerely thank the reviewer for raising concerns about hardware scope and robustness. HE-100 has 576×128 signed logical weights per array, consistent with physical-chip array scales reported by Wang et al. (Nature Electronics, 2025) and Yu et al. (Nature Communications, 2025). This scale balances capacity, analog precision, and feasibility: larger arrays may increase voltage drops and readout costs, offsetting energy gains. We mapped the prediction head's two linear layers onto three hardware tiles and assessed variability through three independent programming cycles. Conductance drift up to one year was simulated, not physically measured. Details: Appendix A.9 and Anonymous repository – Anonymous GitHub.
We chose Matched-MR for controlled evaluation, following TFR-Net and related work with matched train/test MRs; MCT-HFR (ACM TOMM 2024) calls this one-to-one training. It enables comparisons at matched missingness levels. Section 4.6 adds Fixed/Mixed-MR to assess train-test MR mismatch; CELA leads in mean BGSS under both.
Q5: We sincerely thank the reviewer for noting that Section 4.6 covered only MOSI and omitted strong Self-MM. We added SIMS Fixed/Mixed-MR experiments and Self-MM in both protocols.
MOSI results: Anonymous repository – Anonymous GitHub.
SIMS results: Anonymous repository – Anonymous GitHub.
CELA leads in cross-MR mean BGSS on both datasets and protocols, supporting robustness to train-test MR mismatch.