acceptodds
Withdrawn from ICLR 2027

Calibrate Before Disentangling: Robust Multimodal Sentiment Analysis under Missing Observations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Robust inference under corrupted observations is crucial for Multimodal Sentiment Analysis (MSA), as real-world textual, acoustic, and visual inputs are often incomplete or unreliable. Existing methods typically recover missing information through direct cross-modal compensation, but this paradigm leaves two key issues insufficiently addressed. First, unreliable observations can induce representation shifts that propagate through subsequent cross-modal interactions. Second, cross-modal evidence can facilitate the recovery of shared semantics but is inherently less suited to restoring modality-specific information. Motivated by these observations, we propose the Calibrated Disentanglement Network (CaDiNet), following a “calibrate-then-disentangle” paradigm. CaDiNet first performs reliability-aware calibration through a dual-granularity residual offset mechanism that leverages missingness information to capture global corruption patterns and instance-specific deviations before cross-modal interaction. It then disentangles each modality representation into shared and private components and refines them using component-specific evidence. Shared semantics are enhanced using reliable cross-modal information, whereas modality-specific information is refined using evidence retrieved from modality-homogeneous historical memory. Extensive experiments on four MSA benchmarks demonstrate that CaDiNet consistently improves robustness across diverse missing conditions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Sentiment Analysis, Incomplete Multimodal Learning, Representation Calibration, Representation Disentanglement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jV7VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CaDiNet for multimodal prediction under token-level and whole-modality missingness. It first corrects corrupted representations using global and instance-specific residual offsets, then separates shared and private components. Shared components receive dominant-modality evidence, while private components are refined through retrieval from a memory bank populated during training. Experiments cover sentiment regression, dialogue-act classification, and intent recognition.

Critical strengths

The motivation is reasonable: correcting unreliable representations before extensive interaction may reduce the propagation of corruption. The framework also makes an explicit distinction between the evidence used for shared and private refinement.

Critical weaknesses

The central “calibrate-before-disentangling” claim is insufficiently isolated. Removing calibration demonstrates the usefulness of that module, but does not establish that the proposed ordering is necessary. Comparisons with calibration after disentanglement, or a parameter-matched joint correction mechanism, would better support the claimed contribution. The interpretation of memory retrieval as recovering modality-private information is also stronger than the evidence supports. Retrieved values belong to other training instances and may supply useful semantic priors without recovering the missing instance’s specific characteristics. The current ablations do not distinguish these explanations through shuffled-memory, prototype, or simpler retrieval controls.

Questions for authors

Does calibration before disentanglement outperform reversed or joint designs under matched parameters and supervision? How does private refinement compare with shuffled retrieval, modality-specific prototypes, and simpler memory-based prediction? Can the authors rerun relevant baselines, including DERL, under a unified protocol and report uncertainty?

Reviewer_qhDfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CaDiNet, a framework for robust multimodal sentiment analysis under incomplete observations. It introduces a calibrate-then-disentangle paradigm that first corrects missingness-induced representation shifts through global and instance-specific residual offsets. The calibrated representations are then decomposed into shared and private components, which are refined using cross-modal semantic evidence and modality-specific historical memory, respectively. Experiments on four benchmarks report competitive performance under token-level and modality-level missingness, supported by component ablations and representation shift analyses.

Critical strengths
  1. The paper presents a reasonably motivated framework for incomplete multimodal learning. Calibrating representations before subsequent disentanglement and using different evidence sources for shared and private components provide a coherent approach to handling missingness-induced representation shifts and heterogeneous multimodal information.

  2. The paper is easy to read.

Critical weaknesses
  1. The main experiments primarily compare against methods published between 2020 and 2024, omitting several highly relevant approaches from 2025–2026. In particular, HME (NeurIPS 2025) [1] introduces cross-sample retrieval for missing-modality enhancement, which is closely related to the proposed memory-based refinement. CMAD (ICCV 2025) [2] addresses representation inconsistency through feature distillation, while RECAP (AAAI 2026) [3] explicitly combines missing-modality recovery with shared/private information decomposition. Other relevant methods, such as TF-Mamba (EMNLP 2025) [4], are also absent. These omissions make it difficult to establish the advantages of CaDiNet over recent approaches addressing similar challenges. I would like to see more comprehensive comparisons under matched missingness protocols, or clearer justification where direct comparisons are not feasible.

  2. Section 3.3 retrieves modality-specific private representations from historical samples using the shared dominant-modality representation as the query. However, similarity in shared semantics does not necessarily imply similarity in modality-specific information, which may depend on individual acoustic or visual characteristics. This raises questions about whether the retrieved representations recover meaningful sample-specific private information or primarily introduce class-level priors from semantically related samples. Although Table 3 demonstrates the benefit of private refinement and Table 4 supports the proposed evidence assignment, these results do not directly establish the semantic validity of the retrieved private information. More targeted retrieval analyses, such as comparisons with random or label-matched neighbors and measurements of similarity to complete-input private representations, would help substantiate the proposed mechanism.

Refs:

[1] Yan Zhuang et al. Hyper-Modality Enhancement for Multimodal Sentiment Analysis with Missing Modalities. NeurIPS 2025.

[2] Yan Zhuang et al. CMAD: Correlation-Aware and Modalities-Aware Distillation for Multimodal Sentiment Analysis with Missing Modalities. ICCV 2025.

[3] Huiting Huang et al. Recovering Coherent Affective Patterns: Addressing Modality Missing in Multimodal Sentiment Analysis. AAAI 2026.

[4] Xiang Li et al. TF-Mamba: Text-enhanced Fusion Mamba with Missing Modalities for Robust Multimodal Sentiment Analysis. Findings of EMNLP 2025.

Questions for authors
  1. What evidence supports the assumption that samples with similar shared semantic representations also contain transferable modality-specific private information? How do the authors distinguish genuine private information recovery from the introduction of class-level or sentiment-related priors through retrieval?

  2. Does the retrieved private evidence align with the corresponding complete-input private representation of the current sample? Have the authors examined retrieval quality independently of downstream prediction performance, particularly under severe missingness?

  3. Appendix H provides evidence that DRO reduces representation shifts in MAG-BERT. Does the proposed calibrate-then-disentangle ordering provide advantages over applying calibration after disentanglement, or over a joint calibration and disentanglement design? Such a comparison would help clarify the importance of the proposed processing order.

  4. Please expand the baseline comparison to include recent relevant methods such as HME [1], CMAD [2], RECAP [3], and TF-Mamba [4], where comparable evaluation protocols are available. A more explicit discussion of the methodological differences would also strengthen the paper's novelty positioning.

  5. I would like to see more targeted retrieval experiments, such as comparisons with random or label-matched neighbors, alternative retrieval queries, or direct evaluation against complete-input private representations. These analyses would help establish whether the memory bank genuinely provides modality-specific information rather than generic semantic augmentation.

Reviewer_kNsNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CaDiNet improves MSA under missing or corrupted inputs using a calibrate-then-disentangle strategy. It first corrects representation shifts through reliability-aware residual calibration, then separates shared and private modality features for tailored refinement. Shared semantics use cross-modal evidence, while modality-specific information is recovered from modality-homogeneous memory. Experiments on four benchmarks show consistent robustness gains.

Critical strengths

A key strength is the calibrate-then-disentangle design, which addresses both corruption-induced representation shifts and the different recovery needs of shared and modality-specific information. A second strength is the consistent robustness across four MSA benchmarks and diverse missing conditions, supporting the generalizability of the proposed framework.

Critical weaknesses

I have several questions about this manuscript:

  1. How is the memory bank in the private evidence retrieval module updated during training? In addition, does the memory-bank size differ across modalities, and if so, what is the rationale for using modality-specific memory capacities?

  2. No baselines published in 2025 or 2026 are included in the comparison. The authors are encouraged to incorporate more recent methods to better demonstrate the competitiveness of the proposed approach against the latest advances.

  3. Do the global offset and instance-specific offset remain the same across the three modalities, or are they learned separately for each modality? The authors should clarify this design choice and explain its rationale.

  4. What is the specific role of the text modality compared with the visual and acoustic modalities? The authors should further analyze its relative contribution and explain whether text plays a dominant role in cross-modal interaction and final sentiment prediction.

  5. My major concern is the technical novelty of the proposed method. The current design appears to introduce only limited methodological improvements over existing approaches. The authors should provide a more insightful discussion to clearly articulate the novel technical contributions and explain how the proposed framework advances beyond prior work.

Questions for authors

Please clarify several points that could materially affect my assessment: first, how is the memory bank in the private evidence retrieval module updated during training, and does its capacity vary across modalities; second, are the global and instance-specific offsets shared across modalities or learned separately, and what is the rationale for this design; third, what is the specific contribution of the text modality relative to the visual and acoustic modalities, particularly in cross-modal interaction and final prediction; and fourth, given my concern about limited technical novelty, what are the key methodological advances of CaDiNet beyond existing calibration, disentanglement, and memory-based recovery approaches? As a suggestion for future work, the authors should also consider comparing with more recent 2025–2026 baselines to better position the method against the latest advances.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.