- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that target-relevant cross-modal redundancy explains why multimodal models stay robust when a modality is missing. It defines redundancy as , derives upper and lower bounds on the average error when one modality is missing, and proposes ROM.
ROM has two losses: L_LMR (cross-modal contrastive loss) and L_CCD (within-class cross-covariance penalty)
Experiments were ran on Food-101, CUB-200 and HatefulMemes, with ViLT and CLIP, and shows gains
Critical strengths
- The question of what information makes representations robust to missing modalities is important.
- The method is simple, cheap, and works with both single-stream and dual-stream models.
- Gains are large in several mismatched train/test settings.
- The ablation is informative and honestly reports where L_CCD does not help.
Critical weaknesses
-
W1. The central question is not fully answered. The introduction asks what property makes representations robust. The paper proposes redundancy, but the evidence is mainly a correlation, and the theory (see W2) points to a simpler property: how well each modality predicts the label on its own. No experiment separates redundancy from unimodal predictive strength.
-
W2. The link between the bound and redundancy is unclear. Φ_X is defined as the average of H(Y|X1) and H(Y|X2), so the bound depends only on how informative each modality is alone. Redundancy appears only through an identity, and only when the joint information I(X1,X2;Y) is held fixed. For example, two datasets with the same unimodal informativeness but different redundancy give the same bound. Also, R_X is the co-information, which equals redundancy minus synergy. R_X can therefore increase because synergy decreases, and it can be negative on synergy-heavy tasks such as HatefulMemes. Synergy is never discussed. Meanwhile the experiments measure redundancy with a different, PID-based measure (Redundancy Bottleneck), so the paper defines one quantity and validates with another.
-
W3. The bound has limited practical value. The lower bound is meaningful only when a modality is weak; for a reasonably good modality it becomes negative and says nothing. The upper bound is very loose. The bound is never computed on any dataset. It describes an ideal single-modality classifier, not the trained multimodal model with a missing input. It depends on R_X, which training cannot change; no result connects the learned R_θ to the model's error.
-
W4. Novelty is overstated. Robult (Nguyen et al., IJCAI 2025) already targets redundancy for missing-modality robustness, using a label-aware contrastive loss. It is cited but not compared or discussed in depth. UME-MMA (arXiv 2310.06383) already gives an information-theoretic analysis of missing modalities. FactorCL (Liang et al., NeurIPS 2023) uses the same decomposition I(X1;X2) − I(X1;X2|Y) with a similar two-term strategy. It is not a missing-modality method, but it should be cited and the difference stated. L_LMR has the standard form of a cross-modal supervised contrastive loss. The bounds follow standard Fano and Feder–Merhav arguments. The genuinely new part seems to be L_CCD, which has mixed results (it hurts on CUB)
-
W5. Baselines are incomplete. Robult should be compared directly. It reports on overlapping benchmarks. A unimodal-ensemble baseline (e.g. UME-MMA in the CLIP setting) and simple controls (modality dropout, per-modality auxiliary cross-entropy) are needed. If these match ROM, the gains come from unimodal strength, not redundancy. SimMLM and gradient-guided modality decoupling are cited as key prior approaches but not evaluated. MM-IMDb, standard in the MAP/MSP line, is missing.
-
W6. The redundancy measurement needs clarification. How is the Redundancy Bottleneck estimated on high-dimensional embeddings? Since redundancy is bounded by unimodal informativeness, the r = 0.97 correlation may largely reflect unimodal accuracy. Please show the correlation after controlling for it. The y-axis of Figure 4 (right) is undefined, and the redundancy values in Figure 1 (0.27 vs 0.50) look inconsistent with Figure 4 (0 to 5 bits).
Questions for authors
I am open to update my score provided authors could answer the following questions
- Does ROM outperform per-modality auxiliary cross-entropy, modality dropout, and a unimodal ensemble under the same backbone and budget?
- How does ROM compare with Robult, and what is the technical difference between L_LMR and Robult's contrastive loss?
- How do you reconcile R_X (redundancy minus synergy) with the PID redundancy measure used in the experiments? Can ROM reduce useful synergy?
- What are the numerical bound values on Food-101 and CUB-200?
- Which parameters are trained for each method in the ViLT setting?
- How is L_CCD computed for classes with only one or two samples in a batch (about 2.5 samples per class on CUB with batch size 512)?