- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies multimodal domain generalization. It starts from the observation that a modality's contribution to the fused prediction can shrink or even reverse in an unseen domain; on EPIC-Kitchens D2+D3→D1, for example, audio's net contribution drops from +3.27 to +0.14. The proposed method, CSI, zeroes out each modality's features per sample and measures the resulting change in logits. These changes are turned into a sampling distribution, so modalities the current prediction depends on more are removed more often. Training anneals from complete-input to masked-input supervision, adds unimodal losses, and uses an EMA teacher that distills its complete-input predictions into the full, masked and unimodal outputs. Experiments on EPIC-Kitchens and HAC cover multi-source and single-source settings with four modality combinations, plus ablations, a comparison of selection strategies, loss landscape plots, robustness to noise and missing modalities, and an efficiency comparison.
Critical strengths
-
The paper is motivated by a concrete measurement rather than a generic claim. Fig. 1 shows class-level contribution reversals across domains and compares removal with other interventions, which makes the story easy to follow. The figures in general are clear and well designed.
-
The evaluation is complete for the benchmarks used: multi-source and single-source settings, all modality combinations, three seeds, robustness to noise and missing modalities, and efficiency. CSI is also cheaper than MBCD in training time and memory.
Critical weaknesses
-
CSI is very close to MBCD. MBCD already pairs an EMA teacher, which distills into both the fused and unimodal outputs, with adaptive dropout that favours the dominant modality. CSI keeps this teacher and distillation structure and borrows MBCD's feature augmentation. What it changes is the selection signal, from batch-level confidence to a per-sample logit gap, plus an annealed objective. The gains are small. Against the best baseline, the average change is between +0.16 and +1.47 on EPIC-Kitchens and between −0.68 and +1.35 on HAC. CSI also trails on the Cartoon domain in most settings. This looks fundamental, and the paper needs to show clearly what per-sample selection adds over MBCD.
-
The evidence for several claims is weak. The motivating analysis comes from a single split and a single model, and zeroing a modality at test time is itself out of distribution for that model. The method removes whichever modality is most influential, not the one whose benefit fails to transfer, so the link back to Fig. 1 is loose. The flatness claim rests on a single 2D landscape plot. The manuscript also contains leftovers from another version: the HAC robustness figure is labelled "FOMP" rather than CSI, Appendix B.2 refers to a "direction-aware regularization", and an undefined threshold δ appears in the implementation details. On top of that, some numbers don't match across figures and tables. This is fixable, but it makes it hard to tell which results belong to this method.
Questions for authors
-
Is Fig. 6 actually showing CSI, and what are "FOMP" and the "direction-aware regularization" mentioned in Appendix B.2?
-
What value of τ is used, and what does the threshold δ in the implementation details refer to?
-
How does CSI compare with MBCD's confidence-based dropout plugged into the same framework, on both datasets and all modality settings? This would most directly show what the per-sample selection adds.
-
Fig. 1(b) and Fig. 4(d) suggest that removal itself helps a lot, but in Table 4 random removal (61.59) is slightly below no removal (62.31). How should these be reconciled, and what does "Baseline" in Fig. 1(b)/4(d) refer to, given that ERM is 52.04?
-
Why does Fig. 4(d) report 61.78 for CSI while Table 1 reports 60.24 ± 0.45 for the same setting?