- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SelRef to handle missing modalities at test time. It first combines predictions from the available modalities, then uses a refiner to improve them. During training, it aligns the refiner’s correction direction with that of a full-modality teacher, using only selective guidance judged to be helpful. The method is comprehensively evaluated on BraTS2023, UPMC Food-101, and MM-IMDb.
Critical strengths
The evaluation is fairly comprehensive, covering three tasks spanning brain tumor segmentation and image-text classification. It includes comparisons with recent methods, and SelRef achieves the best average performance among the compared baselines on all three main benchmarks: BraTS2023, UPMC Food-101, and MM-IMDb.
Critical weaknesses
-
The benefit of selective privileged guidance, the core contribution of SelRef, is not sufficiently discussed. In Table 3, task-only refinement already improves average Dice substantially over the Prediction Anchor (83.42% vs. 80.55%) and comes close to full SelRef (83.86%). A comparison using the same refiner and full-modality teacher with a stronger general distillation method, such as RLD (Knowledge Distillation with Refined Logits; ICCV 2025), would help clarify why SelRef’s guidance design is particularly useful for missing-modality learning and what it offers beyond standard teacher supervision. More analysis of how Privileged Task Gain and Current-State Compatibility affect teacher guidance and the resulting predictions would further support this core contribution.
-
It is unclear whether the gains on the main task, medical image segmentation, hold across datasets. On BraTS2020 (Table 8), SelRef performs worse than the best competing methods on ET and TC, while its WT advantage is also marginal. The paper should discuss why the improvements on BraTS2023 do not carry over consistently to BraTS2020.
Questions for authors
My questions mainly reflect the concerns in Critical Weaknesses. I hope the authors can address these concerns either by answering the questions below or in another proper way.
-
How would SelRef compare with using the same refiner and full-modality teacher, but replacing selective privileged guidance with a stronger general distillation method? For example, Knowledge Distillation with Refined Logits (RLD, ICCV 2025) also handles incorrect predictions from strong teachers. This comparison would help show why SelRef’s guidance is particularly useful when modalities are missing. Further discussion of this point would also be welcome.
-
What happens if Current-State Compatibility is removed while Privileged Task Gain is kept? Section E.5 shows that 40.97% of task units have positive gain, while 40.91% pass both conditions. Since Compatibility removes very few additional units, what benefit could it provide specifically?
-
Why are the results different on BraTS2020 and BraTS2023? On BraTS2020 (Table 8), SelRef is below the best competing methods on ET and TC segmentation, and its WT advantage is also marginal. What might explain why the gains on BraTS2023 do not carry over consistently?
-
Could the authors explain SimMLM’s Food-101 results in Table 2? It has the highest text-only accuracy (86.27%), but its image-only accuracy (65.70%) is much lower than all the other methods (around 77% or higher). This large gap seems unusual. Could it be due to a reporting error, insufficient training, differences in the experimental settings, or a limitation of SimMLM itself?