- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how supervision noise in medical VLM pre-training affects downstream adaptation, even with clean labels. Through controlled experiments, the authors identify persistent performance degradation and weakened Fisher discriminative structure. To address this, they propose DGTune, which combines weak Fisher-spectrum strengthening with cross-sample Fisher transfer to improve class discrimination and generalization. Experiments across different noise levels, medical VLMs, and other pretrained backbones demonstrate consistent improvements on ID and OOD tasks.
Critical strengths
- The paper addresses a meaningful problem: the persistent impact of noisy pre-training supervision on downstream adaptation, particularly in medical VLMs.
- The proposed method is well motivated by the Fisher discriminative analysis.
- The experimental results are strong and extensive, showing consistent improvements across different noise levels, backbones, and ID/OOD settings.
Critical weaknesses
-
NMTune has already investigated the impact of noisy pre-training on downstream tasks, while weak Fisher-spectrum strengthening is closely related to DeepLDA. The main novelty appears to lie in the cross-sample Fisher transfer component.
-
DGTune and NMTune use different training budgets and adaptation architectures. While the authors follow the official NMTune configuration, a comparison under matched training budgets would help better isolate the contribution of the proposed regularizers.
-
The main figure is not clear for me.
Questions for authors
The component ablation studies are currently limited to Kvasir-v2. I wonder whether the contributions of the two regularizers are similarly consistent across other ID datasets and OOD settings, particularly given that cross-sample generalization is a key motivation of the proposed method. Additional evidence on this would help strengthen the paper. I would be happy to consider raising my score if the authors can address my concerns.