- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CaMLAP, an out-of-distribution generalization method that uses probability of necessity and sufficiency to reduce reliance on spurious correlations. Through cross-modal logic alignment and textual logic alignment, the method improves the robustness of vision-language models. Experiments on several benchmarks show promising results, with CaMLAP outperforming multiple prompt-tuning baselines in various benchmark.
Critical strengths
-
The proposed cross-modal logic alignment and textual logic alignment seems interesting and promising.
-
The proposed approach outperform various prompt tuning methods on the selected benchmarks.
Critical weaknesses
-
The experimental evidence is missing comparisons with many related works, such as [1, 2, 3], which makes the results less convincing.
-
The evaluation benchmarks used in the paper are limited compared with other related work [1, 2, 3]
Reference:
[1] CLIPood: Generalizing CLIP to Out-of-Distributions, ICML 23
[2] LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models, CVPR 23
[3] Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition, ICLR 25
Questions for authors
Provide more benchmarks results as outline in [1, 2, 3], and compare the performance.
Reference:
[1] CLIPood: Generalizing CLIP to Out-of-Distributions, ICML 23
[2] LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models, CVPR 23
[3] Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition, ICLR 25
Response to Reviewer kmre
We thank the reviewer for recognizing the promise of textual and cross-modal logic alignment and the improvements over prompt-tuning baselines on the evaluated benchmarks. We address the requested comparisons and benchmark coverage below.
1. Additional comparisons with related methods
We have added comparisons with CLIPood, LASP, and PromptSRC†+SAS. The last baseline incorporates SAS from Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition (ICLR 2025), addressing the reviewer's third reference. All reported values are means over three random seeds. The tables report Worst and Avg accuracy (%). Bold denotes the best result and underlining denotes the second-best result in each column. The ImageNet subset contains two classes.
ResNet-50
ViT-B/32
CaMLAP achieves the highest worst-group/domain accuracy in all eight settings among these methods. Compared with the strongest added baseline, the gains range from 0.4 to 6.9 percentage points, averaging 3.26 points across the eight settings. Average accuracy is higher in seven settings; on Waterbirds with ViT-B/32, it is slightly lower (80.3% versus 80.5%). These results strengthen the empirical comparison while preserving the main emphasis on performance in the worst group or domain.
2. Benchmark coverage and generalization scope
The four benchmarks cover distinct sources of distribution shift: background association in Waterbirds, gender association in CelebA, object co-occurrence in the two-class ImageNet subset, and visual style in PACS. Using both ResNet-50 and ViT-B/32 further examines whether the observed gains persist across different encoder architectures.
We agree that this evaluation does not cover the full benchmark suites or all generalization protocols considered in the cited works. Our conclusions are restricted to the evaluated group and domain shifts. Extending CaMLAP to more settings is a valuable direction for future work. We will incorporate the added comparisons and make the evaluation scope explicit in a future version.
We thank the reviewer again for recognizing the dual-alignment design and benchmark improvements, and for the constructive suggestions. We will incorporate the additional results and these clarifications into a future version, and use the broader benchmark recommendations to guide future improvements.