- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CoverCLIP, a coverage-aware vision-language alignment framework that aims to improve CLIP's sensitivity to graded semantic coverage. The authors first identify a limitation of existing CLIP models: their difficulty in consistently assigning higher image-text similarity scores to captions containing more correct visual information. To systematically evaluate this issue, they introduce CoverageBench, a manually verified benchmark containing 1,300 images with captions at four progressively increasing coverage levels. They further construct Coverage-1M, a large-scale training dataset with coverage-graded captions, and propose a mixed training framework incorporating coverage-ranking and hard-negative separation objectives. Experiments on image-text retrieval, zero-shot classification, and text-to-image generation demonstrate improvements in coverage sensitivity while largely maintaining general vision-language alignment capabilities.
Critical strengths
- Valuable problem identification and a systematic research framework.
The paper identifies a meaningful and relatively underexplored limitation of existing CLIP models regarding graded semantic coverage. More importantly, the authors provide a comprehensive framework that integrates problem formulation, benchmark construction, training data preparation, methodological design, and experimental validation. The introduction of CoverageBench and Coverage-1M provides valuable resources for studying coverage-aware vision-language alignment. Overall, the work is well organized and presents a coherent research pipeline.
- Reasonably comprehensive and informative experimental evaluation.
The paper conducts experiments across multiple tasks, including image-text retrieval, zero-shot classification, and text-to-image generation. The results demonstrate consistent improvements on CoverageBench while maintaining competitive performance on conventional downstream tasks. In addition, the ablation studies investigate the contributions of different training data sources, loss functions, ranking strategies, and hyperparameters. These experiments provide reasonable empirical support for the effectiveness of the proposed framework.
Critical weaknesses
- Insufficient evidence of broader fine-grained understanding and advantages over alternative vision-language models.
Although CoverCLIP achieves clear improvements on the proposed CoverageBench, its advantages in broader fine-grained vision-language understanding remain insufficiently demonstrated. The main comparisons are primarily conducted against CLIP and its fine-tuned variants, such as Long-CLIP and SmartCLIP. Comparisons with stronger or architecturally different vision-language alignment models are limited. Moreover, the improvements on the proposed coverage metric do not necessarily establish corresponding improvements in general fine-grained semantic understanding. While the text-to-image generation experiments provide additional evidence, the quantitative gains over the strongest baseline are relatively modest. More comprehensive comparisons and evaluations on independent fine-grained or compositional understanding benchmarks would strengthen the practical significance of the proposed approach.
- Limited methodological novelty.
The proposed framework mainly combines several established techniques, including mixed-data training, conventional contrastive learning, margin-based ranking, hard-negative separation, and stop-gradient operations. Although their integration is reasonable and empirically effective, the individual components are largely based on existing approaches, and the overall framework does not introduce a substantially new modeling mechanism. Consequently, the methodological contribution appears relatively incremental. The primary novelty lies more in the problem formulation and dataset construction than in the proposed learning algorithm.
Questions for authors
-
Generalization of CoverageBench. CoverageBench is constructed exclusively from Visual Genome and uses exactly four semantic atoms per image. How well does the benchmark generalize to images from different domains or scenarios involving substantially more semantic information? Could the authors discuss the limitations of this fixed four-level evaluation protocol?
-
Potential linguistic shortcuts. Although the authors control caption lengths, the average caption length still increases slightly with semantic coverage. Could the authors evaluate a text-only or length-based baseline to further rule out potential linguistic shortcuts in CoverageBench?
-
Trade-off between coverage and retrieval. In Table 16(a), the Coverage + ShareGPT-4V configuration achieves a higher Exact Chain accuracy (54.8%) than the full CoverCLIP configuration (51.2%), although the latter achieves better retrieval performance. Could the authors further explain this trade-off and clarify how the final data mixing ratio was selected?
-
Contribution of coverage-specific objectives. Table 16(b) shows that mixed-data CLIP training without the proposed auxiliary objectives already improves Exact Chain accuracy from 29.7% to 44.7%. Could the authors further clarify the relative importance of the coverage-aware training data and the proposed optimization objectives?
-
Practical significance of generation improvements. Although the generation results show statistically significant improvements, the numerical gains in Table 4 are relatively small. Could the authors provide additional evaluations based on human judgments or explicit concept-presence verification to better establish the practical benefits of CoverCLIP in text-to-image generation?
Polite follow-up: request for reviewers’ feedback on our responses!
Hi, we sincerely thank you very much for these constructive comments and evaluation of our manuscript. We would like to kindly ask you to take a look at our responses and reevaluate our work based on our clarifications. Please let us know whether our response addresses your concerns or whether there is any further detail we can provide to help address these concerns.
Thank you again for dedicating your time to reviewing our paper.