- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CrossCheck is a training-free framework for composed image retrieval that combines global relevance from multi-view retrieval and image–text matching with fine-grained semantic checks. Each check pairs a target statement with a contrast statement, scored through conditional token likelihoods. The evidence is calibrated across candidates, aggregated using mean–softmin, and fused with global relevance to rerank a shortlist. Experiments on CIRCO, CIRR, and Fashion-IQ report improvements across three retrieval backbones.
Critical strengths
The paper is clearly written, and the method figure communicates the pipeline effectively. Combining overall target compatibility with explicit modification and preservation requirements is well motivated. The experiments provide useful breadth, with particularly convincing gains on CIRCO. The fixed-shortlist comparisons and per-query analyses offer meaningful evidence that global relevance and semantic scores provide complementary ranking information. The scorer comparison also indicates that the improvements are not solely attributable to upgrading the MLLM. Using public model weights and supporting fully local inference adds practical value.
Critical weaknesses
1. The contribution of the proposed scoring design is not sufficiently isolated within the final system. In Table 3, target-only scoring achieves 29.29 CIRCO mAP@5, adding contrast statements increases this to 30.36, and mean–softmin brings it to 30.43, while the final fusion reaches 41.60. However, the scoring variants are compared only without global fusion. The results therefore leave unclear how much of the final improvement depends on the proposed contrastive formulation and aggregation, versus adding a simpler complementary semantic score. The key comparison is to fuse target-only, per-check yes/no, and paired-statement scores with the same global score, using identical candidates, MLLM, and semantic requirements, with comparable tuning. Evaluating mean–softmin within this final setting would also establish whether its small standalone improvement translates into a meaningful benefit for the complete system.
2. The connection between the semantic verification mechanism and specific retrieval corrections remains unclear. The method uses paired statements to check individual requirements and mean–softmin to emphasize weakly supported checks. However, a low check score may reflect an actual semantic violation, a poorly constructed contrast, or an MLLM scoring error, so emphasizing it need not improve the judgment. The per-query analyses in Figure 3 show beneficial ranking changes but do not establish which errors in attributes, counts, relations, or preservation requirements are corrected by the core design, or why. A systematic analysis connecting generated-check validity, requirement-level verification accuracy, and corrected or degraded retrieval cases would provide a more convincing explanation of when and why paired scoring and weak-check aggregation are effective.
3. Comparisons with closely related methods could be more complete. ImageScope already combines requirement-level verification with overall evaluation, while CoTRR uses semantic assessment for reranking. Discussing ImageScope and including CoTRR in the experimental comparisons would help clarify CrossCheck’s specific advance over existing approaches.
Questions for authors
-
Which semantic errors does paired-statement scoring correct that target-only or yes/no verification misses?
-
When does emphasizing the weakest normalized check help the final ranking, and why?