acceptodds
Withdrawn from ICLR 2027

CrossCheck: Fusing Global Relevance and Fine-Grained Semantic Evidence for Training-Free Composed Image Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Composed image retrieval requires overall target compatibility and precise adherence to a requested modification. However, global multimodal relevance can overlook specific attributes and relations, while fine-grained semantic evaluation may miss broader visual compatibility. We present CrossCheck, a training-free framework that combines these complementary sources of evidence. CrossCheck first estimates global relevance using CLIP-based retrieval and image-text matching. It then constructs semantic checks as target-contrast statement pairs and performs contrastive likelihood scoring: a frozen multimodal large language model compares the conditional token likelihoods of the two statements under the same visual context, yielding fine-grained evidence for individual modification and preservation requirements. We aggregate these signals with sensitivity to weakly supported checks and fuse them with global relevance for final ranking. CrossCheck requires no task-specific training and runs entirely locally with public weights. Across CIRCO, CIRR, and Fashion-IQ and three retrieval backbones, CrossCheck consistently improves over the compared methods. On a fixed CIRCO shortlist, global relevance and fine-grained semantic evidence achieve 29.56 and 30.43 mAP@5 individually, while their fusion reaches 41.60, showing that the gain arises from complementary ranking evidence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Composed Image Retrieval, Multimodal Retrieval, Training-Free Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qgA8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CrossCheck is a training-free framework for composed image retrieval that combines global relevance from multi-view retrieval and image–text matching with fine-grained semantic checks. Each check pairs a target statement with a contrast statement, scored through conditional token likelihoods. The evidence is calibrated across candidates, aggregated using mean–softmin, and fused with global relevance to rerank a shortlist. Experiments on CIRCO, CIRR, and Fashion-IQ report improvements across three retrieval backbones.

Critical strengths

The paper is clearly written, and the method figure communicates the pipeline effectively. Combining overall target compatibility with explicit modification and preservation requirements is well motivated. The experiments provide useful breadth, with particularly convincing gains on CIRCO. The fixed-shortlist comparisons and per-query analyses offer meaningful evidence that global relevance and semantic scores provide complementary ranking information. The scorer comparison also indicates that the improvements are not solely attributable to upgrading the MLLM. Using public model weights and supporting fully local inference adds practical value.

Critical weaknesses

1. The contribution of the proposed scoring design is not sufficiently isolated within the final system. In Table 3, target-only scoring achieves 29.29 CIRCO mAP@5, adding contrast statements increases this to 30.36, and mean–softmin brings it to 30.43, while the final fusion reaches 41.60. However, the scoring variants are compared only without global fusion. The results therefore leave unclear how much of the final improvement depends on the proposed contrastive formulation and aggregation, versus adding a simpler complementary semantic score. The key comparison is to fuse target-only, per-check yes/no, and paired-statement scores with the same global score, using identical candidates, MLLM, and semantic requirements, with comparable tuning. Evaluating mean–softmin within this final setting would also establish whether its small standalone improvement translates into a meaningful benefit for the complete system.

2. The connection between the semantic verification mechanism and specific retrieval corrections remains unclear. The method uses paired statements to check individual requirements and mean–softmin to emphasize weakly supported checks. However, a low check score may reflect an actual semantic violation, a poorly constructed contrast, or an MLLM scoring error, so emphasizing it need not improve the judgment. The per-query analyses in Figure 3 show beneficial ranking changes but do not establish which errors in attributes, counts, relations, or preservation requirements are corrected by the core design, or why. A systematic analysis connecting generated-check validity, requirement-level verification accuracy, and corrected or degraded retrieval cases would provide a more convincing explanation of when and why paired scoring and weak-check aggregation are effective.

3. Comparisons with closely related methods could be more complete. ImageScope already combines requirement-level verification with overall evaluation, while CoTRR uses semantic assessment for reranking. Discussing ImageScope and including CoTRR in the experimental comparisons would help clarify CrossCheck’s specific advance over existing approaches.

Questions for authors
  1. Which semantic errors does paired-statement scoring correct that target-only or yes/no verification misses?

  2. When does emphasizing the weakest normalized check help the final ranking, and why?

Reviewer_dRU6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CrossCheck, a training-free framework for composed image retrieval. It combines CLIP-based global relevance with fine-grained semantic evidence from a frozen MLLM. Target-contrast statement pairs are evaluated through conditional likelihood scoring and fused with global relevance for reranking. Experiments on CIRCO, CIRR, and Fashion-IQ show improvements over existing methods.

Critical strengths
  1. The paper addresses the limitations of global similarity in capturing fine-grained modification requirements.
  2. The method achieves competitive results across three CIR benchmarks and multiple retrieval backbones.
  3. The fixed-shortlist experiments help distinguish ranking improvements from candidate coverage effects.
Critical weaknesses
  1. The methodological novelty is limited. The framework largely combines existing retrieval, semantic verification, and score fusion techniques, without introducing a substantially new retrieval formulation.
  2. The proposed contrastive likelihood scoring offers limited improvements.
  3. The framework relies on multiple generative models, image editing, and expensive MLLM-based reranking. Without end-to-end latency or computational cost comparisons, its practical efficiency remains unclear.
Questions for authors
  1. What distinguishes the proposed semantic verification mechanism from existing MLLM-based verification and reranking approaches beyond the use of paired statements?
  2. What are the end-to-end inference latency and computational costs compared with WISER under comparable hardware settings?
Reviewer_yfvVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CrossCheck, a training-free framework for zero-shot composed image retrieval (ZS-CIR). It fuses global relevance from multi-view retrieval and BLIP-2 ITM with fine-grained semantic evidence from target–contrast likelihood scoring, using calibration and mean–softmin aggregation. The framework supports fully local inference with public weights and improves over WISER on the main metrics across three datasets and three CLIP backbones, raising CIRCO mAP@5 from 36.53 to 41.60 with ViT-G/14.

Critical strengths

1.The paper addresses a practical problem, a configuration that allows for local execution with publicly available weights is valuable. Prior work has noted weaknesses of CLIP-like models in attribute binding and relational understanding [1]. CrossCheck does not require updating model parameters for CIR, nor does it rely on external APIs. In contrast, the methods used for comparison utilize GPT-4o. For scenarios where data must remain local, this approach offers practical utility.

  1. CrossCheck improves over WISER on the main metrics across three datasets and three retrieval backbones, with the largest gains on CIRCO: about 4–5 percentage points in mAP@5. In Appendix A.3, replacing only the MLLM scorer with Qwen2.5-VL-7B, the verifier used by WISER, still gives 39.37 mAP@5
Critical weaknesses
  1. The paper lacks an analysis of actual operational costs and comparisons based on equivalent computational budgets. Appendix A.2 only reports the number of statement-likelihood evaluations, without per-query latency, peak memory, or throughput. It is unclear whether the retrieval gains justify the inference costs.

  2. The specific contributions of the core design and the need for hyperparameter tuning remain unclear. Table 3 reports the ablations of contrast statements and mean–softmin aggregation only for Q alone. It does not include A + target-only, A + contrast-mean, or per-check yes/no verification followed by fusion with A. In addition, different hyperparameters are used across datasets and categories, but the paper does not report a hyperparameter sensitivity analysis

[1] When and why vision-language models behave like bags-of-words, and what to do about it?

Questions for authors
  1. Could you compare paired likelihood scoring with target-only likelihood and per-check yes/no verification after fusion with A, using the same shortlist, MLLM scorer, and evaluation protocol, to clarify the added value of the proposed scoring design?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.