acceptodds
Withdrawn from ICLR 2027

Imagine-then-Reason for Training-free Zero-shot Composed Image Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Composed image retrieval (CIR) retrieves gallery images given a reference image and a modification instruction, whose joint interpretation is a key challenge. Prior work identifies CIR as a task in which the reference content to preserve, modify, or omit is ambiguous. However, our preliminary analysis of instruction types and retrieval performance highlights an underexplored source of fuzziness: . In particular, comparative instructions such as “make it darker” specify relative changes rather than concrete appearances, making them difficult to translate into self-contained target captions. Caption-based retrieval consistently underperforms on these queries. To address this challenge, we propose Imagine-Then-Reason for Composed Image Retrieval (ITR-CIR), a training-free zero-shot framework that (1) uses a visual target hypothesis to guide reasoning and generate concrete target captions; (2) fuses multimodal target estimators’ evidence for retrieval. ITR-CIR first edits the reference image into a possible target image, then uses an MLLM to assess this hypothesis against the original query and generate grounded target captions while disregarding editing errors. The final ranking combines the captions, the synthetic image, and a direct composition of the original query. Experiments on CIRR, FashionIQ, and CIRCO demonstrate the superior performance of ITR-CIR, achieving the best training-free CIRR and FashionIQ full-gallery recall.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
zero-shot composed image retrieval, training-free retrieval, foundation models, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yfUVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training-free zero-shot composed image retrieval, focusing on comparative modification instructions whose intended target appearance depends on the reference image and is therefore difficult to express as a standalone target description. ITR-CIR first uses an image editing model to generate a visual target hypothesis. It then provides the reference image, modification instruction, and generated image to a multimodal large language model to produce target captions, while prompting the model to disregard editing errors. The final ranking combines retrieval scores from the target captions, generated image, and original query composition using fixed weights. Experiments cover CIRR, FashionIQ, and CIRCO, including semantic-aspect analysis, component ablations, alternative model configurations, and qualitative examples.

Critical strengths
  1. A useful diagnosis of comparative instructions. The semantic-aspect statistics establish the practical relevance of comparative instructions, and the paper evaluates different prompting approaches on comparative and other queries using a shared retrieval backbone. Appendix A also validates the automatic auditor against official CIRCO annotations, achieving an F1 score of 87.5 for comparative statements. These results provide empirical support for the motivation.

  2. Meaningful improvements over the included baselines. On FashionIQ, average R@10 improves from OSrCIR’s 32.34 to 39.18 with ViT-B/32 and from 33.26 to 39.17 with ViT-L/14. Together with experiments across retrieval backbones and alternative model configurations, these results suggest practical potential for using generated images to assist target-caption generation.

Critical weaknesses
  1. Limited advance over the closest prior work. The analysis of comparative instructions is useful, but the added value of the method remains unclear. WISER already handles different modification intents through verification and adaptive fusion, and Paracosm already generates an image before describing it. WISER is mentioned but omitted from the quantitative comparisons. The paper does not establish what its focus on comparative instructions adds over WISER in accuracy or cost.

  2. Insufficient validation of the core design. Table 3 varies the fusion branches, while Table 4 tests ITR only in caption-only retrieval. Neither measures the effect of hypothesis-conditioned captions in the full system with the generated images and other branches held fixed. Simply captioning the generated image is also missing as a baseline. ITR-CIR still uses fixed weights to fuse retrieval scores, without accounting for differences in branch reliability across queries, a limitation already discussed by WISER. Finally, Tables 3 and 4 report caption-only CIRCO mAP@5 scores of 29.88 and 25.29, respectively, without explaining the difference.

Questions for authors
  1. How does ITR-CIR compare with WISER and Paracosm when using the same generation/reasoning models? Results on comparative and non-comparative queries would help clarify where the proposed design adds value. If the main benefit is efficiency, please include measured query latency, GPU memory usage, and gallery preprocessing costs.

  2. How does full-system retrieval compare when target captions are generated without the visual target hypothesis, by simply describing it, or using ITR?

  3. Why does caption-only retrieval on CIRCO yield 29.88 mAP@5 in Table 3 but 25.29 in Table 4? Please clarify the settings and report both results under the same configuration.

Reviewer_ZgmeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ITR-CIR, a training-free approach to zero-shot composed image retrieval. It first edits the reference image, then uses the synthetic image together with the original query to generate target captions. Retrieval combines caption, synthetic-image, and reference-modification scores. The paper focuses on comparative instructions and reports gains on CIRR, FashionIQ, and CIRCO results discrimination.

Critical strengths
  1. The paper identifies a useful limitation of target-caption methods. The semantic-aspect audit helps characterize this issue.
  2. The caption-only comparison provides evidence that a synthetic target image can help generate more useful descriptions.
  3. The framework is straightforward and uses frozen, publicly available models.
Critical weaknesses
  1. The methodological contribution seems incremental. IP-CIR [1] already uses imagined proxies, Paracosm [2] generates an image followed by its description, and WISER [3] verifies retrieved candidates. The proposed distinction is to check the synthetic image against the original query before captioning. The paper should clarify what substantive capability this interaction adds beyond these existing designs.
  2. Concerns of the verification step. Table 4 removes both the synthetic image and its associated instructions, so it cannot separate the benefit of additional visual input from that of verification. A comparison with identical inputs but without the verification instruction would better test this claim.
  3. The treatment of comparative instructions may still introduce unsupported specificity. Although the authors acknowledge that a generated image represents only one possible target, all sampled captions share this image and may inherit its arbitrary attributes. An analysis of unsupported attributes in the captions would help assess whether greater specificity remains faithful to the query.
  4. The closest empirical comparisons are missing. IP-CIR, Paracosm, and WISER are cited but absent from the main tables. Comparisons with applicable configurations, with backbone, foundation-model, and inference-cost differences stated, are needed to support the claim of the best training-free full-gallery recall on CIRR and FashionIQ.
  5. The contribution of ITR to the full system remains unclear. Table 3 varies fusion using ITR captions, while Table 4 evaluates caption-only retrieval. A comparison with and without ITR under identical fusion and sampling settings is missing.

References:

[1] Li Y, Ma F, Yang Y. Imagine and seek: Improving composed image retrieval with an imagined proxy[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 3984-3993.

[2] Wang T, Zhao Y, Kong S. Generating a paracosm for training-free zero-shot composed image retrieval[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 454-474.

[3] Wang T, Qu L, Yang T, et al. Wiser: Wider search, deeper thinking, and adaptive fusion for training-free zero-shot composed image retrieval[J]. arXiv preprint arXiv:2602.23029, 2026.

Questions for authors
  1. How does ITR-CIR compare with the closest visual-proxy methods, especially Paracosm, after accounting for model and inference-cost differences?
  2. With the same reference, instruction, and synthetic image, does the verification instruction help avoid editing errors and unsupported attributes in the caption?
  3. Does ITR still help under identical downstream fusion and caption-sampling settings?
Reviewer_SYacOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ITR-CIR, a training-free zero-shot composed image retrieval framework that addresses the difficulty of interpreting underdetermined modification instructions, particularly comparative expressions. Motivated by the limitations of existing caption-based retrieval methods in translating relative changes into concrete target descriptions, ITR-CIR introduces an imagine-then-reason pipeline. Specifically, an image editing model first generates a synthetic target image as a visual hypothesis, which is then provided to an MLLM alongside the reference image and modification instruction to generate target captions. The final retrieval scores combine three sources of evidence: the generated captions, the synthetic target image, and a direct composition of the original query. Experiments on CIRR, FashionIQ, and CIRCO demonstrate improvements in full-gallery retrieval, particularly on FashionIQ, supported by component ablations and additional analyses.

Critical strengths
  1. Clear and relevant motivation. The paper identifies instruction underdetermination as a challenge distinct from preservation ambiguity in composed image retrieval. Its analysis across multiple benchmarks demonstrates the limitations of caption-based methods on comparative instructions, motivating the use of visual hypotheses.

  2. Intuitive and training-free framework. The imagine-then-reason pipeline incorporates synthetic visual evidence into MLLM-based caption generation without additional training. Combining textual, visual, and original-query evidence also reduces reliance on any single target estimator.

  3. Competitive performance with supporting ablations. ITR-CIR achieves strong full-gallery retrieval results on CIRR and FashionIQ, particularly on the latter. Ablation studies demonstrate the complementary contributions of different retrieval estimators and the benefits of visual-hypothesis-guided caption generation.

Critical weaknesses
  1. Limited methodological novelty beyond existing generation-assisted retrieval frameworks. Previous methods such as IP-CIR and WISER have already explored incorporating synthetic target images into composed image retrieval. The main contribution of ITR-CIR is to condition MLLM caption generation on an edited image, followed by a relatively conventional weighted fusion of retrieval scores. While this modification is reasonable, the methodological advancement appears incremental, and the paper does not sufficiently establish a fundamentally different mechanism from existing approaches that combine generated visual evidence with language-based target estimation.

  2. Insufficient evidence for the claimed visual-hypothesis verification mechanism. A central motivation of the method is that the MLLM can examine the synthetic target image, identify useful modifications, and disregard editing errors or unsupported details. However, the experiments primarily demonstrate improvements in downstream retrieval accuracy rather than directly evaluating this verification process. The comparison with caption generation without visual hypotheses establishes the benefit of additional visual context, but does not clearly distinguish genuine verification and reasoning from simpler visual conditioning. Consequently, the evidence supporting the paper's central methodological interpretation remains incomplete.

  3. Unclear practical trade-offs and limited consistency across retrieval settings. The proposed framework introduces an additional image editing stage and requires multiple MLLM caption generations for each query, potentially increasing inference latency and computational cost. However, the paper does not provide a sufficiently detailed efficiency comparison against existing training-free methods. Moreover, despite strong full-gallery retrieval results, ITR-CIR does not consistently outperform competing methods on CIRR subset retrieval or CIRCO. These limitations raise questions about whether the improvements justify the additional complexity and how reliably the approach generalizes across different retrieval scenarios.

Questions for authors
  1. Given that prior methods such as IP-CIR and WISER already incorporate synthetic target images into CIR, could the authors further clarify the methodological distinction and provide a more controlled comparison under matched generation and retrieval settings? In particular, would a simpler approach that directly captions the edited image or conditions an MLLM on the same visual inputs without the proposed verification instructions achieve comparable performance?

  2. Can the authors provide direct evidence that the MLLM actually identifies and corrects errors in the synthetic visual hypothesis, rather than simply benefiting from additional visual context? For example, evaluating caption quality under deliberately imperfect edits or comparing different levels of editing accuracy would help substantiate the claimed verification mechanism.

  3. Could the authors report end-to-end inference latency and computational cost per query compared with representative training-free baselines? Additionally, can the authors further explain the performance degradation on CIRR subset retrieval and the comparatively weaker results on CIRCO? These analyses would help clarify the practical benefits and limitations of the proposed framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.