acceptodds
Under review as a conference paper at ICLR 2027

Diffusion Prior-driven Pseudo Target Extraction for Training-Free Zero-Shot Composed Image Retrieval

Abstract

Zero-shot Composed Image Retrieval (ZS-CIR) aims to retrieve relevant images from a gallery using both a reference image and manipulation text without requiring task-specific training. Recent approaches imagine the target by generating a pseudo target image in pixel space with an image generation model such as Qwen-Image-Edit, and then extract its embedding with the CLIP image encoder. However, the pseudo target image may introduce redundant image details that mislead retrieval, and the image generation and re-encoding incur substantial computational overhead. To address these issues, we propose Diffusion Prior-driven Pseudo Target Extraction (PPT), a training-free framework that extracts the pseudo target embedding directly with a diffusion prior in the CLIP image embedding space. Without any diffusion decoder or CLIP image encoder at query time, PPT achieves competitive performance on object/scene manipulation and attribute manipulation tasks with much lower latency. Our code will be publicly released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
composed image retrieval, zero-shot retrieval, diffusion prior

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ftvEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free diffusion prior-driven pseudo target extraction method for zero-shot composed image retrieval. The key motivation of this work is that existing method Paracosm explicitly generates a pseudo target image and re-encoding it with the CLIP image encoder to derive the target embedding. The authors argue that this incur substantial computational overhead. Overall, the motivation is reasonable. The proposed method takes advantage of diffusion-prior with Kandinsky model. Extensive experiments have been conducted on three public datasets. The proposed method cannot outperform Paracosm regarding R@10 and R@50, but it is more efficient than Paracosm.

My major concerns are listed as follows.

  1. The authors did not explain why they adopt Kandinsky? Any other options? The authors should provide more experiment results on this choice.

  2. Based experiment results in Table 1 and Table 2 as well as the efficiency comparison in Figure 1(b), the proposed method is not more efficient and effective than STiTch. Although Figure 1(b) shows that StiTch performs worse than PPT, this may be due to the dataset choice. On the other datasets, PPT cannot outperform StiTch.

  3. Analysis of miu only takes one value miu=0. The sensitive analysis of this hyperparameter is needed.

  4. The authors did not explain why the results of Paracosm on FashionIQ is missing. The original paper of Paracosm gives this results. By checking the results on original paper, the proposed method also cannot outperform Paracosm on this benchmark.

Critical strengths

The motivation is clear and good.

Critical weaknesses
  1. The authors did not explain why they adopt Kandinsky? Any other options? The authors should provide more experiment results on this choice.

  2. Based experiment results in Table 1 and Table 2 as well as the efficiency comparison in Figure 1(b), the proposed method is not more efficient and effective than STiTch. Although Figure 1(b) shows that StiTch performs worse than PPT, this may be due to the dataset choice. On the other datasets, PPT cannot outperform StiTch.

Questions for authors

none

Reviewer_xFJROpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies training-free Zero-Shot Composed Image Retrieval (ZS-CIR) and proposes the PPT framework. The proposed method leverages a pretrained diffusion prior to generate pseudo-target features directly in the CLIP image embedding space, avoiding pixel-level generation and re-encoding. These features are then fused with the textual query for retrieval. Experiments on CIRCO, CIRR, and FashionIQ evaluate the retrieval performance and computational efficiency of the proposed method.

Critical strengths
  1. The motivation is clear and reasonable, addressing the limitations of pixel-space pseudo-target generation in terms of computational cost and irrelevant content.
  2. The proposed method follows a simple training-free design without requiring additional training.
  3. The experimental analysis is relatively comprehensive, including comparisons of feature generation strategies and evaluations across different MLLMs, random seeds, and denoising steps.
Critical weaknesses
  1. The proposed method mainly employs a pretrained diffusion prior for pseudo-target feature generation and uses linear fusion for query construction. Although the design is practical, the overall methodological novelty remains relatively limited.
  2. Although PPT reduces inference latency, its retrieval performance remains below some existing training-free methods on FashionIQ and CIRR. Therefore, whether the efficiency gains justify the performance trade-off requires further evaluation.
  3. The paper attributes the performance degradation after decoding mainly to redundant image details. However, the current experiments do not sufficiently distinguish this explanation from other possible factors, such as semantic information loss or feature shifts introduced by re-encoding.
  4. The ablation studies of the diffusion prior are limited to CIRCO and CIRR, without further validation on FashionIQ. In addition, the effects of different conditioning inputs and fusion weights are not sufficiently analyzed.
  5. The implementation details of the ViT-B/32 experiments are insufficiently described, particularly regarding the diffusion prior and feature alignment. It is unclear how the prior-generated features are aligned with the retrieval embeddings, which limits reproducibility.
Questions for authors
  1. Further analysis of the performance-efficiency trade-off is suggested to clarify whether the reduced inference latency justifies the retrieval performance gap observed in some settings.
  2. Further analysis is suggested to determine whether the performance degradation after decoding mainly results from redundant image details, semantic information loss, or feature shifts introduced by re-encoding.
  3. Additional text-only versus pseudo-target comparisons on FashionIQ are suggested, together with ablations of different prior conditioning inputs and fusion weights.
  4. Further implementation details of the ViT-B/32 experiments are needed, particularly regarding the diffusion prior, feature dimensions, and alignment between prior-generated features and retrieval embeddings. Clarification of whether a mapping module is used is also suggested to improve reproducibility.
Reviewer_KoPzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduced PPT, which combines a frozen diffusion prior with MLLM-based query composition for training-free ZS-CIR. It fuses the generated image embedding with text features, bypassing image decoding and re-encoding. Experiments show gains over the text-only baseline on CIRCO and CIRR and lower query latency than Paracosm.

Critical strengths
  1. The method addresses a useful efficiency problem in training-free ZS-CIR, obtaining a visual query representation without rendering and re-encoding an image.
  2. The gains over the text-only baseline and across several caption-based pipelines support the usefulness of the prior.
  3. The same-prior decoding comparison, additional MLLM results and diffusion-seed statistics provide useful experimental controls.
Critical weaknesses
  1. The technology contribution of the method appears limited. CompoDiff already generates retrieval embeddings, while un2CLIP provide text-conditioned image embeddings. PPT mainly adapts a frozen prior to an MLLM-based CIR pipeline with weighted fusion. Avoiding CIR-specific training is useful, but the additional contribution remains modest.
  2. The explanation based on redundant image details is not fully established. The same-prior comparison in Table 4 supports bypassing decoding and re-encoding, but does not distinguish irrelevant details from missing requested content or embedding distortion. The selected Grad-CAM examples do not isolate the cause.
  3. PPT trails strong caption-based methods on FashionIQ, where a quantitative comparison with the same text-only pipeline is missing. The speedup over Paracosm is useful, but Figure 1's comparisons with other methods use different GPUs.
  4. The ViT-B/32 implementation is unclear. The specified priors produce ViT-L/14 or ViT-bigG/14 embeddings, whereas Tables 9-10 report ViT-B/32 results. Please specify the compatible prior or mapping used for Eq. (5), including any additional training or calibration.
Questions for authors
  1. What evidence distinguishes redundant details from missing requested content or embedding distortion in the decoding comparison?
  2. Does the prior improve overall FashionIQ performance over the same pipeline without it?
  3. Which prior or mapping is used for ViT-B/32, and does it require additional training or calibration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.