- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates training-free zero-shot compositional image retrieval, a task that involves retrieving a target image based on a reference image and a modification text—ensuring the target retains the reference image's content while satisfying the specified modifications. The authors propose "Sharpen," a method that globally adjusts relevance scores across the database by modifying scaling and popularity suppression factors. It then employs a Multimodal Large Language Model (MLLM) to evaluate candidate images based on modification consistency and overall alignment with the reference image, subsequently re-ranking the candidates using these scores. This approach eliminates the need to train new retrieval models, instead leveraging the MLLM to assess candidates against the intended target. Experiments conducted on the CIRCO, CIRR, and FashionIQ datasets demonstrate significant performance gains over baselines utilizing the same MLLM and CLIP models; the paper also presents component ablation studies, model analysis, and an evaluation of inference costs.
Critical strengths
- The problem is clearly defined, and the method design aligns strongly with the problem. To address a specific failure mode in composed image retrieval—where results retain the content of the reference image without actually executing the requested modification—Sharpen introduces three steps: modified text augmentation, popular image suppression, and candidate image verification. The overall workflow is simple and modular, and it does not rely on additional training.
- Compared to the controlled baseline, the method demonstrates significant performance gains and includes component-level validation. Using identical MLLM and CLIP configurations, Sharpen substantially outperforms the baseline across key metrics on the CIRCO, CIRR, and FashionIQ datasets. Furthermore, through experiments involving component ablation, varying MLLMs, candidate set sizes, and weight sensitivity, the authors demonstrate that both global adjustment and local candidate evaluation contribute to the final results.
Critical weaknesses
- The transductive query bank used for evaluation in the paper consists of the target-description embeddings of all evaluated composed queries, excluding only the query currently being scored. Consequently, the model leverages information from other test queries during evaluation. This differs from standard query-independent scenarios or settings involving new, online queries. It is recommended that the authors report results obtained without the query bank, or construct the query bank using an independent set of training or validation queries, to assess the impact of this setup on performance.
- Table 5 does not explicitly state whether the reported inference time accounts for processes such as query bank construction and similarity computation. Furthermore, details regarding the validation split used for hyperparameter tuning are missing. The authors are encouraged to provide these missing details concerning efficiency experiments and tuning protocols, enabling readers to evaluate the actual deployment costs and the extent to which the results depend on hyperparameter tuning.
Questions for authors
- Please report the results obtained when constructing the query bank using only independent training/validation queries (excluding evaluation queries) to assess the impact of the transductive setting on performance.
- Does the figure of 3.19 seconds in Table 5 account for costs such as query bank construction, preprocessing, similarity computation, caching, and storage? If not, please report the offline construction costs and online query costs separately. Additionally, please provide details regarding the size of the validation split, the parameter search range, and the method used to isolate the test set.
4 comments
Q1 / W1
To address your concern about using evaluated queries to compute popularity scores, we replaced the transductive bank with a fixed disjoint bank of 10,000 human-written COCO captions, keeping all other settings unchanged. The bank contains no evaluated queries or target labels. Popularity scores can be computed once before queries arrive, and each query is processed independently. For comparison, Table 3 reports Sharpen without a query bank, with popularity suppression disabled.
| Query bank | CIRCO mAP@5 | CIRR R@1 | FashionIQ R@10 |
|---|---|---|---|
| None (no popularity suppression) | 35.88 | 53.91 | 43.53 |
| Transductive query bank | 38.41 | 53.96 | 45.34 |
| Fixed disjoint query bank (COCO captions) | 37.44 | 54.15 | 43.95 |
With the COCO-caption bank, popularity suppression improves CIRCO / CIRR / FashionIQ by 1.56 / 0.24 / 0.42 points over no suppression. These gains are obtained without access to the evaluated queries. Using independent banks of training-query descriptions gives 53.41 CIRR R@1 and 45.37 FashionIQ average R@10. We have added the independent-bank comparison and construction details in Appendix D.6 (Table 12), with a reference in Sec. 4.1.
Q2 / W2
The 3.19 s in Table 5 is the total online time per composed query, averaged over CIRCO queries processed in batches on one RTX 5080, excluding model loading. It includes target-description generation, CLIP text encoding, database scoring and candidate selection, 100 MLLM scoring passes at N=50, and score fusion. Candidate-only assessment takes 1.33 s and reference-conditioned assessment takes 1.81 s. The remaining stages take about 0.06 s.
Database image embeddings are computed offline and cached. Encoding takes 25.6 ms per image. Computing popularity scores takes about one second or less on CPU and is repeated when the bank changes. CIRCO's cached embeddings require 482 MB in fp32, and its popularity scores require 0.47 MB.
Constructing a bank of generated descriptions requires description generation and CLIP text encoding. Description generation takes 0.045 s per bank query under batched processing. A bank of existing COCO captions requires only text encoding.
We selected the hyperparameters on a small sample of 5% of the validation split, searching γ ∈ [0.1, 1.0], γ_h ∈ [0.25, 1.0], k ∈ [1, 16], w_s ∈ [0.2, 0.5], and w_a, w_r ∈ [0, 0.7]. We have clarified the validation sampling in Sec. 4.1 and the timing boundary in Sec. 4.5, and added the stage breakdown, offline costs, and search ranges in Appendix D.12 (Table 17).
Thank you again for your review. We have responded to each of your comments and revised the paper accordingly (changes marked in blue). As the discussion period ends on November 18, we would appreciate knowing whether our response has addressed your concerns, and we would be glad to clarify anything that remains.
Thank you for your positive assessment of the method and its gains over the controlled baseline. We address both questions below. The manuscript revisions are marked in blue.