- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The task studied in this paper is combined image retrieval, which involves retrieving the target image from the candidate images based on the reference image and the modified text. Recent supervised CIR methods usually further incorporate specialized local modeling on top of strong visual-language pre-training (VLP) representations to complement the information provided by the global features. However, it is still unclear whether the benefits of local modeling depend on the specially designed local modeling process when strong global representations are already available. The authors first conducted a controlled analysis on the ENCODER and found that replacing its specialized local branch with lightweight token compression resulted in only a slight performance decline. At the same time, after controlling the global-local branch weighting on FashionIQ, the performance of dense and compressed local representations was relatively close. Based on these observations, the authors proposed DART, which allocates the limited compact local capacity to different reference image evidence according to the semantics of the modified text. The authors evaluated this method on FashionIQ and CIRR, covering three supervised CIR frameworks: ENCODER, COMBINER, and OFFSET, and achieved corresponding performance improvements overall.
Critical strengths
- This paper conducts a controlled experiment to analyze the individual roles of strong global VLP representations and compact local modeling. It does not make any direct assumptions, thus establishing the method's motivation on a clearer empirical basis.
- This paper proposes the DART module, which allocates reference-token weights token-wise based on modification semantics, enabling the compact local representation to adapt to the current modified text instead of performing fixed compression on all reference tokens. And The paper evaluates on three representative supervised CIR frameworks, namely ENCODER, COMBINER, and OFFSET, and covers FashionIQ and CIRR, demonstrating a certain degree of framework compatibility.
Critical weaknesses
- This paper only conducted a sensitivity analysis on the regularization coefficient β. However, there was a lack of corresponding parameter sensitivity experiments for the equally important hyperparameter compact-token count K. Moreover, the paper only studied β in the ENCODER framework and the FashionIQ dataset, lacking experiments in other frameworks and datasets, and failing to conduct sufficient parameter sensitivity experiments. Additionally, it did not provide corresponding Case Studies or baseline comparison retrieval success/failure cases, making it impossible to intuitively demonstrate from which types of text modifications the performance improvement of DART mainly comes.
- According to the description in the paper and the experimental presentation, DART seems more like a general enhancement plugin. Currently, the paper only integrates it into three supervised CIR frameworks: COMBINER, ENCODER, and OFFSET, ignoring other local modeling structure methods based on CLIP, and conducting experiments only on the FashionIQ and CIRR datasets, also neglecting the Shoes dataset in COMBINER, ENCODER, and OFFSET, which affects the applicability of the proposed DART module.
Questions for authors
- Besides β, can the author also conduct sensitivity experiments on K and carry out corresponding experimental analysis?
- Can the author provide the original framework and the successful and failed cases of DART's search in terms of retrieval?
- Do the current three representative frameworks and two datasets represent the main claims of the author regarding the generalization of the method? If the author intends to use DART as a more generalized plug-in module, can it be further verified on other local modeling frameworks based on CLIP and related benchmarks?
3 comments
Response to Reviewer GK2G: Expanded Dataset Evaluation
Q3: Generalizability Across Datasets and Frameworks
Thank you for raising concerns about the generalizability of DART.
Additional Evaluation on Shoes. We have included experiments on Shoes using all three frameworks (Appendix A.1, Table 6). The performance changes after introducing DART are summarized below.
| Framework | ΔR@1 | ΔR@10 | ΔR@50 | ΔAvg. Recall |
|---|---|---|---|---|
| COMBINER | −0.87 | −0.19 | −0.02 | −0.36 |
| ENCODER | +2.11 | +2.28 | +1.56 | +1.99 |
| OFFSET | +3.04 | +2.08 | +0.32 | +1.82 |
Δ denotes the performance change after adding DART to each reproduced baseline, calculated from the mean results over three independent runs. All values are in percentage points.
DART improves retrieval performance for ENCODER and OFFSET on Shoes but slightly reduces COMBINER’s performance. These results provide further evidence of DART's applicability across datasets, while highlighting that its benefits vary across host frameworks.
Additional Cross-Dataset Evaluation on i-CIR and CIRCO. We have additionally evaluated CIRR-trained ENCODER and COMBINER models and their DART variants on i-CIR and CIRCO without further fine-tuning (Appendix A.1, Tables 7 and 8).
As shown below, DART improves all reported metrics for both frameworks on i-CIR, as well as all reported mAP metrics on the CIRCO validation and test sets, with larger gains for ENCODER on both datasets. These results provide additional evidence of DART’s applicability beyond the training dataset.
| Framework | mAP | mmAP | minmAP |
|---|---|---|---|
| ENCODER | 37.80 | 35.82 | 12.62 |
| ENCODER + DART | 39.96 | 38.36 | 13.76 |
| COMBINER | 19.79 | 20.13 | 5.47 |
| COMBINER + DART | 20.61 | 20.69 | 5.72 |
Cross-dataset retrieval results on i-CIR (%). All models are trained on CIRR and evaluated without additional fine-tuning.
| Validation | Test | |||||||
|---|---|---|---|---|---|---|---|---|
| Framework | mAP@5 | mAP@10 | mAP@25 | mAP@50 | mAP@5 | mAP@10 | mAP@25 | mAP@50 |
| ENCODER | 22.28 | 23.30 | 25.22 | 26.13 | 21.91 | 22.40 | 24.42 | 25.32 |
| ENCODER + DART | 25.34 | 26.13 | 28.19 | 29.13 | 23.96 | 24.53 | 26.67 | 27.58 |
| COMBINER | 21.26 | 22.05 | 23.95 | 24.80 | 20.29 | 20.87 | 22.63 | 23.55 |
| COMBINER + DART | 22.36 | 23.03 | 24.86 | 25.70 | 20.42 | 20.89 | 22.73 | 23.58 |
Cross-dataset retrieval results on CIRCO (%). All models are trained on CIRR and evaluated without additional fine-tuning.
Evaluation on Additional Host Frameworks. We apologize that the adaptation and baseline reproduction required for a new host framework could not be completed within the revision period. Our conclusions remain limited to the three evaluated frameworks.
Response to Reviewer GK2G: Additional Beta-Sensitivity Analysis
Q1 - 2: Extended Hyperparameter Sensitivity Analysis
Following our previous response, we extend the -sensitivity analysis to COMBINER and OFFSET on FashionIQ (Appendix A.2, Figure 6(a)).
ENCODER and COMBINER achieve their best results at , while OFFSET performs best at . DART exhibits relatively stable performance across the tested values in all three frameworks, supporting the use of as a common setting.
We update Figure 6(a) and the corresponding analysis in the revised manuscript. Thank you again for your constructive suggestion.
Response to Reviewer GK2G: K Sensitivity, Qualitative Results
We thank the reviewer for the detailed and constructive feedback on hyperparameter sensitivity, qualitative analysis, and the applicability of DART across frameworks and datasets. All revisions are highlighted in blue in the revised manuscript.
Q1: Extended Hyperparameter Sensitivity Analysis
We thank the reviewer for highlighting the need for a more comprehensive hyperparameter sensitivity analysis.
Sensitivity to the Number of Local Tokens K: We have included a K-sensitivity analysis on FashionIQ within the ENCODER framework (Appendix A.2, Figure 6 (b)), comparing ENCODER, TL+BA, and DART across K=2,4,6,8.
The results show that the original ENCODER exhibits a substantial performance drop at larger values of K, whereas TL+BA and DART remain relatively stable.
DART consistently outperforms TL+BA across the tested capacities, suggesting that modification-conditioned allocation provides benefits under different local token budgets.
Additional β Sensitivity Analysis. We apologize that time constraints have prevented us from completing the β sensitivity analysis across other frameworks or datasets. We will complete the analysis on FashionIQ with other frameworks during the remaining discussion period.
Q2: Qualitative Retrieval Results
Thank you for suggesting a qualitative analysis to better understand the types of queries where DART performs well and where it remains limited.
We have included qualitative retrieval comparisons between the original ENCODER framework and DART, including both successful and failed cases on FashionIQ (Section 5.4, Figure 5) , CIRR (Appendix A.3, Figure 7) and i-CIR (Appendix A.3, Figure 8).
The successful cases show that DART better satisfies modification requirements while retaining relevant information from the reference image. The failed cases reveal a remaining limitation in fine-grained discrimination of visual attributes, such as the degree of shininess, fur darkness, and illumination conditions.