- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper solves the issue of multi-branch retrieval score fusion in Composed Image Retrieval (CIR). The authors argue that score consistency across different retrieval branches does not necessarily imply that a candidate image satisfies the text-based modification requirements, as such consistency might stem merely from shared visual features. To address this, the paper proposes Contrastive Evidence Calibration (CEC), which models score discrepancies across five channels: parent model scores, query branch differences, text-modification matching, reference image similarity, and composed query matching. Building upon fixed score offsets, the method employs five bounded, learnable coefficients for linear adjustment and is optimized using a global gallery retrieval loss, an optional local candidate set loss, and regularization constraints. Experiments were conducted on the FashionIQ and CIRR datasets. CEC achieved an average recall of 68.82 on FashionIQ and 82.43 on the official CIRR test set, representing improvements of approximately 0.31 and 2.21 percentage points, respectively, over the respective parent models. While the results suggest that score adjustment can enhance retrieval performance, the work lacks sufficient innovation, and the reported experiments fail to adequately demonstrate the inherent effectiveness and independent contribution of the proposed learnable calibration mechanism.
Critical strengths
- The experimental setup is relatively standardized and provides some valuable controlled comparisons.
- Negative results and the limitations of the methods are disclosed quite adequately.
Critical weaknesses
- The core method lacks innovation; fundamentally, it remains a form of constrained linear score fusion.
- Key ablation studies on the FashionIQ dataset indicate that the learnable calibration module yields no additional performance gains.
- CIRR serves as the primary experimental basis for demonstrating the effectiveness of the proposed full calibration method. Table 4 reports that CEC achieves a mean performance improvement of approximately 2.21 percentage points over the parent model. However, this comparison conflates multiple changes—specifically, fixed score offsets, five learnable calibration coefficients, and local candidate-set supervision—without providing sufficient controlled experiments to isolate the individual contribution of each factor.
- The method relies on a complex and computationally expensive parent model, raising concerns regarding both practical efficiency advantages and the fairness of experimental comparisons.
- The theoretical analysis is rudimentary and fails to establish a substantive link to the core research motivation.
Questions for authors
- Table 2 shows that the "fixed offset" approach achieves an Ensemble Mean identical to that of the full CEC, while the learned coefficients in Table 7 are all close to zero. Could the authors explicitly explain what irreplaceable advantage the five learnable coefficients offer over fixed weights on this dataset?
- Could the authors provide a complete ablation study for CIRR, comparing the parent model, the fixed offset approach, the version with learnable calibration (using the full gallery only), and the full CEC (incorporating local candidate group supervision)? It would be particularly valuable to understand the individual contribution of each component to the 2.21 percentage point gain and whether these contributions are stable.
- Why is there almost no need for learned calibration coefficients on FashionIQ, whereas CIRR requires significant coefficients and local group supervision? Does this suggest that the proposed method fails to establish a stable, universal rule for evidence calibration? Could the authors further analyze the reasons behind this discrepancy?