- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses noisy triplets setting in composed image retrieval. It estimates triplet reliability using agreement between the requested and observed reference-anchored transformations, together with separate modification-target and reference-target evidence. It then weights bidirectional ranking and adds alignment and uniformity regularizers. On Fashion-IQ and CIRR, the authors test several synthetic corruption rates and an extreme corruption variant.
Critical strengths
-
Well-motivated and coherent methodology. The motivation is convincing, and the proposed method appropriately addresses the identified limitations of query–target-level noise discrimination by explicitly modeling the semantic relations among the reference image, modification text, and target image. The paper follows a clear logical progression from motivation to methodology and experiments, making it easy to read and understand.
-
Strong robustness under severe corruption. The experiments demonstrate that TRUST remains effective even under extremely high synthetic noise rates. Its performance degrades more slowly than that of competing methods, providing strong evidence that the proposed reliability estimation and reliability-guided optimization improve robustness under severe triplet corruption.
Critical weaknesses
Limited realism of the XNTC setting.
I do not find XNTC sufficiently well-motivated as a novel noise setting. Although independently shuffling all three components makes the corruption extreme in magnitude, it does not appear to reflect the structured, partially matched noise likely to arise in practice. Real-world web crawling or VLM-based annotation is more likely to preserve some valid relations among , , and , while introducing hallucinated modifications or subtle target-attribute mismatches. Plus, fully unrelated triplets may even be easier to identify because all relational cues become inconsistent. Indeed, the paper’s own results show that modification-only corruption is substantially more damaging than XNTC. Therefore, introducing XNTC should not be treated as a strong novelty contribution without empirical evidence of its real-world relevance. I think it is better characterized as a synthetic stress test.
Questions for authors
Q1. Could the authors clarify why the Riemannian log map is needed for the transformation agreement score in Eq. (2)? Since the factor arccos(z^T z') / sqrt(1 - (z^T z')^2) in Eq. (1) is a positive scalar for each vector, it cancels in the cosine similarity of Eq. (2). Eq. (2) is therefore just the cosine between the tangent projections q_i - (q_i^T r_i) r_i and t_i - (t_i^T r_i) r_i, and no Riemannian machinery is required. The simpler Euclidean displacements q_i - r_i and t_i - r_i differ from these only by a component of size (1 - q_i^T r_i) along r_i, which is second order in the angle.
Empirically, I replaced Eq. (2) with 1 - cos(q_i - r_i, t_i - r_i) (the "chord" option in the released code) and ran FashionIQ (validation, mean AVG over 3 seeds):
noise 20%: 65.58 (log map) vs. 65.61 (Euclidean displacement) noise 80%: 62.25 (log map) vs. 62.18 (Euclidean displacement)
The seed standard deviation is 0.04-0.23, so the two are indistinguishable. For reference, plain query-target cosine is worse (65.20 at 20%, 61.03 at 80%), so the reference-anchored direction seems to help, but the log map does not.
What does the log-map formulation provide for reliability estimation beyond this simpler form? A controlled comparison across noise rates (0/20/50/80%) and datasets in the paper would materially affect my assessment of this design choice.
Q2. Could the authors provide stronger evidence that both components of TARE are necessary and complementary? In Table 5, removing either component leads to only small changes in retrieval performance. Could the authors show which noise types each component identifies that the other misses, and whether the gains are consistent across runs? To be honest, as currently presented, TARE seems to add considerable methodological complexity for only marginal performance gains. A clearer account of what each component contributes would materially affect my assessment of the method.
Q3. Could the authors provide evidence that the target uniformity loss L_uniform (Eq. 8) is actually necessary? The motivation (preventing target representations from contracting under alignment) is plausible in theory, but the empirical support appears thin. In Table 5, the "w/o L_uniform" row changes the FashionIQ average by only about -0.27, the smallest effect among the RGAO components, and the term is weighted 10x lower than the alignment term (lambda_u = 0.01 vs. lambda_a = 0.1) and further capped by the budget zeta_b. In my own single-seed reproduction on FashionIQ (20% noise, validation), removing L_uniform did not hurt: the average was slightly higher than the full model (65.92 vs. 65.54). I do not claim this is conclusive from one seed, but it makes me unsure that the term matters in practice.
It would help to see (i) the "w/o L_uniform" ablation with multiple seeds and at higher noise rates, (ii) a direct measurement of the claimed contraction (e.g., the mean pairwise distance or the uniformity value of the target tokens) with and without L_uniform, showing that contraction actually occurs under L_align alone, and (iii) the gradient magnitude of L_uniform relative to L_rank during training, given the budget in Eq. (11). Without such evidence, L_uniform looks like an additional hyperparameter and loss term whose benefit is not demonstrated. A clearer account of when it matters would materially affect my assessment of this component.
If the authors satisfactorily address the questions above, in particular the necessity of the log-map formulation, of both TARE components, and of L_uniform, with controlled comparisons across noise rates and multiple seeds, I would be willing to raise my score.