acceptodds
Under review as a conference paper at ICLR 2027

TRUST: Semantic Relation-Aware Robust Learning Framework for Composed Image Retrieval

Abstract

Composed Image Retrieval enables flexible image search using a reference image and a modification text, yet suffers from Noisy Triplet Correspondence caused by mismatched triplet components. Existing methods typically discriminate noisy samples via query-target-level signals, while overlooking the intrinsic semantic relations among the reference, modification, and target. Specifically, noisy triplets may still appear reliable when query-target similarity is dominated by unchanged visual content or when the composed query obscures invalid component-target relations. To address these limitations, we propose TRUST: a Semantic Relation-Aware Robust Learning framework, which comprises two coherent and synergistic processes. Transformation-Aware Reliability Estimation first measures continuous triplet reliability on the unit hypersphere by jointly modeling Reference-Anchored Transformation Agreement and Target-oriented Query Decoupling. Then, Reliability-Guided Asymmetric Optimization uses the estimated reliability to weight bidirectional ranking, while shaping the spherical representation space through transformation alignment and target uniformity. We further introduce eXtreme Noisy Triplet Correspondence as a more challenging setting. Experiments on Fashion-IQ and CIRR show that TRUST achieves state-of-the-art performance, with substantial advantages at high noise rates.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Composed Image Retrieval, Noisy Triplet Correspondence, Robust Multimodal Learning, Noisy Correspondence Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4R1ZOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses noisy triplets setting in composed image retrieval. It estimates triplet reliability using agreement between the requested and observed reference-anchored transformations, together with separate modification-target and reference-target evidence. It then weights bidirectional ranking and adds alignment and uniformity regularizers. On Fashion-IQ and CIRR, the authors test several synthetic corruption rates and an extreme corruption variant.

Critical strengths
  1. Well-motivated and coherent methodology. The motivation is convincing, and the proposed method appropriately addresses the identified limitations of query–target-level noise discrimination by explicitly modeling the semantic relations among the reference image, modification text, and target image. The paper follows a clear logical progression from motivation to methodology and experiments, making it easy to read and understand.

  2. Strong robustness under severe corruption. The experiments demonstrate that TRUST remains effective even under extremely high synthetic noise rates. Its performance degrades more slowly than that of competing methods, providing strong evidence that the proposed reliability estimation and reliability-guided optimization improve robustness under severe triplet corruption.

Critical weaknesses

Limited realism of the XNTC setting.

I do not find XNTC sufficiently well-motivated as a novel noise setting. Although independently shuffling all three components makes the corruption extreme in magnitude, it does not appear to reflect the structured, partially matched noise likely to arise in practice. Real-world web crawling or VLM-based annotation is more likely to preserve some valid relations among RR, MM, and TT, while introducing hallucinated modifications or subtle target-attribute mismatches. Plus, fully unrelated triplets may even be easier to identify because all relational cues become inconsistent. Indeed, the paper’s own results show that modification-only corruption is substantially more damaging than XNTC. Therefore, introducing XNTC should not be treated as a strong novelty contribution without empirical evidence of its real-world relevance. I think it is better characterized as a synthetic stress test.

Questions for authors

Q1. Could the authors clarify why the Riemannian log map is needed for the transformation agreement score in Eq. (2)? Since the factor arccos(z^T z') / sqrt(1 - (z^T z')^2) in Eq. (1) is a positive scalar for each vector, it cancels in the cosine similarity of Eq. (2). Eq. (2) is therefore just the cosine between the tangent projections q_i - (q_i^T r_i) r_i and t_i - (t_i^T r_i) r_i, and no Riemannian machinery is required. The simpler Euclidean displacements q_i - r_i and t_i - r_i differ from these only by a component of size (1 - q_i^T r_i) along r_i, which is second order in the angle.

Empirically, I replaced Eq. (2) with 1 - cos(q_i - r_i, t_i - r_i) (the "chord" option in the released code) and ran FashionIQ (validation, mean AVG over 3 seeds):

noise 20%: 65.58 (log map) vs. 65.61 (Euclidean displacement) noise 80%: 62.25 (log map) vs. 62.18 (Euclidean displacement)

The seed standard deviation is 0.04-0.23, so the two are indistinguishable. For reference, plain query-target cosine is worse (65.20 at 20%, 61.03 at 80%), so the reference-anchored direction seems to help, but the log map does not.

What does the log-map formulation provide for reliability estimation beyond this simpler form? A controlled comparison across noise rates (0/20/50/80%) and datasets in the paper would materially affect my assessment of this design choice.

Q2. Could the authors provide stronger evidence that both components of TARE are necessary and complementary? In Table 5, removing either component leads to only small changes in retrieval performance. Could the authors show which noise types each component identifies that the other misses, and whether the gains are consistent across runs? To be honest, as currently presented, TARE seems to add considerable methodological complexity for only marginal performance gains. A clearer account of what each component contributes would materially affect my assessment of the method.

Q3. Could the authors provide evidence that the target uniformity loss L_uniform (Eq. 8) is actually necessary? The motivation (preventing target representations from contracting under alignment) is plausible in theory, but the empirical support appears thin. In Table 5, the "w/o L_uniform" row changes the FashionIQ average by only about -0.27, the smallest effect among the RGAO components, and the term is weighted 10x lower than the alignment term (lambda_u = 0.01 vs. lambda_a = 0.1) and further capped by the budget zeta_b. In my own single-seed reproduction on FashionIQ (20% noise, validation), removing L_uniform did not hurt: the average was slightly higher than the full model (65.92 vs. 65.54). I do not claim this is conclusive from one seed, but it makes me unsure that the term matters in practice.

It would help to see (i) the "w/o L_uniform" ablation with multiple seeds and at higher noise rates, (ii) a direct measurement of the claimed contraction (e.g., the mean pairwise distance or the uniformity value of the target tokens) with and without L_uniform, showing that contraction actually occurs under L_align alone, and (iii) the gradient magnitude of L_uniform relative to L_rank during training, given the budget in Eq. (11). Without such evidence, L_uniform looks like an additional hyperparameter and loss term whose benefit is not demonstrated. A clearer account of when it matters would materially affect my assessment of this component.

If the authors satisfactorily address the questions above, in particular the necessity of the log-map formulation, of both TARE components, and of L_uniform, with controlled comparisons across noise rates and multiple seeds, I would be willing to raise my score.

Reviewer_5w6FOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper points out that existing methods typically distinguish noisy samples through query target level signals, while ignoring the inherent semantic relationships of triplets in combined image retrieval. Therefore, TRUST is proposed, which is robustly trained through two modules, TARE and RGAO. We have achieved the current basic SOTA performance on two datasets, FashionIQ and CIRR. And proposed XNTC, a more stringent noise setting method.

Critical strengths
  1. The authors propose TRUST, a robust learning framework designed specifically for noisy Composed Image Retrieval (CIR). This framework integrates TARE to estimate triplet reliability via transformation agreement and query decoupling, along with RGAO to perform reliability-guided ranking and spherical regularization.
  2. The authors introduce a more challenging eXtreme Noisy Triplet Correspondence (XNTC) setting. Extensive experiments on FashionIQ, CIRR, and the newly proposed XNTC benchmark demonstrate that TRUST effectively mitigates performance degradation and achieves substantial gains under high-noise conditions.
  3. Experiments on FashionIQ and CIRR datasets have achieved basic SOTA performance.
Critical weaknesses
  1. The author summarizes the existing NTC methods as mainly relying on query-target-level signals,while overlooking the intrinsic semantic relations among the reference, modification, and target. But TME[1] has already explicitly modeled the relationship between triples, so this statement is inappropriate. Perhaps the author can converge on this statement.
  2. When calculating the transformation disagreement, the author used Riemann logarithms, which actually function similarly to vector differences. Perhaps the author can explain the advantages of using Riemann logarithms compared to vector differences.

[1] Learning with noisy triplet correspondence for composed image retrieval[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Questions for authors

Please see the Critical Weaknesses section for details.

Additionally, I would like to point out that the core formula (2) of the paper, when dealing with common small transformations in fine-grained retrieval, its tangent vector norm approaching zero can cause serious numerical instability and extreme sensitivity of geometric direction to noise, fundamentally shaking the reliability of the TARE module in real fine-grained scenarios. Especially on fine-grained modified datasets like FashionIQ, will such problems occur during the actual training process? Perhaps the author can count the number of occurrences.

Reviewer_yW8jOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TRUST, a robust framework for Composed Image Retrieval under noisy triplet correspondence. It introduces TARE to estimate triplet reliability through semantic relation modeling and RGAO to improve retrieval learning via reliability-weighted ranking and spherical regularization. Experiments on Fashion-IQ and CIRR demonstrate competitive performance, particularly under high noise rates.

Critical strengths

1.The paper addresses an important limitation of noisy-triplet learning by highlighting that high query-target similarity does not necessarily indicate correct modification semantics. 2.TRUST demonstrates competitive robustness, particularly under high noise rates, supported by extensive experiments. The paper is well-organized, with visually appealing and informative figures that clearly illustrate the motivation and proposed framework.

Critical weaknesses

1.Under non-degenerate conditions, the proposed transformation agreement based on the Riemannian logarithmic map is mathematically equivalent to cosine similarity between reference-orthogonally projected features. The paper does not adequately demonstrate its advantages over simpler geometric alternatives, raising concerns about the novelty and necessity of this design. 2.All experiments are conducted using BLIP-2, without evaluating more recent unified multimodal embedding models such as VLM2Vec-V2[1]. Given the stronger representation capabilities of modern models, it remains unclear whether the identified semantic correspondence issues are still significant and whether the proposed method can provide meaningful improvements. Moreover, its effectiveness across different embedding architectures remains unverified. [1]Meng R, Jiang Z, Liu Y, et al. Vlm2vec-v2: Advancing multimodal embedding for videos, images, and visual documents[J]. arXiv preprint arXiv:2507.04590, 2025.

Questions for authors

1.What advantages does the Riemannian formulation offer over simpler geometric alternatives, given their mathematical equivalence under non-degenerate conditions? 2.Do the identified semantic correspondence issues remain significant in modern multimodal embedding models such as VLM2Vec-V2, and can TRUST still provide meaningful improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.