Beyond Agreement: Contrastive Evidence Calibration for Composed Image Retrieval

Composed image retrieval requires preserving relevant reference-image attributes while satisfying a textual modification. Multiple query branches provide complementary retrieval signals, but their agreement may reflect shared appearance cues rather than support for the requested change. We propose Contrastive Evidence Calibration (CEC), a score-level approach that distinguishes branch disagreement, modification matching, reference resemblance, and composed-query compatibility. Starting from a fixed contrastive offset, CEC learns five bounded coefficients over frozen retrieval signals. A regularized full-gallery objective controls the adjustment, while an optional group objective emphasizes distinctions among competing candidates. We characterize the objective in effective coefficient space and derive a sufficient condition for preserving pairwise rankings. With independent training on each dataset, three-seed score ensembles achieve mean recall of 68.82 on the complete FashionIQ validation galleries and 82.43 on the official CIRR test. Matched comparisons show that the fixed offset accounts for the FashionIQ gain, while the complete calibrator improves its CIRR parent by 2.21 points. Applying CIRR’s group objective to automatically constructed FashionIQ groups reduces recall, motivating separate choices for the score representation and its local supervision.