GeoRx: A Reference-Based Benchmark for Attribution Diagnosis of Cross-Modal Embeddings

Cross-modal encoders map multimodal data into a shared embedding space for semantic alignment and retrieval. However, geometric deviations in this space can cause cosine-based rankings to diverge from semantic relevance, limiting retrieval accuracy. Existing benchmarks primarily measure end-to-end retrieval accuracy, offering limited support for attributing retrieval failures to geometric causes. To address this limitation, we introduce GeoRx, a reference-based benchmark for geometric attribution diagnosis in cross-modal retrieval. For each $⟨\mathrm{encoder},\mathrm{workload}⟩$ pair, GeoRx constructs a geometric reference model matched to the encoder's embedding dimension and the workload's corpus size. Five diagnostic metrics derived from candidate–corpus, candidate–candidate, and within-list relationships identify geometric deviations against this reference without requiring additional information. These relationships respectively characterize candidate occurrence frequency across queries, similarity among retrieved candidates, and variation in query–candidate scores. We validate diagnostic accuracy through controlled deviation injection. Guided by the diagnoses, we map correction methods from existing work to the identified deviations for individual and joint correction. Experiments demonstrate target detection rates of 98.93–100.0% and an overall hit rate of 32.35% across diagnosis–correction mappings. Code is available at https://anonymous.4open.science/r/GeoRx-3CC3/.