Multi-Granular Semantic Matching with Uncertainty-Aware Calibration for Sign Language Retrieval

Sign language retrieval requires precise cross‑modal alignment between sign and textual semantics. However, existing methods typically rely on either global or fine‑grained alignment between text and sign video, making it difficult to jointly capture overall event semantics and subtle motion details. Moreover, given a text (respectively a sign video), there are multiple semantically consistent sign videos (respectively texts), and the same sign articulation may convey distinct semantics across different contexts. This correspondence multiplicity and contextual ambiguity introduce uncertainty into sign‑text matching, leading to sub‑optimal semantic alignment. To address these challenges, we propose MGUC, a multi‑granular semantic matching framework with uncertainty‑aware calibration for sign language retrieval. Specifically, coarse‑grained global matching captures overall event semantics and ensures holistic sign‑text consistency. Fine‑grained local matching captures subtle motion details by jointly modeling cross‑modal relevance and local feature interactions through bidirectional similarity aggregation, with learnable [Null] tokens mitigating spurious matches arising from transitional clips and words without explicit sign gestures. We further introduce an uncertainty‑aware modeling and calibration module to capture uncertainty in sign‑text correspondences and refine cross‑modal alignment. It represents sign videos and texts as multivariate Gaussian distributions and uses Bhattacharyya distances to quantify their distributional discrepancy. The relative distance between unpaired and paired samples is used to adaptively calibrate the negative contributions during contrastive learning. This reduces excessive repulsion from potentially semantically related samples, thereby refining text‑to‑sign alignment in the shared embedding space. Extensive experiments on three widely used benchmarks demonstrate that MGUC achieves competitive retrieval performance.