- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MGUC, a multi-granular semantic matching framework with uncertainty-aware calibration for sign language retrieval. It combines global semantic matching with fine-grained local matching, using bidirectional interaction-aware similarity aggregation and learnable [Null] tokens to mitigate spurious local correspondences. Sign videos and text queries are also represented as Gaussian distributions, whose relative Bhattacharyya distances calibrate the negative contributions of unpaired samples during contrastive learning. Experiments provide empirical support for effectiveness.
Critical strengths
- The global and fine-grained branches capture complementary aspects of sign–text correspondence, combining overall event semantics with local motion details.
- PDM and UACC form a well-motivated training strategy: Gaussian representations guide the adaptive weighting of unpaired samples, helping reduce excessive repulsion from semantically related candidates.
- Results on three benchmarks, together with ablations and the supplementary human evaluation, support the effectiveness of the method.
Critical weaknesses
- The paper should discuss PCME++ (ICLR 2024), which also studies probabilistic representations, closed-form distances, and false negatives. The differences and additional benefits of MGUC’s distributional modeling and calibration need to be clearer.
- Table 2 and Appendix A.7 support UACC, but the contribution of learned covariances remains unclear. Comparing against mean-only distance-based calibration, while retaining PDM and the remaining training setup, would help isolate this contribution.
- The method is motivated by multiple semantically valid matches, while the main metrics evaluate annotated pairs. The relevance analysis is helpful, but comparison with baselines using multi-positive retrieval metrics would more directly test this motivation.
Questions for authors
See weakness. Two minor points:
- In Figures 6 and 7, are the weight and uncertainty bins formed using equal sample counts or equal-width value intervals? Please clarify the boundaries.
- Line 723 contains “Table ??”.