- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper exposes the limitations in existing semantic correspondence metrics that models can receive perfect performance on sparse keypoints but behave differently everywhere else. Considering the missing evaluation of denser correspondence fields, 2D-3D Correspondence Consistency (C3D) is proposed to measure the structural consistency of dense correspondence fields, by lifting foreground queries to 3D pointmaps. To further investigate the usefulness of dense correspondence field, the authors build dense pseudo ground truths and train a dense-aware correspondence model K2D. Extensive experiments on multiple benchmarks demonstrate that C3D can better predict model rankings on unseen landmarks, and the proposed dense supervision improves correspondence performance.
Critical strengths
- The evaluation problem is clearly motivated and identifies a real limitation within existing metrics.
- The paper comprehensively evaluates across eleven benchmarks of both PCK and the proposed C3D.
- K3D consistently outperforms previous state of the art.
Critical weaknesses
- The extent of how geometric consistency reflects semantic correctness is insufficiently evaluated. Geometric consistency can be affected by different shapes or articulations. Though local weighting can partially reduce the effects, the unweighted annotation-assisted distance brings the sensitivity to global shape and articulation.
- The robustness to the accuracy of geometric prior is not evaluated. Both pseudo label generation and C3D evaluation depend on VGGT generated geometry, but how geometry errors affect them are not analyzed.
Questions for authors
- In pseudo ground truth generation, the cost matrices of semantic and geometric distances are blended with the weights 0.7 and 0.3, respectively. Could the authors show the magnitude analysis of these two distances after weighting and how each term affects the selected correspondences? An ablation of using different weights can provide more transparency. This will also provide more insights on how geometric consistency can influence the model during training.
- Geometric foundation models like VGGT can introduce artifacts to either or both of the images during evaluation. Have the authors checked the quality of generated single-frame geometry on difficult scenarios such as substantial occlusion or low-light settings where applicable? Would substituting VGGT with an alternative model such as VGGT-Omega largely influence the results or rankings?
- One related work [1] that also exploits geometric prior for correspondence is missing in discussion.
[1] Qian, Qiyang, et al. "Bridging viewpoint gaps: Geometric reasoning boosts semantic correspondence." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.