Semantic Correspondence is a Dense Prediction Task: it Should Be Evaluated as One

Semantic correspondence establishes matches between semantically equivalent locations on different object instances. The desired output is a correspondence field that can be queried anywhere on an object, yet the task remains evaluated at sparse annotated landmarks. Current metrics (e.g., PCK) leave predictions between those landmarks unmeasured, leaving the spatial generality of the predicted correspondence field unassessed. We introduce 2D-3D Correspondence Consistency (C3D), an evaluation protocol that probes a fixed correspondence map at independently sampled foreground locations. C3D lifts sampled queries and their predicted matches into 3D via monocular point maps and measures how much the correspondence distorts normalized intra-object distances, either among the samples themselves (annotation-free) or with respect to annotated anchors (annotation-assisted). We evaluate each method’s fixed correspondence map to probe its representation quality, without post-processing or test-time optimization. Computed on SPair-71k, C3D predicts how methods rank on the unseen keypoints of SPair-U better than SPair-71k PCK does, showing that it is a good generalization indicator. Finally, we turn the annotation-assisted structural residual into supervision: we train K2D using pseudo-correspondences obtained by fused semantic–structural transport against ground-truth anchors. This dense supervision improves transfer to unseen keypoint definitions, providing a concrete setting in which dense supervision and dense evaluation can be studied together.