- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CandidateShift is a training objective for descriptor-free 2D–3D matching. The matcher still takes keypoint coordinates, landmark bearings, and RGB, and it is not changed at inference. During training, the same matcher sees a full candidate set and a reduced one. Geometric labels supervise whether any valid landmark survives. When one does, the full-set prediction is restricted to the retained landmarks, renormalized, and used as a stopped-gradient target for the reduced-set prediction. Both outputs also keep ordinary correspondence supervision. The method is tested on top of GeoMix and transferred to A2-GNN, mainly on MegaDepth, with zero-shot results on Cambridge and 7Scenes.
Critical strengths
- The training problem is stated precisely. Deleting landmarks can preserve a correspondence or remove every valid one, and the correct output in the second case is unmatched. CandidateShift uses geometric survival for that availability decision, and it transfers a soft identity target only for queries that remain matchable. That avoids teaching a preference among candidates that are all invalid. Inference adds no module.
Critical weaknesses
- Most of the reported gain comes from supervised candidate perturbations, and the two new terms do not play the roles claimed for them. Availability largely restates the unmatched label already present in the inherited correspondence loss, and the paper states that the ablation does not separate the two. The upper-quartile error does move further than the one-pixel AUC: top-1 translation at 75% falls from 2.15 m for GeoMix to 1.33 m with perturbations and about 0.89 m with identity or the full loss. That tail gain should be attributed to identity consistency, rather than to the joint availability-and-identity objective.
Questions for authors
- If the stopped-gradient target in Eq. 6 is replaced by the renormalized geometric correspondence labels, how do AUC and the deletion metrics in Figure 4 change?
- Are the Table 2 gaps stable across seeds? In particular, is the difference between identity-only and the full objective distinguishable from run-to-run variation?
- Which term accounts for the higher 7Scenes/SuperPoint error of CandidateShift (Mix), and does that ordering reverse for any loss variant?
8 comments
SjJD-W1 (2/2). Attribution of the pose-error tail improvement
The reweighting experiment now addresses the separate ambiguity concerning the event loss:
Revised Table 6 (p. 10).
Availability versus unmatched-loss reweighting. The availability column indicates the added event loss.
| Variant | Added γ | Availability loss | Top-1 AUC@1/5/10px (%) | Top-10 AUC@1/5/10px (%) |
|---|---|---|---|---|
| Perturbations only | 0 | 16.70 / 52.58 / 59.52 | 20.61 / 62.66 / 70.51 | |
| Identity only | 0 | 17.64 / 54.90 / 61.86 | 21.49 / 64.80 / 72.75 | |
| Perturbations+Identity | 0 | ✓ | 17.69 / 54.78 / 61.68 | 21.59 / 64.90 / 72.75 |
| Perturbations + reweighting | 4 | 16.42 / 52.07 / 58.84 | 20.57 / 62.43 / 70.28 | |
| Identity + reweighting | 0.5 | 17.41 / 54.41 / 61.35 | 21.48 / 64.63 / 72.55 |
Both selected reweighting controls have lower AUC than their corresponding unweighted objectives at all six entries. This clarifies the implemented loss comparison, while the multi-seed results in our response to SjJD-Q2 establish that the small identity-only/joint AUC differences should be interpreted separately from the larger gain over perturbations.
Changes in the revised manuscript. Section 4.3 (p. 7); Appendix G (p. 23); Table 18 (p. 20); Table 2 (p. 8); Table 6 (p. 10).
Table 2 compares the pair objective with supervised perturbations, holding the matcher, FiLM, and group weighting fixed. Both added terms improve localization; identity gives the larger gain and most of the error-tail reduction. The identity-only variant still uses the geometric survival gate: it removes while retaining the availability condition that defines valid identity targets.
Table 25 summarizes seeds 93, 94, and 95. Both identity-bearing objectives improve over supervised perturbations. Identity-only has higher mean AUC; CandidateShift has lower sample standard deviation than identity-only.
Table 26 compares models within each seed. The original seed-93 models remain the reference outside this section; the marked same-seed repeat is excluded from the summary statistics.
SjJD-Q1 (1/2). Geometric identity targets instead of a stopped soft teacher
We appreciate your targeted geometric-label alternative. It provides a direct test of whether the full-context soft prediction contributes information beyond another application of the correspondence labels.
Reviewer concern.
If the stopped-gradient target in Eq. 6 is replaced by the renormalized geometric correspondence labels, how do AUC and the deletion metrics in Figure 4 change?
Response and evidence.
We trained that alternative. For a query with at least one surviving geometric match, its identity target becomes
The survival condition, availability term, direct correspondence supervision, perturbations, FiLM, and group weighting remain the same. Normalizing the binary labels assigns equal mass to the geometrically valid retained landmarks when more than one exists.
MegaDepth SIFT reprojection AUC (%).
| Model | Top-1 AUC@1 / 5 / 10px | Top-10 AUC@1 / 5 / 10px |
|---|---|---|
| CandidateShift | 17.69 / 54.78 / 61.68 | 21.59 / 64.90 / 72.75 |
| Geometric target | 17.42 / 54.02 / 60.79 | 21.26 / 64.45 / 72.37 |
The stopped full-context target gives higher localization AUC at every threshold and both retrieval depths. Since both variants already receive geometric correspondence supervision, the result supports transferring relative preferences learned from the reference context rather than duplicating only the positive label set.
The complete candidate-deletion comparison is:
Candidate deletion on the fixed validation queries.
| Model | Retained fraction | Conditional KL ↓ | Identity accuracy (%) ↑ | Availability accuracy (%) ↑ | Brier ↓ | BCE ↓ | Completed inputs |
|---|---|---|---|---|---|---|---|
| CandidateShift | 0.1 | 1.1400 | 40.97 | 98.56 | 0.0151 | 0.0787 | 503/512 |
| CandidateShift | 0.2 | 0.9457 | 41.21 | 97.15 | 0.0285 | 0.1269 | 512/512 |
| CandidateShift | 0.3 | 0.8058 | 41.82 | 95.68 | 0.0405 | 0.1654 | 512/512 |
| CandidateShift | 0.4 | 0.6623 | 42.64 | 94.32 | 0.0510 | 0.1969 | 512/512 |
| CandidateShift | 0.5 | 0.5409 | 43.45 | 92.99 | 0.0603 | 0.2236 | 512/512 |
| CandidateShift | 0.6 | 0.4257 | 45.39 | 91.79 | 0.0683 | 0.2460 | 512/512 |
| CandidateShift | 0.7 | 0.3169 | 46.55 | 90.70 | 0.0755 | 0.2656 | 512/512 |
| CandidateShift | 0.8 | 0.2171 | 47.50 | 89.63 | 0.0822 | 0.2836 | 512/512 |
| CandidateShift | 0.9 | 0.1128 | 48.34 | 88.72 | 0.0882 | 0.2996 | 512/512 |
| Geometric target | 0.1 | 1.1181 | 42.63 | 98.56 | 0.0152 | 0.0786 | 503/512 |
| Geometric target | 0.2 | 0.9555 | 40.89 | 97.14 | 0.0285 | 0.1267 | 512/512 |
| Geometric target | 0.3 | 0.8296 | 41.50 | 95.69 | 0.0405 | 0.1650 | 512/512 |
| Geometric target | 0.4 | 0.6812 | 41.43 | 94.30 | 0.0510 | 0.1969 | 512/512 |
| Geometric target | 0.5 | 0.5590 | 42.89 | 92.99 | 0.0604 | 0.2241 | 512/512 |
| Geometric target | 0.6 | 0.4345 | 44.71 | 91.76 | 0.0685 | 0.2463 | 512/512 |
| Geometric target | 0.7 | 0.3254 | 45.70 | 90.63 | 0.0758 | 0.2661 | 512/512 |
| Geometric target | 0.8 | 0.2158 | 47.07 | 89.61 | 0.0824 | 0.2837 | 512/512 |
| Geometric target | 0.9 | 0.1111 | 47.85 | 88.71 | 0.0882 | 0.2994 | 512/512 |
At 10% retention, 503 of the 512 inputs complete the forward diagnostic; identity metrics further require surviving ground-truth matches. For CandidateShift, 499 images meet that identity-eligibility condition. At the other retained fractions all inputs complete. Identity accuracy tests the most probable real candidate only on queries with a valid surviving match. Availability metrics use all keypoints in the completed inputs, with for the binary accuracy. Conditional KL is the full-to-reduced divergence between each model's restricted full prediction and its recomputed subset prediction, with unmatched entries removed from that conditional comparison.
CandidateShift has higher label-based identity accuracy from 20% through 90% retention; the geometric target is stronger at the most severe 10% retention. Conditional KL measures self-consistency rather than geometric correctness, so its ordering need not match identity accuracy at every fraction. Figure 4 now includes the geometric-target identity curves, and Figure 5 provides the complementary availability diagnostics.
SjJD-W1 (1/2). Attribution of the pose-error tail improvement
We appreciate your careful analysis of the upper-quartile pose errors. It helped us explain more precisely how supervised perturbations and conditional identity transfer contribute to the improvement.
Reviewer concern.
Response and evidence.
We agree that most of the further tail reduction beyond supervised perturbations is attributable to identity consistency. The revised Section 4.3 now states that identity provides the larger localization gain and most of the reduction in the error tail. The complete loss comparison is:
MegaDepth SIFT reprojection AUC (%).
The progression separates the benefit of candidate perturbations from the added pair supervision. Identity-only accounts for almost all of the further top-1 upper-quartile translation improvement; adding availability changes the translation tail only slightly while reducing invalid top-1 poses. The correspondence metrics also favor identity-only.