- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FlexGlue, which performs sparse-to-dense image matching and visual disambiguation in a single model: given two images and sparse query points in one, it predicts their subpixel correspondences in the other and whether the pair is covisible. Its core idea is to decouple dense context aggregation from sparse querying — a frozen DINOv3 backbone feeds 9 Transformer layers over stride-8 token grids of both images, with correspondences read out only afterwards at query points via a windowed soft-argmax supervised by bilinear ground-truth weights. Per-token covisibility votes supervised at every layer allow early exit on unmatchable pairs (the opposite of LightGlue), and the two tasks are shown to reinforce each other. The paper also introduces LaMAR-DG, the first indoor doppelganger benchmark with hard cross-floor/cross-day/cross-device negatives mined via retrieval. Experiments across relative pose, SfM (ETH3D, IMC2025), localization (InLoc), and disambiguation show accuracy competitive with dense matchers at sparse-matcher cost, with large gains on ETH3D SfM and indoor disambiguation.
Critical strengths
-
Design is well-motivated and validated. Decoupling dense context aggregation from sparse querying addresses a real limitation of both matcher families, and the ablation confirms it is the decisive choice (stride 16 costs 12+ points). The result matches dense matchers on ScanNet-1500 and InLoc at half the runtime of SP+RoMAv2 (26.7 vs 63.2 ms).
-
Early exit on negative pairs is genuinely novel and practical. It inverts LightGlue's strategy, saving compute where it is currently wasted: on ETH3D,
Ours + Ours (early)runs at 45 pairs/s — faster than SP+LightGlue — while raising 1° pose AUC from 29.5 to 67.5. The joint formulation is also evidence-backed (removing the matching loss costs up to 9.3 AUROC), and LaMAR-DG fills a real gap where all prior methods degrade sharply.
Critical weaknesses
-
The claims overshoot what the tables show. The abstract says SotA, but MegaDepth-1500 is well below RoMAv2 (56.4 vs 62.8 @5°), and the best HardMatch number in Table 1 is actually the ''+RoMAv2 refine'' row (55.6 vs the paper's 54.9). ''SotA on 5/6 InLoc thresholds'' counts ties as wins — at [email protected] they lose to RoMAv2 by 5.1 points. Similarly, "efficiency of sparse matchers" doesn't hold: SP+Ours is 2× slower than SP+LightGlue, and 30/s vs 40/s in the SfM pipeline. The defensible claim — faster than dense matchers and dedicated disambiguators, and faster than sparse matchers only with early exit — is still a good one; it just needs to be worded that way.
-
The SfM numbers are hard to trust without completeness metrics. Only pose AUC is reported, but any filter that drops hard true positives will make AUC go up while the reconstruction gets worse. Without registered images, track counts, and the fraction of pairs rejected, I can't tell whether 68.9 vs 62.4 on ETH3D is a genuine gain or just selective discarding. Related: no cheap baselines (retrieval score, RANSAC inliers), and it's never stated whether SfM runs s2s or s2d — if s2d, the mutual-NN snapping in Eq. (3) means tracks are still bounded by detector repeatability, which is exactly what the introduction criticizes dense methods for.
Questions for authors
-
In Tables 3, 4, and 5, is "Ours" running in s2s or s2d mode? If SfM uses s2d, isn't it still bounded by detector repeatability in the second image, since Eq. (3) snaps correspondences back to detected keypoints?
-
What is the early-exit covisibility threshold, at which layers is it checked, and on what set was it tuned? Among pairs rejected by early exit, what fraction are actually true positives?
-
During stage-1 training (first 40 epochs), all pairs are sampled with overlap in [0.05, 0.8]. Where do the negative labels for ''L_covis'' come from then?
-
Which DINOv3 ConvNeXt variant is used, and what are the total parameters, GFLOPs, and peak memory compared to LightGlue, LoMA, and RoMAv2?
-
What are the full MegaDepth-1500 / ScanNet-1500 / InLoc numbers for the label-free (flip-only) variant, so we can see what disambiguation fine-tuning costs in matching accuracy?
5 comments
Thank you for the response. My concerns have been partially addressed.
Dear reviewer,
Please be more specific about what concerns have been addressed and what concerns remain.
Thank you, /AC
Claims on matching. We thank the reviewer for carefully comparing our claims to the numbers. FlexGlue is now more accurate than in the initial submission (stronger augmentations, better match scoring, Sec. 3.1). On HardMatch, it clearly outperforms LoMa-G (56.4 vs. 54.3 mAA@10px) and further improves over it on ScanNet-1500 (+2.6 AUC@5°, Tab. 1). "+RoMav2 refine" is a post-processing on top of FlexGlue (clarified in App. E.1). Full RoMav2 with the same refinement model scores 46.5 on HardMatch. Our SotA claims now exclude MegaDepth-1500. For InLoc, we changed the claims to "best or tied on 5/6 thresholds".
Efficiency claims. FlexGlue is indeed slower than LightGlue and LoMa-B (29.6 vs. 21.7 ms), but more robust (+4.6 pp on HardMatch), and faster than the strongest sparse baseline LoMa-G (49.8 ms). We reworded the claims to "close to the efficiency of sparse matchers".
SfM completeness and pair filtering. SfM pose AUC already includes completeness: relative poses to non-registered images get the maximum error (180°). Tab. 4 now reports registered images, and we register 100% with disambiguation. On the 13 ETH3D scenes with GT depth, we keep 97.8% of positive pairs vs. 76.8% for DG++, at a slightly lower negative rejection rate (95.9 vs. 99.3%), and 96.3% with early exit (Tab. 10).
Cheap pair-filtering baselines. As requested, we add pair-filtering baselines for SfM: top-20 MegaLoc retrieval and MegaLoc similarity ≤ 0.3. Both reduce accuracy, as MegaLoc rates hard negatives as highly similar (Fig. 4). An oracle from sparse GT SfM covisibility boosts LoMa, but stays below our method with our filter. We omit RANSAC-inlier filtering, as COLMAP already performs it.
Matching mode in SfM and detector repeatability. We appreciate the attention to detail of the reviewer here. We do snap to sparse detections and remain limited by detection repeatability. We keep this S2S setting for ease of comparison. But a simple experiment shows how S2D matches can lift this limit: we add "virtual" keypoints where at least two S2D matches from S2S-matched source keypoints agree within a radius r in a third image. This recovers missing detections, increases track lengths, and improves SfM accuracy on ETH3D (App. B.6, Fig. 10 & 11). InLoc runs in S2D.
Early-exit operating point. We reused threshold t=0.8 from DG++, which is slightly suboptimal (Fig. 9, App. B.5). The decay γ=0.8 was selected on ViSym-F and LaMAR-DG, and generalizes to ETH3D. The full sweep with retention rates, stopping layers and ETH3D pose accuracy is in our response to Reviewer 66uv. The decay is needed because earlier layers assign lower scores to positives (Fig. 8, bottom). We check early exit on every layer except the first, which is not confident enough (Fig. 5, right).
Covisibility supervision in stage 1. We do not use L_covis in the first stage and add it only during fine-tuning with negatives, which we clarified in the paper.
Backbone, parameters, and memory. We use DINOv3 ConvNeXt-B, vs. ViT-L in RoMav2. All detectors run at 1024px. Clarified in Appendix B.1.
Matching accuracy of the label-free variant. The label-free variant also trains with negatives, so we isolate disambiguation fine-tuning by removing all negatives (and thus L_covis). Both show similar matching performance, so adding negatives comes at no notable cost in matching accuracy.