acceptodds
Under review as a conference paper at ICLR 2027

FlexGlue: Efficient Sparse-to-Dense Image Matching and Visual Disambiguation

Abstract

Estimating robust correspondences across images is a fundamental building block of geometric computer vision. Existing approaches force a trade-off as they match either sparse points, which is highly efficient, or dense pixel grids, which is more robust but expensive. They also often struggle to reject unrelated image pairs that exhibit visual symmetries. We bridge these gaps by introducing a neural network for asymmetric sparse-to-dense image matching and view-graph filtering. Given two images and query points in the first one, we predict whether the image pair is covisible as well as the subpixel coordinates of the corresponding points in the second image. By searching for correspondences over the entire target image, the sparse-to-dense formulation is not limited by the repeatability of keypoint detectors. Furthermore, aggregated context from images provides global understanding necessary for both robust geometric matching and effective visual disambiguation, while jointly learning these tasks leverages their inherent synergy. Experiments on relative pose estimation, Structure-from-Motion, and visual localization show that our method retains subpixel precision and adds robustness to doppelgangers, while achieving computational efficiency of sparse matchers. Together, our method achieves state-of-the art accuracy on IMC2025, InLoc, Doppelgangers, ViSym, ETH3D SfM with COLMAP, and our proposed indoor doppelgangers benchmark. Weights and code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image matching, visual disambiguation, two-view estimation, dense correspondences

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QZM7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FlexGlue, which performs sparse-to-dense image matching and visual disambiguation in a single model: given two images and sparse query points in one, it predicts their subpixel correspondences in the other and whether the pair is covisible. Its core idea is to decouple dense context aggregation from sparse querying — a frozen DINOv3 backbone feeds 9 Transformer layers over stride-8 token grids of both images, with correspondences read out only afterwards at query points via a windowed soft-argmax supervised by bilinear ground-truth weights. Per-token covisibility votes supervised at every layer allow early exit on unmatchable pairs (the opposite of LightGlue), and the two tasks are shown to reinforce each other. The paper also introduces LaMAR-DG, the first indoor doppelganger benchmark with hard cross-floor/cross-day/cross-device negatives mined via retrieval. Experiments across relative pose, SfM (ETH3D, IMC2025), localization (InLoc), and disambiguation show accuracy competitive with dense matchers at sparse-matcher cost, with large gains on ETH3D SfM and indoor disambiguation.

Critical strengths
  1. Design is well-motivated and validated. Decoupling dense context aggregation from sparse querying addresses a real limitation of both matcher families, and the ablation confirms it is the decisive choice (stride 16 costs 12+ points). The result matches dense matchers on ScanNet-1500 and InLoc at half the runtime of SP+RoMAv2 (26.7 vs 63.2 ms).

  2. Early exit on negative pairs is genuinely novel and practical. It inverts LightGlue's strategy, saving compute where it is currently wasted: on ETH3D, Ours + Ours (early) runs at 45 pairs/s — faster than SP+LightGlue — while raising 1° pose AUC from 29.5 to 67.5. The joint formulation is also evidence-backed (removing the matching loss costs up to 9.3 AUROC), and LaMAR-DG fills a real gap where all prior methods degrade sharply.

Critical weaknesses
  1. The claims overshoot what the tables show. The abstract says SotA, but MegaDepth-1500 is well below RoMAv2 (56.4 vs 62.8 @5°), and the best HardMatch number in Table 1 is actually the ''+RoMAv2 refine'' row (55.6 vs the paper's 54.9). ''SotA on 5/6 InLoc thresholds'' counts ties as wins — at [email protected] they lose to RoMAv2 by 5.1 points. Similarly, "efficiency of sparse matchers" doesn't hold: SP+Ours is 2× slower than SP+LightGlue, and 30/s vs 40/s in the SfM pipeline. The defensible claim — faster than dense matchers and dedicated disambiguators, and faster than sparse matchers only with early exit — is still a good one; it just needs to be worded that way.

  2. The SfM numbers are hard to trust without completeness metrics. Only pose AUC is reported, but any filter that drops hard true positives will make AUC go up while the reconstruction gets worse. Without registered images, track counts, and the fraction of pairs rejected, I can't tell whether 68.9 vs 62.4 on ETH3D is a genuine gain or just selective discarding. Related: no cheap baselines (retrieval score, RANSAC inliers), and it's never stated whether SfM runs s2s or s2d — if s2d, the mutual-NN snapping in Eq. (3) means tracks are still bounded by detector repeatability, which is exactly what the introduction criticizes dense methods for.

Questions for authors
  1. In Tables 3, 4, and 5, is "Ours" running in s2s or s2d mode? If SfM uses s2d, isn't it still bounded by detector repeatability in the second image, since Eq. (3) snaps correspondences back to detected keypoints?

  2. What is the early-exit covisibility threshold, at which layers is it checked, and on what set was it tuned? Among pairs rejected by early exit, what fraction are actually true positives?

  3. During stage-1 training (first 40 epochs), all pairs are sampled with overlap in [0.05, 0.8]. Where do the negative labels for ''L_covis'' come from then?

  4. Which DINOv3 ConvNeXt variant is used, and what are the total parameters, GFLOPs, and peak memory compared to LightGlue, LoMA, and RoMAv2?

  5. What are the full MegaDepth-1500 / ScanNet-1500 / InLoc numbers for the label-free (flip-only) variant, so we can see what disambiguation fine-tuning costs in matching accuracy?

5 comments

Claims on matching. We thank the reviewer for carefully comparing our claims to the numbers. FlexGlue is now more accurate than in the initial submission (stronger augmentations, better match scoring, Sec. 3.1). On HardMatch, it clearly outperforms LoMa-G (56.4 vs. 54.3 mAA@10px) and further improves over it on ScanNet-1500 (+2.6 AUC@5°, Tab. 1). "+RoMav2 refine" is a post-processing on top of FlexGlue (clarified in App. E.1). Full RoMav2 with the same refinement model scores 46.5 on HardMatch. Our SotA claims now exclude MegaDepth-1500. For InLoc, we changed the claims to "best or tied on 5/6 thresholds".

Efficiency claims. FlexGlue is indeed slower than LightGlue and LoMa-B (29.6 vs. 21.7 ms), but more robust (+4.6 pp on HardMatch), and faster than the strongest sparse baseline LoMa-G (49.8 ms). We reworded the claims to "close to the efficiency of sparse matchers".

SfM completeness and pair filtering. SfM pose AUC already includes completeness: relative poses to non-registered images get the maximum error (180°). Tab. 4 now reports registered images, and we register 100% with disambiguation. On the 13 ETH3D scenes with GT depth, we keep 97.8% of positive pairs vs. 76.8% for DG++, at a slightly lower negative rejection rate (95.9 vs. 99.3%), and 96.3% with early exit (Tab. 10).

Cheap pair-filtering baselines. As requested, we add pair-filtering baselines for SfM: top-20 MegaLoc retrieval and MegaLoc similarity ≤ 0.3. Both reduce accuracy, as MegaLoc rates hard negatives as highly similar (Fig. 4). An oracle from sparse GT SfM covisibility boosts LoMa, but stays below our method with our filter. We omit RANSAC-inlier filtering, as COLMAP already performs it.

MatcherPair filterPairs rejectedAUC@1°/3°/5°Reg.
LoMa-Bnone0%59.5 / 77.6 / 82.4100%
LoMa-BMegaLoc top-2053.9%58.9 / 75.1 / 79.3100%
LoMa-BMegaLoc sim. <= 0.348.0%55.0 / 71.4 / 75.796.0%
LoMa-BDG++62.2%66.7 / 84.1 / 88.499.1%
LoMa-Boracle from sparse GT SfM62.4%67.8 / 85.3 / 89.599.6%
Oursnone0%66.3 / 82.2 / 86.099.7%
OursOurs48.0%72.2 / 88.9 / 92.9100%

Matching mode in SfM and detector repeatability. We appreciate the attention to detail of the reviewer here. We do snap to sparse detections and remain limited by detection repeatability. We keep this S2S setting for ease of comparison. But a simple experiment shows how S2D matches can lift this limit: we add "virtual" keypoints where at least two S2D matches from S2S-matched source keypoints agree within a radius r in a third image. This recovers missing detections, increases track lengths, and improves SfM accuracy on ETH3D (App. B.6, Fig. 10 & 11). InLoc runs in S2D.

ETH3D, DaD 2k, AUC@1°sparse-to-sparsewith virtual keypoints
w/o disambiguation66.369.7
w/ disambiguation72.275.0

Early-exit operating point. We reused threshold t=0.8 from DG++, which is slightly suboptimal (Fig. 9, App. B.5). The decay γ=0.8 was selected on ViSym-F and LaMAR-DG, and generalizes to ETH3D. The full sweep with retention rates, stopping layers and ETH3D pose accuracy is in our response to Reviewer 66uv. The decay is needed because earlier layers assign lower scores to positives (Fig. 8, bottom). We check early exit on every layer except the first, which is not confident enough (Fig. 5, right).

Covisibility supervision in stage 1. We do not use L_covis in the first stage and add it only during fine-tuning with negatives, which we clarified in the paper.

Backbone, parameters, and memory. We use DINOv3 ConvNeXt-B, vs. ViT-L in RoMav2. All detectors run at 1024px. Clarified in Appendix B.1.

MethodParams [M] (extr. / match)GFLOPs (extr. / match)Peak VRAM [GB] (total / match)
ALIKED + LightGlue0.7 / 11.936 / 8121.2 / 0.5
DaD + LoMa-B329.7 / 11.87535 / 8122.4 / 0.5
DaD + LoMa-G329.7 / 189.27535 / 54683.8 / 1.5
RoMa v2303.2 / 122.32450 / 22352.7 / 2.2
SP + Ours90.9 / 52.2597 / 19901.2 / 0.7

Matching accuracy of the label-free variant. The label-free variant also trains with negatives, so we isolate disambiguation fine-tuning by removing all negatives (and thus L_covis). Both show similar matching performance, so adding negatives comes at no notable cost in matching accuracy.

ModelMD-1500 (5°/10°/20°)SN-1500 (5°/10°/20°)InLoc DUC1 (0.25/0.5/1 m)InLoc DUC2 (0.25/0.5/1 m)
no negatives56.2 / 72.4 / 84.133.1 / 55.9 / 73.762.1 / 84.8 / 93.974.0 / 90.1 / 94.7
with negatives55.8 / 72.0 / 83.633.5 / 56.5 / 73.962.6 / 84.3 / 94.477.9 / 88.5 / 94.7

Thank you for the response. My concerns have been partially addressed.

Dear reviewer,

Please be more specific about what concerns have been addressed and what concerns remain.

Thank you, /AC

Reviewer_66uvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

FlexGlue combines sparse-to-dense feature matching with visual disambiguation. It aggregates context over dense feature grids from both images, then predicts continuous correspondences through lightweight readout and refinement at sparse query locations. The same representation supports pair-level covisibility prediction and early rejection of non-overlapping pairs. The paper also introduces an indoor and cross-device disambiguation benchmark derived from LaMAR. Experiments evaluate relative pose estimation, visual disambiguation, structure-from-motion, and visual localization, with component and training ablations.

Critical strengths
  1. A useful unified formulation with evidence beyond architectural integration. Sharing context between correspondence estimation and pair-level disambiguation addresses an important practical failure mode: visually similar but unrelated images can produce convincing incorrect matches. Table 7 shows that removing matching supervision substantially reduces disambiguation performance, while Table 3 demonstrates that filtering improves downstream reconstruction.

  2. Broad empirical validation and a valuable evaluation setting. The experiments cover matching, disambiguation, reconstruction, and localization. On LaMAR, the reported AUROC improves from 78.6 for DG++ to 88.8, and the indoor/cross-device benchmark exposes challenging cases beyond conventional outdoor landmarks. Together with the ETH3D results, this supports the practical significance of the proposed integration, despite its reliance on established components.

Critical weaknesses
  1. The efficiency and early-exit claims need a more explicit evaluation protocol. It is unclear whether feature-extraction costs are treated consistently across methods in Table 2. Early-exit thresholds, their selection procedure, and positive-pair retention rates are also missing. Since Table 3 shows a small accuracy decrease with early exit, the speed–accuracy trade-off should be quantified more explicitly.

  2. The claimed bidirectional task synergy is not fully isolated. Table 7 supports matching supervision improving disambiguation. However, Table 6 removes the entire disambiguation fine-tuning stage, changing data and training conditions together. It therefore does not isolate whether the matching improvement comes from the covisibility objective itself or from the modified training procedure.

Questions for authors
  1. Comparable timing. Does Table 2 include feature extraction for every method? Please specify the resolution, batch size, precision, and caching assumptions, and report comparable timings with and without feature extraction.

  2. Early-exit operating point. What early-exit and final filtering thresholds are used, and how are they selected? Please report positive-pair retention and false-acceptance rates at the reported operating point, particularly for low-overlap positives. A small threshold sweep showing accuracy versus throughput would clarify the practical trade-off.

  3. Joint-training attribution. Does the matching improvement persist when the training data, negative samples, initialization, and fine-tuning schedule are held fixed and only the covisibility loss is removed? Please also specify the construction and sampling proportions of the flip-generated negatives used in the label-free variant.

1 comment

Timing protocol in visual disambiguation. In the table on visual disambiguation (Tab. 3, previously Tab. 2), we report the pairwise disambiguation runtimes of all methods, excluding extraction (i.e., assuming cached per-image encodings. as is common in SfM). Below, we also separate extraction, context aggregation & disambiguation, and match readout times (for positives). FlexGlue is more efficient than DG++ (based on MASt3R) on each. The initial version included extraction times for DG and match decoding for ours, which we fixed now. We use a batch size of 1 here, and measure with warmup. We use the default image resolutions of DG (long side 1024px) and DG++ (512px), and 512x512 px for ours. We use mixed precision for all models and enable flash attention for DG++ and ours. Extraction includes the encoders of each model. DG++ is slow because it requires two asymmetric transformer passes and performs voting on both.

Time per pair [ms]ExtractionTransformer + DisambiguationMatch decoding
DG35.5104.3–
DG++62.2124.7109.2
Ours11.821.77.0

Early-exit operating point. We never tuned the covisibility threshold and reused t=0.8 from DG++. An additional ablation showed this is actually slightly suboptimal (Fig. 9 in appendix). We added a plot of positive and negative retention rates over thresholds in the appendix, Fig. 8, compared to DG++ and earlier layers. We keep more positives than DG++ at similar rejection rates. Earlier layers tend to be less confident on positive pairs, especially on LaMAR-DG, which has many hard positives with little overlap and large temporal changes.

Consequently, we need a lower covisibility threshold on earlier layers, which motivated the early stop decay rate. We selected this on the disambiguation datasets, and summarize the analysis in the table below: $\gamma{=}0.8$ strikes a good trade-off between accuracy and speed.

Method, pos. kept / neg. rejected / mean stop layerViSym-FLaMAR-DG
Ours (final layer)88.3 / 98.0 / 9.068.2 / 95.5 / 9.0
+ early exit, $\gamma{=}0.5$88.3 / 98.0 / 7.768.1 / 95.5 / 7.4
+ early exit, $\gamma{=}0.8$ (ours)82.9 / 98.1 / 5.564.1 / 95.8 / 5.2
+ early exit, $\gamma{=}0.9$76.2 / 99.2 / 4.956.8 / 96.1 / 4.5
+ early exit, $\gamma{=}1$59.1 / 99.7 / 4.140.7 / 97.5 / 3.6
DG++70.3 / 93.2 / –40.5 / 95.2 / –

As requested, we further show the impact of early stopping on ETH3D SfM in Fig. 9 and the table below. In addition to pose AUC and throughput, we report positive retention and negative rejection rates on the 13 train scenes with GT depth. The trend is similar to disambiguation datasets: early stopping rejects slightly more positives, which does not impair pose accuracy, but increases throughput. Without a decay, pose accuracy is lower because the earlier layers reject too many positives.

Method, ETH3D SfMStop layerPos. keptNeg. rejectedPairs/sAUC@3°
no disambiguation910003082.2
$t{=}0.8$, final layer997.895.93388.9
+ early exit, $\gamma{=}0.8$ (ours)6.196.396.04288.9
+ early exit, $\gamma{=}1$4.781.899.34986.2

Among low-overlap positives (<20%) from ETH3D scenes with GT depth, DG++ / FlexGlue at final layer / FlexGlue with $\gamma{=}0.8$ keep 49.8% / 87.0% / 82.5% of covisible pairs.

Joint-training attribution. Removing L_covis leaves matching performance almost unchanged (-0.4pp on HardMatch), and we clarified in the revision that the synergy is not bijective. We show that matching helps disambiguation, which reduces overfitting observed in pure disambiguation training (Tab. 8). In addition, we show in our experiments that disambiguation helps a matcher in downstream SfM (Tab. 4 and 5).

In the label-free variant, 10% of all images are non-overlapping negatives from the matching datasets (easy negatives derived from depth), and 40% are flipped images of overlapping positives, resulting in a balanced split. Without the easy negatives, the model struggled to reject very different-looking pairs. For comparison, the full disambiguation run has 50% data from doppelganger datasets (DG / NaverLabs, DG only for Tab. 3), of which 50% are negatives, and another 25% are flip negatives from the covisible matching datasets (sampled uniformly).

Reviewer_x6K2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduce a matcher of solving feature matching and visual disambiguation. Exactly, it matches sparse queries against dense context. It is an efficient sparse-to-dense matcher. This method achieves state-of-the art accuracy by testing on a series of benchmarks. Meanwhile, this paper introduces an indoor doppelganger benchmark.

Critical strengths
  1. Motivation is good. The method of involving query could solve the limitation of spase-to-dense matching method.
  2. Combine matching and disambiguation, this idea is insight.
  3. Cover a lot of tasks on several benchmarks: relative pose estimation, visual disambiguation, multi-view estimation and visual location. It's very convincing.
  4. The presentation and soundness is great.
Critical weaknesses
  1. This paper uses frozen DINOv3 ConvNeXt, but DINOv3 itself may already have strong semantic and domain capabilities. Has it been compared with other backbone methods that also possess strong semantic and domain capabilities?
  2. 'label-free' can clarify that it involves pairs of doppelgangers without the use of manual annotations.
Questions for authors

Can it be demonstrated that the increase in model performance is not mainly dependent on the DINOv3 backbone?

1 comment

Dependence on the DINOv3 backbone. We use the DINOv3 ConvNext-B backbone. The RoMa v2 baseline also uses a DINOv3 backbone (even larger, ViT-L vs. ConvNext-B for us), and we clearly improve over this baseline on challenging benchmarks like HardMatch or SatAst (Tables 1 and 2). Consequently, given the similar backbones, the improvements are attributed to our sparse-to-dense matching architecture. In earlier experiments, weaker backbones generally resulted in lower accuracy. We will add an ablation with the ImageNet-pretrained ConvNeXt in the camera-ready version.

Wording of "label-free". We acknowledge the ambiguity and have clarified this in the paper by explicitly stating the datasets used for training in Tab. 3. This variant uses no manually annotated doppelganger pairs, only flip augmentations (40%) and non-overlapping pairs derived from the GT depth in our matching datasets (10%).

Comments on the paper

1 comment

We thank the reviewers for their valuable feedback and attention to detail. Reviewers found the sparse-to-dense design well motivated (x6K2, QZM7), the joint formulation of matching and visual disambiguation valuable (x6K2, 66uv, QZM7), the early exit on unmatchable pairs novel and practical (QZM7), the LaMAR-DG benchmark a useful addition (66uv, QZM7), and the evaluation broad and convincing (x6K2, 66uv). The main concerns were the precision of some claims, the evaluation protocol of efficiency and early exit, and details on training and SfM, which we address in the individual responses.

We incorporated the feedback in the revision, with changes colored in blue. Table, figure and section numbers refer to the revised version. We have also improved the model itself with stronger augmentations (more in-plane rotations) and better match confidences (Section 3.1), which together result in increased performance across most benchmarks. We further added evaluations on a wide range of two-view estimation benchmarks (Tab. 2), showing that FlexGlue performs well in many scenarios. For consistency with the baselines, we now use DaD keypoints for all keypoint-based methods in ETH3D SfM and IMC2025 (Tab. 4 and 5), except SuperPoint+LightGlue. Additionally, we have found that the ViSym dataset of Doppelgangers++ is noisy (derived from GPS poses), so we have manually cleaned the test set, ViSym-F (Tab. 3, 11 and Fig. 12). The updated training recipe barely affects disambiguation (LaMAR-DG 91.5 vs. 92.3), so we only re-evaluated the disambiguation ablations on ViSym-F. We detail our 1.5-entmax readout (Sec. 3.1) and ablate it against a softmax (Tab. 7). The updated version further expands on the early-exit rule and provides a detailed analysis of its impact on the accuracy-speed trade-off (Fig. 9). We address the note of a reviewer on the limitation of detector repeatability in SfM by showing in Section B.6 how sparse-to-dense matches can alleviate this issue. We also revised some claims regarding efficiency and accuracy and added a detailed limitations section (Appendix F).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.