acceptodds
Withdrawn from ICLR 2027

FlowMatcher: Heteroscedastic Conditional Transport for Dense Correspondence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A dense matcher should report not only where each pixel corresponds, but how reliable that correspondence is. Coarse-to-fine matchers expose this reliability at the coarse stage, where matching scores spread across candidate regions, yet de- code a single warp and refine it deterministically. We introduce FlowMatcher, a stochastic refiner that makes this coarse ambiguity the source of a transport pro- cess: candidate target coordinates are drawn from a Gaussian centered on the coarse warp, with a per-pixel scale predicted from the coarse matching scores, and a time-conditioned multiscale refiner moves each candidate toward the cor- rect correspondence. Samples converge where the evidence is decisive and stay apart where it is not, so their disagreement serves as an uncertainty signal. Built on the frozen RoMaV2 coarse matcher, FlowMatcher remains competitive with RoMaV2 in accuracy, while its sample spread ranks errors better than a RoMaV2- style precision head trained on our architecture, improves relative pose as residual weights, and yields held-out 90% regions after split-conformal calibration. The samples also contain correct answers that a single prediction misses: an oracle choosing among them raises one-pixel accuracy by 5–10 points over their mean, and a ground-truth-free cycle-consistency criterion recovers part of this gain in dense accuracy and pose. Although both refiners are trained at 640 × 640, when both methods are run directly at 1024 × 1024, FlowMatcher exceeds RoMaV2 in one-pixel accuracy on all dense benchmarks tested.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Dense Image Matching, Flow Matching, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yWVYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a stochastic refinement module called FlowMatcher to replace the refiner in RoMaV2. This module preserves RoMaV2's frozen DINOv3 coarse matcher and VGG fine features. For each source pixel, candidate target coordinates are taken from an isotropic Gaussian distribution. The mean of this distribution is the coarse distortion, and its pixel scale is obtained by fixed-log linear regression on six handcrafted statistics of the coarse matching distribution (entropy, modal quality, dispersion, and distance to the modality and mean).

Subsequently, a three-stage (step size 4/2/1) temporal conditional refiner predicts a sharp endpoint based on each hypothesis. It uses a RAFT-style relevance lookup to find the current state and integrates the endpoint along a linear path with N Euler steps. During training, a robust endpoint loss for a single true distortion is used, where the step-unfolded state is constructed by the model's own t=0 prediction, rather than by the true distortion.

A set of candidate deformations is obtained by repeating the cascade K times. These candidate deformations are used in three ways: (i) their mean; (ii) pixel-wise selection based on round-trip error and deterministic back-projection (“cyclic”); and (iii) their empirical covariance as an uncertainty signal. This signal is evaluated using AUSE as a residual weight in pose refinement and after split conformal or Student-t calibration.

Experiments cover dense correspondences of seven reserved populations, relative pose, uncertainty ranking and calibration, homoscedastic source ablation, and runtime on impaired HPatches, MegaDepth-1500, ScanNet-1500, and ETH3D. Key findings are as follows: FlowMatcher's deterministic processing lags RoMaV2 by 1–2 PCK@1 at 640 pixels, while cyclic selection with K=32 roughly reduces the gap. On the AUSE dataset, FlowMatcher outperforms the jointly trained RoMaV2-style exact matching head in sample diffusion performance across two of the three populations. With 1024 input samples, FlowMatcher's performance degradation is less than that of RoMaV2.

Critical strengths

The empirical research is unusually thorough and candid. It covers AUSE's seven dense crowds, three posture benchmarks, corruption robustness, and scene-guided intervals. The paper clearly reports the weaknesses of the approach. Examples include public grid analysis showing RoMaV2 is still most accurate at 640, cross-domain calibration failure, and pixel noise corruption. It also gives failure cases for loop selection. The appendix details the architecture, source scale factors, data mix, and protocols for replication.

The core idea is simple and practical: Coarse information hypothesis perturbations are combined with correlation finding of estimated correlations to produce a sample set that acts as both an uncertainty and a proposal. Distribution-based weights lead to large gains when refining poses on the unfiltered set of matches (e.g., +11.0/+14.2 AUC@5 on MegaDepth/ETH3D with squared loss), and ETH3D poses improve across all baselines. The analysis of what the raw spread does and does not measure is careful: it shows that the spread covers only 5-27% of the nominal 90%, i.e. it reflects sensitivity to initialization rather than the full error distribution. This analysis is valuable.

Critical weaknesses

The core claim that random transmission generates useful uncertainties and hypotheses is not isolated from the correct baseline. Lacking the most natural control: take RoMaV2's own deterministic refiner, or use a FlowMatcher trained with N=1 and noise-free, perturb its input warp with the same heteroscedastic Gaussian, and measure expansion, AUSE, oracle headroom and period selection gain. Without this control, it is unclear whether stream-matching-style training would increase anything other than test-time input perturbation to a lookup-based refiner. The paper's own evidence suggests as much. Homoscedastic ablation ranks errors the same as or slightly better than heteroscedastic errors (AUSE 0.195 vs 0.200), with the conclusion stating that "uncertainty comes from transmission" rather than from nominal heteroscedastic sources. Without such a baseline, the GT-oracle headroom (5-10 PCK@1) is equally uninformative, since choosing the best one among 33 perturbation candidates per pixel versus the true value will improve the PCK@1 of almost any reasonable perturbation scheme. The "traffic matching" framework is also very loose. The model is trained on a single true endpoint with a robust loss, and its samples are not conditionally distributed, as evidenced by the raw coverage of 12-27%. In effect, it is an iterative refinement of injected noise. Heteroscedastic source accuracy gain (+1.5 PCK@1) from single seed, 100k steps, MegaDepth only run.

The practical accuracy and cost trade-offs are negative, and some title claims are generous in their framing. Deterministic delivery at PCK@1 for each population is lower than RoMaV2 at matched input resolution (640). Exceeding it requires K=32 draws plus backward passes, which costs around 50-100x the RoMaV2 latency (9x for K=4, N=4). The loop gain is small (+0.5–1.1 PCK@1), and selection still lags behind RoMaV2 on MegaDepth. On MegaDepth-1500 poses, this method lags behind RoMaV2+HR by 2.4 AUC@5. Summary Statement FlowMatcher outperforms RoMaV2 on 1024, comparing both models outside their training resolution with RoMaV2's HR pass disabled. As acknowledged in the appendix, it shows robustness to resolution changes rather than higher accuracy. Due to uncertainty, K ≥ 8-16 is required to beat the trained head, and it is only bound to MegaDepth (two test scenarios); no comparison or ensemble/MC-dropout baseline for computational matching is given. A split-conformal 90% coverage of commutative random halves of the same population holds by construction and should not be taken as evidence of quality. Student-t calibration must be rescaled based on domain and settings. All results are from a single training seed (~4,270 GPU hours), so the significance of many sub-point differences is uncertain.

Questions for authors

Can you report the distribution-based AUSE, oracle margin, and period selection gain of a deterministic refiner (such as the refiner released by RoMaV2 or the refiner you trained with no noise and N=1) and apply it to an input perturbed by the same σ-scale Gaussian? This is the key to testing whether transportation training is important.

What are the latency and accuracy for the full K=32 plus backpass loop configuration used in Table 1? How does propagating AUSE and cyclic PCK compare to RoMaV2 or a training head for matching computation, such as a deep ensemble of 4 deterministic refiners with K=4 draws?

Since homoscedastic sources rank errors equally well, what exactly is the role of σ apart from modest accuracy gains? Has the homoskedasticity constant scale been adjusted? How much does the +1.5 PCK@1 gain vary from seed to seed?

Can you report the oracle headroom for a simple candidate set, such as RoMaV2's predictions plus a random Gaussian shift from the matching distribution, so that the 5-10 point oracle gain can be accounted for?

For Table 4, can you clarify why the "all 5,000" setting is a realistic deployment scenario? Can you also report distribution weights combined with RoMaV2+HR channels? Is this the strongest RANSAC-only baseline?

How sensitive are the results to the six hand-crafted rough statistics and their least-squares fits compared to, say, individually learned scale heads or entropy?

Reviewer_1QqtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FlowMatcher, a stochastic dense-matching refiner using coarse-conditioned Gaussian proposals and time-conditioned refinement. Multiple samples support correspondence selection and uncertainty estimation. However, the contribution is insufficiently isolated, and substantial presentation problems undermine the manuscript’s technical clarity.

Critical strengths
  1. Connecting coarse matching evidence to the refinement search radius is a reasonable design choice. The heteroscedastic source provides a practical way to allocate different proposal widths across pixels. The training-time comparison in Table 6 supports an accuracy benefit over the tested constant-scale source.
  2. The evaluation covers several useful aspects of correspondence quality. Beyond matching accuracy, the paper examines uncertainty ranking, downstream pose refinement, calibration, and failure cases. The explicit acknowledgement that sample spread measures initialization sensitivity and can miss shared errors is valuable.
Critical weaknesses
  1. The central contribution is insufficiently isolated. The experiments do not clearly distinguish the benefit of the transport formulation from stochastic initialization and repeated local refinement. The inference-time interventions in Table 15 cannot replace a matched, retrained baseline without time conditioning.
  2. The computational overhead weakens the practical contribution. At 640 resolution, the 32-sample mean underperforms RoMaV2 in PCK@1 across all seven evaluated populations; cycle selection exceeds it on only three. Even the four-step, four-sample configuration incurs approximately nine times the latency. Comparable-budget comparisons are needed to justify this cost.
  3. Figure 2 requires clarification and revision. Its input images depict the same scene as Figure 3’s third row but appear visually altered, raising questions about their provenance. The connector routing also makes conditioning dependencies and iteration boundaries difficult to follow. The authors should distinguish illustrative imagery from actual model inputs and outputs.
  4. The experimental presentation is insufficiently focused. Both the main text and appendix repeatedly enumerate scores and configurations without consistently explaining the hypotheses, decisive controls, and implications. This makes the evidence for the central contribution unnecessarily difficult to assess.
Questions for authors
  1. How was Figure 2 produced? Were generative tools used to modify its imagery, and which images represent actual model inputs or outputs?
  2. Would an otherwise matched refiner, retrained without time conditioning and evaluated with the same proposals, refinement budget, and selection procedure, achieve comparable results?
Reviewer_J1kkOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents the FlowMatcher, a framework to compute dense pixel correspondence between images. The framework provides an initial data-dependent warp distribution, which is then fed into flow matching for refinement. The uncertainty helps improve the matching accuracy.

Critical strengths
  1. The paper presents a way to estimate uncertainty of the warp.
  2. The paper shows that incorporating the uncertainty information helps improve matching accuracy.
Critical weaknesses
  1. Lack novelty. (a) The idea of data-dependent source distribution has been used in prior works, such as PriorGrad [1]. The idea of sampling multiple warps to estimate uncertainty has been discussed in [2].
  2. The method requires high computational cost.
  3. In Equation 3, the Gaussian assumption for warp is limited. In practice, the distribution is often multi modal [3].

[1] PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior. [2] Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow. [3] SMD-Nets: Stereo Mixture Density Networks.

Questions for authors
  1. Is it possible to provide theoretical analysis for the change in line 228-232? How will this affect the training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.