acceptodds
Withdrawn from ICLR 2027

P2G: From Point-wise Matching to Relational Graphs for Training-Free Cross-Modal Image Registration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cross-modal image registration is challenging because modality discrepancies and large geometric transformations weaken the reliability of point-wise matching evidence. Rather than further strengthening individual point descriptors, we observe that intra-image relations among corresponding points remain consistent across modalities, providing complementary evidence of compatibility. Based on this observation, we propose P2G (Points-to-Graph), a task-training-free framework that progressively transforms corresponding point matches into a weighted correspondence relational graph. P2G first estimates a coarse affine transformation and preserves multiple plausible matches under geometric guidance, and then resolves their ambiguity using complementary structural and frozen foundation-model features to reach one-to-one correspondences as weighted graph nodes. Graph edges are constructed from multi-layer features of a frozen foundation model and encode how consistently intra-image query feature relations are preserved across modalities. Node reliability and edge compatibility jointly guide precise correspondence-triplet sampling for robust affine estimation. Experiments across multiple cross-modal datasets and geometric difficulty settings demonstrate that P2G achieves strong registration accuracy and robustness to shifts in scene distribution and geometric transformation than dataset-specific alternatives, offering a new graph perspective for cross-modal feature representation. Our source codes will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-modal Image registration, Feature representation, Foundation model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5C8pOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

P2G proposes affine registration of cross-modal images without task-specific training. GRID-FAST/LPSD and FSC estimate a coarse transform, after which structural descriptors and frozen DINOv3 features select one-to-one matches from multiple candidates. These become weighted graph nodes; edges measure agreement of within-image feature similarities across modalities. Node and edge scores guide three-point sampling and affine estimation. On OSDataset Hard, AUC@3px is 44.73%, compared with 33.33% for the best baseline (Table 1). Tests also cover visible–infrared and other modality pairs.

Critical strengths
  1. The premise is useful: relationships among points may be more stable across modalities than direct descriptor similarity. On fixed correspondences, Appendix D (Table 4) reports ROC-AUC of 0.729 for graph scores against 0.670 for point-wise DINOv3 scores on WHU-OPT-SAR Hard.
  2. The reported Hard-setting results are competitive without task-specific fine-tuning. On OSDataset Hard, AUC@3px is 44.73% versus 33.33% for RoMaV2, and AUC@5px is 63.89% versus 56.07% (Table 1).
Critical weaknesses
  1. The affine estimation procedure is not clearly specified. In Sec. 3.4, the sampling weights in Eqs. (9)–(10) can become negative, while the final hypothesis-selection criterion in Eq. (12) appears inconsistent with the objective of minimizing registration error. Since these steps are essential to the proposed method, the authors should clarify the actual implementation and ensure that the mathematical formulation matches the reported experiments.
  2. The contribution of relational edge modeling could be further clarified. Although Table 3 provides comprehensive component-level ablations, the relation-guided estimator is compared with FSC, making it difficult to isolate the benefit of relational edge weights from the sampling strategy itself. Additional comparisons with node-only guided sampling and shuffled-edge guidance under identical correspondence sets and sampling budgets would provide stronger evidence for the proposed graph representation.
Questions for authors
  1. Could the authors clarify how the sampling probabilities in Eqs. (9)–(10) are computed and how the final affine hypothesis is selected in Eq. (12)? Please provide the corresponding pseudocode and confirm whether the reported experiments follow the described implementation.
  2. Under identical correspondence sets and sampling budgets, how does the proposed relation-guided estimation compare with node-only guided sampling and shuffled-edge guidance? Could the authors also report the runtime of the main stages?
Reviewer_QDo3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes P2G, a task-training-free framework for cross-modal image registration that extends point-wise correspondence matching to a relational graph representation. The key motivation is that, although individual feature descriptors may be inconsistent across modalities, the relationships between features within each image can remain relatively stable. The proposed framework consists of four stages. First, it estimates a coarse affine transformation using handcrafted structural descriptors and retains multiple candidate correspondences for each keypoint. Second, it combines local structural descriptors with frozen DINOv3 features through adaptive fusion to select reliable one-to-one correspondences as weighted graph nodes. Third, it constructs relational edges by measuring the cross-modal consistency of intra-image feature similarities across multiple feature layers. Finally, node reliability and edge compatibility jointly guide correspondence-triplet sampling for robust affine transformation estimation. The method is evaluated on four datasets covering visible–SAR, visible–infrared, and other modality combinations under different geometric transformation settings. The results demonstrate competitive registration performance, particularly under large geometric transformations. Ablation studies and additional correspondence-reliability analyses further investigate the contributions of the proposed components.

Critical strengths
  1. A meaningful relational perspective on cross-modal correspondence reliability. The paper addresses an important limitation of conventional point-wise matching by exploiting the consistency of intra-image feature relationships across modalities. Instead of relying exclusively on direct cross-modal descriptor similarity, the proposed edge representation evaluates whether relationships between corresponding features are preserved. This provides a potentially useful complementary signal when appearance discrepancies make individual correspondences ambiguous. Importantly, the additional analysis in Appendix D provides empirical support for this motivation: the feature-graph scores achieve higher ROC-AUC and average precision than point-wise semantic scores, while shuffling edge weights substantially reduces their discriminative ability. These observations suggest that the proposed relational representation captures meaningful correspondence information.
  2. Encouraging empirical performance supported by component-level ablations. The method demonstrates strong registration accuracy across multiple modality combinations, particularly under challenging geometric transformations. For example, on WHU-OPT-SAR (Hard), P2G achieves an RMSE* of 0.98 pixels and an AUC@1px of 15.36%, outperforming the compared methods on these metrics. The component ablations in Table 3 also show consistent improvements from candidate generation, node construction, and relation-guided estimation. The convergence comparison in Figure 7 further suggests that relation-guided sampling can improve hypothesis estimation under a limited iteration budget. These results provide useful empirical evidence for the effectiveness of the overall framework.
Critical weaknesses
  1. Potential inconsistencies in the core sampling and affine estimation formulations raise concerns about technical correctness. The relation-guided sampling formulation in Section 3.4 appears mathematically problematic. In Eqs. (9) and (10), the conditional sampling probabilities are proportional to the negative logarithm of sums of exponentiated negative node and edge weights. Since the node and edge weights are bounded by (0, 1] and [0, 1], respectively, the expression in Eq. (10) is always negative, while Eq. (9) can also produce negative values. As written, these expressions cannot directly define valid nonnegative sampling probabilities. Furthermore, Eq. (12) maximizes a score defined as the sum of weighted inlier reprojection errors. For a fixed inlier set, this objective favors larger errors rather than more accurate transformations, which appears inconsistent with robust geometric estimation. These may be typographical errors, but they affect the central estimation algorithm. Without clarification of the actual implementation, it is difficult to assess the correctness and reproducibility of the reported results.
  2. The independent contribution of relational modeling is not sufficiently isolated, and the evaluation leaves some robustness claims under-supported. Although the ablation studies demonstrate improvements from relation-guided estimation, they mainly compare the proposed approach against FSC and RANSAC. It remains unclear how much of the improvement specifically originates from the proposed semantic relational compatibility, rather than from reliability-weighted or otherwise guided sampling. More controlled comparisons with node-only sampling and alternative pairwise compatibility measures would strengthen the novelty and effectiveness claims. Additionally, most quantitative evaluations rely on synthetically generated affine transformations, while the real-transformation multimodal evaluation contains only 50 image pairs. The reported RMSE* also excludes registration failures exceeding 10 pixels, limiting its usefulness as a standalone robustness metric. Finally, the reported inference time of 4.05 seconds per image pair is substantially higher than that of the learning-based baselines. These limitations weaken the evidence for broad practical generalization and computational efficiency.
Questions for authors

(1) Clarification of the sampling probabilities and hypothesis evaluation. Could the authors clarify the exact sampling probabilities used in Eqs. (9) and (10)? Under the stated ranges of node and edge weights, the expression in Eq. (10) is always negative. How are these quantities converted into valid sampling probabilities in the implementation? Additionally, Eq. (12) maximizes the sum of weighted inlier reprojection errors. Is this the actual hypothesis-selection criterion, or should the objective involve minimizing reprojection errors or maximizing weighted inlier support? Please provide the implemented formulations and clarify whether the reported experiments were conducted using them. This clarification is essential to my assessment of the method's soundness.

(2) Isolating the contribution of relational compatibility. Could the authors provide a controlled comparison between the proposed relation-guided sampling and alternative sampling strategies under identical correspondence sets, hypothesis budgets, and evaluation criteria? In particular, comparisons with node-reliability-only sampling and conventional pairwise geometric compatibility would help establish whether the reported improvements originate specifically from the proposed cross-modal semantic relations. The existing comparisons with FSC and RANSAC do not fully isolate this contribution.

(3) Robustness to coarse-registration errors and overall registration failures. Since candidate generation and semantic feature extraction depend on the initial affine transformation, how sensitive is P2G to errors in the coarse estimate? Could the authors quantify the Top-K candidate recall and final registration success rate as a function of initial registration error? Additionally, reporting overall failure rates or SR@10px alongside RMSE* would help clarify whether improvements in conditional registration accuracy translate into consistently robust performance across all test pairs.

Reviewer_QU1zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes P2G, a framework for cross-modal image registration that requires no task-specific training. It first generates multiple candidate matches using structural descriptors and a coarse affine estimate, then combines structural and frozen DINOv3 features to select and weight correspondences. These correspondences form graph nodes, while edge weights measure the cross-modal consistency of within-image feature similarities. The resulting graph guides correspondence-triplet sampling for affine estimation. Validation includes four datasets, different geometric difficulty settings, component ablations, and analyses of relational compatibility. The reported results show competitive registration accuracy, particularly under large geometric transformations.

Critical strengths

(1)The relational formulation is well motivated and supported by targeted evidence. Comparing within-image feature relationships provides a plausible source of correspondence evidence when direct cross-modal feature similarity is unreliable. Appendix D evaluates the same correspondences using point-wise and relational scores: the relational score improves ROC-AUC from 0.670 to 0.729 on WHU-OPT-SAR and from 0.651 to 0.688 on OSDataset. Shuffling edge weights removes this advantage. These results support the central observation that relational compatibility contains useful information beyond individual semantic matches.

(2)The reported accuracy under challenging geometric transformations is practically meaningful. Without task-specific training, P2G achieves the best reported results across all four metrics on OSDataset Hard, including an AUC@3px of 44.73%, compared with 33.33% for the strongest evaluated baseline. Table 3 also shows substantial improvements after adding relation-guided estimation. This combination of competitive accuracy and deployment without task-specific optimization is a substantive strength.

Critical weaknesses

(1)The core sampling and hypothesis-selection rules contain unresolved mathematical inconsistencies. Under the stated ranges of node and edge weights, the right-hand side of Eq. (9) can have mixed signs, while that of Eq. (10) is always negative, with an upper bound of (1-\log 3<0). These expressions therefore do not directly define the intended sampling probabilities. Although the text refers to a log-probability function, the required conversion to valid probabilities is unspecified. Separately, Eq. (12) maximizes (S(H_t)=\sum_{i\in\Phi}w_i e_i(H_t)), which rewards larger reprojection errors for a fixed inlier set. These may be presentation errors, but they concern the estimator responsible for the final results. Without clarification of the actual implementation, I cannot establish the soundness or reproducibility of the proposed procedure.

(2)The comparative evidence for cross-modal generalization is incomplete. On RoadScene, all learning-based baselines use weights trained on WHU visible–SAR data; on MRSIDataset2, comparisons include only handcrafted methods. These experiments provide useful evidence under the selected transfer conditions, but leave the advantage over directly applicable pretrained cross-modal matchers unresolved. In particular, MatchAnything is cited but not evaluated, despite its relevance to generalization across modalities. This gap limits my assessment of the practical significance of the claimed generalization advantage.

Questions for authors

(1)Clarification of the implemented algorithm: Are the quantities in Eqs. (9)–(10) probabilities or log-scores, and how are they converted into a valid sampling distribution? What complete hypothesis-scoring function is actually used in Eq. (12)? Please provide pseudocode and confirm that it corresponds to the reported tables and Figure 7.

(2)Targeted validation of the relational contribution: With correspondences, node weights, hypothesis scoring, and sampling budget held fixed, how does final registration performance change when edge compatibility is removed or shuffled? Table 3 changes the estimator, while Appendix D evaluates correspondence classification rather than final registration. This comparison would clarify whether relational edges provide an independent end-to-end benefit.

(3)Targeted validation of the generalization claim: Under the same evaluation protocol, how does P2G compare with publicly released general-purpose cross-modal pretrained models, such as MatchAnything, without adaptation to RoadScene or MRSIDataset2? This comparison would materially affect my assessment of the claimed deployment and generalization advantages.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.