acceptodds
Withdrawn from ICLR 2027

RVHomo: Unsupervised Cross-Modal Homography Estimation with Geometry-Aware VFM Features

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unsupervised cross-modal homography estimation is challenging because modality-dependent appearances obscure shared geometric structures, while common surrogate objectives do not explicitly enforce geometry-preserving behavior. We propose the Recurrent VFM-guided Homography Estimation Network (RVHomo), which combines broad structural cues from a frozen vision foundation model with the spatial precision of local CNN features. Rather than directly injecting coarse VFM tokens, RVHomo transforms hierarchical DINOv2 features into scale-aligned geometric representations and fuses them with CNN features for coarse-to-fine recurrent estimation. A dual-scale reconstruction objective further encourages the adapted representations to retain spatial structure. We additionally enforce geometric consistency at the modality-transfer and homography-prediction levels, imposing warp equivariance and composition-consistent predictions without cross-modal transformation labels. To reduce model complexity, we derive RVHomo-D through two-stage progressive distillation, eliminating modality transfer and online VFM extraction at inference. Experiments on GoogleMap, OPT-SAR, FSD, and VIS-IR demonstrate that RVHomo achieves MACE values of 0.65, 1.95, 4.32, and 4.55, reducing error by 17.4–46.7% relative to the strongest competing estimator on each dataset. RVHomo-D remains more accurate than all evaluated competitors while reducing RVHomo's total parameter count by 98.5% and computation by 70.4%.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-modal homography estimation, Unsupervised image registration, Vision foundation models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ShHHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RVHomo, an unsupervised framework for cross-modal homography estimation that combines hierarchical features from a frozen DINOv2 vision foundation model (VFM) with local CNN features. The method introduces geometry-aware feature upsampling and multi-level fusion, a dual-scale reconstruction objective, and two geometric consistency losses that regularize modality transfer and homography prediction. A two-stage progressive distillation strategy is further developed to obtain a lightweight model, RVHomo-D, without modality transfer or online VFM extraction.​ The method is evaluated on four datasets covering different modality pairs. The problem is relevant, and the reported results are promising. However, the novelty of the individual components and their relationship to prior work needs to be established more convincingly. The experimental evaluation, although reasonably broad within the unsupervised setting, could better isolate the contributions of the proposed components and contextualize the method against recent VFM-based approaches. The presentation of the architecture and qualitative results could also be improved.

Critical strengths

Relevant problem and clear motivation. Unsupervised cross-modal homography estimation is challenging because substantial appearance differences make conventional correspondence and reconstruction objectives unreliable. Combining VFM representations with spatially precise CNN features is a reasonable direction. Promising quantitative results. The method is evaluated on four datasets involving substantially different modality pairs. RVHomo reports MACE reductions of 17.4%–46.7% relative to the strongest evaluated competitor on each dataset. The paper also includes architectural ablations and experiments examining the effects of the two geometric consistency losses. Useful model-compression strategy. The two-stage distillation design addresses the considerable cost of using a large frozen VFM at inference. The reported reductions in total parameters and GFLOPs are substantial, and RVHomo-D maintains competitive estimation accuracy. Some evidence for geometric consistency. The appendix includes controlled perturbation experiments and aggregate measurements of spatial preservation and prediction robustness. These analyses provide additional support for the proposed consistency objectives beyond the main benchmark table.

Critical weaknesses
  1. The main concern is that the framework largely integrates established ideas without clearly establishing a distinct methodological contribution. It builds on SSHNet's reformulation and split optimization, adds hierarchical DINOv2 features into a CNN-based estimator, and applies known techniques: multi-scale fusion, reconstruction regularization, warp equivariance, composition consistency, and teacher–student distillation. Each component is plausible, and the combination may be effective, but the paper does not make clear which part is the principal advance and why this is more than an integration of existing techniques. The ablations support the contributions of the VFM features, GFU, hierarchical aggregation, and consistency losses, but do not isolate the dual-scale reconstruction objective, nor show that the consistency losses help beyond applying similar constraints to a plain SSHNet baseline.
  2. The baselines are reasonably broad for the unsupervised setting, but two gaps remain. First, recent VFM-based methods (GFNet, DistillMatch) discussed in related work are not benchmarked; their supervision differs, so a fair comparison may be limited, but this should be explained and attempted where possible. Second, all SSHNet baselines omit the consistency losses, so the main comparison conflates architecture and objectives; a controlled comparison is needed to attribute each contribution. The ablations use only OPT-SAR (consistency analysis only on Google Maps); results are single-point estimates without multi-seed variance, and efficiency reports parameters/GFLOPs rather than measured latency or memory.
  3. The related-work section would benefit from a clearer comparison of the assumptions, supervision, representation-learning mechanisms, and evaluation protocols of recent VFM-based methods. This would help establish precisely what remains unresolved by prior work and sharpen the novelty claim.
  4. Figure 1 provides a reasonably clear high-level overview, whereas Figure 2 combines several conceptually different processes and detailed architecture blocks in a dense layout. Many labels and symbols are difficult to read at the paper's displayed scale. Separating the geometric-consistency mechanisms from the detailed feature-fusion architecture, or enlarging the individual panels, would make the method easier to understand. Figure 4 provides a useful side-by-side comparison, but the individual image panels and numerical annotations are small. Enlarged crops or additional visualizations of challenging regions would make differences in alignment more apparent. Moreover, the supplementary qualitative section explicitly presents examples in which RVHomo performs favorably. These examples are useful for illustration but do not establish typical performance or characterize failure cases.
Questions for authors

Can the authors provide a controlled comparison that separates the effects of the proposed consistency losses from those of the VFM-guided architecture, ideally by applying the losses to an appropriate SSHNet-based baseline? What is the independent contribution of the dual-scale reconstruction objective? How sensitive are the results to the selected DINOv2 layers and the feature-fusion design? Can the main results and key ablations be reported over multiple random seeds, with variability estimates, to establish the stability of the improvements? How do the proposed method and RVHomo-D compare with recent VFM-based homography or matching approaches under clearly specified, comparable supervision and evaluation protocols? Can the authors provide measured inference latency, throughput, and memory usage, together with fuller details of the training setup and dataset splits, to improve reproducibility?

6 comments
AuthorsOpenReview ↗

Response to Reviewer ShHH (Part 1/6)

We thank the reviewer for recognizing the relevance of the task, the cross-modal accuracy improvements, and the practical value of the compact student. We have strengthened the experimental controls and clarified the methodological scope, related work, reproducibility, efficiency, and visualization. All references below use the revised manuscript numbering.

W1: Principal contribution and relationship to established techniques

Concern. The framework integrates established ideas, and the principal advance and its relationship to previous work need to be established more convincingly.

Response. We thank the reviewer for requesting a clearer account of our methodological contribution. RVHomo develops a geometry-aware adaptation pathway that converts hierarchical VFM features into spatially precise, multiscale representations for unsupervised cross-modal homography estimation, together with component-specific geometric constraints within SSHNet’s split optimization. The contribution concerns how these representations and constraints are constructed for this learning setting, building on established VFM and geometric-consistency principles.

The representation challenge is to retain both cross-modal structural cues and the spatial precision required for four-corner homography prediction without cross-modal transformation labels in the training objective. Deep VFM features provide broad structural context, whereas local CNN descriptors preserve finer spatial information. RoMa uses deepest-layer DINOv2 features for coarse dense matching followed by CNN-based refinement. GFNet introduces cross-view interaction on DINOv2 features and conditions CNN/FPN refinement on the coarse representation for grid-based homography estimation. RVHomo reorganizes these information sources: it applies gated spatial refinement and top-down aggregation to multiple DINOv2 depths, then fuses the adapted representations with local CNN descriptors at both estimation scales. Structural context and local detail are therefore jointly available throughout recurrent global-homography estimation.

Feature-use comparison (Table 4, OPT-SAR). GFNet is a supervised reference; the other rows share the unsupervised estimator and objectives, with both consistency terms disabled:

ApproachFeature-use strategyRefinementMACE
Supervised reference
GFNet——2.498
Unsupervised comparisons
RoMa-style featuresDINOCNN features23.47
GFNet-style featuresCross-view DINODINO-conditioned CNN features26.91
Separated DINO/CNNDINOLocal CNN features24.37
RVHomoMultilevel DINO–CNN fusion—2.38

Table 4 examines why a dedicated adaptation pathway is needed. We transfer RoMa-style and GFNet-style feature pathways to the shared recurrent homography estimator and additionally evaluate a Separated DINO/CNN configuration, which uses deepest-layer DINO features at the first scale and RVHomo’s local CNN branch at the second. Under the shared unsupervised protocol, with both proposed consistency losses disabled, these alternatives obtain end-of-training OPT-SAR MACE values of 23.47, 26.91, and 24.37, compared with 2.38 for RVHomo. Despite completing the full 120,000-iteration training schedule, these adapted configurations retain high estimation errors. Together with the high final error of Separated DINO/CNN, these results show that none of the three alternatives achieves accurate homography estimation under the evaluated protocol. Their finite MACE values are reported rather than marked NC, consistent with Appendix A.4. In particular, retaining RVHomo’s local CNN refinement branch does not recover the accuracy of its fused representations. These results support dedicated spatial adaptation and multilevel integration in the evaluated setting. They concern adapted feature pathways within our framework, rather than the performance of the complete original supervised matchers. This diagnostic comparison does not isolate the coarse-stage VFM features as the sole cause of the performance gap.

AuthorsOpenReview ↗

Response to Reviewer ShHH (Part 2/6)

Continued topic: W1: Principal contribution and relationship to established techniques

The component controls then examine the proposed adaptation design. Table 5 evaluates VFM removal, conventional scale adaptation, and removal of top-down aggregation. Appendix C.1 retains four adaptation branches while varying their feature sources: using blocks [6,12,18,24] obtains 1.95 MACE, compared with 2.67 when all branches receive block 24. Appendix C.2 further compares GFU with conventional convolutional adaptation at nearly equal parameter counts—2,696,192 and 2,698,022—obtaining 1.95 and 2.34 MACE, respectively. These full-objective experiments support complementary feature depths and the proposed spatial-adaptation mechanism, while controlling adapter parameter capacity.

For geometric consistency, our distinctions concern the supervision targets and update paths. NeMAR uses registration-predicted deformations to encourage translation–registration compatibility through reconstruction. Our transfer constraint instead uses an independently sampled known perturbation. Relative to Honkamaa et al.’s direct commutation objective, it employs a detached warped reference translation, so gradients update the transfer network only through the perturbed-input branch. For prediction consistency, Warp Consistency derives the composition relation and reconstructs a known warp through predicted dense mappings. We instead compose a detached reference homography with the known perturbation to construct a four-corner prediction target, without a reverse matching path; gradients through modality transfer are blocked. The two constraints therefore update different learning components: one encourages spatial preservation during transfer, while the other specifies the estimator’s response to known geometric changes.

Table 3 addresses whether the gains can be explained solely by adding these objectives to an existing estimator. RVHomo without either consistency term outperforms all three consistency-regularized SSHNet variants on every dataset. Adding both terms further improves RVHomo. With architecture and reconstruction fixed, Table 2 independently evaluates the two constraints across all four datasets, with their joint configuration achieving the lowest error on each. These results provide complementary evidence for the representation pathway and geometric regularization.

Reconstruction serves as auxiliary representation regularization, while progressive distillation transfers the resulting estimator to a lightweight student. Their contributions are evaluated separately below. The principal methodological advance is the spatial and hierarchical organization of VFM–CNN representations for this unsupervised estimation setting, complemented by geometric constraints tailored to the two components of split learning.

W2 / question on controlled attribution: Architecture versus consistency losses

Concern. SSHNet comparisons omit the consistency losses, so the main comparison confounds the estimator with the objectives.

Response. Table 3 now applies the same joint consistency regularization to SSHNet-IHN, SSHNet-RHWF, and SSHNet-MCNet, and compares each estimator with the regularization disabled and enabled:

EstimatorGoogleMapOPT-SARFSDVIS-IR
−-++−-++−-++−-++
IHN1.231.112.942.485.325.145.515.23
RHWF1.291.033.082.805.645.585.605.82
MCNet1.221.142.872.856.066.166.226.01
RVHomo0.930.652.381.954.524.324.754.55

For a fixed estimator, this comparison identifies the effect of adding the joint consistency terms. The terms improve IHN and RVHomo on all four datasets. Effects are mixed for RHWF and MCNet: RHWF worsens on VIS-IR and MCNet worsens on FSD. We report these exceptions explicitly.

RVHomo without either term still outperforms all three regularized SSHNet variants on each dataset; adding the terms further improves RVHomo. Thus, the gain cannot be attributed solely to adding the consistency objectives to an otherwise unchanged SSHNet estimator.

AuthorsOpenReview ↗

Response to Reviewer ShHH (Part 3/6)

Continued topic: W2 / question on controlled attribution: Architecture versus consistency losses

The component-wise loss ablation in Table 2 also retains architecture and reconstruction while independently enabling each consistency term. It now covers all four datasets:

Ltrans\mathcal{L}_{\mathrm{trans}}Lhomo\mathcal{L}_{\mathrm{homo}}GoogleMapOPT-SARFSDVIS-IR
×\times×\times0.932.384.524.75
×\times✓\checkmark0.822.194.354.70
✓\checkmark×\times0.692.044.444.63
✓\checkmark✓\checkmark0.651.954.324.55

W1 / questions on reconstruction, DINOv2 layers, and fusion design

Concern. Isolate reconstruction regularization and examine sensitivity to the selected DINOv2 layers and fusion architecture.

Response. We have added three complementary analyses in Appendix C.

Independent reconstruction contribution (Appendix C.3, Table C.3). Architecture and the remaining objectives are fixed:

Reconstruction lossGoogleMapOPT-SARFSDVIS-IR
Disabled0.732.124.364.78
Enabled0.651.954.324.55

The contribution varies across modality pairs, including a small change on FSD. Table C.4 reports the OPT-SAR weight sensitivity:

λrecon\lambda_{\mathrm{recon}}MACE
02.12
51.960
101.948
201.943

The retained coefficient is 10; its unrounded 1.948 result rounds to the reported 1.95.

Layer selection, Appendix C.1 / Table C.1.

ConfigurationFeature blocksBranchesMACE
Deepest layer, four branches[24,24,24,24][24,24,24,24]42.67
Shifted four-depth hierarchy[3,9,15,21][3,9,15,21]42.26
Deepest layer, single branch[24][24]12.78
RVHomo[6,12,18,24][6,12,18,24]41.95

The four-branch comparisons support complementary feature depths and the reported layer selection. The single-branch variant also simplifies adaptation and aggregation, so its change is not attributed to feature depth alone. The indices identify the extracted block outputs; all variants retain the full DINOv2 backbone.

Fusion and scale adaptation (Table 5, OPT-SAR).

ConfigurationMACE
w/o VFM2.54
w/o GFU3.74
w/o top-downNC
RVHomo1.95

The w/o GFU control retains top-down aggregation and uses bilinear resizing plus convolutional projection. The w/o top-down control retains GFU and uses parallel resizing and concatenation. The full model's remaining training objectives are retained.

Parameter-matched adapter control (Appendix C.2).

Scale-adaptation moduleAdapter parametersMACE
Bilinear resize + widened convolutions2,698,0222.34
GFU (RVHomo)2,696,1921.95

Both adapters share the optimization settings, downstream hierarchy, CNN fusion, and estimator. Their parameter counts differ by about 0.068%, supporting GFU's design over the evaluated conventional adapter at nearly equal parameter capacity.

W2 / question on recent VFM methods and supervision

Concern. GFNet and DistillMatch are discussed but not benchmarked. Attempt comparisons where possible and explain differences in supervision and protocol, including the position of RVHomo-D.

Response. We have added GFNet adapted to our 128 × 128 patch benchmark as a supervised reference in Table 4, obtaining 2.498 MACE on OPT-SAR. Appendix A.3 clarifies the resolution adaptation: the official GFNet implementation uses 448 × 448 for initial matching and provides an additional 560 × 560 refinement path, while our reference uses the benchmark's 128 × 128 patches and pixel-coordinate convention. It is a resolution-adapted reference, rather than a reproduction of GFNet's original-resolution evaluation. Table 1 reports 1.95 for RVHomo and 2.04 for RVHomo-D on the same dataset. The GFNet result is explicitly separated from the unsupervised feature-path controls because it uses geometric supervision; we do not interpret the numbers as a comparison under matched supervision.

Reviewer_f15rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies cross-modal homography estimation without ground-truth transformations for the original cross-modal training pairs. Building on SSHNet, RVHomo combines hierarchical frozen DINOv2 features with local CNN features through learned upsampling and multiscale fusion, together with dual-scale reconstruction regularization. It introduces consistency objectives for modality transfer and homography prediction under known geometric perturbations. A two-stage distillation procedure removes the modality-transfer network and replaces the VFM with a lightweight student. Experiments on GoogleMap, OPT-SAR, FSD, and VIS-IR report improved mean average corner error (MACE), supported by architectural and loss ablations, consistency analyses, and model-complexity measurements.

Critical strengths
  1. Consistent reported improvements across several modality combinations, with a compact student. Table 1 reports lower MACE than the strongest listed comparison on all four benchmarks, with relative reductions of approximately 17.4%–46.7% for the full model. The distilled model retains much of this accuracy while removing the online VFM and modality-transfer network; Table 5 reports 4.877M parameters and 35.552 GFLOPs for the student. Together, these results suggest practical value beyond improving a single benchmark.

  2. The geometric-consistency objectives receive meaningful empirical support. Table 2 includes the full two-by-two ablation of the two consistency losses, and Appendix C further examines spatial preservation during modality transfer and prediction robustness under controlled perturbations. This provides more informative evidence than aggregate performance improvements alone and supports the usefulness of addressing both failure modes.

Critical weaknesses
  1. The methodological novelty is insufficiently established.** Combining robust VFM features with precise local features has close precedents in RoMa and GFNet, while geometry-preserving translation and warp consistency are also established ideas. Although the existing ablations demonstrate usefulness, they do not sufficiently isolate the contribution of RVHomo’s specific adaptation beyond these principles. In particular, the reconstruction objective lacks an independent ablation.

  2. Experimental details and controls are insufficient.** The construction of unaligned training/test pairs, complete training configurations, baseline tuning procedures, and several key hyperparameters are insufficiently documented. Moreover, Table 4 is ambiguous about the model associated with the 2.31 result. Clear model identification, comparable training budgets, and sensitivity to the feature-distillation weight are needed to substantiate the proposed explanation for progressive distillation.

Questions for authors
  1. Experimental setting and reproducibility. How are the training, validation, and test pairs constructed for each dataset? Please clarify the original pairs’ alignment status, split sizes and units, evaluation ground-truth generation, and training/test transformation distributions. Please also provide the complete training settings and baseline tuning protocol, including the criterion for reporting non-convergence. These details are necessary to assess what learning setting is actually tested and whether the comparisons are comparable.

  2. Methodological contribution and its isolation. What is the specific technical advance over the feature-combination principles in RoMa/GFNet and the geometric-consistency principles in the cited prior work? For GFU, please clarify whether the existing replacement baseline controls for capacity and optimization settings. An independent reconstruction-loss ablation, with its weight reported and a small sensitivity analysis, would help establish whether the claimed geometric adaptation depends on the proposed mechanism or on auxiliary regularization more generally.

7 comments
AuthorsOpenReview ↗

Response to Reviewer f15r (Part 1/5)

We thank the reviewer for the careful assessment and for recognizing the cross-dataset improvements, compact student, and empirical support for geometric consistency. We have expanded the methodological comparisons and added controlled experiments, reproducibility details, and distillation analyses. All table and appendix numbers below refer to the revised manuscript; the original distillation Table 4 is now Table 6.

Q1 / W2: Experimental setting, reproducibility, and baseline protocol

Concern. Clarify original alignment, pair construction, split sizes and units, evaluation ground truth, train/test transformations, training settings, baseline tuning and model selection, and NC.

Response. Appendix A now documents each part of the protocol.

Original alignment and split units. All four datasets store pre-aligned cross-modal reference pairs. Splits are made at the stored-pair level, before stochastic patch extraction or perturbation. The training and held-out evaluation pools are:

DatasetSource / targetTrain pairsEval. pairsInput canvas
GoogleMapSatellite / map8,822888192×192192\times192
OPT-SAROptical / SAR8,0002,000192×192192\times192
FSDRGB / disparity4,0001,000192×192192\times192
VIS-IRVisible / infrared15,0002,990256×256256\times256

GoogleMap uses train2014 and val2014; the other datasets use their stored train/test partitions. Corresponding filenames pair the source and target modalities. RVHomo and SSHNet do not create an additional validation subset.

Pair construction and evaluation ground truth. Our sample construction follows a widely used homography benchmark protocol: randomly perturb the four patch corners and apply the resulting projective warp. This construction is used in early deep estimators such as DHN and UDHN, in perceptual unsupervised estimation such as biHomE, in recurrent methods such as IHN, RHWF, and MCNet, and in unsupervised cross-modal methods such as SCPNet and SSHNet. Following the cross-modal benchmark setting used by SCPNet and SSHNet, we use 128 × 128 patches with corner perturbations in [−32, 32] pixels. Images are loaded through a three-channel interface and scaled to [0,1]. GoogleMap uses its stored 192 × 192 canvas; OPT-SAR/FSD are resized to that size, and VIS-IR to 256 × 256. A common crop location defines a 128 × 128 patch with a 32-pixel context margin. The 192 × 192 canvas gives the central patch; VIS-IR permits uniformly sampled valid crop locations.

For each patch corner ci\mathbf c_i, each component of δi\boldsymbol\delta_i is independently sampled from the discrete uniform distribution on {−32,…,32}\{-32,\ldots,32\}. We define Hsyn:ci↦ci+δi\mathbf H_{\mathrm{syn}}:\mathbf c_i\mapsto\mathbf c_i+\boldsymbol\delta_i, warp the source context with its inverse, and keep the target unchanged. Because the reference pair is pre-aligned, the forward mapping from perturbed source to target has ground-truth corner offsets Dgt,i=δi\mathbf D_{\mathrm{gt},i}=\boldsymbol\delta_i. Equation (A.1) specifies this construction. Training and evaluation use the same perturbation distribution on their respective image partitions.

The stored alignment defines the reference relation from which the synthetic warp generates each pair. RVHomo and SSHNet do not use the resulting cross-modal offsets in their training loss. They use known synthetic intra-modal transformations for self-supervision. Thus, the benchmark provides evaluation ground truth without using cross-modal transformation labels for training.

AuthorsOpenReview ↗

Response to Reviewer f15r (Part 2/5)

Continued topic: Q1 / W2: Experimental setting, reproducibility, and baseline protocol

Training, tuning, and checkpoint reporting. Both target modalities are available to baselines retrained on each target dataset. RVHomo, all SSHNet variants, and UDHN, CA-UDHN, biHomE, BasesHomo, and SCPNet use the following configuration on all four datasets, retaining their own architectures and objectives:

SettingConfiguration
Input patch size128×128128\times128
Batch size16
Configured training length120,000 iterations
OptimizerAdamW
Maximum learning rate4×10−44\times10^{-4}
Weight decay10−510^{-5}
Learning-rate scheduleOneCycleLR: 5% increasing phase, linear annealing
Gradient clippingMaximum gradient norm of 1.0

AltO retains its alternating procedure: AdamW, learning rate 3×10−43 \times 10^{-4}, weight decay 10−510^{-5}, batch size 16, OneCycleLR, and 200 epochs.

For RVHomo, all SSHNet variants, and these five generic baselines, the main comparison reports the final 120,000-iteration checkpoint. Failed runs follow the NC criterion below.

The added RVHomo coefficients are λrecon=10\lambda_{\mathrm{recon}}=10, λtrans=1\lambda_{\mathrm{trans}}=1, and λhomo=0.25\lambda_{\mathrm{homo}}=0.25. Appendix B.1 separately specifies the consistency perturbations: corner-coordinate ranges of [−32,32] for transfer consistency and [−8,8] for prediction consistency. These are separate source-view perturbations from those used to construct the original estimator input pairs.

Evaluation and NC. MACE uses the final four-corner prediction in the shared source-to-target pixel convention. Five evaluation passes aggregate sampled patches and perturbations for the same trained model; they are distinct from independently trained random seeds. AltO's rectification homography is inverted to match the convention.

NC is reserved for a failed run that cannot produce usable finite corner predictions because of non-finite values, invalid homographies, or a documented training collapse. Whenever finite corner predictions are available, we report their actual MACE, regardless of how large the error is or whether it decreases during training. High or flat error alone does not qualify as NC.

Q2 / W1: Technical contribution beyond existing feature-combination and consistency principles

1. VFM feature adaptation and the relationship to RoMa / GFNet

Concern. Combining robust VFM features with precise CNN features has precedents in RoMa and GFNet. What is the specific advance, and what isolates the proposed adaptation?

Response. We agree that introducing VFM features or combining semantic and local information is not itself our contribution. The principal contribution is a feature-adaptation pathway for unsupervised cross-modal homography estimation: hierarchical frozen DINOv2 features undergo learnable spatial adaptation and top-down aggregation, then integrate with local CNN descriptors at both estimation scales.

The revised Related Work distinguishes how these representations are used. RoMa uses deepest-layer DINOv2 features for coarse dense matching and CNN features for local refinement. GFNet introduces cross-view interaction on deepest-layer DINOv2 features and uses DINO-conditioned CNN/FPN features for refinement. RVHomo instead constructs multilevel VFM–CNN fused representations at both estimation scales. The prediction setting also differs: RVHomo directly estimates the four-corner parameterization of a global homography using intra-modal self-supervision, without cross-modal transformation labels.

To examine whether existing feature-use strategies suffice in this setting, Table 4 adds the following OPT-SAR comparison:

ApproachFeature-use strategyRefinementMACE
Supervised reference
GFNet——2.498
Unsupervised comparisons
RoMa-style featuresDINOCNN features23.47
GFNet-style featuresCross-view DINODINO-conditioned CNN features26.91
Separated DINO/CNNDINOLocal CNN features24.37
RVHomoMultilevel DINO–CNN fusion—2.38
AuthorsOpenReview ↗

Response to Reviewer f15r (Part 3/5)

Continued topic: 1. VFM feature adaptation and the relationship to RoMa / GFNet

For the four unsupervised configurations, the recurrent homography estimator and training objectives are shared, and both proposed consistency losses are disabled. The transferred pathways use scale and channel adapters to meet the estimator's feature interfaces. Separated DINO/CNN uses the local CNN branch employed in RVHomo for the second estimation scale, while retaining deepest-layer DINO features at the first scale without feature fusion. These are adapted feature-path controls; the separately labeled supervised GFNet row evaluates GFNet adapted to our 128 × 128 patch benchmark. Appendix A.3 distinguishes this reference from GFNet's official 448 × 448 initial matching and additional 560 × 560 refinement setting. It now also specifies the adapted matching grids [9, 9, 18, 36, 72], disabled high-resolution refinement, frozen pretrained DINO backbone, and randomly initialized remaining modules. GFNet uses batch size 16, 120,000 updates, AdamW with peak learning rate 4×10−44 \times 10^{-4} and weight decay 10−510^{-5}, linear OneCycleLR with a 5% increasing phase, norm-1 clipping, and FP16 mixed precision. Its native dense-regression and certainty losses use synthetic cross-modal labels. We report the final checkpoint and convert forward correspondences to a homography by threshold-balanced sampling and 3-pixel RANSAC. Appendix A.3 provides the complete settings and failure-handling rule.

The results show that transferring these feature-use strategies, including replacing the refinement branch with RVHomo's local CNN, does not recover RVHomo's accuracy under the evaluated protocol. They support the need for geometrically precise VFM representations in this learning setting. RVHomo addresses this requirement through spatial adaptation of hierarchical VFM features and their integration with local CNN descriptors at both scales, making structural context and localization cues jointly available throughout estimation. We interpret this as evidence for the adaptation pathway within our framework, not a universal limitation of the original supervised methods or proof that one stage alone explains the gap.

Component controls (Table 5, OPT-SAR). Unlike Table 4, these ablations retain the full model's consistency regularization:

ConfigurationMACE
w/o VFM2.54
w/o GFU3.74
w/o top-downNC
RVHomo1.95

The w/o GFU control retains the hierarchy and top-down aggregation, replacing GFU with bilinear resizing and convolutional projection. The w/o top-down control retains GFU but uses parallel resizing and concatenation.

DINOv2 feature selection (Appendix C.1).

ConfigurationFeature blocksBranchesMACE
Deepest layer, four branches[24,24,24,24][24,24,24,24]42.67
Shifted four-depth hierarchy[3,9,15,21][3,9,15,21]42.26
Deepest layer, single branch[24][24]12.78
RVHomo[6,12,18,24][6,12,18,24]41.95

The four-branch controls vary feature depth while retaining the adaptation branches. The single-branch variant also simplifies adaptation and aggregation, so its change is not attributed to feature depth alone.

2. Relationship to established geometric-consistency methods

Concern. Geometry-preserving translation and warp consistency are established ideas. How do the individual objectives differ, and why constrain both components?

Response. We agree that commutativity and mapping composition are established geometric principles. Our distinctions concern the construction of the supervision targets and the optimization paths within SSHNet's split learning.

For modality-transfer consistency, NeMAR uses registration-predicted deformations and encourages preservation through reconstruction along the translate–register and register–translate paths. We sample a known perturbation independently of the registration prediction and directly constrain the transfer network's response. Relative to the direct commutation objective of Honkamaa et al., we use an asymmetric target: the warped reference translation is detached, and only the perturbed-input branch receives gradients.

Reviewer_EhW8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed RVHomo, an unsupervised cross-model homography estimator. It combines VFM features and CNN features, and leverages unaligned multi-modal paired data by adopting a framework similar to that of SSHNet. The authors also proposed to distill RVHomo into a light-weight model, RVHomo-D, that does not need to perform modality transfer or online VFM extraction during inference.

Critical strengths
  1. The proposed method successfully addressed the unsupervised cross-modal homography estimation problem, demonstrating SOTA performance.
  2. The design of the method (e.g., network modification) is overall reasonable and clear.
Critical weaknesses
  1. (Most important, could raise the score if properly addressed) The fairness of comparison experiments is unclear: No information is provided regarding whether the target modalities are visible to all methods during training, and whether the baselines are retrained (and for how long) on the same dataset as RVHomo.
  2. It is unclear why the 2 additional consistency loss terms are introduced: These loss terms are designed to encourage the modal-transfer network to commute with geometry warping, and encourage the predicted homography to satisfy the homography-composition relation, but it wasn't explicitly explained why these properties would contribute to increased performance.
  3. Some figures (e.g., Fig.2 (a) (b)) are not easy to follow, some figures (e.g., Fig.C.2) have wrong and misleading captions (e.g., line color and their meanings do not match what is in the figure).
Questions for authors
  1. (Address W1, Most important, could raise the score if properly addressed) The fairness of comparison experiments is unclear, please address the following questions: The benchmark is performend on multiple combinations of modalities, are these combinations visible to all baseline methods during training? Did the authors retrain the baseline methods on datasets of the target modalities? Are the baseline methods trained on the same datasets as RVHomo?
  2. (Address W2) Please explicitly explain why satisfying the commutative and compositive properties is important for the performance of the method. This would improve the soundness of the design of these loss terms.
  3. (W3) Please improve the clarity of the figures (mostly the diagrams), and correct the caption of Figure C.2.
5 comments
AuthorsOpenReview ↗

Response to Reviewer EhW8 (Part 1/3)

We thank the reviewer for recognizing RVHomo's effectiveness and the rationale of its architecture. We have clarified the comparison protocol, connected the consistency objectives to the two learning components, and revised the figures. Table and appendix numbers below refer to the revised manuscript.

W1 / Q1: Target-modality visibility, baseline retraining, and comparison fairness

Concern. Are the target modality combinations visible to all comparison methods during training? Were the baselines retrained on the corresponding target datasets using the same training data as RVHomo?

Response. Yes. The baselines were retrained on the corresponding target datasets. For each benchmark, the training images from both target modalities are available to the comparison methods, as they are to RVHomo. They use the same dataset training pools and corresponding held-out evaluation partitions. We have added Appendix A to document the dataset partitions and pair counts, input construction, training configurations, and evaluation protocol.

RVHomo, all SSHNet variants, and the five generic unsupervised baselines—UDHN, CA-UDHN, biHomE, BasesHomo, and SCPNet—share the configuration below on all four datasets. Each method retains its own architecture and training objectives. Results use the final 120,000-iteration checkpoint; failed runs follow Appendix A.4's NC criterion:

SettingConfiguration
Input patch size128×128128\times128
Batch size16
Configured training length120,000 iterations
OptimizerAdamW
Maximum learning rate4×10−44\times10^{-4}
Weight decay10−510^{-5}
Learning-rate scheduleOneCycleLR: 5% increasing phase, linear annealing
Gradient clippingMaximum gradient norm of 1.0

AltO retains its original alternating-optimization procedure and official training configuration: AdamW, learning rate 3×10−43 \times 10^{-4}, weight decay 1×10−51 \times 10^{-5}, batch size 16, OneCycleLR, and 200 epochs.

Our sample construction follows a widely used homography benchmark protocol: randomly perturb the four patch corners and apply the resulting projective warp. This construction is used in early deep estimators such as DHN and UDHN, in perceptual unsupervised estimation such as biHomE, in recurrent methods such as IHN, RHWF, and MCNet, and in unsupervised cross-modal methods such as SCPNet and SSHNet. Following the cross-modal benchmark setting used by SCPNet and SSHNet, we use 128 × 128 patches with corner perturbations in [−32, 32] pixels. We apply this protocol to stored, pre-aligned cross-modal reference pairs, warping the source and keeping the target fixed. The known warp provides the evaluation ground truth. These cross-modal labels do not enter the training loss; RVHomo and SSHNet obtain self-supervision from additional synthetic intra-modal pairs. All methods use the same source-to-target coordinate convention, and MACE is averaged over five evaluation passes.

Appendix A.1–A.4 documents the dataset partitions, pair construction, training settings, and evaluation protocol.

AuthorsOpenReview ↗

Response to Reviewer EhW8 (Part 2/3)

W2 / Q2: Why do the commutative and compositive properties improve estimation?

Concern. The properties enforced by the losses are understandable, but why should enforcing them improve homography estimation?

Response. The motivation follows from SSHNet's decomposition of learning into modality transfer and homography estimation. Separating their optimization does not ensure that the transfer network preserves source geometry or that the estimator responds correctly to known geometric changes. A transfer network may change spatial structure while matching target appearance. The estimator may also make inconsistent predictions for geometrically related source views. Our losses constrain these two behaviors.

Modality-transfer consistency protects the geometry of the estimator's input. Modality transfer should change appearance while retaining spatial layout. We compare “warp, then transfer” with “transfer, then warp” under a known sampled perturbation. The discrepancy penalizes spatial changes introduced by modality transfer. This encourages the transferred input to preserve the geometric structures that the homography estimator must align. The warped reference translation is detached, so this term updates the transfer network through the perturbed-input branch.

Homography-prediction consistency constrains the estimator's response to geometry. Following Section 3.3, let Ghomo∈R4×2\mathbf G_{\mathrm{homo}}\in\mathbb R^{4\times2} encode the known source perturbation as four-corner displacements, and let Dref∈R4×2\mathbf D_{\mathrm{ref}}\in\mathbb R^{4\times2} be the detached reference prediction. Let Π\Pi convert four-corner displacements to a homography matrix and Π−1\Pi^{-1} convert it back. The matrix Π(Ghomo)\Pi(\mathbf G_{\mathrm{homo}}) maps the perturbed source view to the reference source view, while Π(Dref)\Pi(\mathbf D_{\mathrm{ref}}) maps the reference source to the target. The composed prediction target is

=\mathcal C(\mathbf D_{\mathrm{ref}},\mathbf G_{\mathrm{homo}}) =\Pi^{-1}\!\left[\Pi(\mathbf D_{\mathrm{ref}})\Pi(\mathbf G_{\mathrm{homo}})\right].$$ As defined in Appendix B.1, composition is performed in homography-matrix space rather than by adding corner displacements. The loss $\mathcal L_{\mathrm{homo}}$ is the mean absolute difference between the perturbed-view prediction $\mathbf D_{\mathrm{pert}}$ and this detached target. The transferred perturbed input is also detached, so this objective updates only the homography estimator. It provides a known relation between predictions without supplying the cross-modal transformation label. The two constraints are complementary: geometry-preserving transfer does not automatically enforce composition-consistent predictions, and prediction consistency does not directly enforce geometry preservation in the transfer network. We retain SSHNet's separate optimization roles and add a geometric requirement for each component. The revised experiments connect this motivation to performance (all MACE values are in pixels; lower is better). **Controlled loss ablation (Table 2).** Architecture and reconstruction regularization are fixed while each consistency term is independently enabled: | $\mathcal{L}_{\mathrm{trans}}$ | $\mathcal{L}_{\mathrm{homo}}$ | GoogleMap | OPT-SAR | FSD | VIS-IR | |---|---|---|---|---|---| | $\times$ | $\times$ | 0.93 | 2.38 | 4.52 | 4.75 | | $\times$ | $\checkmark$ | 0.82 | 2.19 | 4.35 | 4.70 | | $\checkmark$ | $\times$ | 0.69 | 2.04 | 4.44 | 4.63 | | $\checkmark$ | $\checkmark$ | **0.65** | **1.95** | **4.32** | **4.55** | The joint configuration has the lowest MACE on all four datasets. **Spatial preservation (Appendix B.2, Table B.1).** The aggregate median discrepancies are: | Evaluation measure | Median w/o $\mathcal{L}_{\mathrm{trans}}$ | Median w/ $\mathcal{L}_{\mathrm{trans}}$ | Mean reduction | Improved samples | |---|---|---|---|---| | Path-wise image discrepancy | 0.0188 | **0.0134** | 28.9% | 99.4% | | Structural edge discrepancy | 0.2693 | **0.2403** | 10.0% | 95.0% | These medians are distinct from the mean-based reduction percentages in Table B.1. The illustrated sample's local residual decreases from 2.24 to 0.48 pixels.
AuthorsOpenReview ↗

Response to Reviewer EhW8 (Part 3/3)

Continued topic: W2 / Q2: Why do the commutative and compositive properties improve estimation?

Prediction robustness (Appendix B.3, Table B.2). Over 888 GoogleMap samples, the complete perturbation results are:

Perturbation (pixel)MACE w/o Lhomo\mathcal{L}_{\mathrm{homo}}MACE w/ Lhomo\mathcal{L}_{\mathrm{homo}}Degradation w/o Lhomo\mathcal{L}_{\mathrm{homo}}Degradation w/ Lhomo\mathcal{L}_{\mathrm{homo}}
01.5350.7050.0000.000
21.7590.7680.2240.063
41.9760.8110.4410.106
62.1340.8920.5990.187
82.3520.9670.8170.261

Degradation is measured relative to the zero-perturbation condition and reported from the unrounded results. These analyses describe checkpoint behavior; Table 2 supplies the component-wise controlled comparison.

The controlled ablation assesses the contribution of each consistency term, while the visualizations illustrate spatial preservation and prediction robustness. We use the constraints as regularizers alongside SSHNet's learning objectives, rather than assuming that its two objectives necessarily exhibit conflicting gradients.

W3 / Q3: Diagram readability and the incorrect Figure C.2 caption

Concern. Improve the clarity of Figure 2, especially panels (a) and (b), and correct the misleading caption of Figure C.2.

Response. We have revised Figure 2 to distinguish the feature-adaptation architecture and GFU details from the two geometric-consistency mechanisms. The consistency mechanisms have separate panels with image examples, and the enlarged labels improve readability at the displayed scale.

We have corrected the original Figure C.2, now Figure B.2 after reorganizing the appendix. Its caption now matches the actual visual encoding: green circles indicate evaluation-only ground truth, blue squares indicate predictions without homography-prediction consistency, and red triangles indicate predictions with it. The caption also identifies the numerical values as sample ACEs.

We additionally enlarged the numerical annotations in Figure 4 and supplied corner-level enlarged comparisons in Figure F.1 to make registration differences easier to inspect.

Comments on the paper

2 comments

Dear Reviewers,

The authors have posted their rebuttal responses. I would kindly encourage you to review the authors' responses and continue the discussion by providing follow-up comments where appropriate. In particular, please indicate whether the authors' responses have adequately addressed your concerns and update your assessment if necessary.

Thank you for your time and continued efforts!

Your AC

Dear Reviewers,

There are fewer than four days left in the discussion period. The authors have submitted their rebuttal, but there have been no replies yet.

Please read the rebuttal and let the authors know if it answers your concerns. If you still have concerns, please share them soon so the authors have enough time to respond.

Thank you for your help.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.