- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RVHomo, an unsupervised framework for cross-modal homography estimation that combines hierarchical features from a frozen DINOv2 vision foundation model (VFM) with local CNN features. The method introduces geometry-aware feature upsampling and multi-level fusion, a dual-scale reconstruction objective, and two geometric consistency losses that regularize modality transfer and homography prediction. A two-stage progressive distillation strategy is further developed to obtain a lightweight model, RVHomo-D, without modality transfer or online VFM extraction. The method is evaluated on four datasets covering different modality pairs. The problem is relevant, and the reported results are promising. However, the novelty of the individual components and their relationship to prior work needs to be established more convincingly. The experimental evaluation, although reasonably broad within the unsupervised setting, could better isolate the contributions of the proposed components and contextualize the method against recent VFM-based approaches. The presentation of the architecture and qualitative results could also be improved.
Critical strengths
Relevant problem and clear motivation. Unsupervised cross-modal homography estimation is challenging because substantial appearance differences make conventional correspondence and reconstruction objectives unreliable. Combining VFM representations with spatially precise CNN features is a reasonable direction. Promising quantitative results. The method is evaluated on four datasets involving substantially different modality pairs. RVHomo reports MACE reductions of 17.4%–46.7% relative to the strongest evaluated competitor on each dataset. The paper also includes architectural ablations and experiments examining the effects of the two geometric consistency losses. Useful model-compression strategy. The two-stage distillation design addresses the considerable cost of using a large frozen VFM at inference. The reported reductions in total parameters and GFLOPs are substantial, and RVHomo-D maintains competitive estimation accuracy. Some evidence for geometric consistency. The appendix includes controlled perturbation experiments and aggregate measurements of spatial preservation and prediction robustness. These analyses provide additional support for the proposed consistency objectives beyond the main benchmark table.
Critical weaknesses
- The main concern is that the framework largely integrates established ideas without clearly establishing a distinct methodological contribution. It builds on SSHNet's reformulation and split optimization, adds hierarchical DINOv2 features into a CNN-based estimator, and applies known techniques: multi-scale fusion, reconstruction regularization, warp equivariance, composition consistency, and teacher–student distillation. Each component is plausible, and the combination may be effective, but the paper does not make clear which part is the principal advance and why this is more than an integration of existing techniques. The ablations support the contributions of the VFM features, GFU, hierarchical aggregation, and consistency losses, but do not isolate the dual-scale reconstruction objective, nor show that the consistency losses help beyond applying similar constraints to a plain SSHNet baseline.
- The baselines are reasonably broad for the unsupervised setting, but two gaps remain. First, recent VFM-based methods (GFNet, DistillMatch) discussed in related work are not benchmarked; their supervision differs, so a fair comparison may be limited, but this should be explained and attempted where possible. Second, all SSHNet baselines omit the consistency losses, so the main comparison conflates architecture and objectives; a controlled comparison is needed to attribute each contribution. The ablations use only OPT-SAR (consistency analysis only on Google Maps); results are single-point estimates without multi-seed variance, and efficiency reports parameters/GFLOPs rather than measured latency or memory.
- The related-work section would benefit from a clearer comparison of the assumptions, supervision, representation-learning mechanisms, and evaluation protocols of recent VFM-based methods. This would help establish precisely what remains unresolved by prior work and sharpen the novelty claim.
- Figure 1 provides a reasonably clear high-level overview, whereas Figure 2 combines several conceptually different processes and detailed architecture blocks in a dense layout. Many labels and symbols are difficult to read at the paper's displayed scale. Separating the geometric-consistency mechanisms from the detailed feature-fusion architecture, or enlarging the individual panels, would make the method easier to understand. Figure 4 provides a useful side-by-side comparison, but the individual image panels and numerical annotations are small. Enlarged crops or additional visualizations of challenging regions would make differences in alignment more apparent. Moreover, the supplementary qualitative section explicitly presents examples in which RVHomo performs favorably. These examples are useful for illustration but do not establish typical performance or characterize failure cases.
Questions for authors
Can the authors provide a controlled comparison that separates the effects of the proposed consistency losses from those of the VFM-guided architecture, ideally by applying the losses to an appropriate SSHNet-based baseline? What is the independent contribution of the dual-scale reconstruction objective? How sensitive are the results to the selected DINOv2 layers and the feature-fusion design? Can the main results and key ablations be reported over multiple random seeds, with variability estimates, to establish the stability of the improvements? How do the proposed method and RVHomo-D compare with recent VFM-based homography or matching approaches under clearly specified, comparable supervision and evaluation protocols? Can the authors provide measured inference latency, throughput, and memory usage, together with fuller details of the training setup and dataset splits, to improve reproducibility?
6 comments
Response to Reviewer ShHH (Part 2/6)
Continued topic: W1: Principal contribution and relationship to established techniques
The component controls then examine the proposed adaptation design. Table 5 evaluates VFM removal, conventional scale adaptation, and removal of top-down aggregation. Appendix C.1 retains four adaptation branches while varying their feature sources: using blocks [6,12,18,24] obtains 1.95 MACE, compared with 2.67 when all branches receive block 24. Appendix C.2 further compares GFU with conventional convolutional adaptation at nearly equal parameter counts—2,696,192 and 2,698,022—obtaining 1.95 and 2.34 MACE, respectively. These full-objective experiments support complementary feature depths and the proposed spatial-adaptation mechanism, while controlling adapter parameter capacity.
For geometric consistency, our distinctions concern the supervision targets and update paths. NeMAR uses registration-predicted deformations to encourage translation–registration compatibility through reconstruction. Our transfer constraint instead uses an independently sampled known perturbation. Relative to Honkamaa et al.’s direct commutation objective, it employs a detached warped reference translation, so gradients update the transfer network only through the perturbed-input branch. For prediction consistency, Warp Consistency derives the composition relation and reconstructs a known warp through predicted dense mappings. We instead compose a detached reference homography with the known perturbation to construct a four-corner prediction target, without a reverse matching path; gradients through modality transfer are blocked. The two constraints therefore update different learning components: one encourages spatial preservation during transfer, while the other specifies the estimator’s response to known geometric changes.
Table 3 addresses whether the gains can be explained solely by adding these objectives to an existing estimator. RVHomo without either consistency term outperforms all three consistency-regularized SSHNet variants on every dataset. Adding both terms further improves RVHomo. With architecture and reconstruction fixed, Table 2 independently evaluates the two constraints across all four datasets, with their joint configuration achieving the lowest error on each. These results provide complementary evidence for the representation pathway and geometric regularization.
Reconstruction serves as auxiliary representation regularization, while progressive distillation transfers the resulting estimator to a lightweight student. Their contributions are evaluated separately below. The principal methodological advance is the spatial and hierarchical organization of VFM–CNN representations for this unsupervised estimation setting, complemented by geometric constraints tailored to the two components of split learning.
W2 / question on controlled attribution: Architecture versus consistency losses
Concern. SSHNet comparisons omit the consistency losses, so the main comparison confounds the estimator with the objectives.
Response. Table 3 now applies the same joint consistency regularization to SSHNet-IHN, SSHNet-RHWF, and SSHNet-MCNet, and compares each estimator with the regularization disabled and enabled:
| Estimator | GoogleMap | OPT-SAR | FSD | VIS-IR | ||||
|---|---|---|---|---|---|---|---|---|
| IHN | 1.23 | 1.11 | 2.94 | 2.48 | 5.32 | 5.14 | 5.51 | 5.23 |
| RHWF | 1.29 | 1.03 | 3.08 | 2.80 | 5.64 | 5.58 | 5.60 | 5.82 |
| MCNet | 1.22 | 1.14 | 2.87 | 2.85 | 6.06 | 6.16 | 6.22 | 6.01 |
| RVHomo | 0.93 | 0.65 | 2.38 | 1.95 | 4.52 | 4.32 | 4.75 | 4.55 |
For a fixed estimator, this comparison identifies the effect of adding the joint consistency terms. The terms improve IHN and RVHomo on all four datasets. Effects are mixed for RHWF and MCNet: RHWF worsens on VIS-IR and MCNet worsens on FSD. We report these exceptions explicitly.
RVHomo without either term still outperforms all three regularized SSHNet variants on each dataset; adding the terms further improves RVHomo. Thus, the gain cannot be attributed solely to adding the consistency objectives to an otherwise unchanged SSHNet estimator.
Response to Reviewer ShHH (Part 3/6)
Continued topic: W2 / question on controlled attribution: Architecture versus consistency losses
The component-wise loss ablation in Table 2 also retains architecture and reconstruction while independently enabling each consistency term. It now covers all four datasets:
| GoogleMap | OPT-SAR | FSD | VIS-IR | ||
|---|---|---|---|---|---|
| 0.93 | 2.38 | 4.52 | 4.75 | ||
| 0.82 | 2.19 | 4.35 | 4.70 | ||
| 0.69 | 2.04 | 4.44 | 4.63 | ||
| 0.65 | 1.95 | 4.32 | 4.55 |
W1 / questions on reconstruction, DINOv2 layers, and fusion design
Concern. Isolate reconstruction regularization and examine sensitivity to the selected DINOv2 layers and fusion architecture.
Response. We have added three complementary analyses in Appendix C.
Independent reconstruction contribution (Appendix C.3, Table C.3). Architecture and the remaining objectives are fixed:
| Reconstruction loss | GoogleMap | OPT-SAR | FSD | VIS-IR |
|---|---|---|---|---|
| Disabled | 0.73 | 2.12 | 4.36 | 4.78 |
| Enabled | 0.65 | 1.95 | 4.32 | 4.55 |
The contribution varies across modality pairs, including a small change on FSD. Table C.4 reports the OPT-SAR weight sensitivity:
| MACE | |
|---|---|
| 0 | 2.12 |
| 5 | 1.960 |
| 10 | 1.948 |
| 20 | 1.943 |
The retained coefficient is 10; its unrounded 1.948 result rounds to the reported 1.95.
Layer selection, Appendix C.1 / Table C.1.
| Configuration | Feature blocks | Branches | MACE |
|---|---|---|---|
| Deepest layer, four branches | 4 | 2.67 | |
| Shifted four-depth hierarchy | 4 | 2.26 | |
| Deepest layer, single branch | 1 | 2.78 | |
| RVHomo | 4 | 1.95 |
The four-branch comparisons support complementary feature depths and the reported layer selection. The single-branch variant also simplifies adaptation and aggregation, so its change is not attributed to feature depth alone. The indices identify the extracted block outputs; all variants retain the full DINOv2 backbone.
Fusion and scale adaptation (Table 5, OPT-SAR).
| Configuration | MACE |
|---|---|
| w/o VFM | 2.54 |
| w/o GFU | 3.74 |
| w/o top-down | NC |
| RVHomo | 1.95 |
The w/o GFU control retains top-down aggregation and uses bilinear resizing plus convolutional projection. The w/o top-down control retains GFU and uses parallel resizing and concatenation. The full model's remaining training objectives are retained.
Parameter-matched adapter control (Appendix C.2).
| Scale-adaptation module | Adapter parameters | MACE |
|---|---|---|
| Bilinear resize + widened convolutions | 2,698,022 | 2.34 |
| GFU (RVHomo) | 2,696,192 | 1.95 |
Both adapters share the optimization settings, downstream hierarchy, CNN fusion, and estimator. Their parameter counts differ by about 0.068%, supporting GFU's design over the evaluated conventional adapter at nearly equal parameter capacity.
W2 / question on recent VFM methods and supervision
Concern. GFNet and DistillMatch are discussed but not benchmarked. Attempt comparisons where possible and explain differences in supervision and protocol, including the position of RVHomo-D.
Response. We have added GFNet adapted to our 128 × 128 patch benchmark as a supervised reference in Table 4, obtaining 2.498 MACE on OPT-SAR. Appendix A.3 clarifies the resolution adaptation: the official GFNet implementation uses 448 × 448 for initial matching and provides an additional 560 × 560 refinement path, while our reference uses the benchmark's 128 × 128 patches and pixel-coordinate convention. It is a resolution-adapted reference, rather than a reproduction of GFNet's original-resolution evaluation. Table 1 reports 1.95 for RVHomo and 2.04 for RVHomo-D on the same dataset. The GFNet result is explicitly separated from the unsupervised feature-path controls because it uses geometric supervision; we do not interpret the numbers as a comparison under matched supervision.
Response to Reviewer ShHH (Part 1/6)
We thank the reviewer for recognizing the relevance of the task, the cross-modal accuracy improvements, and the practical value of the compact student. We have strengthened the experimental controls and clarified the methodological scope, related work, reproducibility, efficiency, and visualization. All references below use the revised manuscript numbering.
W1: Principal contribution and relationship to established techniques
Concern. The framework integrates established ideas, and the principal advance and its relationship to previous work need to be established more convincingly.
Response. We thank the reviewer for requesting a clearer account of our methodological contribution. RVHomo develops a geometry-aware adaptation pathway that converts hierarchical VFM features into spatially precise, multiscale representations for unsupervised cross-modal homography estimation, together with component-specific geometric constraints within SSHNet’s split optimization. The contribution concerns how these representations and constraints are constructed for this learning setting, building on established VFM and geometric-consistency principles.
The representation challenge is to retain both cross-modal structural cues and the spatial precision required for four-corner homography prediction without cross-modal transformation labels in the training objective. Deep VFM features provide broad structural context, whereas local CNN descriptors preserve finer spatial information. RoMa uses deepest-layer DINOv2 features for coarse dense matching followed by CNN-based refinement. GFNet introduces cross-view interaction on DINOv2 features and conditions CNN/FPN refinement on the coarse representation for grid-based homography estimation. RVHomo reorganizes these information sources: it applies gated spatial refinement and top-down aggregation to multiple DINOv2 depths, then fuses the adapted representations with local CNN descriptors at both estimation scales. Structural context and local detail are therefore jointly available throughout recurrent global-homography estimation.
Feature-use comparison (Table 4, OPT-SAR). GFNet is a supervised reference; the other rows share the unsupervised estimator and objectives, with both consistency terms disabled:
Table 4 examines why a dedicated adaptation pathway is needed. We transfer RoMa-style and GFNet-style feature pathways to the shared recurrent homography estimator and additionally evaluate a Separated DINO/CNN configuration, which uses deepest-layer DINO features at the first scale and RVHomo’s local CNN branch at the second. Under the shared unsupervised protocol, with both proposed consistency losses disabled, these alternatives obtain end-of-training OPT-SAR MACE values of 23.47, 26.91, and 24.37, compared with 2.38 for RVHomo. Despite completing the full 120,000-iteration training schedule, these adapted configurations retain high estimation errors. Together with the high final error of Separated DINO/CNN, these results show that none of the three alternatives achieves accurate homography estimation under the evaluated protocol. Their finite MACE values are reported rather than marked NC, consistent with Appendix A.4. In particular, retaining RVHomo’s local CNN refinement branch does not recover the accuracy of its fused representations. These results support dedicated spatial adaptation and multilevel integration in the evaluated setting. They concern adapted feature pathways within our framework, rather than the performance of the complete original supervised matchers. This diagnostic comparison does not isolate the coarse-stage VFM features as the sole cause of the performance gap.