- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a zero-shot cross-modal homography estimation framework called TMO. Its core innovation lies in using test-time adaptation (TTA) to bridge the gap between pre-training data and real-world data. Experimental results across six benchmark datasets demonstrate consistent performance improvements, with the MACE metric reduced by 16.9%–50.8% following adaptive adjustment.
Critical strengths
-
The motivation is strong. Training data simply doesn't match real-world data, and this gap is most evident in zero-shot scenarios.
-
Prediction-centric data synthesis offers a powerful approach for generating training data. It enables appearance adaptation while minimizing geometric distortion, requiring only a single pair of unaligned samples (lacking ground truth). Ablation studies and experiments presented in the appendix further demonstrate its effectiveness.
-
The results demonstrate strong convergence across all six datasets. Ablation studies further confirm that this paradigm generalizes well across various network architectures.
Critical weaknesses
-
Certain ablation studies are missing. The paper lacks ablation studies for several parameters. For instance, the sampling range for perturbations, the impact of different LoRA ranks, and the weighting of the loss function.
-
Some baseline comparisons are missing. The metrics for CCNet are cited directly from existing results rather than being re-evaluated under the experimental setup used in this paper; please verify whether the corresponding code has been made publicly available during the rebuttal phase to ensure a fair comparison. Furthermore, if any Test-Time Adaptation (TTA) methods exist within this field, they should also be included in the baseline comparisons.
-
The paper lacks a rationale for updating only the modality-transfer network while keeping the homography network frozen. Please provide quantitative experiments to justify this design choice.
-
The paper lacks evaluations on larger-scale models. It remains unclear whether TMO remains effective when paired with a homography estimator that has larger capacity and stronger baseline generalization.
Questions for authors
Could the authors provide additional ablation studies on key hyperparameters, clarify the fairness of the CCNet comparison and the availability of relevant TTA baselines, and offer quantitative evidence supporting the decision to freeze the homography network? Additionally, does TMO remain effective when applied to larger-capacity homography estimators with stronger baseline generalization? The idea behind this paper is interesting; however, if the authors cannot provide sufficient justification and experimental evidence to address these concerns, I would consider lowering my score.
4 comments
Part 2: Estimator Freezing and Larger-Capacity Models
Dear Reviewer 6V3N, thank you for your review. This response addresses W3 (estimator freezing), W4 (model capacity), and your question.
W3. The paper lacks a rationale for updating only the modality-transfer network while keeping the homography network frozen. Please provide quantitative experiments to justify this design choice.
Why freeze the homography estimator? Pretraining aims to learn modality-invariant alignment; uncertain test-time supervision may disrupt it. Translator adaptation changes appearance while preserving this alignment. Table R3 compares five updates from the same baseline: estimator-only updates give mixed results, while joint updates improve all six. Translator-only TMO beats joint updates on five and trails by 0.01 px on RGB-NIR.
Table R3. Updating different components of Translator+MCNet (MACE ↓).
| Update | OPT-SAR | DPDN | Google Map | Google Earth | RGB-NIR | M3FD |
|---|---|---|---|---|---|---|
| Translator+MCNet (no TTA) | 13.66 | 8.63 | 3.29 | 1.30 | 1.36 | 5.85 |
| FNet only | 13.52 | 8.56 | 3.41 | 1.27 | 1.26 | 5.86 |
| Homography head only | 13.58 | 8.93 | 3.63 | 1.22 | 1.37 | 6.15 |
| Full homography estimator | 13.73 | 9.20 | 3.59 | 1.22 | 1.13 | 6.05 |
| Homography estimator + translator LoRA | 8.23 | 7.51 | 1.71 | 1.13 | 0.68 | 5.61 |
| Translator LoRA only (TMO) | 8.00 | 6.10 | 1.62 | 1.08 | 0.69 | 4.09 |
W4. The paper lacks evaluations on larger-scale models. It remains unclear whether TMO remains effective when paired with a homography estimator that has larger capacity and stronger baseline generalization.
Larger-capacity estimator. We retain RoMa's full backbone [1], add a homography head, and train on MS COCO. We pair this estimator with the translator in TMO, adapting only translator LoRA while freezing the estimator. Standard RoMa has about 111M parameters [2] versus MCNet's 0.85M [3] (about 130×). Unadapted Translator+RoMa beats Translator+MCNet on four datasets; 300-step TMO lowers MACE on all six (Table R4). This supports TMO's applicability to a larger estimator while preserving its pretrained alignment function.
Table R4. Translator+RoMa under TMO before adaptation (RoMa$0$) and after 300 steps (RoMa${300}$), with only translator LoRA updated (MACE ↓).
| Method | OPT-SAR | DPDN | Google Map | Google Earth | RGB-NIR | M3FD |
|---|---|---|---|---|---|---|
| 8.07 | 4.42 | 2.54 | 1.73 | 1.59 | 3.63 | |
| 6.20 | 3.86 | 1.85 | 1.55 | 1.03 | 3.22 |
Q. Questions for Authors. Could the authors provide additional ablation studies on key hyperparameters, clarify the fairness of the CCNet comparison and the availability of relevant TTA baselines, and offer quantitative evidence supporting the decision to freeze the homography network? Additionally, does TMO remain effective when applied to larger-capacity homography estimators with stronger baseline generalization? The idea behind this paper is interesting; however, if the authors cannot provide sufficient justification and experimental evidence to address these concerns, I would consider lowering my score.
Please see W1–W2 in our first response for hyperparameters and baselines, and W3–W4 above for the estimator and capacity experiments.
References
[1] Edstedt et al. RoMa: Robust Dense Feature Matching. CVPR, 2024.
[2] Lu and Du. Raising the Ceiling: Conflict-Free Local Feature Matching with Dynamic View Switching. ECCV, 2024.
[3] Zhu et al. MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation. CVPR, 2024.
Response to Authors' Rebuttal
Thank you for your detailed response. The rebuttal has thoroughly addressed all of my concerns. Accordingly, I have raised my score to 4.
Response to Reviewer 6V3N
We sincerely appreciate your positive reassessment and for raising your score. Your insightful comments have been tremendously helpful in refining our framing, and we are pleased that our rebuttal addressed your questions. We will carefully revise the manuscript according to your advice to make our presentation even clearer and more rigorous. Thank you once again for your dedication and thoughtful review.
Part 1: Hyperparameter Ablations and Additional Baseline Comparisons
Dear Reviewer 6V3N, thank you for recognizing our motivation, prediction-centered synthesis, and consistent gains. This response addresses W1 (hyperparameters) and W2 (baseline comparisons).
Hyperparameter ablations. Table R1 varies perturbation range, LoRA rank, and image/feature loss weights around the baseline: ±24 px, rank 8, and weights (1, 1.5). Each loss sweep fixes the other weight; the shared baseline is listed once. MACE is rounded to 0.01 px from the reported relative changes.
Table R1. Hyperparameter ablations (MACE ↓).
Conclusion. Higher LoRA rank can help but may increase compute; ±8 perturbations hurt five datasets relative to ±24. Different datasets favor different loss weights, yet all tested settings improve over unadapted Translator+MCNet on all six datasets. These variations do not alter our core finding: adaptation on a single test pair without ground truth can improve homography estimation.
To our knowledge, CCNet's homography code has not been publicly released, so we use its published results.
Additional TTA baselines. We surveyed registration and TTA studies and selected Reg-TTR and SGTTA as the most suitable baselines to adapt. Reg-TTR [1] refines a predicted warp using NCC/SSIM and smoothness; our Reg-TTR-H optimizes an eight-DoF homography with MIND-like [3] and HOG [4] losses (weights 1/0.5), capturing modality-independent local structure and gradient orientations, respectively. SGTTA [2] uses feature-statistics, HOG, and model-history guidance; our SGTTA-H updates per-pair MCNet channel scale/bias with HOG [4]/MIND-like [3] losses (2/0.5) and weak statistics (0.01). We tried multiple configurations, selected each method's best on synthetic COCO, and report 300-step MACE on all 5,731 pairs (Table R2). TMO leads on all six datasets.
Table R2. Homography estimation before and after adaptation (MACE; lower is better).
References
[1] Chen et al. Reg-TTR, Test-Time Refinement for Fast, Robust and Accurate Image Registration. arXiv, 2026.
[2] Zhou et al. Test-Time Adaptation via Style and Structure Guidance for Histological Image Registration. AAAI, 2024.
[3] Heinrich et al. MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration. Medical Image Analysis, 2012.
[4] Dalal and Triggs. Histograms of Oriented Gradients for Human Detection. CVPR, 2005.