- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes to use coarse GPS measurements from satellite imagery and vehicle heading estimates as an external metric reference for feed-forward 3D reconstruction. The coarse satellite images are retrieved from Google Maps using publicly available GPS annotations from which heading-oriented local crops are extracted.
The main idea is to train a coupled architecture built on a pretrained feed-forward 3D reconstruction model, where both branches are initialized with the pretrained model, keeping frozen the ground branch and the satellite branch is refined. The two branch is connected through an asymmetric cross-attention mechanism that operates at different depths of the decoder and progressively aggregates ground-view geometry building a rich crossview representation for accurate scale and coordinate estimation. Furthermore a lightweight reverse interaction on the final layers injects satellite-derived metric and structural cues into the ground-view representation to refine scene geometry point by point. The model trained with LiDAR-derived metric depth and camera-pose supervision learns to jointly estimates absolute camera poses and metric scene geometry.
Critical strengths
Overall, I remain unconvinced by the paper and do not identify a sufficiently compelling strength to support acceptance. Nevertheless, the paper has several positive aspects worth acknowledging:
-
Using a retrieved satellite image patches to help recovering metric scale, to estimate absolute camera pose, and to refine dense geometry within a feed-forward reconstruction framework seems an interesting direction.
-
The main contribution is the progressive update of the satellite branch using ground-view features, combined with reverse interaction introduced only in the final decoder layers. The ablations suggest that with this per-point geometric correction one can obtain better reconstruction, but still below the results obtained with ground-truth scale alignment.
-
I appreciate that the model was tested with different feed-forward architectures (π3, VGG, VGGT-Ω).
-
The paper proposes cross-dataset evaluation on three tasks: monocular metric depth estimation, multi-view metric depth estimation, and multi-view metric point-cloud estimation. In addition, a cross-view localization experiment on KITTI and Oxford RobotCar is added.
Critical weaknesses
Main concerns supporting rejection:
-
The experiments do not convincingly establish the benefit of the proposed architecture beyond providing additional satellite information, given the limited comparisons with metric reconstruction methods and the absence of an inference-cost analysis.
-
Limited conceptual novelty and the substantial gap between predicted-scale and ground-truth-scale performance, this leaves insufficient evidence of a compelling methodological advance.
Detailed remarks:
-
Using retrieved satellite imagery as a reference offers limited conceptual novelty, as cross-attention between satellite and ground views has already been explored (see [a,b]). The most original contribution appears to be the progressive update of the satellite branch using ground-view features, combined with reverse interaction introduced only in the final decoder layers.
-
The paper should discuss the relation to [a] proposing cross-attention between ground and satellite features, and to SGFormer [b] that uses ground geometry guiding of a satellite branch before feature fusion.
-
The experiments illustrates that adding scale estimation, coordinate offset and correction heads help improving the vanilla models, but do but emphasize enough that the proposed model uses extra information (satellite images) compared to the vanilla models. Note that the cost of the model is doubled with coupling two π3 architectures.
-
It is appreciable that the experiments are done cross-datasets, still it is rather pity that the experiments are mainly limited to training the model on KITTI and tested on nuScenes and vice-versa. How would these models perform on Oxford RobotCar (depth maps and point clouds could be constructed from the raw sensor data)? Also how would the model be improved if trained on a collection of several datasets?
-
The model is mainly compared to models which do not have metric output (except DA3 and MASt3R) but not compared to CUT3R [c], MUSt3R [d] , AMB3R, DVGT[e], MapAnything [f]. Nevertheless those models seem to perform less well or they use much more ground images (128 in DVGT) so numbers are not directly comparables. Still the paper should mention/discuss these works.
-
Why not using for nuScenes the aligned geographic images from [g] https://huggingface.co/datasets/SpatialRetrievalAD/nuScenes-Geography-Data.
-
In section 4.1, the difference between the results obtained with GT vs predicted scale in Table 3 suggest that the estimated scale is not very accurate.
8.The paper should provide an inference cost comparison between the vanilla model and the proposed coupled architecture.
References: [a] Yuan et al, Cross-Attention Between Satellite and Ground Views for Enhanced Fine-Grained Robot Geo-Localization, WACV’24 [b] Guo et al, SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion, CVPR’25. [c] Wang et al, Continuous 3D Perception Model with Persistent State, CVPR'25 [d] Cabon et al, MUSt3R: Multi-view Network for Stereo 3D Reconstruction, CVPR'25 [e] Zuo et al, DVGT: Driving Visual Geometry Transformer, CVPR'26 [f] Keetha et al, MapAnything: Universal Feed-Forward Metric 3D Reconstruction, 3DV'26 [g] Jia et al, Spatial Retrieval Augmented Autonomous Driving, CVPR'26
Questions for authors
-
What is happening if the satellite image is completely wrong (beyond the considered perturbation) and has no overlap with the ground images ? Is the model able to ignore the correction terms falling back to the backbone mode or it yields to worse results. This would be interesting to investigate. What about longer sequences (e.g 64 or 128 images), where the satellite image covers only 1 or 2 frames?
-
Was the satellite image added as input together with the ground images when the vanilla Feed-forward methods were tested? It would be interesting to see if the satellite image would help or on the contrary would degrade the performance of those models.
-
In Section 4.2 requires some clarification. If I well understood in this case the model is trained and tested on the same dataset (same or different areas). Would the model trained on KITTI work on Oxford RobotCar and vice-versa? Or on the VIGOR dataset without training on the VIGOR dataset (maybe a model trained on KITTI, NuScenes and maybe Oxford RobotCar.
Minor correction needs:
Line 034: For SFM I would cite Snavely et al. Photo Tourism: Exploring Photo Collections in 3D ACM TOG 2006 and Schönberger and Frahm, Structure-from-Motion Revisited. CVPR 2016.
Line 308: I would use another letter for cost to avoid confusion with with alignment parameters. Also providing more details (formulas) for the cost, maybe in an Appendix, would make clearer the loss in Eq(9).
Line 446-447 - please cite method by method either here to more easily connect with Tables 4 and 5.
Line 540 : reference the same as in 539
In the bib references putting the method names into {} allows to keep the capital letters. nuscenes-> nuScenes, Vggt ->VGGT, Cvlnet -> CVLNet , etc
6 comments
Response to Reviewer sWXA (2/3)
Q4.Results are mainly KITTI↔nuScenes. How does the model perform on Oxford RobotCar, and if trained on several datasets together?
Thank you for this valuable suggestion. We did not conduct Oxford RobotCar point-cloud experiments because its point-cloud projection is not sufficiently accurate; please also refer to issue [2] in the official robotcar-dataset-sdk repository. Nevertheless, we recognize the importance of further cross-dataset evaluation. We therefore conducted additional experiments on A2D2, using the 1,000 samples from each of its three cities for generalization testing. These results have been added to the appendix.
| Model | Scale | Acc Mean ↓ | Acc Med. ↓ | Comp Mean ↓ | Comp Med. ↓ | N.C. Mean ↑ | N.C. Med. ↑ |
|---|---|---|---|---|---|---|---|
| DA3 | GT | 2.3090 | 1.3024 | 2.3163 | 1.6639 | 0.7523 | 0.9057 |
| KITTI only | GT | 0.9852 | 0.4908 | 1.3985 | 0.9163 | 0.7714 | 0.9233 |
| nuScenes only | GT | 0.9679 | 0.4719 | 1.3384 | 0.8728 | 0.7723 | 0.9239 |
| KITTI + nuScenes | GT | 0.9519 | 0.4672 | 1.3688 | 0.8890 | 0.7733 | 0.9251 |
| DA3 | Pred | 3.7004 | 1.6436 | 3.0968 | 2.4976 | 0.7397 | 0.8942 |
| KITTI only | Pred | 2.2543 | 3.2949 | 3.4443 | 2.2540 | 0.7387 | 0.8872 |
| nuScenes only | Pred | 2.2130 | 1.4549 | 3.6773 | 2.4633 | 0.7437 | 0.8890 |
| KITTI + nuScenes | Pred | 2.2031 | 1.3863 | 3.0968 | 2.1611 | 0.7422 | 0.8892 |
All three training settings substantially outperform DA3 on the unseen A2D2 dataset. Combining the two training datasets further reduces accuracy error, while predicted-scale Comp Med. decreases from 2.254 and 2.463 for the single-dataset settings to 2.161.
Q5.Please discuss CUT3R, MUSt3R, AMB3R, DVGT, and MapAnything, even if the numbers are not directly comparable.
We thank the reviewer for pointing out these methods. CUT3R keeps a recurrent state and accumulates point maps online. MUSt3R extends two-view reconstruction to multiple views and outputs them in a shared coordinate frame. AMB3R places a sparse voxel backend and a scale head behind the feed-forward frontend. DVGT focuses on driving scenes and predicts point maps from surround-view sequences; the timing it reports uses 128 images. MapAnything returns depth, rays, pose, and scale in one forward pass, and can optionally take calibration, pose, or depth as input.
We evaluated these methods on KITTI, where our method achieves better results. This is because normalized relative geometry is easier to learn than absolute metric depth, so , which uses normalized geometry as its prior, is more accurate than models that directly estimate metric geometry. Through satellite imagery, our method incorporates metric information at low cost while preserving this feed-forward prior. The discussion has been added to the main text, and the results are provided in the appendix.
| Model | Scale | Acc Mean ↓ | Acc Med. ↓ | Comp Mean ↓ | Comp Med. ↓ | N.C. Mean ↑ | N.C. Med. ↑ |
|---|---|---|---|---|---|---|---|
| MapAnything | GT | 0.4199 | 0.1700 | 0.5609 | 0.1939 | 0.8021 | 0.9568 |
| DVGT-1 | GT | 0.3499 | 0.1536 | 0.3118 | 0.1344 | 0.8116 | 0.9577 |
| CUT3R | GT | 0.4935 | 0.2472 | 0.4408 | 0.1554 | 0.7900 | 0.9372 |
| MUSt3R | GT | 0.3799 | 0.1636 | 0.4866 | 0.1929 | 0.7964 | 0.9464 |
| AMB3R | GT | 0.2926 | 0.1540 | 0.3149 | 0.1503 | 0.8475 | 0.9653 |
| Ours | GT | 0.2239 | 0.0894 | 0.2122 | 0.0929 | 0.8626 | 0.9807 |
| MapAnything | Pred | 0.5653 | 0.2554 | 0.5981 | 0.2594 | 0.7816 | 0.9396 |
| DVGT-1 | Pred | 5.7786 | 6.0324 | 11.0013 | 7.4800 | 0.6170 | 0.6798 |
| CUT3R | Pred | 0.7082 | 0.3965 | 0.9220 | 0.2986 | 0.7705 | 0.9189 |
| MUSt3R | Pred | 0.7824 | 0.5838 | 1.5550 | 0.7772 | 0.7604 | 0.9104 |
| AMB3R | Pred | 0.5841 | 0.3888 | 0.7020 | 0.4058 | 0.8008 | 0.9392 |
| Ours | Pred | 0.2757 | 0.1100 | 0.2623 | 0.1125 | 0.8443 | 0.9752 |
Q6.Why not use the aligned nuScenes geographic images from Jia et al. [g]?
Our approach is similar to nuScenes-Geography-Data [g]. As explained by the nuscenes-devkit maintainers in [1], vehicle latitude and longitude can be obtained by combining each map's origin coordinates with the metric translations in ego_pose. Both [g] and our method use this conversion to retrieve satellite imagery.
Response to Reviewer sWXA (3/3)
Q7.If the satellite image has no overlap with the ground views, does the model fall back to the backbone or get worse? What about long sequences (64 or 128 frames) covered by only 1–2 frames?
Thank you for the suggestion. These two scenarios differ from the problem setting considered in this work. We address coarse GPS position errors of , following the same setting as [1], whereas completely non-overlapping satellite and ground imagery, or long sequences with only one or two frames within satellite coverage, are beyond the scope of this work. Nevertheless, we understand that the proposed experiment aims to examine the role of satellite imagery in scale estimation. We therefore conducted the following additional experiments on 1,000 KITTI images.
The results show that scale can be estimated normally when all three frames are within satellite coverage. When there is no overlap, scale estimation deteriorates markedly, with Acc Mean increasing from 0.362 to 0.602. For a 64-frame sequence with only one or two frames within coverage, scale estimation is misled by these few frames and deteriorates severely, with Comp Mean rising from 0.646 to 4.465.
| Setting | Scale | Acc Mean↓ | Acc Med↓ | Comp Mean↓ | Comp Med↓ | N.C. Mean↑ | N.C. Med↑ |
|---|---|---|---|---|---|---|---|
| 3 frames, within coverage | Pred | 0.3618 | 0.1935 | 0.2897 | 0.1558 | 0.8028 | 0.9415 |
| 3 frames, outside coverage | Pred | 0.6019 | 0.4146 | 0.8811 | 0.3568 | 0.7830 | 0.9335 |
| 64 frames, within coverage | Pred | 0.5739 | 0.3945 | 0.6461 | 0.3475 | 0.7270 | 0.8736 |
| 64 frames, only 1 frame within coverage | Pred | 0.6622 | 0.5179 | 4.4649 | 0.6531 | 0.7818 | 0.9036 |
| 64 frames, only 2 frames within coverage | Pred | 0.6702 | 0.5162 | 4.4477 | 0.6486 | 0.7806 | 0.9034 |
This experiment has been added to the appendix.
Q8.Were vanilla feed-forward models also given the satellite image as input, and does it help or hurt them?
Thank you for this suggestion. We further evaluated vanilla and VGGT on nuScenes by directly adding the satellite image as an input, and included this experiment in the supplementary material. The results are shown below.
| Method | Acc Mean ↓ | Acc Med. ↓ | Comp Mean ↓ | Comp Med. ↓ | N.C. Mean ↑ | N.C. Med. ↑ |
|---|---|---|---|---|---|---|
| PI3 + Sat frame | 2.9189 | 2.9526 | 21.1088 | 16.7188 | 0.7515 | 0.8862 |
| VGGT + Sat frame | 5.6205 | 5.6736 | 24.5153 | 20.6166 | 0.7182 | 0.8251 |
| Ours | 1.4982 | 0.8089 | 1.7042 | 0.9590 | 0.8702 | 0.9792 |
Directly adding satellite imagery to the vanilla models does not yield good results. This is because satellite and ground images have different viewpoints: satellite images often show rooftops, whereas ground images show building facades. As a result, the vanilla models cannot establish correspondences between the two views in most cases, so directly using satellite images as input does not help ground-view reconstruction.
Q9.In Section 4.2, are training and testing on the same dataset? Does a KITTI model transfer to Oxford RobotCar or to VIGOR without training on them?
Section 4.2 follows the same experimental setting as existing methods, with training and testing performed on the same dataset. VIGOR uses panoramic images, whose projection differs from that of pinhole images. Feed-forward 3D reconstruction models such as Pi3 and VGGT are trained for pinhole cameras and cannot accommodate panoramic inputs; therefore, our method cannot be directly transferred to VIGOR. During the rebuttal period, Oxford RobotCar could not be downloaded from its official website. We therefore added a transfer experiment on A2D2 using 1,000 images from three cities in the dataset. Both our method and the baselines are trained exclusively on KITTI and evaluated on A2D2.
| Method | Distance Mean (m) ↓ | Distance Med. (m) ↓ | Yaw Mean (°) ↓ | Yaw Med. (°) ↓ |
|---|---|---|---|---|
| FG² | 13.27 | 12.20 | 6.72 | 4.53 |
| BevSplat | 14.45 | 12.07 | 4.90 | 4.97 |
| BevSplat | 14.45 | 11.74 | 4.90 | 4.97 |
| Ours | 10.87 | 10.82 | 2.87 | 1.81 |
When trained only on KITTI, our method achieves a mean distance error of 10.87 m and a mean yaw error of 2.87°, outperforming FG² and BevSplat.
Q10.Minor fixes: SfM citations, the cost symbol and formula, per-method citations for Tables 4 and 5, the duplicate reference, and BibTeX capitalization.
Thank you for pointing these out. We have corrected all of these formatting issues.
[1] https://github.com/nutonomy/nuscenes-devkit/issues/144
[2] https://github.com/ori-mrg/robotcar-dataset-sdk/issues/20
[g] Spatial Retrieval Augmented Autonomous Driving
Response to Reviewer sWXA (1/3)
We thank Reviewer sWXA for the valuable comments. Our responses are provided below.
Thank you for this suggestion. We added a computational-cost experiment to the appendix. Both models are evaluated on the same NVIDIA H200 GPU, with ground-image resolution 252×1008 and satellite-image resolution 518×518. After 10 warm-up runs, we measure 50 consecutive runs. The results are shown below.
These results show that incorporating satellite imagery does not introduce a large increase in memory usage or inference latency. Moreover, as the number of ground images increases, the additional latency becomes negligible: with 50 ground images, it accounts for 1.6% of the total latency.
Cross-attention between ground and satellite views is not introduced for the first time in this work. Our work is the first to apply it to augment feed-forward reconstruction models, enabling metric-scale scene reconstruction with limited adaptation data while keeping the pretrained backbone frozen.
Method [a] fuses local satellite and ground features with self- and cross-attention to localize the robot within a known patch. Method [b] encodes ground images as voxels and satellite images as BEV features, guides the satellite branch with compressed ground features, and fuses the two streams for voxel semantic completion.
Our cross-attention operates on a frozen feed-forward backbone for metric reconstruction. In the primary direction, satellite features query ground features throughout decoding, allowing the satellite branch to progressively aggregate ground geometry and predict global scale β and coordinate transform C. Reverse interaction is introduced only in the final decoder layers: ground features query satellite features, and the resulting features are supplied only to the pointwise correction head. The base point maps before correction are still produced by the frozen backbone. Thus, [a] and [b] address localization within a known patch and voxel semantic completion, respectively, whereas our asymmetric, staged interaction jointly estimates metric scale, absolute pose, and pointwise geometric correction.
A gap remains between the predicted and ground-truth reference scales. Nevertheless, our method already substantially outperforms existing methods without using ground-truth test-set scale. Table 1 compares our method with dedicated monocular metric-depth methods: our AbsRel is 0.137, lower than DepthPro's 0.183, MoGeV2's 0.182, and DA3's 0.226; our δ1 is 0.854, higher than all three.
In the predicted-scale block of Table 2, our AbsRel and δ1 are 0.123 and 0.859, compared with 0.245 and 0.359 for DA3. In the predicted-scale block of Table 3, our Acc./Comp. are 1.498/1.704, compared with 2.091/2.541 for DA3. After reference-scale alignment, our method also outperforms the compared feed-forward reconstruction methods in Tables 2 and 3. Therefore, although the predicted scale does not fully match the reference scale, it already yields useful and more accurate metric results.