acceptodds
Under review as a conference paper at ICLR 2027

Empowering Feed-Forward Reconstruction Models with Metric Scale via Satellite Images

Abstract

Feed-forward 3D reconstruction models have recently shown strong generalization across diverse scenes, yet most of them recover geometry only up to an unknown global scale. This scale ambiguity limits their use in applications that require metric understanding of the environment. Existing metric reconstruction methods commonly rely on large-scale metric annotations or accurate camera calibration, both of which are costly or unreliable in many real-world settings. We propose a satellite-guided framework for resolving scale ambiguity in outdoor feed-forward 3D reconstruction. The key idea is to use readily available satellite imagery as a global metric reference. Given coarse position and heading estimates, our method retrieves a local satellite patch and integrates it with a feed-forward reconstruction backbone through bidirectional cross-view interaction. By enforcing consistency between the reconstructed scene and the satellite reference, the model infers absolute scale, refines scene geometry, and estimates camera pose in a metric coordinate frame. Experiments on KITTI, nuScenes, and Oxford RobotCar demonstrate improvements in metric depth estimation, multi-view point-cloud reconstruction, and camera localization under the evaluated cross-dataset and geographic transfer protocols.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
metric 3D reconstruction, feed-forward reconstruction, satellite imagery, cross-view localization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sWXAOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes to use coarse GPS measurements from satellite imagery and vehicle heading estimates as an external metric reference for feed-forward 3D reconstruction. The coarse satellite images are retrieved from Google Maps using publicly available GPS annotations from which heading-oriented local crops are extracted.

The main idea is to train a coupled architecture built on a pretrained feed-forward 3D reconstruction model, where both branches are initialized with the pretrained model, keeping frozen the ground branch and the satellite branch is refined. The two branch is connected through an asymmetric cross-attention mechanism that operates at different depths of the decoder and progressively aggregates ground-view geometry building a rich crossview representation for accurate scale and coordinate estimation. Furthermore a lightweight reverse interaction on the final layers injects satellite-derived metric and structural cues into the ground-view representation to refine scene geometry point by point. The model trained with LiDAR-derived metric depth and camera-pose supervision learns to jointly estimates absolute camera poses and metric scene geometry.

Critical strengths

Overall, I remain unconvinced by the paper and do not identify a sufficiently compelling strength to support acceptance. Nevertheless, the paper has several positive aspects worth acknowledging:

  1. Using a retrieved satellite image patches to help recovering metric scale, to estimate absolute camera pose, and to refine dense geometry within a feed-forward reconstruction framework seems an interesting direction.

  2. The main contribution is the progressive update of the satellite branch using ground-view features, combined with reverse interaction introduced only in the final decoder layers. The ablations suggest that with this per-point geometric correction one can obtain better reconstruction, but still below the results obtained with ground-truth scale alignment.

  3. I appreciate that the model was tested with different feed-forward architectures (π3, VGG, VGGT-Ω).

  4. The paper proposes cross-dataset evaluation on three tasks: monocular metric depth estimation, multi-view metric depth estimation, and multi-view metric point-cloud estimation. In addition, a cross-view localization experiment on KITTI and Oxford RobotCar is added.

Critical weaknesses

Main concerns supporting rejection:

  • The experiments do not convincingly establish the benefit of the proposed architecture beyond providing additional satellite information, given the limited comparisons with metric reconstruction methods and the absence of an inference-cost analysis.

  • Limited conceptual novelty and the substantial gap between predicted-scale and ground-truth-scale performance, this leaves insufficient evidence of a compelling methodological advance.

Detailed remarks:

  1. Using retrieved satellite imagery as a reference offers limited conceptual novelty, as cross-attention between satellite and ground views has already been explored (see [a,b]). The most original contribution appears to be the progressive update of the satellite branch using ground-view features, combined with reverse interaction introduced only in the final decoder layers.

  2. The paper should discuss the relation to [a] proposing cross-attention between ground and satellite features, and to SGFormer [b] that uses ground geometry guiding of a satellite branch before feature fusion.

  3. The experiments illustrates that adding scale estimation, coordinate offset and correction heads help improving the vanilla models, but do but emphasize enough that the proposed model uses extra information (satellite images) compared to the vanilla models. Note that the cost of the model is doubled with coupling two π3 architectures.

  4. It is appreciable that the experiments are done cross-datasets, still it is rather pity that the experiments are mainly limited to training the model on KITTI and tested on nuScenes and vice-versa. How would these models perform on Oxford RobotCar (depth maps and point clouds could be constructed from the raw sensor data)? Also how would the model be improved if trained on a collection of several datasets?

  5. The model is mainly compared to models which do not have metric output (except DA3 and MASt3R) but not compared to CUT3R [c], MUSt3R [d] , AMB3R, DVGT[e], MapAnything [f]. Nevertheless those models seem to perform less well or they use much more ground images (128 in DVGT) so numbers are not directly comparables. Still the paper should mention/discuss these works.

  6. Why not using for nuScenes the aligned geographic images from [g] https://huggingface.co/datasets/SpatialRetrievalAD/nuScenes-Geography-Data.

  7. In section 4.1, the difference between the results obtained with GT vs predicted scale in Table 3 suggest that the estimated scale is not very accurate.

8.The paper should provide an inference cost comparison between the vanilla model and the proposed coupled architecture.

References: [a] Yuan et al, Cross-Attention Between Satellite and Ground Views for Enhanced Fine-Grained Robot Geo-Localization, WACV’24 [b] Guo et al, SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion, CVPR’25. [c] Wang et al, Continuous 3D Perception Model with Persistent State, CVPR'25 [d] Cabon et al, MUSt3R: Multi-view Network for Stereo 3D Reconstruction, CVPR'25 [e] Zuo et al, DVGT: Driving Visual Geometry Transformer, CVPR'26 [f] Keetha et al, MapAnything: Universal Feed-Forward Metric 3D Reconstruction, 3DV'26 [g] Jia et al, Spatial Retrieval Augmented Autonomous Driving, CVPR'26

Questions for authors
  1. What is happening if the satellite image is completely wrong (beyond the considered perturbation) and has no overlap with the ground images ? Is the model able to ignore the correction terms falling back to the backbone mode or it yields to worse results. This would be interesting to investigate. What about longer sequences (e.g 64 or 128 images), where the satellite image covers only 1 or 2 frames?

  2. Was the satellite image added as input together with the ground images when the vanilla Feed-forward methods were tested? It would be interesting to see if the satellite image would help or on the contrary would degrade the performance of those models.

  3. In Section 4.2 requires some clarification. If I well understood in this case the model is trained and tested on the same dataset (same or different areas). Would the model trained on KITTI work on Oxford RobotCar and vice-versa? Or on the VIGOR dataset without training on the VIGOR dataset (maybe a model trained on KITTI, NuScenes and maybe Oxford RobotCar.

Minor correction needs:

Line 034: For SFM I would cite Snavely et al. Photo Tourism: Exploring Photo Collections in 3D ACM TOG 2006 and Schönberger and Frahm, Structure-from-Motion Revisited. CVPR 2016.

Line 308: I would use another letter for cost to avoid confusion with with alignment parameters. Also providing more details (formulas) for the cost, maybe in an Appendix, would make clearer the loss in Eq(9).

Line 446-447 - please cite method by method either here to more easily connect with Tables 4 and 5.

Line 540 : reference the same as in 539

In the bib references putting the method names into {} allows to keep the capital letters. nuscenes-> nuScenes, Vggt ->VGGT, Cvlnet -> CVLNet , etc

6 comments
AuthorsOpenReview ↗

Response to Reviewer sWXA (1/3)

We thank Reviewer sWXA for the valuable comments. Our responses are provided below.

Q1.Please compare inference cost with the vanilla model.

Thank you for this suggestion. We added a computational-cost experiment to the appendix. Both models are evaluated on the same NVIDIA H200 GPU, with ground-image resolution 252×1008252\times1008 and satellite-image resolution 518×518518\times518. After 10 warm-up runs, we measure 50 consecutive runs. The results are shown below.

Number of views NModel variantLatency (ms)Peak memory (MiB)
1Vanilla Pi31256565
1Full Dual-Branch2617598
10Vanilla Pi311149084
10Full Dual-Branch143610528
50Vanilla Pi31440720280
50Full Dual-Branch1464823930

These results show that incorporating satellite imagery does not introduce a large increase in memory usage or inference latency. Moreover, as the number of ground images increases, the additional latency becomes negligible: with 50 ground images, it accounts for 1.6% of the total latency.

Q2.Cross-attention between satellite and ground views is not new. Please discuss Yuan et al. [a] and SGFormer [b].

Cross-attention between ground and satellite views is not introduced for the first time in this work. Our work is the first to apply it to augment feed-forward reconstruction models, enabling metric-scale scene reconstruction with limited adaptation data while keeping the pretrained backbone frozen.

Method [a] fuses local satellite and ground features with self- and cross-attention to localize the robot within a known patch. Method [b] encodes ground images as voxels and satellite images as BEV features, guides the satellite branch with compressed ground features, and fuses the two streams for voxel semantic completion.

Our cross-attention operates on a frozen feed-forward backbone for metric reconstruction. In the primary direction, satellite features query ground features throughout decoding, allowing the satellite branch to progressively aggregate ground geometry and predict global scale β\beta and coordinate transform CC. Reverse interaction is introduced only in the final decoder layers: ground features query satellite features, and the resulting features are supplied only to the pointwise correction head. The base point maps before correction are still produced by the frozen backbone. Thus, [a] and [b] address localization within a known patch and voxel semantic completion, respectively, whereas our asymmetric, staged interaction jointly estimates metric scale, absolute pose, and pointwise geometric correction.

Q3.The gap between predicted scale and GT scale in Table 3 suggests that the estimated scale is not very accurate.

A gap remains between the predicted and ground-truth reference scales. Nevertheless, our method already substantially outperforms existing methods without using ground-truth test-set scale. Table 1 compares our method with dedicated monocular metric-depth methods: our AbsRel is 0.137, lower than DepthPro's 0.183, MoGeV2's 0.182, and DA3's 0.226; our δ1\delta_1 is 0.854, higher than all three.

In the predicted-scale block of Table 2, our AbsRel and δ1\delta_1 are 0.123 and 0.859, compared with 0.245 and 0.359 for DA3. In the predicted-scale block of Table 3, our Acc./Comp. are 1.498/1.704, compared with 2.091/2.541 for DA3. After reference-scale alignment, our method also outperforms the compared feed-forward reconstruction methods in Tables 2 and 3. Therefore, although the predicted scale does not fully match the reference scale, it already yields useful and more accurate metric results.

AuthorsOpenReview ↗

Response to Reviewer sWXA (2/3)

Q4.Results are mainly KITTI↔nuScenes. How does the model perform on Oxford RobotCar, and if trained on several datasets together?

Thank you for this valuable suggestion. We did not conduct Oxford RobotCar point-cloud experiments because its point-cloud projection is not sufficiently accurate; please also refer to issue [2] in the official robotcar-dataset-sdk repository. Nevertheless, we recognize the importance of further cross-dataset evaluation. We therefore conducted additional experiments on A2D2, using the 1,000 samples from each of its three cities for generalization testing. These results have been added to the appendix.

ModelScaleAcc Mean ↓Acc Med. ↓Comp Mean ↓Comp Med. ↓N.C. Mean ↑N.C. Med. ↑
DA3GT2.30901.30242.31631.66390.75230.9057
KITTI onlyGT0.98520.49081.39850.91630.77140.9233
nuScenes onlyGT0.96790.47191.33840.87280.77230.9239
KITTI + nuScenesGT0.95190.46721.36880.88900.77330.9251
DA3Pred3.70041.64363.09682.49760.73970.8942
KITTI onlyPred2.25433.29493.44432.25400.73870.8872
nuScenes onlyPred2.21301.45493.67732.46330.74370.8890
KITTI + nuScenesPred2.20311.38633.09682.16110.74220.8892

All three training settings substantially outperform DA3 on the unseen A2D2 dataset. Combining the two training datasets further reduces accuracy error, while predicted-scale Comp Med. decreases from 2.254 and 2.463 for the single-dataset settings to 2.161.

Q5.Please discuss CUT3R, MUSt3R, AMB3R, DVGT, and MapAnything, even if the numbers are not directly comparable.

We thank the reviewer for pointing out these methods. CUT3R keeps a recurrent state and accumulates point maps online. MUSt3R extends two-view reconstruction to multiple views and outputs them in a shared coordinate frame. AMB3R places a sparse voxel backend and a scale head behind the feed-forward frontend. DVGT focuses on driving scenes and predicts point maps from surround-view sequences; the timing it reports uses 128 images. MapAnything returns depth, rays, pose, and scale in one forward pass, and can optionally take calibration, pose, or depth as input.

We evaluated these methods on KITTI, where our method achieves better results. This is because normalized relative geometry is easier to learn than absolute metric depth, so π3\pi^3, which uses normalized geometry as its prior, is more accurate than models that directly estimate metric geometry. Through satellite imagery, our method incorporates metric information at low cost while preserving this feed-forward prior. The discussion has been added to the main text, and the results are provided in the appendix.

ModelScaleAcc Mean ↓Acc Med. ↓Comp Mean ↓Comp Med. ↓N.C. Mean ↑N.C. Med. ↑
MapAnythingGT0.41990.17000.56090.19390.80210.9568
DVGT-1GT0.34990.15360.31180.13440.81160.9577
CUT3RGT0.49350.24720.44080.15540.79000.9372
MUSt3RGT0.37990.16360.48660.19290.79640.9464
AMB3RGT0.29260.15400.31490.15030.84750.9653
OursGT0.22390.08940.21220.09290.86260.9807
MapAnythingPred0.56530.25540.59810.25940.78160.9396
DVGT-1Pred5.77866.032411.00137.48000.61700.6798
CUT3RPred0.70820.39650.92200.29860.77050.9189
MUSt3RPred0.78240.58381.55500.77720.76040.9104
AMB3RPred0.58410.38880.70200.40580.80080.9392
OursPred0.27570.11000.26230.11250.84430.9752

Q6.Why not use the aligned nuScenes geographic images from Jia et al. [g]?

Our approach is similar to nuScenes-Geography-Data [g]. As explained by the nuscenes-devkit maintainers in [1], vehicle latitude and longitude can be obtained by combining each map's origin coordinates with the metric translations in ego_pose. Both [g] and our method use this conversion to retrieve satellite imagery.

AuthorsOpenReview ↗

Response to Reviewer sWXA (3/3)

Q7.If the satellite image has no overlap with the ground views, does the model fall back to the backbone or get worse? What about long sequences (64 or 128 frames) covered by only 1–2 frames?

Thank you for the suggestion. These two scenarios differ from the problem setting considered in this work. We address coarse GPS position errors of ±20 m\pm20\,\mathrm m, following the same setting as [1], whereas completely non-overlapping satellite and ground imagery, or long sequences with only one or two frames within satellite coverage, are beyond the scope of this work. Nevertheless, we understand that the proposed experiment aims to examine the role of satellite imagery in scale estimation. We therefore conducted the following additional experiments on 1,000 KITTI images.

The results show that scale can be estimated normally when all three frames are within satellite coverage. When there is no overlap, scale estimation deteriorates markedly, with Acc Mean increasing from 0.362 to 0.602. For a 64-frame sequence with only one or two frames within coverage, scale estimation is misled by these few frames and deteriorates severely, with Comp Mean rising from 0.646 to 4.465.

SettingScaleAcc Mean↓Acc Med↓Comp Mean↓Comp Med↓N.C. Mean↑N.C. Med↑
3 frames, within coveragePred0.36180.19350.28970.15580.80280.9415
3 frames, outside coveragePred0.60190.41460.88110.35680.78300.9335
64 frames, within coveragePred0.57390.39450.64610.34750.72700.8736
64 frames, only 1 frame within coveragePred0.66220.51794.46490.65310.78180.9036
64 frames, only 2 frames within coveragePred0.67020.51624.44770.64860.78060.9034

This experiment has been added to the appendix.

Q8.Were vanilla feed-forward models also given the satellite image as input, and does it help or hurt them?

Thank you for this suggestion. We further evaluated vanilla π3\pi^3 and VGGT on nuScenes by directly adding the satellite image as an input, and included this experiment in the supplementary material. The results are shown below.

MethodAcc Mean ↓Acc Med. ↓Comp Mean ↓Comp Med. ↓N.C. Mean ↑N.C. Med. ↑
PI3 + Sat frame2.91892.952621.108816.71880.75150.8862
VGGT + Sat frame5.62055.673624.515320.61660.71820.8251
Ours1.49820.80891.70420.95900.87020.9792

Directly adding satellite imagery to the vanilla models does not yield good results. This is because satellite and ground images have different viewpoints: satellite images often show rooftops, whereas ground images show building facades. As a result, the vanilla models cannot establish correspondences between the two views in most cases, so directly using satellite images as input does not help ground-view reconstruction.

Q9.In Section 4.2, are training and testing on the same dataset? Does a KITTI model transfer to Oxford RobotCar or to VIGOR without training on them?

Section 4.2 follows the same experimental setting as existing methods, with training and testing performed on the same dataset. VIGOR uses panoramic images, whose projection differs from that of pinhole images. Feed-forward 3D reconstruction models such as Pi3 and VGGT are trained for pinhole cameras and cannot accommodate panoramic inputs; therefore, our method cannot be directly transferred to VIGOR. During the rebuttal period, Oxford RobotCar could not be downloaded from its official website. We therefore added a transfer experiment on A2D2 using 1,000 images from three cities in the dataset. Both our method and the baselines are trained exclusively on KITTI and evaluated on A2D2.

MethodDistance Mean (m) ↓Distance Med. (m) ↓Yaw Mean (°) ↓Yaw Med. (°) ↓
FG²13.2712.206.724.53
BevSplat λ1=0\lambda_1=014.4512.074.904.97
BevSplat λ1=1\lambda_1=114.4511.744.904.97
Ours10.8710.822.871.81

When trained only on KITTI, our method achieves a mean distance error of 10.87 m and a mean yaw error of 2.87°, outperforming FG² and BevSplat.

Q10.Minor fixes: SfM citations, the cost symbol and formula, per-method citations for Tables 4 and 5, the duplicate reference, and BibTeX capitalization.

Thank you for pointing these out. We have corrected all of these formatting issues.

[1] https://github.com/nutonomy/nuscenes-devkit/issues/144
[2] https://github.com/ori-mrg/robotcar-dataset-sdk/issues/20
[g] Spatial Retrieval Augmented Autonomous Driving

Reviewer_mfDUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a satellite-guided framework for metric-scale feed-forward 3D reconstruction. The method uses bidirectional cross-view attention to integrate ground-view and satellite features. It predicts a global scale factor to recover metric-scale geometry and a coordinate transformation to align the reconstruction with the satellite reference frame, enabling absolute camera pose estimation. It also predicts per-point geometric corrections to refine the reconstructed 3D geometry. Experiments on various datasets show improvements in metric reconstruction and camera localization. More experiments show such method also works on different backbones.

Critical strengths
  1. Proposes a novel idea that use satellite images to recover the metric scale, provides a pratical way to address the limitation of current feed-forward reconstruction models.
  2. Beside the scale, the proposed method also predicts the geometric corrections to improve the reconstruction quality.
  3. The proposed method is backbone-agnostic, works on various 3D vision models.
Critical weaknesses

The method uses metric depth and camera pose supervision during training. It is unclear how much of the improvement comes from satellite images rather than metric supervision. Although Table 6 provides an ablation study, it does not isolate these two factors. A controlled comparison with the same metric supervision but without satellite images would help clarify the contribution of satellite guidance.

Questions for authors

The main concern is listed in "Critical Weaknesses" section. I noticed that the authors also acknowledge this limitation in Section 4.3. If the authors can further clarify this issue, I would consider raising my score.

Another minor question is how to get satellite images for dataset without GPS annotations? Is there a practical way to obtain the corresponding satellite images without known geographic locations?

3 comments
AuthorsOpenReview ↗

Response to Reviewer mfDU

We thank Reviewer mfDU for the valuable comments. Our responses are provided below.

Q1.How much of the gain comes from satellite images rather than metric depth and pose supervision? Please compare under the same supervision without satellite images.

Thank you for this suggestion. We added a ground-only control to the ablation study in Section 4.3. This model and the full model use the same metric supervision and training budget, with both trained on KITTI and tested on nuScenes. The only difference is whether satellite imagery is incorporated.

MethodScaleAcc. Mean ↓Acc. Med. ↓Comp. Mean ↓Comp. Med. ↓N.C. Mean ↑N.C. Med. ↑
w/o satGT1.27920.70921.47290.90450.88560.9810
OursGT1.08790.59591.19670.70860.89560.9886
w/o satPred2.78661.40662.16821.49340.84450.9632
OursPred1.49820.80891.70420.95900.87020.9792

As the table shows, under the same supervision and training budget, the model with satellite imagery yields a more accurate metric scale and better geometry. This gain comes from the satellite content.

Q2.How can satellite images be obtained for datasets without GPS annotations?

Our method retrieves satellite images based on coarse geographic location. For KITTI, we use the GPS-aligned ground–satellite image pairs provided by [1]. For nuScenes, we use the dataset's publicly available GPS information to retrieve satellite images. Precise GPS is not required at inference; the retrieved patch only needs to cover the vehicle's location.

For data without GPS annotations, global image geolocation methods such as [2] and [3] can estimate coarse coordinates from a single image, which can then be used to retrieve satellite imagery. However, their typical localization errors currently range from kilometers to tens of kilometers, exceeding the retrieval range of our satellite patches. They therefore cannot yet directly replace the coarse GPS used in our method. As their accuracy improves, estimating coarse location from an image and then retrieving the corresponding satellite imagery could become a feasible extension.

[1] Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Images
[2] G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models
[3] GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization

Thank you for the detailed response and additional experiments. The new controlled ablation addresses my main concern and provides clearer evidence of the contribution of satellite guidance. The clarification regarding satellite image retrieval is also helpful. I have raised my score accordingly.

AuthorsOpenReview ↗

Reply to Reviewer mfDU

We sincerely thank Reviewer mfDU very much for the positive response. We are excited to know that your concerns have been addressed. Your invaluable suggestions greatly enhance the quality of the paper.

Reviewer_awFnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses georeferenced satellite images to recover metric scale in feed-forward 3D reconstruction. Given ground images and a coarse pose estimate, the model uses cross-view attention to estimate scale, refine the pose, and correct the reconstructed geometry. Experiments on outdoor driving datasets report improvements in reconstruction and localization, with results on several reconstruction backbones.

Critical strengths
  1. Satellite imagery provides a useful reference for resolving scale ambiguity. I find the joint treatment of scale recovery, localization, and geometry correction practically relevant. In relation to prior work such as Cross-View Splatter, this joint formulation is the most interesting aspect of the paper.
  2. The reported improvements cover both reconstruction and localization. Experiments across datasets and backbones suggest that the approach has value beyond a single setting, although differences in input information should be kept in mind when interpreting the comparisons.
Critical weaknesses
  1. The experiments do not isolate the benefit of satellite imagery from that of additional metric supervision. Both are introduced together, so the current ablations leave the main source of improvement unclear. A ground-image-only model trained with the same supervision and a comparable training budget is needed to support the central claim.
  2. The geometry ablation is difficult to interpret. If the base point maps are fixed and only a global similarity transform is applied, similarity-aligned geometry errors should remain essentially unchanged under the same evaluation protocol. Yet they change before the explicit correction module is added. It is unclear whether this comes from changes to the base geometry or the evaluation procedure.

The reference-scale estimator and the correction loss are also insufficiently specified. These details matter for understanding and reproducing the results.

Questions for authors
  1. Can you compare against a ground-image-only model using matched metric supervision and training resources? A test with mismatched satellite patches would also help establish whether the model uses the satellite content.
  2. Does cross-view attention alter the base point maps before geometric correction? Please clarify why the aligned geometry errors change in the ablation and whether point selection and alignment are consistent across variants.
  3. How are the reference scale and correction loss defined? Please also clarify the reconstruction evaluation setup, particularly the data splits and input views.
1 comment
AuthorsOpenReview ↗

Response to Reviewer awFn

We thank Reviewer awFn for the valuable comments. Our responses are provided below.

Q1.Can you compare against a ground-image-only model with the same metric supervision and training budget, and test mismatched satellite patches to show that the model uses the satellite content?

Thank you for this suggestion. We added a ground-only control to the ablation study in Section 4.3. This model and the full model use the same metric supervision and training budget, with both trained on KITTI and tested on nuScenes. The only difference is whether satellite imagery is incorporated.

MethodScaleAcc. Mean ↓Acc. Med. ↓Comp. Mean ↓Comp. Med. ↓N.C. Mean ↑N.C. Med. ↑
w/o satGT1.27920.70921.47290.90450.88560.9810
OursGT1.08790.59591.19670.70860.89560.9886
w/o satPred2.78661.40662.16821.49340.84450.9632
OursPred1.49820.80891.70420.95900.87020.9792

As shown in the table, Under the same supervision and training budget, satellite imagery therefore provides both more accurate metric scale and better geometry, indicating that the gains come from the satellite content.

Q2.Does cross-view attention change the base point maps before geometric correction, and why do similarity-aligned geometry errors change before the correction module is added?

To preserve the base model's prior, cross-view attention does not alter the base point maps before correction. The original ablation description was insufficiently clear, and we have revised Section 4.3 accordingly: the fourth row introduces the loss Ltriplet\mathcal L_{\mathrm{triplet}} for ΔXN\Delta X_N, rather than the correction module itself.

The correction module is present throughout the adapted configurations and is continuously updated through comparison with the ground-truth point cloud, so different configurations produce different results. The first row is the base model; scale learning starts in the second row; the third adds feature-similarity learning; and the fourth adds Ltriplet\mathcal L_{\mathrm{triplet}}, yielding further improvements.

Q3.How are the reference scale and the correction loss defined?

We have further clarified Section 3.3.

Reference scale: The reference scale β^\hat{\beta} is the global scale obtained from valid ground-truth points X^N\hat{X}_{N} and the corresponding point XNX_{N} predictions by minimizing β^=∥X^N−sXN∥1\hat{\beta}=\lVert\hat{X}_{N}-sX_{N}\rVert_{1}. The reference-scale loss is the scale loss in the paper, namely the ℓ2\ell_{2} loss between the predicted scale β\beta and the reference scale β^\hat{\beta} in log space: Lscale=(log⁡β−log⁡β^)2\mathcal{L}_{\mathrm{scale}}=\left(\log\beta-\log\hat{\beta}\right)^{2}.

Correction loss: The correction term is ΔXN\Delta X_{N} in the reconstruction equation XN′=Cr[β(XN+ΔXN)]+CtX_{N}'=C^{r}\left[\beta\left(X_{N}+\Delta X_{N}\right)\right]+C^{t}. In Stage B, the scale and pose heads are frozen, and the correction head learns the pointwise term ΔXN\Delta X_{N} by minimizing Ltriplet=1HW−1∑(u,v)≠(u^i,v^i)∣log⁡(1+exp⁡(α[Di(u^i,v^i)−Di(u,v)]))∣\mathcal{L}_{\mathrm{triplet}}=\frac{1}{HW-1}\sum_{(u,v)\neq(\hat{u}_{i},\hat{v}_{i})}\left|\log\left(1+\exp\left(\alpha\left[\mathcal{D}_{i}(\hat{u}_{i},\hat{v}_{i})-\mathcal{D}_{i}(u,v)\right]\right)\right)\right|, where Di\mathcal{D}_{i} is the normalized cross-correlation between BEV-projected ground features and satellite features, (u^i,v^i)(\hat{u}_{i},\hat{v}_{i}) is the positive camera location, and (u,v)(u,v) denotes a sampled negative location.

Q4.What are the data splits and input views in the reconstruction evaluation?

Each reconstruction test uses three ground images and one satellite image. Reconstruction is evaluated across datasets: models trained on nuScenes are tested on KITTI, and models trained on KITTI are tested on nuScenes.

KITTI follows the training/test split in [1], with no overlap between the training and test sets. Test1 contains trajectories in the same area as training, while Test2 contains trajectories in a different area. For nuScenes, training uses the official v1.0-trainval training scenes with CAM_FRONT and CAM_BACK; evaluation uses the official test set with CAM_FRONT only. Each test sample also includes one satellite image, with initial position and heading perturbations of ±20 m\pm20\,\mathrm m and ±10∘\pm10^\circ.

[1] Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Images

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.