acceptodds
Under review as a conference paper at ICLR 2027

SplatAnywhere: 3D Gaussian Splatting from Sparse and Inconsistent Street Panoramas

16 views

Abstract

3D Gaussian Splatting (3DGS) produces photorealistic scene models, yet reconstructing street scenes still requires dedicated capture for every target site, making reconstruction costs grow with every new location. Google Street View (GSV) already covers a large share of the world's roads with panoramas captured repeatedly over the years. Reconstruction from GSV would enable worldwide 3D digital coverage at no additional capture cost. However, GSV panoramas are sparse in camera centers, which leaves depth weakly constrained and yields models that look acceptable at training viewpoints but show floating artifacts under extrapolated views. They are also inconsistent in appearance across capture sessions, which adds photometric error that no shared color can remove. Standard photometric optimizers, ill-suited for this problem, compensate for the photometric error with wrong geometry. To reconstruct robustly from GSV, we present SplatAnywhere, an optimization-based pipeline that builds structure-aligned 3DGS scenes directly from GSV panoramas. SplatAnywhere adds structural supervision losses in the regions that Structure from Motion under-represents, suppresses splits where the views disagree, reducing the number of Gaussians while improving the geometry, and uses a session appearance adapter that takes up the photometric differences across capture sessions. Across street scenes sampled from five representative cities, SplatAnywhere reaches the best average PSNR, SSIM, and LPIPS compared with 3DGS baselines, lowers their 3D structural error by 26% to 89%, and holds these gains even for extrapolated views.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Real2Sim, Sim2Real, Novel View Synthesis, Structure from Motion

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_yDD4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SplatAnywhere, a 3D Gaussian Splatting framework for reconstructing street scenes from sparse and temporally inconsistent Google Street View panoramas. It combines structural supervision, consensus-gated densification, and session-aware appearance modeling. Experiments across five cities demonstrate improved rendering quality and substantial reductions in 3D structural errors compared with existing 3DGS methods.

Critical strengths
  1. The paper addresses the practical challenge of reconstructing 3D street scenes from sparse and inconsistent real-world observations. The reported improvements in geometric accuracy are substantial, especially for extrapolated viewpoints.

  2. The paper includes comparisons with multiple baselines, component ablations, and evaluations using airborne LiDAR as geometric reference. The consensus-gated densification strategy is particularly effective in reducing the number of Gaussians while improving geometric consistency.

Critical weaknesses
  1. The proposed framework combines several existing techniques, including SfM-based depth supervision, monocular depth priors, and appearance embeddings. Although the integration is effective, the individual components are mostly incremental. The novelty of the overall framework could be better justified.

  2. Despite the worldwide reconstruction motivation, experiments cover only five relatively small street scenes. Furthermore, the main evaluation derives test views from panoramas already used for training. Although Appendix A.4 provides a held-out panorama experiment, it is limited to London, and camera poses are still estimated using all panoramas.

Questions for authors
  1. What is the main methodological contribution beyond combining existing geometry supervision and appearance modeling techniques?
  2. How does the method perform when both the test panoramas and their observations are excluded from the SfM reconstruction?
2 comments

We sincerely thank you for your feedback and questions. In the general response above, we have toned down the worldwide claim, and conducted additional experiments to cover unseen new camera positions. Below, we respond to additional comments.

1) Novelty

Thank you for the careful reading. Our main methodological contribution is the analysis of why reconstruction from Street View fails, together with a set of mechanisms designed against that failure.

First, the problem. To our knowledge this is the first work that reconstructs 3DGS street scenes from public Street View panoramas alone, without dedicated capture or LiDAR. Section 3.1 shows that its two difficulties are coupled. The appearance differs between capture sessions and the cameras lie on a narrow strip of road, so the optimizer can lower the photometric error by moving geometry. Reviewers ZtSY and v8Kd also list this formulation and analysis among the strengths of the paper. Under this coupling, existing techniques are not sufficient. When the baselines that use monocular depth supervision (Vid2Sim), per-image appearance embeddings (WildGaussians), and time conditioning (ChronoGS) are trained with our SfM and inpainted crops, their structural error stays 1.6 to 9 times as large as ours (Appendix A.5).

Second, the mechanisms. Each one closes one route by which this coupled error enters, and three of them depart from their closest prior work.

(a) Monocular depth. Vid2Sim supervises the rendered depth and normals with monocular predictions. We take the structure from multi-view evidence (SfM points and a road grid) and use the monocular depth only as a one-sided bound on free space, never as a depth target, because it is relative and unreliable at object boundaries. In an additional London run, replacing the bound with a depth target raises the 3DSE from 0.202 to 0.640 m, as shown in the table below.

Use of the monocular depth (London)PSNR3DSE (m)
Ray clearance (ours)17.120.202
Depth target15.500.640
Ray clearance and depth target17.210.209

The first row is our model. The second row replaces the one-sided bound with a depth target, which supervises the rendered depth on both sides. Its 3DSE is 3.2 times higher and its PSNR is 1.6 dB lower. The third row keeps the bound and adds the depth target on top. The monocular depth therefore helps as a conservative bound on free space, and using it as a depth target harms the geometry.

(b) Appearance. Per-image appearance models, such as the embeddings of WildGaussians, usually give every image its own parameters. That is safe when many viewpoints fix the geometry, but under Street View these parameters can also absorb residuals that should correct the geometry. Our adapter follows Eq. 1 instead. It has one code per capture session, which is enough to take up the between-session term E_sess, and a penalty keeps it near the identity. It is designed so that the optimizer does not have to explain session differences with geometry, and in the ablation it lowers the 3DSE by 9 to 25% (Table 6), and in every view group of the four cities tested (Table 21). Since the code depends only on the session, it also needs no pixels of the test view and applies at new camera positions (Section 3.6).

(c) Densification. Consensus-gated densification splits less where the gradients of different crops disagree, the opposite of GDAGS, which splits more under conflict. Replacing our gate with GDAGS doubles the model (2.20 M against 1.11 M Gaussians) and raises the London 3DSE from 0.202 to 0.256 m at similar image quality (Appendix A.14).

Third, the mechanisms are needed together, which is why the paper presents a complete pipeline. SfM depth anchors are the one standard ingredient, and on their own they are not enough. With them the London 3DSE is still 0.544 m, because SfM points concentrate on textured facades and leave the road almost unsupervised. The ray clearance and the road grid, which the baselines have no counterpart for, lower it to 0.464 and then 0.202 m (Table 6). The consensus gate is equally necessary in the full model: removing it raises the 3DSE by 44 to 89% and the model from 1.11 M to 2.49 M Gaussians (Appendix A.14). The session adapter lowers the 3DSE as well, from 1.348 to 1.211 m on Interpolation and from 1.055 to 0.789 m on Pitch Shift (Table 6).

Empirically, on eight scenes with independent LiDAR (five cities and three Argoverse sites), our method has the lowest structural error of all eight baselines on every mean, with 3DSE lower by 26 to 95%, and the highest PSNR on the held-out views and at the Argoverse sites.

The reviewers have resolved most of my questions and doubts. I have decided to maintain my score ( 3: Weak acceptance)

Reviewer_v8KdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SPLATANYWHERE, a method for building street-level 3D scenes from Google Street View panoramas. The main challenge is that the images provided are sparsely captured and often look different because they were captured at different times. The method removes vehicles and pedestrians, estimates camera poses using GPS information and GLOMAP, and reconstructs the scene using Gaussian Splatting. It adds depth and road constraints, accounts for appearance changes across capture sessions, and controls Gaussian splitting to reduce artifacts caused by inconsistent images. Experiments on scenes from five cities report improved geometric accuracy and rendering quality compared with several existing methods.

Critical strengths

This paper is well-motivated with practical value. Google Street View’s broad coverage of streets worldwide makes it a valuable resource for 3D reconstruction. The paper addresses the challenges of using this existing imagery, offering a potentially useful way to reconstruct many locations without requiring dedicated data collection.

The analysis of data captured from one side of a scene helps explain why plausible renderings at training viewpoints can coexist with inaccurate geometry. This provides useful insight into the limitations of street-view capture and motivates evaluating reconstruction from viewpoints away from the original camera trajectory.

The method explicitly addresses regions such as roads, where limited texture makes reconstruction difficult using image matching and photometric supervision alone. Its combination of depth supervision and road geometry priors provides additional constraints that help recover more accurate surfaces and reduce floating artifacts.

Critical weaknesses

The majority of the experiments use all panoramas for training, and both the Interpolation and Pitch Shift tests retain the camera centers of the training panoramas. Changing the crop heading does not provide an independently captured observation. The Lateral Shift evaluation tests novel camera positions but lacks corresponding ground-truth images. The held-out panorama experiment in Appendix A.4 provides a more informative evaluation, but it is limited to London and still uses the held-out images during SfM. It remains unclear whether points triangulated using these images contribute to Gaussian initialization or to the subsequent geometric supervision. Clarifying this distinction and extending the held-out evaluation to additional scenes would strengthen the paper.

Section 3.2 argues that hallucinated content introduced by FLUX rarely survives SfM triangulation. However, the generated pixels appear to still involved in the supervision of the photometric loss, L_rgb. Consequently, hallucinated content could still influence the reconstructed appearance or geometry even if its correspondences are rejected during SfM.

According to Section 3.1, the rendered depth is a normalized average of Gaussian depths weighted by their alpha-compositing contributions. Contributions in front of and behind a true surface can therefore produce an accurate average depth without being concentrated near that surface. Although 3DSE is useful for assessing rendered-depth accuracy, a low score alone does not establish that floating or layered geometry has been eliminated.

The paper only conducted experiments in 5 metropolitan areas with 80m radius, which have a comparatively dense panorama imagery coverage. This can weaken the statement that the proposed approach has the capability to reconstruct 3D scenes at world scale.

Questions for authors

Figure 2 includes optional refinement with Difix3D+, and Appendix A.7 reports refinement results separately. Could the authors clarify whether Figure 4 shows direct Gaussian renderings or images refined with Difix3D+? If refinement was applied, please identify this explicitly in the caption and provide the corresponding unrefined renderings.

The stated reason for excluding sparse-view reconstruction baselines is that they rely on diffusion models and cannot scale. However, methods such as InstantSplat [1] and SPARS3R[2] perform sparse-view reconstruction without diffusion. Could the authors compare against methods from this category?

The pipeline converts each input panorama into 12 perspective crops. Could the author explain the choice of crop coverage and compare with methods that directly optimizes from equirectangular images, such as OmniGS[3]?

[1] Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, Zhangyang Wang, Yue Wang: InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds. CoRR abs/2403.20309 (2024) [2] Yutao Tang, Yuxiang Guo, Deming Li, Cheng Peng: SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction. CVPR 2025: 26810-26821 [3] Longwei Li, Huajian Huang, Sai-Kit Yeung, Hui Cheng: OmniGS: Fast Radiance Field Reconstruction Using Omnidirectional Gaussian Splatting. WACV 2025: 2260-2268

3 comments

We genuinely thank you for the detailed review. We have conducted additional experiments in the general comments above to respond to 1) extrapolation to unseen camera positions and 2) the worldwide claim. Below, we respond to additional comments.

1) 3DSE does not by itself measure floating or layered geometry

Thank you for this valuable suggestion. We added a second structural metric, 3DSE-M, as in the 2DGS paper (https://arxiv.org/abs/2403.17888). 3DSE-M replaces the expected depth in 3DSE with the median depth along each ray (the depth at which the accumulated opacity reaches 50%). Mass behind the median does not affect it, and mass in front moves it off the surface only once that mass holds half of the opacity. Contributions in front of and behind the surface therefore cannot average out as they can in 3DSE. The median depth is also the depth that 2DGS fuses into its meshes, because it is less sensitive to outliers than the expected depth, so 3DSE-M measures the surface that can be extracted from a model. We define it in Section 4 and report it next to 3DSE in Table 3, in the new held-out results (Table 4, Appendix A.4) and in the Argoverse results (Table 5, A.25).

Under 3DSE-M our method remains the best of all trained methods. On the five-city mean of the held-out views it is 0.135 m for ours against 0.195 m for Vid2Sim, the best baseline, and on the three-site mean at Argoverse it is 0.166 m against 0.324 m.

To further measure floaters, we also directly calculate the floaters in the free space given by the LiDAR points of Argoverse, which is denser and provides better ground-truth free space than airborne LiDAR of the main set. The table below gives the share of this free space, between 0.5 and 5 m above the ground, that each model fills with opacity above 0.1 (in %).

Share of free space filled by each model (%, lower is better)

MethodPittsburghMiamiWashington DCMean
Vanilla 3DGS5.282.992.803.69
Scaffold-GS3.931.661.342.31
WildGaussians (stabilized)5.492.762.313.52
Vid2Sim5.312.943.824.02
ChronoGS2.501.511.541.85
SPARS3R6.597.657.227.15
OmniGS8.264.173.005.14
InstantSplat (stabilized)73.9784.5866.2074.92
Ours1.361.300.601.08

From the result we can tell that ours fills the least free space at every site.

2) More baselines

Thank you for pointing us to these three baselines. We added InstantSplat, SPARS3R, and OmniGS to Tables 3 to 5. The results are shown below.

Held-out views (five cities)│Argoverse sites
MethodPSNR3DSE (m)3DSE-M (m)│PSNR3DSE (m)3DSE-M (m)
SPARS3R14.421.1021.088│12.753.5163.079
OmniGS14.131.3291.451│13.722.6702.138
InstantSplat (stabilized)10.051.9401.958│9.266.2986.331
Ours (unconditioned)15.490.1920.135│14.820.3370.166

Please note that InstantSplat is designed and evaluated with 3 to 12 views in its original paper, which states “However, InstantSplat faces limitations when applied to scenes with more than hundreds of images due to its reliance on MVS for globally aligned point clouds”. As released, it diverges on our hundreds of crops (9.22 dB on Seattle), so we report a stabilized variant with the 3DGS learning rates for scaling and rotation, with which its training converges. However, even the stabilized variant reaches only 12.6 to 15.2 dB on its own training views and has the worst image and structural results of all baselines. By design, InstantSplat has no adaptive density control and never prunes Gaussians, which is a likely reason for this result on hundreds of views. We did not add pruning or further changes to its optimization, because we think it would then be very close to SPARS3R, which also starts from MASt3R point maps and is already among the baselines. The other two baselines train normally.

3) Clarification of Figure 4

We have revised Figure 4 to improve clarity. Figure 4 shows direct Gaussian renderings. Difix3D+ is not applied to them, and the caption now states this explicitly. All quantitative results and Figure 4 are rendered without refinement. Only the teaser (Figure 1) and Appendix A.7 show refined renderings, and their captions are revised to say so now.

4) Handling hallucinations of FLUX

Thank you for this observation, which led us to a more comprehensive study of the inpainted content in L_rgb. As a simple ablation, we mask the inpainted pixels out of L_rgb during 3DGS optimization. Both settings share the same front end, and the table below gives their means over the three sites of the Argoverse set.

Inpainted pixels in L_rgbPSNR3DSE (m)3DSE-M (m)
With14.820.3370.166
Without14.660.2970.164

Removing the inpainted content from L_rgb costs a little PSNR, improves 3DSE a little, and leaves 3DSE-M almost unchanged, which indicates a slight gain in structure. In the renderings, however, certain areas become very low in quality, with large artifacts (Figure 14 in Appendix A.26). A likely reason is that, without the inpainted pixels in L_rgb, few of the sparse GSV views supervise these largely masked areas, so they are poorly constrained. We therefore keep the inpainted content for now. Future work may select the FLUX-inpainted content adaptively or address the hallucination and view inconsistency of the inpainting models.

5) Choice of crop coverage

Thank you for this question. We ran London with 6, 12, and 18 crops per panorama, and the table below shows the results. We think the number of crops is not a sensitive parameter, and 12 crops is a good balance.

Crops per panoramaPSNR3DSE (m)3DSE-M (m)
616.970.2160.173
1217.120.2020.164
1817.110.2010.165

OmniGS, which optimizes directly on the equirectangular panoramas, is now added as a baseline (see response 2 above), with a held-out 3DSE of 1.329 m against 0.192 m for ours.

In summary, the revision and this response add a held-out protocol and an independent evaluation on Argoverse 2 (General 1), narrow the claims (General 2), measure the surface position and the floating opacity separately (point 1), add three baselines (point 2), clarify Figure 4 (point 3), and study the inpainted pixels and the number of crops (points 4 and 5). We hope these changes address your concerns and would be grateful if you would consider them in your final assessment.

Reviewer_v8KdOpenReview ↗

Satisfied with the rebuttal

The authors have provided a robust rebuttal to my comments and concerns and also performed additional experiments. I will raise my score to weak accept.

Reviewer_ZtSYOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SplatAnywhere reconstructs street-level 3D Gaussian scenes from sparse Google Street View (GSV) panoramas whose appearance varies across capture years. It converts panoramas into perspective crops and estimates poses with SfM under soft GPS and shared-panorama-center constraints. SfM depth anchors, a semantic road-height grid, and conservative ray clearance supervise geometry; cross-view gradient consensus controls Gaussian splitting; and a capture-session appearance adapter absorbs photometric variation. Experiments on 691 panoramas across five cities report image and structural metrics for heading interpolation, downward pitch, and lateral shifts, with component ablations and a held-out-panorama study in London.

Critical strengths
  1. Treats sparse capture and cross-session inconsistency as a coupled problem. SfM/road/free-space supervision, session appearance adaptation, and suppression of splits driven by conflicting gradients each address a specific failure mechanism. In the London ablation, interpolation 3D structural error falls from 1.425 m in the base configuration to 0.202 m, and removing the consensus gate raises the Gaussian count from 1.11M to 2.49M.
  2. Evaluates structure independently of image similarity. Public airborne LiDAR in five cities provides a geometric reference. In Table 3, interpolation 3DSE is 0.209 m versus 0.305 m for the strongest street-reconstruction baseline, Vid2Sim. The appendix additionally reports per-class errors, shared-SfM and shared-inpainting controls, three training seeds, and a London held-out-panorama test. These checks make the geometric gain more credible.
Critical weaknesses
  1. The headline view-synthesis protocol does not fully establish extrapolation to unseen camera positions. Interpolation and pitch-shift test views share camera centers with training panoramas; twelve 90° training crops substantially overlap crops at new headings. Lateral-shift views have no corresponding photographs, so only distribution-level appearance and LiDAR structure can be assessed there. Appendix A.4 importantly holds out 24 London panoramas, but their poses are still estimated using SfM over the full set, including those panoramas. Performance on wholly unseen positions therefore remains entangled with reuse of the source panoramas and test-view geometry.
Questions for authors
  1. Can the authors evaluate photographs and geometric ground truth at unseen camera centers while excluding the held-out panoramas from SfM, point initialization, and training? This would directly test the claimed extrapolation.
  2. Can the authors report failure rates and structural errors for facades, curbs, slopes, and other challenging geometry across more independent neighborhoods, together with end-to-end acquisition and processing costs as scene extent grows?
2 comments

We sincerely thank you for your constructive feedback. We have conducted additional experiments in the general comments above to establish extrapolation to unseen camera positions. Below, we respond to additional comments.

1) Computational scalability

Thank you for raising this question. The compute cost can be broken down into multiple components. SuperGlue matching takes 2 h 19 min per scene on average and transient removal 1.7 to 2 min per panorama, against 28 min for 3DGS training. The bottleneck is not training but matching and FLUX inpainting. We have reported the running time of each stage of the pipeline in Appendix A.21 (Table 25), and the revised main text now points to it (Section 4).

To show how the cost grows with the scene extent, we additionally ran the full pipeline on Seattle with fetching radii of 20 m (9 panoramas), 40 m (96 panoramas), and 80 m (149 panoramas). The total reconstruction time is 0.7 h, 4.3 to 4.8 h, and 6.7 to 7.5 h, respectively, growing linearly with the number of panoramas and crops. Downloading the panoramas takes about 15 s per panorama (about 40 min at 80 m), small next to the processing. However, since our pipeline is optimization-based, its front end is flexible. Changing it to a fast variant with SIFT instead of SuperGlue and masks instead of inpainting cuts the time by about 75% at 40 and 80 m, at the cost of registering fewer panoramas (83% and 87%) and a PSNR lower by 1.5 to 1.8 dB. For the best quality, SuperGlue and inpainting are therefore still needed.

To reconstruct at scale, we can leverage massive parallelism, building and stitching multiple small scenes. For example, when we split the Seattle 80 m scene into four overlapping regions built independently, the camera positions of the panoramas shared by two regions agree to 0.08 m (median) without any alignment. Since every region runs its own inpainting, matching, SfM, and training, the regions can run on separate machines, and the wall-clock time is that of the largest region instead of the whole area. In this test the four regions took about half the time of the single model (1.3 h against 2.5 h for SfM and training). Including the inpainting at the per-panorama rate of Appendix A.21, the largest region takes 3.0 to 3.3 h against 6.7 to 7.5 h for the single model.

2) More challenging geometry analysis

Thank you for this suggestion. The airborne LiDAR of the government datasets is sparse on facades and curbs, so we used the street-level LiDAR (Argoverse 2) of the three newly added independent sites. The table below gives the structural errors by class averaged over the three sites (3DSE / 3DSE-M in metres):

MethodRoadSidewalkCurbFacade
Vid2Sim0.274 / 0.2060.435 / 0.2850.827 / 0.5601.759 / 0.357
Ours0.118 / 0.0700.177 / 0.0720.425 / 0.1340.681 / 0.249

Our model has the lowest error in every class and on both metrics. On slopes, most ground at these sites is gentle (87% of the ground cells are below 4 degrees, and the steepest reaches 15 degrees). The median error of the ground slope is 1.4 degrees for ours against 1.9 degrees for Vid2Sim and 2.5 to 4.5 degrees for the other baselines, and on the steeper cells (4 degrees and more) it is 1.8 degrees for ours against 2.3 to 4.5 degrees for baselines.

The share of test views whose 3DSE-M exceeds 0.5 m is 2.5% for ours against 5.0% for Vid2Sim and at least 26% for the other baselines on the held-out views of the five cities. At the Argoverse sites, it is 3.7% for ours against 20% for Vid2Sim and at least 26% for the other baselines. For test views whose 3DSE-M exceeds 0.25 m, the shares are 14.4% against 30.5% and at least 64% on the held-out views, and 20.8% against 54.4% and at least 64% at the Argoverse sites.

Reviewer_ZtSYOpenReview ↗

I will remain my review

Thank you for the detailed responses and additional experiments. I have read the replies to both my questions and those of the other reviewers, and I think the new evidence addresses the main concerns.

The fully held-out test clarifies that the test panoramas are excluded from training reconstruction and used only in a separate process to estimate evaluation poses. The independent Argoverse evaluation and the geometry results for different structural categories strengthen the evidence for the reported improvements. I also appreciate the runtime breakdown and failure analysis, which give a clearer picture of when the method is useful and what it costs to run.

What I value most is that the paper addresses an underexplored problem with practical applications and supports its approach with careful experiments. The figures and writing make the method and results easy to follow. The dependence on sufficient street-view coverage and preprocessing remains a limitation, but the authors have made these requirements clearer, and I still find the overall contribution convincing.

I maintain my rating of 4

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers (1/2)

Dear reviewers,

We sincerely thank you for the time and effort you took in reviewing this paper and for your constructive feedback. In particular, we appreciate the recognition of the importance and usefulness of the problem we study (v8Kd, yDD4), of the problem formulation and insights (ZtSY, v8Kd), and of the overall improvements over the baselines (ZtSY, v8Kd, yDD4). We address the concerns point by point below. The corresponding changes are marked in blue in the uploaded PDF.

General problems

1) Evaluation views not held out from training (Reviewers ZtSY, v8Kd, yDD4)

We thank the reviewers for pointing out this problem. We conducted full held-out experiments in all five cities, in which the test panoramas are excluded from the SfM used for training, from Gaussian initialization, from the structural supervision, and from 3DGS optimization. The exclusion is for both our method and for every baseline. We also added a new metric, 3DSE-M. It takes the median depth along each ray in place of the expected depth and better measures where the surfaces are located.

Table 4 and Appendix A.4 now report these results (renderings in Figure 7), and the five-city means are in the table below. Under this protocol, our method still gives the best PSNR, 3DSE, and 3DSE-M of all trained methods, with 3DSE and 3DSE-M lower than those of the baselines by 26 to 90% and 31 to 93%. Google Earth is listed as a reference system, as in Table 3.

MethodPSNRSSIMLPIPS3DSE (m)3DSE-M (m)
Vanilla 3DGS14.740.5080.5120.9691.001
Scaffold-GS15.050.5160.4990.7330.543
WildGaussians (stabilized)15.310.4920.6490.8390.573
Vid2Sim15.140.5020.4470.2600.195
ChronoGS14.640.4900.5340.6030.319
SPARS3R14.420.4980.5321.1021.088
OmniGS14.130.4900.5671.3291.451
InstantSplat (stabilized)10.050.3910.8551.9401.958
Google Earth (reference)11.500.3880.6880.1290.122
Ours (unconditioned)15.490.5150.4540.1920.135
Ours (conditioned)15.750.5170.4520.1920.135

Moreover, to show that the method applies to other locations and to test it against independent data, we used GSV to reconstruct three additional scenes presented in the Argoverse 2 dataset (Pittsburgh, Miami, and Washington DC). We compared our GSV-based reconstruction against the vehicle camera images and street-level LiDAR of that dataset, which were never used in training (Table 5 and Appendix A.25, renderings in Figure 13). These logs give independent held-out views at other camera positions and much denser LiDAR on facades and curbs. The three-site means are in the table below. Our method has the lowest structural error of all trained methods at all three sites. On the three-site mean, its 3DSE and 3DSE-M are lower than those of the baselines by 46 to 95% and 49 to 97%, and its PSNR and LPIPS are the best. Note that the Argoverse cameras differ from Street View in color response, which lowers the image scores of all methods. The table has only Ours (unconditioned), because the Argoverse photographs do not belong to any Street View capture session. It is still trained with the session appearance adapter, so its structure benefits from the adapter. Notably, on 3DSE-M, which measures where the surfaces are located, our error (0.166 m) is slightly lower than that of the Google Earth mesh (0.177 m) at each of the three sites.

MethodPSNRSSIMLPIPS3DSE (m)3DSE-M (m)
Vanilla 3DGS13.880.4800.5212.1331.134
Scaffold-GS14.400.4720.4821.3940.377
WildGaussians (stabilized)14.400.5140.6621.0100.387
Vid2Sim14.440.4280.4740.6200.324
ChronoGS12.200.4200.5471.3210.326
SPARS3R12.750.4350.6363.5163.079
OmniGS13.720.5010.5612.6702.138
InstantSplat (stabilized)9.260.3770.9446.2986.331
Google Earth (reference)13.010.3960.6430.1770.177
Ours (unconditioned)14.820.4560.4620.3370.166
AuthorsOpenReview ↗

General Response to All Reviewers (2/2)

2) Experiment scale vs. world-scale claim (Reviewers v8Kd, yDD4)

We have revised the claims in the paper to be more constrained. The abstract no longer says that reconstruction from GSV “would enable worldwide 3D digital coverage”. It now says that it “would make a 3D model available wherever enough panoramas exist”. In addition, the evaluation now covers eight cities instead of five. The three new sites overlap with the Argoverse 2 dataset, which adds independent photographs and LiDAR to the analysis (Table 5 and Appendix A.25).

Across our eight sites, the number of panoramas ranges from 78 to 192 and the number of sessions from 7 to 24 (Table 2), and at every site our method has a lower 3DSE than all trained baselines (Tables 7 and 33). Since our method is an optimization pipeline, it has the benefit of flexibility but also requires a basic amount of data. To our understanding, the data is not only about density but also about alignment and inconsistency across sessions, which is the focus of our paper. A systematic study of how coverage affects the quality is left to future work, and Appendix A.1 now lists this as a limitation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.