- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SplatAnywhere, a 3D Gaussian Splatting framework for reconstructing street scenes from sparse and temporally inconsistent Google Street View panoramas. It combines structural supervision, consensus-gated densification, and session-aware appearance modeling. Experiments across five cities demonstrate improved rendering quality and substantial reductions in 3D structural errors compared with existing 3DGS methods.
Critical strengths
-
The paper addresses the practical challenge of reconstructing 3D street scenes from sparse and inconsistent real-world observations. The reported improvements in geometric accuracy are substantial, especially for extrapolated viewpoints.
-
The paper includes comparisons with multiple baselines, component ablations, and evaluations using airborne LiDAR as geometric reference. The consensus-gated densification strategy is particularly effective in reducing the number of Gaussians while improving geometric consistency.
Critical weaknesses
-
The proposed framework combines several existing techniques, including SfM-based depth supervision, monocular depth priors, and appearance embeddings. Although the integration is effective, the individual components are mostly incremental. The novelty of the overall framework could be better justified.
-
Despite the worldwide reconstruction motivation, experiments cover only five relatively small street scenes. Furthermore, the main evaluation derives test views from panoramas already used for training. Although Appendix A.4 provides a held-out panorama experiment, it is limited to London, and camera poses are still estimated using all panoramas.
Questions for authors
- What is the main methodological contribution beyond combining existing geometry supervision and appearance modeling techniques?
- How does the method perform when both the test panoramas and their observations are excluded from the SfM reconstruction?
2 comments
The reviewers have resolved most of my questions and doubts. I have decided to maintain my score ( 3: Weak acceptance)
We sincerely thank you for your feedback and questions. In the general response above, we have toned down the worldwide claim, and conducted additional experiments to cover unseen new camera positions. Below, we respond to additional comments.
1) Novelty
Thank you for the careful reading. Our main methodological contribution is the analysis of why reconstruction from Street View fails, together with a set of mechanisms designed against that failure.
First, the problem. To our knowledge this is the first work that reconstructs 3DGS street scenes from public Street View panoramas alone, without dedicated capture or LiDAR. Section 3.1 shows that its two difficulties are coupled. The appearance differs between capture sessions and the cameras lie on a narrow strip of road, so the optimizer can lower the photometric error by moving geometry. Reviewers ZtSY and v8Kd also list this formulation and analysis among the strengths of the paper. Under this coupling, existing techniques are not sufficient. When the baselines that use monocular depth supervision (Vid2Sim), per-image appearance embeddings (WildGaussians), and time conditioning (ChronoGS) are trained with our SfM and inpainted crops, their structural error stays 1.6 to 9 times as large as ours (Appendix A.5).
Second, the mechanisms. Each one closes one route by which this coupled error enters, and three of them depart from their closest prior work.
(a) Monocular depth. Vid2Sim supervises the rendered depth and normals with monocular predictions. We take the structure from multi-view evidence (SfM points and a road grid) and use the monocular depth only as a one-sided bound on free space, never as a depth target, because it is relative and unreliable at object boundaries. In an additional London run, replacing the bound with a depth target raises the 3DSE from 0.202 to 0.640 m, as shown in the table below.
The first row is our model. The second row replaces the one-sided bound with a depth target, which supervises the rendered depth on both sides. Its 3DSE is 3.2 times higher and its PSNR is 1.6 dB lower. The third row keeps the bound and adds the depth target on top. The monocular depth therefore helps as a conservative bound on free space, and using it as a depth target harms the geometry.
(b) Appearance. Per-image appearance models, such as the embeddings of WildGaussians, usually give every image its own parameters. That is safe when many viewpoints fix the geometry, but under Street View these parameters can also absorb residuals that should correct the geometry. Our adapter follows Eq. 1 instead. It has one code per capture session, which is enough to take up the between-session term E_sess, and a penalty keeps it near the identity. It is designed so that the optimizer does not have to explain session differences with geometry, and in the ablation it lowers the 3DSE by 9 to 25% (Table 6), and in every view group of the four cities tested (Table 21). Since the code depends only on the session, it also needs no pixels of the test view and applies at new camera positions (Section 3.6).
(c) Densification. Consensus-gated densification splits less where the gradients of different crops disagree, the opposite of GDAGS, which splits more under conflict. Replacing our gate with GDAGS doubles the model (2.20 M against 1.11 M Gaussians) and raises the London 3DSE from 0.202 to 0.256 m at similar image quality (Appendix A.14).
Third, the mechanisms are needed together, which is why the paper presents a complete pipeline. SfM depth anchors are the one standard ingredient, and on their own they are not enough. With them the London 3DSE is still 0.544 m, because SfM points concentrate on textured facades and leave the road almost unsupervised. The ray clearance and the road grid, which the baselines have no counterpart for, lower it to 0.464 and then 0.202 m (Table 6). The consensus gate is equally necessary in the full model: removing it raises the 3DSE by 44 to 89% and the model from 1.11 M to 2.49 M Gaussians (Appendix A.14). The session adapter lowers the 3DSE as well, from 1.348 to 1.211 m on Interpolation and from 1.055 to 0.789 m on Pitch Shift (Table 6).
Empirically, on eight scenes with independent LiDAR (five cities and three Argoverse sites), our method has the lowest structural error of all eight baselines on every mean, with 3DSE lower by 26 to 95%, and the highest PSNR on the held-out views and at the Argoverse sites.