acceptodds
Under review as a conference paper at ICLR 2027

GeoForward: Geometrically Consistent Multi-View Representations for Feedforward 3D Gaussian Splatting

Abstract

Feedforward 3D reconstruction has largely followed an appearance-driven paradigm: multi-view images are encoded into visual representations, cross-view correspondence is implicitly inferred from learned features, and 3D primitives are subsequently predicted from these representations. While effective, this paradigm leaves a fundamental source of information underexploited: calibrated cameras and sparse metric measurements already provide explicit constraints on where the same physical point should appear across views. We propose a novel paradigm, Geometry-Conditioned Feedforward Reconstruction, in which metric geometry establishes cross-view correspondence before the 3D representation is formed, while learned visual features refine rather than discover this correspondence. Based on this paradigm, we introduce GeoForward. Sparse LiDAR points are unprojected into 3D and analytically projected across calibrated cameras to provide metric correspondence anchors. A bounded continuous local search then refines these anchors using visual features, and the resulting geometry-conditioned representation is fused with global multi-view context from VGGT. The shared representation drives separate metric-depth and Gaussian-attribute prediction, establishing a direct path from explicit geometry to both Gaussian centers and spatial attributes. A sparse 3D refinement module subsequently improves local attribute compatibility while preserving the geometrically determined centers. For temporal reconstruction, endpoint Gaussians are composed using vehicle masks and motion priors. On nuScenes, GeoForward achieves the lowest cross-camera depth-consistency error and Chamfer distance among the compared methods while maintaining competitive reconstruction quality. These results demonstrate that explicitly conditioning correspondence formation on metric geometry provides an effective alternative to purely appearance-driven feedforward 3D reconstruction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, 3D Gaussian Splatting, scene reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_apimOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoForward, a geometry-guided feedforward 3D Gaussian reconstruction framework for autonomous driving scenes. It combines LiDAR-based cross-view projection, learned local feature refinement, VGGT representations, separate depth and Gaussian prediction, and sparse 3D attribute refinement. Experiments on nuScenes and Waymo report strong geometric reconstruction performance.

Critical strengths

The paper addresses an important problem in geometrically consistent feedforward 3D reconstruction, and the use of metric observations to guide cross-view feature alignment is well motivated.

The method achieves encouraging geometric accuracy on multiple driving-scene datasets, and the evaluation includes several reconstruction and cross-view consistency metrics.

Critical weaknesses
  1. Limited methodological novelty. The central alignment mechanism mainly combines calibrated LiDAR projection with conventional learned local feature refinement. Other components, including VGGT feature fusion, Gaussian prediction, sparse 3D convolution, and temporal composition, largely follow established approaches. The claimed new reconstruction paradigm is therefore not sufficiently distinguished from prior LiDAR-guided multi-view reconstruction methods.

  2. Potentially unfair comparisons and insufficient ablations. The proposed method uses sparse LiDAR and calibrated cameras at inference, as well as offline dense metric-depth supervision during training. Several competing methods operate under different input and supervision settings. Moreover, the current ablations do not adequately isolate the contributions of geometric projection, local correspondence refinement, LiDAR-token conditioning, and sparse 3D attribute refinement.

  3. Inconsistencies in the reported training methodology. Appendix A.6 explicitly states that the temporal reprojection loss provides no gradient to the model, although the main text presents it as an effective temporal consistency constraint. The checkpoint initialization descriptions for nuScenes and Waymo also require clarification. These issues weaken confidence in the reproducibility and interpretation of the results.

Questions for authors

Can the authors provide input- and supervision-matched comparisons against existing LiDAR-guided reconstruction approaches, along with independent ablations of the major modules?

Why is the temporal reprojection loss included as a training objective when the implementation provides no gradient through this term? What is the exact checkpoint training sequence across nuScenes and Waymo?

Could the authors report results on the complete validation sets and provide inference time, memory consumption, and separate evaluation of dynamic regions? It would also be useful to report rendering quality before the endpoint-specific color calibration.

Reviewer_JUAdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GeoForward, a feed-forward 3D Gaussian Splatting framework for autonomous driving that explicitly establishes cross-view correspondences using sparse LiDAR and calibrated cameras before feature fusion. The geometry-conditioned features are then used to predict Gaussian representations, followed by a lightweight 3D refinement module and temporal composition for dynamic scenes. Experiments on nuScenes and Waymo demonstrate improved geometric consistency and competitive rendering quality.

Critical strengths
  • The motivation is clear and well justified. The paper highlights that existing feed-forward methods mainly rely on appearance-based correspondence, while sparse LiDAR already provides explicit geometric constraints that can simplify cross-view matching.
  • The proposed geometry-guided correspondence is simple and effective. Using analytical projection followed by local refinement is an intuitive design, and the experimental results consistently improve geometric accuracy and cross-view consistency on both nuScenes and Waymo.
Critical weaknesses
  • The novelty is somewhat incremental. The main contribution is introducing geometry earlier in the correspondence pipeline through LiDAR-guided feature alignment, rather than proposing a fundamentally new reconstruction representation or learning framework. The method also relies on strong assumptions, including calibrated cameras, sparse LiDAR, vehicle masks, and motion annotations, which limit its applicability beyond autonomous driving scenarios.
  • The comparisons are not entirely convincing. Most compared methods do not have access to sparse LiDAR during inference, making it difficult to separate the gains from the proposed algorithm itself versus the additional geometric information provided by the input modality.
  • The related work and experimental comparisons could be more comprehensive. The paper does not discuss or compare against several recent representative feed-forward 3D Gaussian Splatting methods, such as YoNoSplat [1], NoPoSplat [2], and SPFSplat [3]. Although these methods are designed for generic sparse-view reconstruction rather than autonomous driving, they represent important advances in feed-forward 3DGS and would help better position the proposed method within the broader literature. Moreover, it would be interesting to discuss whether the proposed geometry-guided correspondence could be integrated into the cross-view attention mechanisms adopted by these methods, potentially combining explicit geometric constraints with learned feature interactions.

[1] Ye, Botao, et al. "Yonosplat: You only need one model for feedforward 3d gaussian splatting." International Conference on Learning Representations. 2026.

[2] Ye, Botao, et al. "No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images." International Conference on Learning Representations. 2025.

[3] Huang, Ranran, and Krystian Mikolajczyk. "No pose at all: Self-supervised pose-free 3d gaussian splatting from sparse views." IEEE/CVF International Conference on Computer Vision. 2025.

Questions for authors

Mentioned above.

Reviewer_FEYPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes GeoForward, a feed-forward 3DGS framework for driving scenes that uses sparse LiDAR to establish cross-view correspondence before predicting Gaussians. In particular, LiDAR points are projected across calibrated cameras as anchors, refined by a bounded learned offset, and fused with VGGT features. The fused features then predict metric depth and Gaussian attributes. Experiments on nuScenes and Waymo, with zero-shot transfer to KITTI and Argoverse 2, report the best geometric accuracy among the compared methods.

Critical strengths
  • The idea of using the calibrated cameras and sparse LiDAR of driving platforms to anchor cross-view feature association is reasonable and well motivated. It narrows the correspondence search, which is relevant for practical driving systems.

  • Experiments are conducted on two large-scale driving benchmarks, with point-cloud, depth and cross-camera consistency metrics beyond rendering quality. This matters, since feed-forward 3DGS works rarely evaluate geometry directly.

Critical weaknesses
  • The comparisons are not conducted under equivalent inputs. GeoForward takes sparse LiDAR at inference, both as projected anchors and as raw point tokens (App. A.3). In contrast, all feed-forward baselines in Tabs. 1, 2 and 5 are camera-only, and DGGT and StreetForward do not take input poses. Hence, the lower Chamfer distance and depth errors are expected. I am not convinced by this comparison that the gains come from the proposed alignment. OmniRe, which also exploits LiDAR, is already close on nuScenes (CD 1.625 vs. 1.435) with better completeness. Despite the extra input, novel-view PSNR on nuScenes also stays below ReconDrive (23.191 vs. 23.986, Tab. 3). It would be better to give the feed-forward baselines the same LiDAR, e.g., through LiDAR-completed depth. How the reference geometry for evaluation is obtained should also be stated.

  • The main paper and the appendix describe different models and training procedures. Sec. 3.5 states that "The VGGT encoder remains frozen throughout both stages" (L306), yet App. A.1 trains "Low-rank adapters" on it (L642). Sec. 4 says "For Waymo, we start from a nuScenes-trained checkpoint" (L351), while App. A.6 describes the opposite order (L737-738). The temporal reprojection constraint of Sec. 3.5 is also described as having "no gradient with respect to the model parameters" (L759). A depth adapter and a FiLM color head, which likely contribute to the 36.615 dB input-view PSNR, only appear in the appendix. Hence, it is unclear which model configuration produced the results in Tabs. 1-5. It is highly recommended to reconcile the method description with the actual implementation.

Questions for authors
  1. The row without LiDAR alignment in Tab. 4 reproduces the 0% LiDAR point of Fig. 5 exactly (PSNR 22.189, RMSE 7.034). Does this variant remove all LiDAR input, including the depth prior and point tokens? If so, the claim that metric-guided correspondence "improves both rendering quality and depth accuracy" (L431-432) is not isolated. An ablation that keeps the LiDAR depth prior and removes only the alignment would help.

  2. How are the "Offline dense metric-depth targets" (L639) built? Do they share LiDAR sweeps with the sparse input LiDAR at the same timestamps?

  3. Why does the main nuScenes comparison use only 14 validation scenes (L771)? The full-validation audit mentioned in App. B is not reported, and those results would make the comparison more reliable.

  4. The abstract presents "a novel paradigm" (L037) of geometry-conditioned reconstruction. However, conditioning depth and multi-view geometry models on sparse depth or LiDAR prompts is already a common practice in recent work. It would be better to discuss this line of work and clarify what is new beyond the cross-view anchoring.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.