- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RoGe, a unified reconstruction-generation framework for novel view synthesis (NVS) from sparse posed images. The method builds an implicit scene representation using a camera-conditioned VGGT, and queries its scene tokens with target camera rays to obtain per-view geometric features. These features are injected into a pretrained video diffusion model, together with context image latents and Plücker ray maps. The reconstruction and generation modules are jointly optimized with flow matching and photometric losses, without introducing an explicit 3D intermediate representation.
Experiments on RE10K, DL3DV, and ACID demonstrate improvements over reconstruction-based, generation-based, and hybrid NVS baselines, particularly in image quality and video-level fidelity. Additional experiments evaluate trajectory extrapolation on ScanNet, while ablations investigate the effectiveness of ray-queried geometric features and joint training.
Critical strengths
- Effective use of implicit geometric features for video generation.
RoGe directly conditions video generation on target-view-aligned geometric features, avoiding explicit 3D reconstruction or rendered RGB intermediates. The ablations in Table 5 demonstrate that ray-queried geometric features provide more effective conditioning than raw VGGT tokens or rendered/predicted RGB maps. This supports the effectiveness of the proposed integration.
- Good empirical performance.
RoGe achieves consistent improvements in image-level metrics over strong reconstruction and generation baselines across three datasets, including the unseen ACID dataset. The video-level evaluations also show competitive geometric consistency and camera controllability. These results demonstrate the practical effectiveness of the proposed framework.
Critical weaknesses
- Limited methodological novelty and insufficient differentiation from prior work.
The proposed framework primarily integrates existing reconstruction and generation techniques. In particular, LagerNVS already extracts target-view-aligned features from implicit VGGT scene tokens through ray–scene attention, while related work such as Gen3R (CVPR 2026) has explored tight integration between reconstruction and diffusion. The main contribution of RoGe appears to be adapting these techniques to condition a pretrained video diffusion model, rather than introducing a substantially new reconstruction-generation mechanism. Moreover, the closely related Gen3R is not discussed, making it difficult to establish the novelty and significance of the proposed integration.
- Insufficient evidence for the significance of the proposed integration.
Although RoGe achieves strong quantitative results, the ablations in Table 5 show that end-to-end joint training provides only a modest improvement (20.07 to 20.40 PSNR), while subsequent higher-resolution fine-tuning contributes a larger gain. This raises questions about the relative importance of joint optimization compared with the strong pretrained reconstruction and generation backbones. Overall, the experiments demonstrate improved performance but do not sufficiently establish a substantial methodological advance over existing reconstruction-generation approaches.
Questions for authors
See weaknesses.