- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper’s goal is world-consistent novel view synthesis from sparse images, which requires both faithfully reconstructing observed regions and plausibly completing unseen ones. Geometry-based methods preserve structure but cannot generate missing content, while video diffusion models offer rich appearance priors but drift across views. GeoVerse addresses this by performing diffusion in the geometric latent space of a frozen 3D foundation model (DA3). It injects appearance priors from a frozen video model (Wan2.2 VACE) through a ControlNet-style adapter. It also maintains a persistent point-cloud spatial memory, re-projected into each target view as RGB-D guidance, to keep multi-round generation consistent. The work shows that video priors can enhance geometric latent generation without adopting a sequential video pipeline, offering an efficient path toward world-consistent scene synthesis.
Critical strengths
Consistent gains across in-domain and out-of-domain benchmarks. GeoVerse outperforms a broad set of generative NVS baselines on RealEstate10K, DL3DV, Mip-NeRF 360 and ScanNetv2, with clear improvements over GLD in both visual quality and pose accuracy (e.g., +2.45 dB PSNR and 32.4% lower ATE on Mip-NeRF 360). This is consequential to my assessment because the gains hold on out-of-domain data, which suggests that enriching the geometric latent space with video priors improves generalization rather than fitting the training distribution.
Spatial memory that extends single-pass consistency to multiple rounds. The persistent point-cloud memory re-projects accumulated content into each target view as RGB-D guidance with a validity mask, anchoring new predictions to a shared scene representation. This matters because drift in sequential view generation is a key weakness of prior video-based approaches. The ablation shows that removing the memory clearly degrades both fidelity and cross-view alignment, and the long-sequence results on ScanNetv2 (15.33 → 19.65 dB PSNR over GLD) support its effectiveness.
Critical weaknesses
1. Limited novelty, and the contribution of the video prior is not fully isolated. Most of the pipeline is built from existing components: the GLD backbone, a point-cloud memory similar to ViewCrafter, a ControlNet-style adapter, and piecewise reflow distillation. The main new idea is injecting single-pass Wan2.2 features into the geometric latent space. The ablation in Table 3 suggests that this prior is the most important component, but it is reported only on ScanNetv2, and the paper does not state whether the ablated variants were trained on the same 15 datasets. Since the training data was also expanded from the 4 datasets used by GLD, I wonder how much of the overall gain over GLD comes from the video prior and how much from the larger training set. This matters because the video prior is the core contribution, and a clearer isolation of its effect would make the claim much more convincing.
2. Mixed evidence for world consistency. GeoVerse achieves the best ATE and MEt3R across benchmarks, which is a clear strength. However, on reprojection error it is behind some baselines on several datasets, and the gains on cross-view consistency metrics are generally smaller than those on image quality. Given that world consistency is the headline claim, I wonder whether the improvements mainly reflect better per-view image quality rather than better 3D consistency. A short discussion of why the reprojection error does not improve would help clarify this.
Questions for authors
Ablation settings. Were the variants in Table 3, especially w/o Wan2.2 prior, trained on the same 15 datasets and with the same distillation as the full model? If possible, could you also report the main ablations on one more benchmark, such as DL3DV or Mip-NeRF 360? Does the Wan prior help in unobserved regions? Could you report metrics separately on the valid (projected) and invalid (unprojected) target regions, with and without the Wan prior? This would directly support the claim that video features help complete unseen content. Choice of extraction timestep. How was t = 500 chosen for Wan feature extraction? A brief explanation, or a small sweep over a few timesteps, would help show how sensitive the results are to this choice. Source of the speedup. The speedup over GLD seems to come mainly from reflow distillation and the one-step L0 cascade, which could also be applied to GLD. Could you provide a per-component runtime breakdown and the peak GPU memory, and comment on how GLD would perform with the same few-step sampling?
2 comments
Q4.1 :
Table 8 provides a per-component runtime breakdown, recording a peak GPU memory of 23.03 GiB on a single NVIDIA A800 80GB GPU using two context images and six target views at 504×504 resolution.
Table 8. Detailed breakdown of CUDA-synchronized model-inference latency.
| Model inference stage | Time (s) |
|---|---|
| Context DA3 geometry | 0.32 |
| Multi-level feature encoding | 0.55 |
| Wan feature extraction | 0.25 |
| L1 generation | 5.94 |
| Cascade + decode | 2.17 |
| Total model inference | 9.23 |
Q4.2 :
To quantify the effect of distillation, we directly applied 3+1 updates to pretrained GLD weights without trajectory distillation. Table 9 shows that full-image quality degrades across all four datasets, while Appendix Fig. 11 further exposes clear losses in newly synthesized detail. Therefore, merely shortening the sampling schedule fails to reproduce the distilled model's joint image and geometry behavior. These results confirm the necessity of explicitly training the geometric generator for the target few-step trajectory.
Table 9. Quantitative comparison of GLD performance using the sampling configuration reported in its paper (50/50, 49+49=98 denoising steps) and few-step sampling (4/2, 3+1=4 denoising steps).
| Dataset | Points | PSNR ↑ | SSIM ↑ | LPIPS ↓ | ATE ↓ | RPEr (°) ↓ | RPEt ↓ | Reproj. ↓ | MEt3R ↓ |
|---|---|---|---|---|---|---|---|---|---|
| DL3DV | 50/50 | 17.38 | 0.546 | 0.383 | 0.058 | 1.538 | 0.125 | 0.652 | 0.262 |
| DL3DV | 4/2 | 17.88 | 0.571 | 0.368 | 0.112 | 3.717 | 0.164 | 0.637 | 0.263 |
| RE10K | 50/50 | 19.66 | 0.709 | 0.299 | 0.037 | 1.338 | 0.067 | 0.608 | 0.183 |
| RE10K | 4/2 | 20.13 | 0.738 | 0.283 | 0.040 | 1.259 | 0.074 | 0.601 | 0.178 |
| Mip-NeRF360 | 50/50 | 17.57 | 0.440 | 0.406 | 0.105 | 2.337 | 0.202 | 0.621 | 0.251 |
| Mip-NeRF360 | 4/2 | 18.10 | 0.462 | 0.394 | 0.345 | 12.276 | 0.727 | 0.602 | 0.261 |
| ScanNetv2 | 50/50 | 15.33 | 0.664 | 0.491 | 0.011 | 0.396 | 0.019 | 0.598 | 0.108 |
| ScanNetv2 | 4/2 | 15.84 | 0.681 | 0.470 | 0.025 | 0.916 | 0.038 | 0.600 | 0.124 |
We sincerely thank Reviewer u7eS for the careful assessment of novelty, fair video-prior attribution, geometric consistency, cross-dataset ablations, and computational cost. The comments encouraged us to provide clearer component attribution, analyze reprojection-error limitations, and add detailed runtime, memory, and few-step diagnostics. Below are our point-by-point responses:
W1 / Q1 :
GeoVerse integrates large-scale geometric-latent modeling, frozen video-prior transfer, persistent spatial memory, and few-step distillation to jointly synthesize RGB and geometry rather than merely refining an explicit 3D reconstruction.
Across ablation variants trained on the same 15-dataset pool, scaling training data improves both in-domain and out-of-domain performance, while incorporating the Wan prior yields an even larger gain. This improvement is especially pronounced on the out-of-domain ScanNetv2 dataset, where multi-round generation progressively exposes unobserved regions and heightens reliance on a strong generative prior. Table 6 further disentangles the contributions of the video prior, target hints, injection design, and training losses on DL3DV.
Table 6. Ablation comparison on the DL3DV dataset.
W2 :
We agree that reprojection error does not uniformly favor GeoVerse. Its evaluation estimates geometry and cameras from the generated images, so the residual also depends on appearance and correspondence support. Smaller realized viewpoint changes or smooth, low-texture outputs can yield lower residuals without more faithful target-view synthesis. Appendix Fig. 10 illustrates these visual confounds.
Q2 :
Among reference-valid target pixels, initially observed reconstruction regions and unobserved generation regions account for 63.19% / 36.81% on DL3DV, 75.40% / 24.60% on RE10K, and 62.32% / 37.68% on Mip-NeRF360, respectively. To reflect both components, we evaluate reconstruction fidelity with PSNR/SSIM and generation quality with FID/KID, as shown in Table 7. Comparing variants with and without the Wan prior further indicates that video features improve both novel-content synthesis and reconstruction fidelity through appearance and viewpoint cues. This region-wise evaluation more clearly separates reconstruction from generation capability, and GeoVerse achieves the strongest performance in both aspects across all three datasets.
Table 7. Quantitative comparison of fidelity in reconstructed regions and quality in generated regions. Recon. reports PSNR ↑ / SSIM ↑; Gen. reports FID ↓ / KID ↓.
Q3 :
We adopt t=500 as a fixed intermediate timestep for extracting video-prior features. Because the input combines reliable projected content with unobserved regions requiring synthesis, an intermediate conditioning level naturally balances content preservation and generative completion. Preliminary experiments indicated low sensitivity around this value, so we kept this setting fixed across all subsequent runs.