- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GeoDYN, a framework is to use 4DGS to establish geometric correspondences and transport learned features across viewpoints and time, instead of directly conditioning video diffusion on potentially corrupted 4DGS-rendered images.
Critical strengths
-
This paper proposes a new approach that uses input video features warped through 4DGS as conditioning for video diffusion, rather than directly using 4DGS-rendered images. The experimental results demonstrate the effectiveness of this approach.
-
The motivation is interesting, as 4DGS rendering can introduce ghosting and noticeable reconstruction artifacts. I think this is an important problem worth investigating.
Critical weaknesses
-
The paper lacks a controlled ablation study to demonstrate that feature warping is better than using 4DGS-rendered images as conditioning. I suggest replacing the warped features with VAE-encoded features from 4DGS renderings while keeping the same network architecture, training data, and fine-tuning settings. This would provide a more direct comparison between the two conditioning strategies.
-
The feature warping process is not clearly explained, particularly in Figure 3. Based on my understanding of Equation (2), (F_{\mathrm{src}}^i(\tau,p)) represents features from the source frame at timestamp (\tau). If so, how does the method maintain view consistency under large viewpoint changes, where some regions are not visible in the corresponding source frame? It is unclear how the proposed pipeline handles this limitation.
-
The results of the full model in the ablation study do not match those reported in the main comparison table. I would expect the full model to achieve the same performance in both tables. Could the authors clarify the difference in experimental settings or explain this discrepancy?
Questions for authors
-
Diffusion model features are not necessarily 3D consistent, which is one reason why most existing methods use rendered images or point cloud rather than features as conditioning. There are any new issues for using features instead of rendering images?
-
I wonder why the better performance in uncovered regions. From my view, the reason is the conditioning in your pipeline is less reliable (Due to the 3D inconsistency feature), causing the model to rely more on diffusion priors rather than reconstruction information. Could you explain the improvement with more details?
-
How is the uncovered region defined in this paper? Does it mean that the region is invisible from all source views? If so, I think evaluating PSNR in these regions is questionable. It should be impossible to accurately recover the ground-truth image in completely unseen regions unless there is some information leakage.