- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a new method, namely View4D, for generating synchronized multi-view video from an input video or a text prompt, along with camera poses. The proposed method adds a multi-view attention module into a pre-trained video diffusion model, while per-frame camera information is injected into the multi-view attention as relative positional encoding. A module named explicit geometric guidance is presented that uses depth maps to first obtain warped target frames and masks, and then fuse them with the original latents via a trainable encoder. In addition, the paper leverages both multi-view videos and multi-view images to train the video diffusion model. Experiments are conducted to show the effectiveness of the proposed method.
Critical strengths
-
The paper explores an important and interesting problem that generates multi-view videos given moving camera trajectories.
-
The paper is well organized and easy to follow.
-
Experimental results show that the proposed method outperforms baselines in terms of geometric consistency and video quality.
Critical weaknesses
-
The proposed View4D is trained on a newly constructed dataset, while the baselines are not trained or finetuned on the same data. Moreover, it seems that the test data differs from that used in baseline methods. Although the authors provide additional ablation studies in Table 4 to analyze the contributions of the training data and proposed architecture, the main comparisons are still based on models trained with different datasets. It remains unclear how much of the performance improvement comes from the proposed architecture rather than the additional training data. A comparison with baselines trained on the same data would make the results more convincing.
-
As shown in the ablation results, the new dataset contributes significantly to the performance improvements. Although the authors provide additional details, more information about the dataset construction process, scene diversity, and data distribution would help readers better understand the dataset and reproduce the results. In addition, publicly releasing the dataset would be a valuable contribution to the community and facilitate future research.
Questions for authors
-
The authors provide additional ablation studies in Table 4 to analyze the contributions of training data and architecture. However, the main baselines are still trained on different datasets. Would it be possible to train or finetune representative baselines on the same constructed dataset for a fairer comparison?
-
The proposed method relies on off-the-shelf geometry models such as Pi3 to estimate camera parameters and depth maps. These estimates may be inaccurate in complex in-the-wild videos, especially those containing fast motion, occlusions, or low-texture regions. Although the authors provide qualitative results with unaligned depth guidance in Figure 11 and evaluate using different geometry estimators in Table 6, a systematic quantitative robustness analysis is still missing. It would be helpful to evaluate how different levels of errors in depth, camera intrinsics, and relative camera configurations affect the generation quality and cross-view consistency.
2 comments
Thank you for the authors' response. As demonstrated in the authors' reply, the dataset itself is a valuable contribution. I encourage the authors to include more details about the dataset construction in the final version and hope they will release the dataset in the future. I will maintain my weak accept score.
Thank you for your detailed comments and insight suggestions. We will revise our paper based on your feedback. Here are our responses to your comments:
1. Comparison with baseline trained on the same dataset (in response to Weakness 1 and Question 1)
We fine-tune SynCamMaster [1], a synchronized multi-view video generation method, on our dataset with the same backbone and training conditions. The results below highlight the importance of both our dataset and architecture. Training on our dataset produces more realistic and consistent videos, while our architecture is crucial for achieving cross-view camera accuracy and appearance consistency. We will include this comparison in the revised paper.
[1] SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints. ICLR 2025
2. Dataset details and release (in response to Weakness 2)
We thank the reviewer for the suggestions. We provide more details about the dataset below:
We are very willing to release our dataset and to help advance progress in this area. To ensure the reproducibility of our work, we will provide a detailed document describing how to collect and construct the training data. We are actively working through the open-sourcing process. However, since the dataset is relatively large, the full release may require some additional time. In the meantime, we can first provide a subset of the data so that the community can quickly iterate on experiments and verify the correctness of the dataset.
3. Robustness analysis of geometric guidance (in response to Question 2)
The relative camera configurations {T0→j}j=1n are specified by the user rather than estimated by a geometry foundation model, and serve as reference for evaluating cross-view camera accuracy in RotErr-V and TransErr-V.
To assess robustness to errors in estimated depth and camera intrinsics, we separately perturb the depth maps and focal lengths by 5%, 10%, and 20% (Ours_depth_x and Ours_focal_x, where x = 05, 10, or 20). We randomly select 10 video clips and run View4D with the perturbed geometric conditions, comparing all settings against the unperturbed baseline on the same clips. As shown below, our method maintains comparable generation quality and cross-view consistency across the tested perturbation levels, supporting its robustness to errors in depth and camera intrinsics from geometry estimation [2].
[2] π³: Permutation-Equivariant Visual Geometry Learning. ICLR 2026