- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This papers proposes a pipeline for camera-controlled autoregressive video generation. It uses an off-the-shelf metric depth estimation model to estimate the depth for the first frame, propagates the estimated geometry to the rest of the frames, and warp the rest of the frames. The camera-controlled problem is therefore reframed as a video inpainting task using a first AR. It further uses another AR to fix the inconsistency between frames, and a third AR to interpolate between the refined frames to recover the original frame rate. The results show that the proposed pipeline can achieve good camera control compared to the baselines.
Critical strengths
- The paper is easy to read, and the motivation of the every proposed component is easy to follow.
- The proposed method achieves good camera control compared to the baseline models.
Critical weaknesses
- I have concerns over the efficiency of the pipeline. It seems that WarpAR is doing the heavy lift of the camera condition part for the video generation, but with limited accuracy since only the key frame has explicit geometry, and the rest of the frames are inpainted without geometry-guidance. This leads to downstream ARs to fix the error. Each AR has the same architecture, and therefore the same computational cost, as the main WarpAR. This makes me wonder if the WarpAR can be substantially improved, so that the subsequent consistency fixes do not have to be heavy. Also, multiple components could potentially make the system fragile if upstream AR fails.
- The geometry part is inferred with an off-the-shelf metric depth model, which could bring another source of error if its estimation is not accurate. Besides, it brings me concern that if it's fair since other baselines do not have the off-the-shelf metric depth model as source of input.
Questions for authors
- Could the authors provide a robustness analysis of how often each component fails, and if the subsequent component can recover the error or not?
- Could you propose potential solutions to reduce the error for the WarpAR estimation itself before any downstream fixes?
3 comments
Reply to the authors
Thank the authors for providing the rebuttal. While I don't have other questions for the authors, I believe this paper achieves good camera control while having "fundamental limitations of geometry-based warping", acknowledge by reviewer 7vzY as well. I will remain my score as Weak Accept.
Thank you again for your constructive feedback and for taking the time to consider our responses.
Official Comment by Authors
Q1: How robust is each component, can downstream stages recover upstream errors, and how efficient is the pipeline?
WanAR (referred to as WarpAR in the review) provides camera accuracy. In the downstream stages, FixAR addresses visual drift accumulated during WanAR rollouts while preserving camera accuracy, and InterpolatAR restores the original frame rate. The newly added camera metrics in revised Tables 1 and 2 further support accurate camera control across both datasets. Revised Table 3 ablation demonstrates the robustness of downstream FixAR through five runs with different seeds: camera metrics remain similar, while mean FID (36.11 → 33.88), KID (0.0022 → 0.0017), and FVD (451.21 → 423.86) consistently improve. It also shows that our FixAR outperforms SeedVR2, a state-of-the-art diffusion-based video restoration baseline, across all reported metrics.
Regarding pipeline efficiency, our full pipeline has moderate runtime and GPU memory usage, comparable to many evaluated baselines (Appendix .2, Table 5).
Q2: How could errors in WanAR be reduced before downstream correction?
We agree that reducing visual drift within WanAR itself is a promising direction. Potential solutions include scaling its model capacity and training data, or adapting diffusion-forcing or self-forcing strategies to improve robustness to imperfect generated histories. These approaches involve tradeoffs in training cost and train-test distribution mismatch, as discussed in “Autoregressive long video generation” in Related Work.
Q3: Is using an off-the-shelf depth estimator fair to the baselines?
Many baselines, including ViewCrafter, TrajectoryCrafter, CogNVS, and VACE, also use off-the-shelf depth estimators in our evaluation. Appendix Table 7 further shows that changing the depth estimator has little effect in our ablation, supporting the fairness of the comparison and limited sensitivity to the particular estimator.