- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes UniGenStereo, a unified framework for stereo video generation that supports 3 geometry-conditioning settings: dense disparity (G0), camera baseline only (G1), and monocular input without geometric information (G2). Built upon Wan2.1, the method introduces a geometry head to predict g=f/Z, which is used to construct warp-based conditioning for right-view generation. A two-stage training strategy is proposed to jointly optimize geometry prediction and video generation through differentiable warping. Experiments demonstrate competitive performance across all three settings.
Critical strengths
-
The framework supports different levels of available geometric information within a single model, providing flexibility for various stereo video generation scenarios.
-
The proposed Differentiable Self-Warp enables end-to-end optimization of geometry prediction and video generation, which is a meaningful design.
Critical weaknesses
My main concern is that most key claims of the manuscript are not sufficiently supported by ablation experiments.
- Table 4 mainly demonstrates the effectiveness of warped latents as conditioning inputs, which is relatively intuitive and commonly adopted in geometry-guided generation. Table 5 only shows the overall benefit of Stage 2. Neither directly validates the effectiveness of joint optimization through Differentiable Self-Warp, which is a central component of the proposed framework. Therefore, it remains unclear whether this key design actually contributes to the performance improvements.
The paper emphasizes unified conditioning across G0/G1/G2 and the advantages of predicting (g=f/Z), but lacks controlled ablations demonstrating the benefits of unified training or this specific geometric representation.
Questions for authors
-
According to Appendix A.1, each stage is trained for 3 epochs. Therefore, the comparison in Table 5 may not be entirely fair, as Stage 2 benefits from additional training iterations. If Stage 1 is extended by another 3 epochs, does Stage 2 still provide significant improvements?
-
Stage 2 introduces both Differentiable Self-Warp and additional stereo datasets. Could the authors compare Predicted Warp (Detached) and Predicted Warp (Joint) under identical training conditions to isolate the benefit of backpropagating the generation loss through the warp?
-
Section 3.3 extensively discusses the advantages of predicting (g=f/Z) rather than depth. However, there is no direct experimental evidence showing that this representation is more beneficial for downstream video generation. Could the authors provide a controlled comparison between predicting (f/Z) and directly predicting depth (Z), using the same architecture and training settings, to substantiate this design choice?
9 comments
Dear Reviewer rcRx,
As the discussion period ends soon, we wanted to check whether the controlled ablations in our response — in particular the matched-budget comparison (Table W1aQ1), the detached-versus-joint comparison (Table W1bQ2), and the f/Z-versus-1/Z comparison (Table W1cQ3) — address your concerns. If any point remains unclear, we would be glad to respond before the deadline. Thank you again for the careful review and valuable feedback.
Thank you for the additional experiments. However, it seems that an important ablation is still missed to demonstrate the benefit of joint optimization over the detached variant. Could the authors compare Detached + External Stereo Data against variant D?
Response to Reviewer rcRx
We appreciate the reviewer’s recognition of the flexibility of our three inference settings and the role of Differentiable Self-Warp in jointly optimizing geometry prediction and video generation. We thank the reviewer for the careful reading, valuable feedback, and the constructive suggestions. In response to the suggestions, we added experiments and analysis. Our point-by-point responses follow.
W1a & Q1. Continuing Stage 1 under a matched training budget - Stage 2 with external stereo data improves PSNR on both benchmarks.
Variants A–D start from the same Stage 1 checkpoint and each train for 7,000 additional iterations. A continues the Stage 1 training protocol with ground-truth warps and the original training data. We evaluate all rows under G1 on the same 50 randomly sampled videos from StereoGenBench (SGB) and 50 from SVD.
Continuing Stage 1 alone raises PSNR by 0.90 dB on SGB and 0.19 dB on SVD. At the same training budget and with the same data as A, joint training with predicted warps (C) raises SGB PSNR by another 0.78 dB. The variant that adds external stereo data for geometry-only supervision (D) improves PSNR over A by 0.28 dB on SGB and 1.04 dB on SVD.
Table W1aQ1. Matched-budget Stage 2 ablation under G1
W1b & Q2. Predicted Warp (Detached) versus Predicted Warp (Joint) - Backpropagating through the warp improves all three StereoGenBench metrics.
B and C use the same training data, predicted-warp schedule, and 7,000-iteration budget. In B, we detach the predicted geometry used to build the conditioning warp. In C, the generation loss backpropagates through the differentiable warp into the geometry head.
Enabling that gradient raises SGB PSNR from 26.15 to 26.54 dB and lowers LPIPS from 0.115 to 0.113 (Table W1bQ2). SVD PSNR changes from 18.26 to 18.22 dB. From C to D, SVD PSNR rises by 1.00 dB and LPIPS drops from 0.306 to 0.261 (Table W1aQ1). The improvement on SVD shows that adding external stereo data helps the model generalize to real-world captures held out from training.
Table W1bQ2. Effect of backpropagating through the predicted warp under G1
We will add these ablation results in the final version.
W1c & Q3. Direct f/Z prediction versus inverse depth - Direct prediction improves geometry and right-view generation.
We trained both variants from scratch with the same architecture and training settings, changing only the prediction target. One predicts g = f/Z directly, and the depth-based variant predicts inverse depth 1/Z. We use inverse depth, a common parameterization in monocular depth estimation, because it maps points at infinity to zero. We multiplied its prediction by the ground-truth focal length f at evaluation to obtain g. We evaluate geometry on the full StereoGenBench test set and evaluate right-view generation under G1 on the same 50 videos randomly sampled from that test set for both variants.
Direct prediction reduces geometry MAE by 18.3% and raises δ1 by 11.18 percentage points. At a fixed baseline B, the warp's disparity error scales with the error in g through
D = B × g. On the 50-video subset, PSNR improves by 2.62 dB, SSIM rises from 0.852 to 0.891, and LPIPS drops from 0.079 to 0.058 (Table W1cQ3).Table W1cQ3. Geometry prediction and right-view generation
We will also include this analysis in our final paper.
The benefit of unified conditioning is that a single checkpoint operates under G0 (disparity), G1 (baseline), or G2 (monocular input) depending on the geometry available at inference, while achieving state-of-the-art results in all three settings on both SGB and SVD (Tables 2 and 3).
We hope these results address the reviewer’s concerns and would be glad to clarify any remaining points during the discussion period.
[1] FoundationStereo: Zero-Shot Stereo Matching CVPR 2025
[2] Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding, CVPR 2026