acceptodds
Under review as a conference paper at ICLR 2027

UniGenStereo: Unified Stereo Video Generation with Flexible Conditioning

Abstract

High-fidelity stereo video generation is crucial for immersive extended reality (XR), robotics, and embodied AI. In practice, the geometric information available at inference time varies widely: dense per-frame depth or disparity is sometimes available, sometimes only the camera baseline is known, and sometimes a monocular clip arrives with neither. Existing methods each commit to one of these cases, and those taking no geometric input cannot control the stereo geometry they produce. We propose UniGenStereo, which synthesizes the right view from a monocular video with one set of weights for all three. Disparity is the camera baseline times the ratio between focal length and metric depth. The ratio does not depend on the baseline, so one prediction serves them all; we predict it from the input video with a lightweight head on the generation backbone. (i) Supplied disparity is used as given and the head is never invoked; (ii) a calibrated baseline scales the predicted ratio; and (iii) with no baseline, a standard interpupillary value stands in. Whatever its source, the disparity warps the input view into the same conditioning channels, so one backbone covers all three and varying the baseline alone controls the synthesized disparity. Experiments show state-of-the-art results in all three settings, including on real-world capture.

Primary area:
generative models
Keywords:
Stereo Video, Video Generation, VR

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_rcRxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniGenStereo, a unified framework for stereo video generation that supports 3 geometry-conditioning settings: dense disparity (G0), camera baseline only (G1), and monocular input without geometric information (G2). Built upon Wan2.1, the method introduces a geometry head to predict g=f/Z, which is used to construct warp-based conditioning for right-view generation. A two-stage training strategy is proposed to jointly optimize geometry prediction and video generation through differentiable warping. Experiments demonstrate competitive performance across all three settings.

Critical strengths
  1. The framework supports different levels of available geometric information within a single model, providing flexibility for various stereo video generation scenarios.

  2. The proposed Differentiable Self-Warp enables end-to-end optimization of geometry prediction and video generation, which is a meaningful design.

Critical weaknesses

My main concern is that most key claims of the manuscript are not sufficiently supported by ablation experiments.

  1. Table 4 mainly demonstrates the effectiveness of warped latents as conditioning inputs, which is relatively intuitive and commonly adopted in geometry-guided generation. Table 5 only shows the overall benefit of Stage 2. Neither directly validates the effectiveness of joint optimization through Differentiable Self-Warp, which is a central component of the proposed framework. Therefore, it remains unclear whether this key design actually contributes to the performance improvements.

The paper emphasizes unified conditioning across G0/G1/G2 and the advantages of predicting (g=f/Z), but lacks controlled ablations demonstrating the benefits of unified training or this specific geometric representation.

Questions for authors
  1. According to Appendix A.1, each stage is trained for 3 epochs. Therefore, the comparison in Table 5 may not be entirely fair, as Stage 2 benefits from additional training iterations. If Stage 1 is extended by another 3 epochs, does Stage 2 still provide significant improvements?

  2. Stage 2 introduces both Differentiable Self-Warp and additional stereo datasets. Could the authors compare Predicted Warp (Detached) and Predicted Warp (Joint) under identical training conditions to isolate the benefit of backpropagating the generation loss through the warp?

  3. Section 3.3 extensively discusses the advantages of predicting (g=f/Z) rather than depth. However, there is no direct experimental evidence showing that this representation is more beneficial for downstream video generation. Could the authors provide a controlled comparison between predicting (f/Z) and directly predicting depth (Z), using the same architecture and training settings, to substantiate this design choice?

9 comments
AuthorsOpenReview ↗

Response to Reviewer rcRx

We appreciate the reviewer’s recognition of the flexibility of our three inference settings and the role of Differentiable Self-Warp in jointly optimizing geometry prediction and video generation. We thank the reviewer for the careful reading, valuable feedback, and the constructive suggestions. In response to the suggestions, we added experiments and analysis. Our point-by-point responses follow.

W1a & Q1. Continuing Stage 1 under a matched training budget - Stage 2 with external stereo data improves PSNR on both benchmarks.

Variants A–D start from the same Stage 1 checkpoint and each train for 7,000 additional iterations. A continues the Stage 1 training protocol with ground-truth warps and the original training data. We evaluate all rows under G1 on the same 50 randomly sampled videos from StereoGenBench (SGB) and 50 from SVD.

Continuing Stage 1 alone raises PSNR by 0.90 dB on SGB and 0.19 dB on SVD. At the same training budget and with the same data as A, joint training with predicted warps (C) raises SGB PSNR by another 0.78 dB. The variant that adds external stereo data for geometry-only supervision (D) improves PSNR over A by 0.28 dB on SGB and 1.04 dB on SVD.

Table W1aQ1. Matched-budget Stage 2 ablation under G1

ModelSGB PSNR ↑SGB SSIM ↑SGB LPIPS ↓SVD PSNR ↑SVD SSIM ↑SVD LPIPS ↓
Stage 1 checkpoint24.860.7860.12417.990.5180.319
A. Continued Stage 125.760.7950.11518.180.5220.308
C. Predicted Warp (Joint)26.540.8040.11318.220.5240.306
D. Joint + external stereo data26.040.7930.11619.220.5550.261

W1b & Q2. Predicted Warp (Detached) versus Predicted Warp (Joint) - Backpropagating through the warp improves all three StereoGenBench metrics.

B and C use the same training data, predicted-warp schedule, and 7,000-iteration budget. In B, we detach the predicted geometry used to build the conditioning warp. In C, the generation loss backpropagates through the differentiable warp into the geometry head.

Enabling that gradient raises SGB PSNR from 26.15 to 26.54 dB and lowers LPIPS from 0.115 to 0.113 (Table W1bQ2). SVD PSNR changes from 18.26 to 18.22 dB. From C to D, SVD PSNR rises by 1.00 dB and LPIPS drops from 0.306 to 0.261 (Table W1aQ1). The improvement on SVD shows that adding external stereo data helps the model generalize to real-world captures held out from training.

Table W1bQ2. Effect of backpropagating through the predicted warp under G1

ModelSGB PSNR ↑SGB SSIM ↑SGB LPIPS ↓SVD PSNR ↑SVD SSIM ↑SVD LPIPS ↓
B. Predicted Warp (Detached)26.150.7970.11518.260.5280.301
C. Predicted Warp (Joint)26.540.8040.11318.220.5240.306

We will add these ablation results in the final version.

W1c & Q3. Direct f/Z prediction versus inverse depth - Direct prediction improves geometry and right-view generation.

We trained both variants from scratch with the same architecture and training settings, changing only the prediction target. One predicts g = f/Z directly, and the depth-based variant predicts inverse depth 1/Z. We use inverse depth, a common parameterization in monocular depth estimation, because it maps points at infinity to zero. We multiplied its prediction by the ground-truth focal length f at evaluation to obtain g. We evaluate geometry on the full StereoGenBench test set and evaluate right-view generation under G1 on the same 50 videos randomly sampled from that test set for both variants.

Direct prediction reduces geometry MAE by 18.3% and raises δ1 by 11.18 percentage points. At a fixed baseline B, the warp's disparity error scales with the error in g through D = B × g. On the 50-video subset, PSNR improves by 2.62 dB, SSIM rises from 0.852 to 0.891, and LPIPS drops from 0.079 to 0.058 (Table W1cQ3).

Table W1cQ3. Geometry prediction and right-view generation

PredictionMAE↓ (px/cm)RMSE↓ (px/cm)AbsRel↓δ1↑PSNR↑SSIM↑LPIPS↓
f/Z0.6523.4090.25174.70%27.570.8910.058
1/Z with ground-truth f0.7983.4750.26063.52%24.950.8520.079

We will also include this analysis in our final paper.

The benefit of unified conditioning is that a single checkpoint operates under G0 (disparity), G1 (baseline), or G2 (monocular input) depending on the geometry available at inference, while achieving state-of-the-art results in all three settings on both SGB and SVD (Tables 2 and 3).

We hope these results address the reviewer’s concerns and would be glad to clarify any remaining points during the discussion period.

[1] FoundationStereo: Zero-Shot Stereo Matching CVPR 2025

[2] Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding, CVPR 2026

Dear Reviewer rcRx,

As the discussion period ends soon, we wanted to check whether the controlled ablations in our response — in particular the matched-budget comparison (Table W1aQ1), the detached-versus-joint comparison (Table W1bQ2), and the f/Z-versus-1/Z comparison (Table W1cQ3) — address your concerns. If any point remains unclear, we would be glad to respond before the deadline. Thank you again for the careful review and valuable feedback.

Thank you for the additional experiments. However, it seems that an important ablation is still missed to demonstrate the benefit of joint optimization over the detached variant. Could the authors compare Detached + External Stereo Data against variant D?

Reviewer_8KsGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

UniGenStereo synthesizes stereo video under three conditioning settings using one checkpoint: supplied disparity, a calibrated baseline, or monocular input alone. An internal geometry head predicts the baseline-independent ratio f/Z, which is scaled by the requested or default baseline to produce disparity. Supplied and predicted disparity feed the same warp-based conditioning interface. Two-stage training first supervises geometry and generation separately, then couples them through differentiable warping while adding external stereo data. Experiments cover synthetic seen/unseen splits, real-world capture, geometry prediction, and baseline control.

Critical strengths
  1. The unified conditioning interface is simple and useful. Predicting baseline-normalized disparity allows the same generator to use external geometry, inferred geometry with a requested baseline, or a default stereo strength. The dedicated geometry pass also explicitly avoids leaking supplied disparity into the predictor, addressing an important implementation concern.

  2. The empirical evaluation covers complementary settings rather than relying solely on synthetic reconstruction scores. Improvements persist on unseen synthetic environments and real SVD captures, and the comparison includes an in-domain fine-tuned ReCamMaster baseline. Geometry prediction and same-scene baseline sweeps further examine whether the conditioning mechanism behaves as intended.

Critical weaknesses
  1. The ablations do not isolate the central training and representation choices. Stage 2 simultaneously adds external stereo data, predicted-warp exposure, and generation gradients through the warp. Table 5 therefore cannot establish how much improvement comes specifically from joint optimization. Likewise, comparison against a separately trained depth model does not isolate direct f/Z prediction from differences in data and adaptation. Table 4 uses inference-time channel removal, which tests dependence on inputs rather than the benefit of training with each component.

  2. Baseline controllability is demonstrated more strongly than metric accuracy. Table 7 reports high per-scene R², but mean response slopes are 0.723 and 1.225 rather than the ideal 1, and the diagnostic permits an offset. Thus, a requested physical baseline need not yield the corresponding stereo geometry. This matters for the calibrated-rig applications motivating G1, even though the appendix acknowledges the limitation.

Questions for authors
  1. With training data and optimization budget fixed, what are the separate effects of external data, predicted-warp exposure with stopped geometry gradients, and full gradient coupling? How does a comparably trained depth-based predictor perform?

  2. What are the baseline-response intercepts and absolute calibration errors across scenes? Do the reported control gains persist on real rigs with multiple measured baselines, beyond the single-baseline SVD evaluation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8KsG (Part1)

We appreciate the reviewer’s recognition of the simplicity and flexibility of the unified conditioning interface. We thank the reviewer for the careful reading, valuable feedback, and the constructive suggestions. In response to the suggestions, we added experiments and analysis. Our point-by-point responses follow.

W1a. Separating the effects of Stage 2 - Predicted warps raise PSNR on both benchmarks, joint optimization adds a gain on StereoGenBench, and external stereo data improves generalization to real-world SVD.

We agree that Table 5 reports the combined effect of the Stage 2 changes. Variants A–D start from the same Stage 1 checkpoint and each train for 7,000 additional iterations. Each successive variant introduces one change.

  • A (continued training) continues the Stage 1 training protocol with ground-truth warps.
  • B (predicted warp, detached) builds the warp from the model’s predicted geometry for a fraction of training samples and detaches that geometry from the generation loss.
  • C (predicted warp, joint) builds the warp as in B and lets the generation loss backpropagate through the differentiable warp into the geometry head.
  • D (joint + external data) adds external stereo data to C for geometry-only supervision.

We evaluate all rows under G1, using the target baseline and predicted geometry. We randomly sampled 50 videos from StereoGenBench (SGB) and 50 from SVD and used the same subsets for all models.

Table W1a. Controlled Stage 2 ablation under G1

ModelContinued trainingPredicted warpJoint optimizationExternal dataSGB PSNR ↑SGB SSIM ↑SGB LPIPS ↓SVD PSNR ↑SVD SSIM ↑SVD LPIPS ↓
Stage 124.860.7860.12417.990.5180.319
A✓25.760.7950.11518.180.5220.308
B✓✓26.150.7970.11518.260.5280.301
C✓✓✓26.540.8040.11318.220.5240.306
D✓✓✓✓26.040.7930.11619.220.5550.261

Continuing Stage 1 raises PSNR by 0.90 dB on SGB and 0.19 dB on SVD (Stage 1 → A). At the matched budget, exposure to predicted warps adds 0.39 dB on SGB and 0.08 dB on SVD (A → B). Joint optimization adds another 0.39 dB on SGB and lowers LPIPS from 0.115 to 0.113 (B → C). At that step, SVD PSNR decreases by 0.04 dB.

From C to D, all three SVD metrics improve, including a 1.00 dB PSNR gain and an LPIPS reduction from 0.306 to 0.261. SGB PSNR decreases by 0.50 dB in the same comparison. The improvement on SVD shows that adding external stereo data helps the model generalize to real-world captures held out from training.

AuthorsOpenReview ↗

Response to Reviewer 8KsG (Part2)

W1b. Predicting f/Z versus inverse depth - Direct f/Z prediction improves both geometry accuracy and right-view generation.

We trained both variants from scratch with the same architecture and training settings, changing only the prediction target. We evaluate geometry on the full StereoGenBench test set and evaluate right-view generation under G1 on the same 50 videos randomly sampled from that test set for both variants. For the inverse-depth variant, we multiplied predicted 1/Z by ground-truth f to obtain g at evaluation. We use inverse depth, a common parameterization in monocular depth estimation, because it maps points at infinity to zero.

Direct prediction reduces MAE by 18.3% and raises δ1 by 11.18 percentage points. On the 50-video sample, it also improves PSNR by 2.62 dB, raises SSIM from 0.852 to 0.891, and lowers LPIPS from 0.079 to 0.058 (Table W1b). The warp uses disparity D = B × g, so at a fixed baseline the disparity error scales directly with the error in g. Predicting g directly also lets the model construct the disparity condition when the focal length is unavailable at inference.

Table W1b. Geometry prediction and right-view generation

PredictionMAE↓ (px/cm)RMSE↓ (px/cm)AbsRel↓δ1↑PSNR↑SSIM↑LPIPS↓
f/Z0.6523.4090.25174.70%27.570.8910.058
1/Z with ground-truth f0.7983.4750.26063.52%24.950.8520.079

W1c. Training-time ablation of warp conditioning - The warped latent has the largest effect on generation quality, consistent with Table 4.

Starting from the same Stage 1 checkpoint, we continued training the full model and the three ablated variants on the same data for the same number of iterations. Each ablated variant omitted the corresponding input during both continued training and inference. We evaluated all four models under G0 on the same 50 videos randomly sampled from StereoGenBench, with ground-truth disparity supplied.

Removing the warped latent reduced PSNR by 4.45 dB, compared with 0.35 dB for the coordinate embedding and 0.54 dB for the disocclusion mask. The warped-latent ablation also caused the largest drop in SSIM and the largest increase in LPIPS.

Table W1c. Training-time ablation under G0

Warp conditionPSNR↑SSIM↑LPIPS↓
Full (27 channels)30.040.89090.0619
w/o warped latent25.590.80170.0894
w/o coordinate embedding29.690.88710.0634
w/o disocclusion mask29.500.88910.0639
AuthorsOpenReview ↗

Response to Reviewer 8KsG (Part3)

W2 & Q2. Baseline controllability and metric calibration - The effective baseline increases monotonically in all three real sequences, with scene-dependent calibration error.

Synthetic scenes. For each target baseline, we fit D_ref = a D_gen + b between reference and generated disparity and compute the effective baseline B_eff = B/a. We compare the Table 7 fit, which estimates the pixel-level intercept b, with a fit that fixes b = 0. In both cases, we then fit B_eff = αB + β across the five target cameras per scene, allowing the baseline-response intercept β to vary. For each family, we average the fitted intercepts across its 20 scenes and compute MAE as the mean of |B_eff − B| across its 100 camera pairs.

Table W2Q2a. Baseline calibration on synthetic scenes

FamilyTarget baseline range (cm)Pixel intercept bMean slope αMean intercept β (cm)MAE (cm)Rel. errorMedian R²
Uniform2.6-465.4Free (Table 7)0.7232.6640.9832.2%0.959
Uniform2.6-465.4Fixed to zero0.6844.5340.4731.6%0.979
IPD-Gaussian5.3-33.8Free (Table 7)1.225−1.929.1039.8%0.965
IPD-Gaussian5.3-33.8Fixed to zero0.8732.303.2318.0%0.986

Fixing the pixel intercept to zero lowers the relative effective-baseline error from 32.2% to 31.6% on Uniform and from 39.8% to 18.0% on IPD-Gaussian. It also raises the median baseline-response R² for both families. The zero-offset fit matches the expected multiplicative scaling of disparity with baseline in rectified stereo. We will use it as the primary fit for estimating effective baselines in the revision.

Uniform baselines reach 465.4 cm. Because D = B × g, a larger baseline amplifies a given error in g = f/Z in the disparity supplied to the warp. This sensitivity likely contributes to the larger absolute calibration errors on Uniform, particularly when the synthetic monocular input provides limited cues to metric scene scale.

Real multi-baseline sequences. Real stereo videos with multiple measured baselines for the same scene are scarce, leaving real-world evaluation of baseline control relatively underexplored. We evaluated three Nagoya University MPEG-FTV sequences [3] under G1. We fixed one source view per sequence and generated 81 frames at each of four measured target baselines (5, 10, 15, and 20 cm), retaining the original focal lengths and resolution. We estimated reference disparity from each real stereo pair with FoundationStereo [1] and fixed both intercepts to zero.

Table W2Q2b. Baseline calibration on real multi-baseline sequences

SequenceB_eff at 5 cm10 cm15 cm20 cmSlopeR²MAE (cm)Mean abs. rel. error
Champagne Tower5.679.9614.0020.220.9900.9880.485.4%
Balloons5.4010.1816.1727.251.2220.9352.2513.5%
Kendo4.658.3811.7015.970.8030.9912.3216.3%

UniGenStereo tracks the requested baseline across all three real sequences, with R² values ranging from 0.935 to 0.991. The per-sequence effective-baseline MAE ranges from 0.48 to 2.32 cm.

We hope these results address the reviewer’s concerns and would be glad to clarify any remaining points during the discussion period.

[1] FoundationStereo: Zero-Shot Stereo Matching CVPR 2025

[2] Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding, CVPR 2026

[3] Fujii Laboratory, Nagoya University. FTV Test Sequences (MPEG-FTV Project).

Reviewer_YfbiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a unified model entitled UniGenStereo that generates stereo (target) video from a source monocular video in three different scenarios: (G0) disparity map available, (G1) calibrated baseline provided and (G2) only the source video is provided (a standard interpupillary baseline is used). The authors proposed a geometry predictor, a lightweight head, that estimates gg the ratio between the focal length and the depth, from the latent representation of the source video. This head is only used when the disparity maps are not provided. Then the disparity maps are obtained by multiplying the ratio gg with a calibrated baseline (case G1) or the standard interpupillary baseline (case G2). A warp-based conditioning is employed to ensure temporal coherence across the generated sequence. This framework is trained following a two-stages scheme: The first stage supervise the geometry predictor using a dense ground-truth disparities. The second stage jointly refine geometry prediction with generation via a differentiable warp. The training dataset is enriched during this stage with assets with missing disparity maps.

The experimental validation shows that UniGenStereo outperforms the baselines across the 3 geometric settings. The approach also transfers well to real world captured dataset (SVD).

Critical strengths

The core idea of this paper (unifying the different geometric settings encountered in stereo generation) is simple, but very elegant and impactful.

The performances shown in this paper are quite convincing.

The clarity of the presentation is quite good.

Critical weaknesses
  • The evaluation metrics used in this paper are mainly reconstruction metrics. The paper will be more convincing by adding stereo-specific metrics that can assess stereoscopic viewing quality. A user study could be a good solution.

  • While the rest of the paper is generally well written, the abstract is comparatively unclear and would benefit from a careful rewrite.

  • Some of the citations are referenced as arXiv, but they have been published.

Questions for authors
  • How the weights of the loss terms are chosen in both stage 1, stage 2? Did the authors tunes these terms to improve the convergence and the model performances?

  • It is not clear why the authors used different comparison methods in Table 2 and Table 3.

  • How would a variant model that predicts depth (instead of the ratio g) using the same architecture and training pipeline perform?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Yfbi (Part1)

We appreciate the reviewer’s recognition of the value of unifying stereo video generation across different geometric inputs. We thank the reviewer for the careful reading, valuable feedback, and the constructive suggestions. In response to the suggestions, we added experiments and analysis. Our point-by-point responses follow.

W1. Stereo-specific metrics - In the extended Table 2 comparison, UniGenStereo has the lowest disparity EPE, scale error, and temporal flicker in every tier.

For each frame, we estimate disparity with FoundationStereo [1] on both the generated pair (input left, generated right) and the reference pair (input left, ground-truth right). We compare the two disparity maps using end-point error (EPE, px), relative EPE (EPE divided by mean reference disparity), Bad-1 and Bad-3 (percentage of pixels with error above 1 and 3 px, respectively), and disparity correlation (Corr). We also report scale error (the mean of |s − 1| across clips, where s is the least-squares scale from reference to generated disparity), temporal flicker in the disparity error, and vertical misalignment (Vert, the median vertical offset of SIFT matches in pixels, with zero indicating ideal alignment).

Table W1. StereoGenBench, Seen split. Best in bold, with the second best in italics in tiers with more than two methods.

TierMethodPSNR↑SSIM↑LPIPS↓EPE↓Rel-EPE↓Bad-1↓Bad-3↓Scale err↓Corr↑Flicker↓Vert↓
G0StereoCrafter21.930.6580.3205.250.15751.221.40.1060.9383.3701.763
G0SVG23.400.7320.2302.340.05624.79.90.0280.9781.7211.221
G0M2SVid27.060.8120.1732.500.04717.49.10.0260.9771.6330.638
G0Ours28.400.8530.0811.900.04426.68.10.0190.9851.3170.382
G1ReCamMaster (fine-tuned)19.970.6220.22519.130.52991.080.40.4750.9342.1550.478
G1Ours22.430.6810.17412.840.23486.066.10.1820.9671.6860.404
G2StereoCrafter-Zero†11.760.3980.700262.9012.97995.994.812.9830.82370.586–
G2StereoPilot13.890.4680.54310.570.47383.760.61.0310.8555.4350.858
G2StereoWorld16.920.6190.31249.431.95093.186.51.8280.33018.2660.537
G2ImmersePro17.660.6030.28268.842.85090.782.82.7010.31424.9120.281
G2Ours24.900.8060.1177.700.24474.539.80.1610.9124.5810.475

† Too few StereoCrafter-Zero outputs yielded reliable left–right matches to report a Vert score.

All methods use the same evaluation code on the StereoGenBench Seen split. Our method performs strongly in both generation quality and stereo geometry. It achieves the best PSNR, SSIM, and LPIPS, as well as the lowest disparity EPE, scale error, and temporal flicker in every tier.

W2. Abstract clarity - The revised abstract introduces the three input settings first and explains their shared generation pipeline.

High-fidelity stereo video generation is crucial for immersive extended reality (XR), robotics, and embodied AI. The geometric information available at inference varies across applications. UniGenStereo uses a single checkpoint to synthesize the right view from a monocular video with per-frame disparity, with a target camera baseline, or with the video alone. When disparity is supplied, we use it directly. In the other two settings, a lightweight geometry head predicts the baseline-independent ratio g=f/Zg = f/Z, where ff is focal length and ZZ is metric depth. For rectified stereo, we compute disparity as D=B⋅gD = B ⋅ g, using the supplied baseline BB or a default interpupillary baseline when only the video is available. In all three settings, the resulting disparity is used to warp the input video and condition the same generator. Varying the target baseline lets users adjust stereo strength. We first train with ground-truth geometry and then jointly refine geometry prediction and generation through a differentiable warp. On StereoGenBench and the real-world Spatial Video Dataset, UniGenStereo achieves state-of-the-art results across all three settings.

AuthorsOpenReview ↗

Response to Reviewer Yfbi (Part2)

W3. Publication venues for preprint citations - We checked the full bibliography and will update six references.

The updates cover the published versions of Depth Anything 3 (ICLR 2026), TrajectoryCrafter (ICCV 2025), What Makes Good Synthetic Training Data for Zero-Shot Stereo Matching? (CVPR 2026), ImmersePro (ICML 2026), and DissolveStereo (ACM Transactions on Graphics, 2026, previously cited as StereoCrafter-Zero), along with the workshop version of ReStereo (CVPR 2026 SynData4CV Workshop).

Q1. How are the loss weights chosen? Were they tuned? - We chose fixed weights before each stage to balance gradient contributions without a weight sweep or benchmark-based tuning.

To balance gradient contributions, we weighted LgL_g at 0.3 in Stage 1 to prioritize the generation gradient on the shared backbone, then reduced it to 0.1 in Stage 2 so LgL_g anchors metric scale as the geometry head also receives gradients from warp consistency (LwcL_{wc} weighted at 0.5) and generation.

Q2. Different comparison methods used in Table 2 and Table 3? - We added SVD results for the remaining Table 2 methods to the Table 3 comparison.

Table Q2 uses the same stereo-specific metrics as Table W1. UniGenStereo achieves the best PSNR and LPIPS and leads on six of the eight stereo-specific metrics.

Table Q2. G2 results on SVD. PSNR, SSIM, and LPIPS use all 159 clips. Stereo-specific metrics use 50 clips selected at evenly spaced disparity ranks from the 106 clips with mean reference disparity ≤ 15 px, using only the real stereo pairs for selection. Best in bold, second best in italics.

MethodPSNR↑SSIM↑LPIPS↓EPE↓Rel-EPE↓Bad-1↓Bad-3↓Scale err↓Corr↑Flicker↓Vert↓
Eye2Eye§20.60.7330.392––––––––
StereoPilot22.250.6280.2053.480.54273.440.00.5370.9300.4830.246
StereoWorld19.850.5420.2478.911.40574.749.11.2890.5272.0980.382
ImmersePro19.600.5370.2729.671.52473.147.80.9050.4412.7210.168
StereoCrafter-Zero21.570.6040.2583.350.52872.835.50.4920.8630.6170.640
Ours22.460.6540.1783.060.48358.331.00.4500.8880.3500.195

§ Eye2Eye generation metrics are from Metzger et al. [2], since it is not open-sourced.

Q3. Direct f/Z prediction versus inverse depth - Direct f/Z prediction improves geometry and right-view generation even when the inverse-depth variant uses ground-truth focal length.

We trained both variants from scratch with the same architecture and training settings, changing only the prediction target. One predicts g = f/Z directly, and the other predicts inverse depth 1/Z. We use inverse depth, a common parameterization in monocular depth estimation, because it maps points at infinity to zero. We multiplied the inverse-depth predictions by the ground-truth focal length f to obtain g. We evaluate geometry on the full StereoGenBench test set and evaluate right-view generation under G1 on the same 50 videos randomly sampled from that test set for both variants.

Direct prediction reduced MAE from 0.798 to 0.652 px/cm (18.3%) and increased δ1 from 63.52% to 74.70% (11.18 percentage points), with smaller improvements in RMSE and AbsRel. For right-view generation, it raised PSNR by 2.62 dB and SSIM from 0.852 to 0.891, and lowered LPIPS from 0.079 to 0.058. Because the warp uses disparity D = B × g, an error in g translates directly into disparity error at a fixed baseline. Direct prediction also lets the model obtain g when focal-length metadata is unavailable at inference.

Table Q3. Geometry prediction and right-view generation

PredictionMAE↓ (px/cm)RMSE↓ (px/cm)AbsRel↓δ1↑PSNR↑SSIM↑LPIPS↓
f/Z0.6523.4090.25174.70%27.570.8910.058
1/Z with ground-truth f0.7983.4750.26063.52%24.950.8520.079

We hope these responses address the reviewer’s comments and would be glad to clarify any remaining points during the discussion period.

[1] FoundationStereo: Zero-Shot Stereo Matching CVPR 2025

[2] Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding, CVPR 2026

The authors answers all my concerns. I will retain my positive rating.

We sincerely thank the reviewer for the valuable feedback and positive rating and are glad that our responses addressed all of your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.