acceptodds
Withdrawn from ICLR 2027

Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern stereo matching models fail when disparity crosses zero, with end-point error (EPE) rising by 4.6–37. Yet stereoscopic content, from cinema 3D to VR, routinely contains objects behind the zero-disparity plane (ZDP), corresponding to negative disparities. The blind spot cascades through datasets, architectures, and evaluation protocols, all of which inherit the non-negative geometry. Rectified parallel cameras place ZDP at infinity, so every finite depth yields by construction, and nothing within the standard pipeline can violate, or even measure, a negative disparity. To measure it, we propose ZDPShift, a benchmark of stereo pairs from seven cinematographer-authored open movies, each frame rendered at five zero-disparity-plane positions with dense signed ground truth. Six state-of-the-art image and video stereo matching models collapse once the plane moves. On identical scene content, FoundationStereo goes from px EPE to px, with every backbone leaving roughly half of all pixels exceeding a three-pixel disparity error. What is missing, however, is not the underlying matching capability. Training on supervision synthesized from SceneFlow, which adds no new data or parameters, keeps the error flat across the signed range. Training only the decoder, with the pretrained matching features frozen, performs comparably across all six backbones, with EPE jittering within px. Thus, the pretrained features already extend to the negative regime they were never trained on, and only the output convention discarded it. Meanwhile, positive-regime accuracy on KITTI, Middlebury, ETH3D, and Sintel is largely preserved.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
stereo matching, negative disparity, zero-disparity plane, stereoscopic 3D, virtual reality

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1QLXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an analysis regarding negative disparities in stereo matching and a simple fix for SOTA deep learning-based stereo matching models. In particular, negative disparities can appear in isolated cases, such as 3D movies or imperfectly calibrated stereo rigs; thus, running most of the SOTA stereo matching methods will fail on a large number of pixels.

To fix existing image and video SOTA stereo matching methods, the authors proposed two fixes: 1) shift the right image to simulate negative pixels during model training; 2) fix non-correlation cost volumes (e.g., GWC cost volumes) using both positive and negative indices (pure correlation volume-based solutions like RAFT-Stereo already support negative disparities natively).

To evaluate the fine-tuned models, the authors proposed a novel benchmark created starting from open-source movies like Spring, with customizations such as the capability to shift the zero disparity plane or the removal of artist effects such as depth-of-field blur (an issue present in already published stereo benchmarks based on open movies like Spring).

The fine-tuned models drastically reduce errors on the negative disparity side (Tab. 3), while keeping the models sufficiently accurate on the positive range (Tab. 4).

Critical strengths

Major

  • A novel stereo benchmark from open movies. The new ZDPShift dataset contains scenes from different open movies like Spring and Settlers, reduces artist effects such as lens flare, and introduces multiple zero-disparity planes.
  • Simple fix for existing SOTA architectures. The GWC indices fix and the right image disparity shift can extend existing SOTA stereo models to the negative disparity regime. Those are very easy edits. Furthermore, the fine-tuning is affordable (i.e., a few hours) on a top-tier GPU (probably even on a smaller GPU), since, as demonstrated by the authors, the matching features are already compatible with the negative disparity regime.

Minor

  • Clean writing. The paper is easy to read and well structured, with clear explanations of the method and the experiments.
Critical weaknesses

Major

  • Given the recent literature, the novelty is quite limited. The main claim of the paper -- i.e., "Restoring negative disparity" -- is already explored in the literature (1). In particular, the right image shift is already used as an augmentation for the S2M2 model family, which supports negative disparity values. Probably, the most novel parts are the novel benchmark and the GWC cost volume fix, which are quite limited for an ICLR paper.

  • The positive regime accuracy is not fully preserved. While the simple fix proposed is interesting, the claim that the positive regime accuracy is preserved when applying the fine-tuning is not well supported by Tab. 4. In particular, we observe a drop on KITTI2015 with all image architectures -- with a ~2x drop when observing RAFT-Stereo -- or we can observe drops in all other benchmarks -- such as Middlebury, where FoundationStereo Bad2 drops from 1.1 to 2.9. I consider this a minor weakness since the fine-tuned stereo models are mainly used for 3D movies: for standard usage, we can use the non-fine-tuned version.

Minor

  • Limited applications of negative disparity values. As written in the paper (rows 40-43), the standard procedure is to place ZDP at infinity, removing de facto negative disparity values. From my experience, the only two cases where negative values appear are in the case of imperfectly calibrated stereo rig setups (1) and the case of 3D movies.

References

(1) Min, Junhong, et al. "S 2 m 2: scalable stereo matching model for reliable depth estimation." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.

Questions for authors

Questions

  • Can the authors provide more details about the usage of stereo matching for 3D movies? The only referenced scenario is in row 161.
  • Can the authors provide more applications for negative disparity other than 3D movies and an imperfect stereo rig?
  • Can the authors provide a comparison with S2M2?

Suggestions

  • Row 102: "[...]though it has been considered essential for decades in 3D movie production.". Please justify this part better.
  • Row 166: FoundationStereo has an iterative recurrent module.
Reviewer_EiexOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces the ZDPShift, a novel benchmark for stereo images with signed disparity, alongside a method to adapt existing stereo matching frameworks to handle signed disparity data. The paper has performed extensive experiments to demonstrate the effectiveness of its method.

Critical strengths
  1. The paper points out a new challenge for stereo matching, where there are some pixels with negative disparities.
  2. The paper presents a benchmark and extensive experiments for evaluating existing stereo matching frameworks on such case.
Critical weaknesses
  1. Limited practical application: The negative-disparity stereo pairs seem to appear only in artificially rendered pairs, such as 3D films, where the ground-truth disparity has already been known by the film producer. Therefore I was wondering if there is other valid practical applications?
  2. Simple alternative to solve negative disparity: as in appendix D.4, re-position the ZDP by shifting the views might also solve the problem well. Although the authors provide an experiment to compare with such an alternative, but I think the experiment is a little bit unfair. Because the re-position method seems to be not trained on negative disparity, while the proposed method of paper has been trained on negative disparity.
Questions for authors
  1. It would be beneficial to present more valid applications of negative disparity.
Reviewer_iowKOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Modern stereo matching models silently fail once disparity crosses zero (the "negative" regime, i.e. content behind the zero-disparity plane / ZDP), which is the normal operating regime of stereoscopic cinema and VR. The authors show six SOTA image/video stereo models (RAFT-Stereo, IGEV-Stereo, FoundationStereo, DynamicStereo, BiDAStereo, Stereo-Any-Video) collapse the instant disparity goes negative, with EPE rising 4.6–37× on identical scene content (e.g. FoundationStereo 2.15px → 75.53px at ZDP shift +32). They trace the blind spot to three interlocking conventions: (i) datasets captured with centered rectified parallel rigs force d≥0 by construction; (ii) cost-volume architectures index a one-sided hypothesis range [0, dmax], making negative disparity inexpressible; (iii) every benchmark only measures the positive regime, so the failure is unobservable. Contributions: (1) a cheap fix — synthesize signed supervision by relabelling SceneFlow as d'=d−∆ and extend the hypothesis range with no added parameters (freezing the matching features and training only the decoder works within 0.2px); (2) ZDPShift, the first public multi-ZDP stereo benchmark (21,375 pairs = 4,275 open-movie frames × 5 ZDP shifts) with dense signed ground truth; (3) signed variants of all six backbones that recover the negative regime while largely preserving positive-regime accuracy.

Critical strengths
  1. Exposes a real, field-wide blind spot with a cheap, effective fix. Showing that essentially every released SOTA stereo model cannot represent behind-screen disparity — and that the remedy is nearly free (label shift + hypothesis-range extension, no new params) — is high-impact and immediately actionable for practitioners and benchmark designers.
  2. ZDPShift is a genuine community asset. It is the first benchmark offering dense signed ground truth across multiple ZDP positions, filling a gap no prior dataset (KITTI/Middlebury/ETH3D/SceneFlow/Holopix50k) addresses. The "freeze features → still works within 0.2px" finding is an insightful, broadly interesting observation about what stereo representations actually encode.
Critical weaknesses
  1. Limited methodological novelty. The technique is essentially a data/label transformation (relabel SceneFlow by ∆) plus extending an index range; there is no new learning method or architectural idea. The paper's value is the observation + benchmark + trivial fix rather than a technical advance, which is a weak fit for ICLR's typical method-driven expectations (though it is well within scope as an analysis/benchmark paper).
  2. Benchmark is synthetic-only and narrow. ZDPShift is rendered from 7 Blender open movies (32 scenes); signed GT cannot exist for captured content, so the headline real-film results are qualitative only. Coverage is highly scene-dependent (Agent 327 stays ~all-positive even at +32; Project Gold hits 93% negative), only 5 discrete ∆ values are used (max +32px), and there is no captured/diverse real stereo with negative disparity. This bounds the benchmark's representativeness and the claimed generality.
Questions for authors
  1. The positive-regime regressions for RAFT-Stereo (Middlebury D1 5.26→10.01, KITTI EPE 1.13→1.92) are substantial. Can you isolate the effect of signed supervision by reporting the original model fine-tuned only on unshifted (positive) SceneFlow as an ablation? This would confirm the regressions are not simply from fine-tuning.
  2. ZDPShift keeps cameras strictly parallel; real 3D films often use convergent (toe-in) rigs, violating the rectified-pair assumption your relabeling relies on. How does the method transfer to such captured content, and is a parallel-only assumption a deployment limitation?
  3. The sign-flip failure mode on low-texture regions (fog/haze) is flagged but not quantified. Can you report its frequency / affected-pixel fraction and any mitigation? This seems material for production use.
  4. Why only ∆∈{−16,0,16,24,32} and a +32px ceiling? Larger behind-screen depths (larger |negative| disparity) are common in cinema — is the range a limitation of the rendering or a deliberate scoping choice?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.