- Rating
- 3(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(−1)
Read review
Paper summary
The paper presents ForceStereo, a stereo matching framework that treats disparity estimation as reliability-guided state evolution. The method first uses scene-level context to calibrate local disparity hypotheses through an Overview-Conditioned Hypothesis Volume (OCHV). It then constructs reliability conditions from state-dependent residuals and structural cues, followed by a Neural ODE-based Residual Continuous Evolution (RCE) module to update the disparity state. The method achieves strong results on several benchmarks and zero-shot evaluations. The ablation study validates the effectiveness of the proposed modules.
Critical strengths
- The paper provides a meaningful perspective on stereo matching by treating observation reliability as part of the inference process, rather than as an auxiliary confidence prediction.
- The experimental results are strong and consistent across standard benchmarks and zero-shot settings. The ablation study is also comprehensive and helps clarify the contribution of the proposed components.
- The paper is clearly written and logically organized, making the proposed method and experimental findings easy to follow.
Critical weaknesses
-
The practical distinction between the Neural ODE formulation and a carefully designed iterative update is not completely clear. The method is implemented with three fixed-step Euler evaluations, although the paper presents it as continuous state evolution. The matched-GRU comparison is helpful, but more discussion would be useful on whether the improvement comes from the continuous formulation itself or from the additional state-dependent conditioning and trajectory supervision.
-
The accuracy improvements are well demonstrated, but the efficiency evaluation is less comprehensive. The paper should report the runtime and memory cost on one or two representative datasets and under different image resolutions. It would also be useful to separate the costs of feature extraction, reliability computation, and RCE updates, and to discuss the resulting accuracy–efficiency trade-off.
Questions for authors
- How sensitive are the results to the number of RCE solver steps and the integration interval? Does increasing the number of steps provide further improvement or mainly increase computation?
- How are the model-independent unreliability scores and their thresholds defined in the zero-shot analysis?
- What is the complete inference cost of ForceStereo compared with BridgeDepth and the other baselines, including memory usage and the additional feature warping and residual computation?
8 comments
W1. Continuous Evolution and Discrete Refinement
W1. Continuous Evolution and Discrete Refinement
Where the gains come from. The key benefit of RCE lies in how state-dependent observations are converted into geometric changes. As disparity evolves, RP regenerates residuals and matching geometry. RCE maps these updated conditions to ambiguity-bounded local velocities and integrates them to refine the geometry state. This couples observation feedback with a rate-based evolution rule.
Continuous evolution vs. discrete GRU refinement. RCE defines this rule through a shared velocity field over a continuous evolution coordinate. Three fixed-step Euler evaluations numerically approximate its integration. A conditioned GRU also receives the updated RP, but directly predicts finite gated state transitions. Its updates depend on learned gates and state features, without explicitly linking local geometric rates to an integration step size. RCE explicitly accumulates regulated local velocities through integration. The key distinction is how the same observations govern state changes, not access to additional conditioning or supervision.
1. Controlled comparison with discrete refinement. To test the benefit of this velocity-field parameterization, we compare RCE with a matched GRU. Both use the same RP regeneration, ambiguity gating, trajectory supervision, and three-step update budget (Table 6; Appendix Table 13). Beyond standard benchmarks, we also evaluate both models on the most unreliable 10% of Middlebury pixels, identified by our model-independent unreliability score.
| Method | Middlebury BP-2 | Unreliable 10% BP-2 | ETH3D BP-1 |
|---|---|---|---|
| Matched GRU (3 steps) | 7.73 | 11.16 | 1.10 |
| RCE (3 steps) | 7.62 | 10.03 | 0.97 |
RCE achieves lower errors under the same conditioning and supervision. In highly unreliable regions, it achieves a 10.1% relative reduction in BP-2 (11.16% to 10.03%). These results support an additional benefit from RCE's velocity-field update rule beyond state-dependent conditioning and trajectory supervision.
2. Contribution of state-dependent observation feedback. We separately examine the benefit of dynamically updating observation conditions within RCE (Appendix Table 12).
| RP Configuration | Middlebury BP-2 | Unreliable 10% BP-2 |
|---|---|---|
| Fixed RP | 7.91 | 11.24 |
| Dynamic Residual | 7.79 | 10.90 |
| Dynamic Geometry | 7.82 | 10.68 |
| Full Dynamic RP | 7.62 | 10.03 |
Updating both residuals and matching geometry reduces Middlebury BP-2 from 7.91% to 7.62%, with a larger relative improvement in unreliable regions (11.24% to 10.03%). This confirms that reinterpreting observations as the geometry state evolves also contributes to performance.
Together, these experiments support two complementary sources of improvement. State-dependent observation feedback improves the conditions guiding geometric refinement. RCE's velocity-field parameterization provides an additional advantage over the matched GRU under the same conditioning and supervision. Their combination enables reliability-guided geometric evolution.
References
[1] Neural Ordinary Differential Equations. Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, David K. Duvenaud.
[2] RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching. Lahav Lipson, Zachary Teed, Jia Deng.
W2 and Q3. Inference Efficiency and Computational Cost
W2 and Q3. Inference Efficiency and Computational Cost
ForceStereo improves stereo accuracy with a 16 ms increase in end-to-end inference time over BridgeDepth on Scene Flow. Latency increases from 75 ms to 91 ms. To quantify this trade-off, we additionally profile inference latency, peak GPU memory, and component costs on an NVIDIA L20 GPU.
End-to-end runtime and memory. We measure both methods at three input resolutions.
| Input Resolution | BridgeDepth Latency | ForceStereo Latency | BridgeDepth Peak Memory | ForceStereo Peak Memory |
|---|---|---|---|---|
| 496 × 720 | 68.34 ms | 84.85 ms | 1.640 GiB | 1.871 GiB |
| 992 × 1440 | 152.14 ms | 184.59 ms | 2.633 GiB | 3.547 GiB |
| 1984 × 2880 | 887.21 ms | 959.75 ms | 6.627 GiB | 8.263 GiB |
The additional latency is 16.51, 32.45, and 72.54 ms, respectively. As input resolution increases, ForceStereo's relative inference latency overhead over BridgeDepth decreases from 24.2% to 8.2%. The corresponding peak memory measurements are reported in the table.
Component-wise profiling. We measure the main operations separately on the L20 GPU.
| Component | Latency |
|---|---|
| Feature Extraction (DAv2 + Stereo) | 25.99 ms |
| Proposal / Hypothesis / State | 15.45 ms |
| OCHV | 7.95 ms |
| Latent Alignment | 24.21 ms |
| RP (3 steps) | 5.91 ms |
| RCE excluding RP (3 steps) | 11.64 ms |
Across three Euler steps, RP takes 5.91 ms, while RCE excluding RP takes 11.64 ms. The latter includes dynamic geometry computation and Euler updates. These are individually profiled component timings, and end-to-end inference is measured separately.
Warping and residual costs. We further decompose the RP computation across three Euler steps.
| RP Operation (3-step total) | Latency |
|---|---|
| Image Warping | 0.54 ms |
| Feature Warping + Normalization | 0.65 ms |
| Residual + Frequency Computation | 0.71 ms |
| Reliability Features + Fusion | 4.01 ms |
| Total RP | 5.91 ms |
Image warping, feature warping with normalization, and residual/frequency computation together account for 1.90 ms across three steps. Reliability feature construction and fusion account for the remaining 4.01 ms. These measurements directly quantify the observation-processing costs of RP.
Accuracy–efficiency trade-off. The additional computation is accompanied by improved accuracy over BridgeDepth (Table 4).
| Benchmark | BridgeDepth | ForceStereo | Relative Error Reduction |
|---|---|---|---|
| ETH3D BP-1 | 1.26 | 0.97 | 23.02% |
| Middlebury BP-2 | 7.84 | 7.62 | 2.81% |
| DrivingStereo BP-3 | 4.28 | 3.59 | 16.12% |
ForceStereo adds 1.01M parameters (0.29%). Table 7 also shows comparable absolute latency increases when applying the framework to other backbones: IGEV-Stereo increases from 375 to 389 ms (+14 ms), and NMRF from 42 to 58 ms (+16 ms).
| Model | Middlebury BP-2 | ETH3D BP-1 | Latency |
|---|---|---|---|
| IGEV-Stereo | 9.95 | 4.05 | 375 ms |
| Force-IGEV | 8.98 | 3.32 | 389 ms |
| NMRF | 8.53 | 3.71 | 42 ms |
| Force-NMRF | 8.01 | 2.56 | 58 ms |
Overall, these measurements quantify the computational cost of dynamic observation feedback and reliability-guided state evolution. With 16 ms additional end-to-end latency on Scene Flow, ForceStereo achieves 23.0% and 16.1% relative error reductions on ETH3D and DrivingStereo, respectively. These results establish a clear accuracy–efficiency trade-off for the proposed refinement mechanism.
We have organized the implementations of OCHV, RP, and RCE, along with the profiling scripts, in a maintained code repository. We plan to release it promptly upon acceptance to support reproducibility and further research.
References
[1] BridgeDepth: Bridging Monocular and Stereo Reasoning with Latent Alignment. Tongfan Guan, Jiaxin Guo, Chen Wang, Yun-Hui Liu.
[2] Iterative Geometry Encoding Volume for Stereo Matching. Gangwei Xu, Xianqi Wang, Xiaohuan Ding, Xin Yang.
[3] Neural Markov Random Field for Stereo Matching. Tongfan Guan, Chen Wang, Yun-Hui Liu.
General Response
We thank the reviewer for the constructive feedback and recognition of our core contribution. We appreciate the thoughtful questions on continuous evolution, computational efficiency, and evaluation methodology.
We have also revised the method description to improve clarity and readability. Additional comparison and ablation experiments have been included, particularly to examine the role of monocular priors. All revised text, figures, and tables are highlighted in blue for easy identification.
Below, we address each concern separately with experimental evidence and additional analyses.