- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes RoboStereo, which performs 4D reconstruction — per-frame depth, camera pose and world point maps — from stereo video, targeting embodied/robotic settings where a calibrated stereo rig is available and metric scale matters. The core mechanism is Geometry-Guided Stereo Fusion, which interleaves a DVLT-style loop-transformer multi-frame reconstruction backbone with WAFT-Stereo-style disparity refinement. A second contribution is data: eleven rendered/synthetic training sources with stated sampling weights, plus the Embodiment Stereo Benchmark.
RoboStereo achieves the lowest scale-aligned depth AbsRel on all seven benchmark groups, the best delta-1/RMSE on five of seven, and the best camera ATE/translational RPE on four of seven.
Critical strengths
(a) A coupled, rather than one-way, integration of stereo matching and multi-frame reconstruction. The bidirectional recurrent loop is well matched to the stated problem (local manipulation geometry is weakly constrained by monocular multi-frame aggregation), and the read/write ablation shows each direction is useful and that the full model beats the baseline (one-way WAFT-Stereo -> Pi3X) on every group.
(b) A data and benchmark contribution targeting an underexplored niche. The rendered collection covers close-range and wrist-mounted embodied interactions absent from Dynamic Replica-style datasets, and the benchmark protocol includes groups (e.g., EBench wrist) excluded from joint training, which partially guards against in-domain overfitting and gives the community a reusable evaluation resource.
Critical weaknesses
(a) No quantitative real-world evaluation. Every scored result is on synthetic renderings from simulator families also used for training; real stereo footage is shown only qualitatively. For an embodied vision system this is a major gap: real stereo failures arise from sensor noise, rectification error, exposure changes, and motion blur, and the metric rescaling assumes a constant focal-baseline product that physical rigs only approximate. The central practical claim therefore remains unverified.
(b) Limited novelty beyond combining two existing systems, and uneven camera results. The architecture is essentially the DVLT backbone plus WAFT-Stereo with gated adapters, the authors explicitly state they cannot separate gains due to joint modeling from gains due to the rendered data, and camera accuracy is best on only four of seven groups (rotational RPE is rarely best). There is also no comparison with Stereo4D, which targets the same dynamic-stereo reconstruction setting. These points weaken both the novelty and the "geometry plus cameras" claims.
Questions for authors
(1) Can you provide quantitative results on real stereo data (e.g., against RGB-D or scanned-scene GT), including robustness to injected rectification error and noise, since metric rescaling depends on a constant f*b?
(2) Why is Stereo4D not included, and how does the model behave on internet or real stereo video rather than simulator renderings?
(3) Please train the strongest baseline (WAFT-Stereo, or WAFT-Stereo -> Pi3X) on the same eleven-source mixture with the same budget. How much of the margin is data and how much is architecture?