- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents StereoPolicy, a framework that learns robotic manipulation policies directly from synchronized stereo images without explicit depth supervision or 3D reconstruction. It combines shared-weight image encoders with a Stereo Transformer that alternates self-attention and cross-attention to produce binocular features trained through action objectives. The framework is integrated into diffusion policies and pretrained VLA models, and evaluated on three simulation benchmarks and seven real-world tasks, with additional ablations, relative-depth probes, and camera-configuration studies.
Critical strengths
- The paper demonstrates improvements across diffusion policies and pretrained VLA models in both simulation and real-world manipulation, including comparisons with native two-image VLA inputs. This breadth supports the practical value of the approach.
- Duplicated-input controls and matched joint-attention comparisons support the contribution of binocular information and the fusion design. Relative-depth probes provide additional evidence that action training makes geometric information more accessible in the learned representations.
Critical weaknesses
-
The architecture combines established components, including shared image encoders, alternating self-attention/cross-attention, positional encoding, and action supervision. While the matched joint-attention comparison supports the effectiveness of the chosen design, its methodological contribution relative to existing direct stereo-control approaches could be more clearly distinguished. Moreover, Table 3 compares a trainable StereoPolicy visual module with frozen FoundationStereo features, making it difficult to separate the effects of representation choice and task adaptation. Overall, the empirical contribution is more compelling than the architectural novelty.
-
Table 8 shows that gripper–target relative depth becomes more linearly decodable after action training, including on held-out trajectories and spatial layouts. However, this establishes access to one task-relevant geometric quantity, leaving unclear whether the features encode broader, transferable spatial relationships and whether the policy uses the probed information to improve action selection. The evidence supports task-adapted features containing geometric information, but provides a more limited explanation of what geometry is learned and how it contributes to control.
Questions for authors
-
In Table 3, StereoPolicy’s visual module is trained with the action loss, while the FoundationStereo feature extractor is frozen. Have the authors tried jointly fine-tuning FoundationStereo and the action policy using the same action loss? This comparison would help clarify whether the performance gap comes mainly from the different architectures or from allowing only StereoPolicy’s visual features to adapt to the task.
-
Appendix C.4 states that the highest success rate observed during training is reported for the Robomimic and OmniGibson experiments. Are these scores reported directly from the periodic evaluations, or are the selected checkpoints evaluated again on fresh, independent episodes? Please also clarify how checkpoint selection is performed across the three training seeds. Reporting the maximum observed evaluation score directly may lead to optimistic performance estimates.
3 comments
I thanks the author for the extra experiments and that solves my concerns. I therefore raise my evaluation to 3.
Thank you for taking the time to look over the additional experiments. We’re glad they addressed your concerns, and we’d be happy to answer any further questions!
Response to Reviewer E3Ez
We thank Reviewer E3Ez for their time and detailed feedback!
Q1. FoundationStereo fine-tuning and task adaptation
We thank reviewer for identifying this important distinction. To disentangle the effect of architecture from task-specific feature adaptation, we additionally experimented by jointly fine-tuning FoundationStereo and the action policy using the same action loss and show the results below (ToolHang-100). Results are reported as mean success rate and standard deviation across three training seeds.
Jointly fine-tuning decreases the gap but still underperforms StereoPolicy-DP. This additional experiment strengthens the evidence for the effectiveness of StereoPolicy’s task-trained stereo representation for manipulation-policy learning.
Q2. Checkpoint selection and independent testing
We fix the previously selected best checkpoints before testing on fresh evaluation seeds, retaining three training seeds and 100 episodes per task and seed. Please see C2 (Tables R1–R2) for the protocol and results.
W1. Contribution and prior stereo-control work
Please see C1.
W2. Interpretation of geometric probes: Table 8 shows that gripper–target relative depth becomes more linearly decodable ...
We use gripper–target relative depth as a focused diagnostic of spatial information relevant to manipulation. On held-out trajectories and layouts, action training reduces linear-probe MAE from 3.1 to 1.1 cm on ToolHang and from 4.0 to 1.5 cm on PourWater (Table 8). Matched policy ablations separately show the control benefits of stereo input and fusion (Table 7). These provide complementary evidence of learned geometric information and improved manipulation performance, without treating probe accuracy as proof of a specific causal mechanism or a general-purpose 3D representation.
We thank the reviewer again for the careful review and constructive suggestions, and welcome any further questions.