- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes GC-WAM, a world-action model that uses future-depth prediction as its main visual prediction objective, bidirectional contrastive alignment of geometrically corresponding local features across camera views, and a geometry-first training curriculum. Its central motivation is to learn geometric information that is accessible to the action policy and remains useful under viewpoint changes. Depth and camera calibration provide training supervision, while deployment retains RGB observations, language, and robot state as inputs without requiring future-depth generation. Evaluation includes frozen-feature geometry diagnostics, component ablations, LIBERO/LIBERO-Plus, and RobotTwin 2.0.
Critical strengths
-
The problem is relevant to manipulation, and the proposed supervision addresses a meaningful distinction. Learning features from which geometry can be recovered is different from ensuring that descriptors of the same physical surface remain matchable across views. Future-depth supervision and calibration-guided local alignment are a coherent way to address these complementary requirements. Using geometry as training supervision while retaining RGB-based deployment is also practically appealing.
-
The method shows substantial gains on tests that directly exercise observation robustness. On LIBERO-Plus, Camera success improves, and the method also improves Light, Background, and Sensor performance relative to that baseline (Tab.2). These are relevant interventions on the observations, rather than only improvements on unperturbed tasks. Tab.4 additionally reports an average success rate increase after adding geometric consistency, providing preliminary evidence that the component is useful. These results support the value of the overall training approach within the evaluated simulation conditions.
Critical weaknesses
-
The targeted experiments do not yet establish the central link from improved geometry to cross-view consistency and robust control. E.g., Fig.1(b) is relevant, but unseen-view depth-probe AbsRel is 0.0355 for GC-WAM versus 0.0356 for RGB and 0.0368 for RGB-D. The difference from RGB is very small. At the control level, Tab.4 reports only average success, so it does not show whether the Geo component specifically improves Camera robustness. Tab.6's camera-augmentation baseline achieves 74.5% on Camera versus 67.7% for GC-WAM, although it uses more data exposure and longer training. Thus, this comparison does not establish the alignment objective's incremental advantage under matched conditions. Finally, the paper motivates geometry through shape, contact surfaces, and spatial relationships, but does not analyze whether gains occur in tasks or conditions where these particular capabilities determine success.
-
The evidence for practically useful robustness is limited by near-failure under randomized conditions and the absence of real-robot validation. RobotTwin success falls from 85.4% on Clean2Clean to 4.7% on Clean2Random, compared with Fast-WAM's 70.2% to 1.2% (Table 3). The improvement therefore remains very limited in absolute terms under randomization. Moreover, all control evidence is simulated. Without physical-robot tests under controlled viewpoint or appearance changes, it remains unclear whether the benefits learned using simulated depth and calibrated view pairs translate to actual manipulation.
Questions for authors
-
Please specify the data splits, probe architecture, feature/token selection, noise conditions, and training protocol for Fig.1, and define its variants and policy evaluation setting. Are the probed features obtained using only information available during RGB-based deployment? What repeated-evaluation or uncertainty estimates are available? Is there evidence that the improved readout corresponds to better handling of relative depth, spatial relations, or contact-relevant geometry in the evaluated tasks?
-
What are the Camera results for the Tab.4 variants under the same view exposure and training budget? Please identify the training and test camera poses and, if available, separate camera distance, position, and orientation changes. Aggregate correspondence results on held-out scenes/views would help connect the Fig.3 observation to this control result.