- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper takes the stance that a policy should not be told where the camera is, but rather infer it by itself. To achieve this, the authors propose CamVLA, that decouples manipulation control from camera geometry by predicting a camera-centric end-effector action in the local camera frame, and a hand-eye matrix relating cameras to the robot base. Experiments show clear improvement relative to baselines without this mechanism.
Critical strengths
This paper tackles what is from my perspective a clear weakness in current VLA methods, which is camera awareness and sensitivity to camera viewpoints.
Critical weaknesses
A clear weakness in this paper, as far as I can see, is the lack of comparison to baselines that use explicit camera information. If the claim is that "the policy should not be told where the camera is", there should be a comparison against methods that are told where the camera is. I agree that this would require privileged information, however geometric foundation models are very accessible and accurate these days, to the point where I assume camera intrinsics and extrinsics could be available if they are beneficial, and such experiments would provide a clear indication whether that is the case or not. The only mention to GT calibration is as part of ablations, and CamVLA shows marginal improvement relative to GT, which to me indicates that if such calibration is available the method does not bring much improvement. Furthermore, calibration is used at training time, and thus it might be sensible to assume that they will also be available at test time.
Questions for authors
Broadly speaking, I would like to hear from the authors some arguments to address my comments in the weakness section: what are the benefits of self-estimating calibration, vs. using off-the-shelf foundation models to produce those? Were there any studies showing the impact of camera miscalibration during training, and how that influences results? I imagine if the method is robust to inaccurate labels it could lead to some meaningful benefits over using those same inaccurate labels at test time.
3 comments
Rebuttal for Reviewer uyfu (2/2)
Q2: Were there any studies showing the impact of camera miscalibration during training, and how that influences results? I imagine if the method is robust to inaccurate labels it could lead to some meaningful benefits over using those same inaccurate labels at test time.
Thank you for this insightful question!
(1) Robustness to extrinsic rotation inaccuracies: Our robustness analysis in Figure 6 evaluates CamVLA's tolerance to inaccurate extrinsic rotations. When rotation errors ranging from to are injected during execution, CamVLA suffers minimal degradation under small errors (64.0% at , 63.3% at , and 58.7% at ) and consistently outperforms the noise-free baseline (36.0%) even under rotation errors of up to . Crucially, our Geometric Head achieves a mean rotation error of only , falling well within this tolerance regime with negligible performance degradation.
(2) Invariance to translation miscalibration and tolerance to real-world label noise: As derived in Appendix Sec. B, our relative-action formulation cancels the hand-eye translation vector analytically, making both camera-centric delta action conversion during training and policy execution completely independent of translation miscalibration. Together with our real-world experiments (Table 2), which already rely on imperfect physical hand-eye calibration yet consistently outperform baselines across all shifts (79.0% vs. 63.3% for and 80.7% vs. 64.7% for GR00T N1.7 at , and 29.3% vs. 16.0% and 33.0% vs. 14.7% at ), these results confirm CamVLA's robustness to moderate label inaccuracies.
(3) Benefits over directly using inaccurate calibration at test time: Rather than directly consuming inaccurate or drifted calibration labels at test time, CamVLA re-estimates extrinsics online from live visual inputs ( error) while gaining stronger camera-aware policy representations through joint training. Consequently, CamVLA delivers clear advantages over test-time reliance on external extrinsics, outperforming even baselines privileged with ground-truth test-time calibration (51.4% vs. 49.4% for OC-VLA and 46.3% for AnyCamVLA).
Kind Reminder of the Discussion Period Ending Soon
Dear Reviewer uyfu,
We hope you are doing well.
As the discussion period will end on Nov 18, 11.59pm AoE, we would like to know whether our response has addressed your concerns, and we would really appreciate it if you could take some time to read our responses and provide any further feedback that you have. Your feedback is important to us, and we would like to have the opportunity to discuss them with you before the discussion period ends.
Thank you again for helping us, we believe our paper has been standing much stronger after incorporating your constructive suggestions.
Best regards,
Rebuttal for Reviewer uyfu (1/2)
We thank the reviewers (fNjd - R1, PF2A - R2, uyfu - R3) for their feedback. We are pleased that they found our work addresses a clear weakness (R3) and an important problem (R2) with compelling motivation (R2), and our method is novel (R2), well-motivated (R2), geometrically appropriate (R1), clear and easy to follow (R1, R2, R3), and easy to understand and potentially adopt (R1). It is also noted that we achieve consistent improvements across strong VLA backbones in simulation and real-world experiments (R1, R2, R3) with useful pose-error and ground-truth analyses (R1). We address here the comments and will incorporate all feedback in revision.
Review uyfu
Q1: What are the benefits of self-estimating calibration, vs. using off-the-shelf foundation models to produce those?
Thank you for this thoughtful question! First, in real-world deployments, test-time camera extrinsics are typically unavailable or invalidated due to camera bumps, remounting, or dynamic camera motion, so training-time calibration generally cannot be relied upon at test time. Moreover, CamVLA† (52.3%) in our ablations represents the Oracle upper bound of our framework using ground-truth extrinsics. The minimal gap between calibration-free CamVLA (51.4%) and CamVLA† (52.3%) confirms that our self-predicted extrinsics are highly accurate (1.41∘ error) and effectively replace GT calibration at test time, while outperforming the base-frame π0 baseline (33.2%) by +18.2%.
To compare against methods that are explicitly "told where the camera is", we evaluate AnyCamVLA and OC-VLA using ground-truth camera extrinsics at test time alongside calibration-free π0 and CamVLA:
These results and geometric considerations highlight three clear conclusions regarding self-estimating calibration:
(1) Practicality for Real-World Deployment: Unlike AnyCamVLA and OC-VLA, which are not calibration-free and fail to apply when cameras experience real-world mounting disturbances or motion, CamVLA operates completely calibration-free at test time while outperforming all evaluated baselines (51.4%).
(2) Implicit Camera-Aware Geometric Understanding: Even when AnyCamVLA (46.3%) and OC-VLA (49.4%) are privileged with ground-truth test-time extrinsics, they still underperform calibration-free CamVLA (51.4%). Jointly predicting hand-eye extrinsics forces the shared VLA backbone to implicitly understand camera geometry, yielding stronger manipulation performance than purely relying on external coordinate transformations.
(3) Inapplicability of Off-the-Shelf Foundation Models: General geometric foundation models like VGGT predict relative camera-to-camera poses with unknown scale rather than camera-to-robot-base extrinsics, so their predicted extrinsics cannot be directly used in practical calibration-free test scenarios where pre-calibrated reference views are unavailable. Meanwhile, baselines that rely on external extrinsics (AnyCamVLA and OC-VLA) already underperform calibration-free CamVLA (51.4%) even with ground-truth calibration, making an external foundation-model pose estimator unnecessary.