acceptodds
Under review as a conference paper at ICLR 2027

View-Robust Vision-Language-Action Model via Camera-Centric Reparameterization

Abstract

Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Viewpoint Robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uyfuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper takes the stance that a policy should not be told where the camera is, but rather infer it by itself. To achieve this, the authors propose CamVLA, that decouples manipulation control from camera geometry by predicting a camera-centric end-effector action in the local camera frame, and a hand-eye matrix relating cameras to the robot base. Experiments show clear improvement relative to baselines without this mechanism.

Critical strengths

This paper tackles what is from my perspective a clear weakness in current VLA methods, which is camera awareness and sensitivity to camera viewpoints.

Critical weaknesses

A clear weakness in this paper, as far as I can see, is the lack of comparison to baselines that use explicit camera information. If the claim is that "the policy should not be told where the camera is", there should be a comparison against methods that are told where the camera is. I agree that this would require privileged information, however geometric foundation models are very accessible and accurate these days, to the point where I assume camera intrinsics and extrinsics could be available if they are beneficial, and such experiments would provide a clear indication whether that is the case or not. The only mention to GT calibration is as part of ablations, and CamVLA shows marginal improvement relative to GT, which to me indicates that if such calibration is available the method does not bring much improvement. Furthermore, calibration is used at training time, and thus it might be sensible to assume that they will also be available at test time.

Questions for authors

Broadly speaking, I would like to hear from the authors some arguments to address my comments in the weakness section: what are the benefits of self-estimating calibration, vs. using off-the-shelf foundation models to produce those? Were there any studies showing the impact of camera miscalibration during training, and how that influences results? I imagine if the method is robust to inaccurate labels it could lead to some meaningful benefits over using those same inaccurate labels at test time.

3 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer uyfu (1/2)

We thank the reviewers (fNjd - R1, PF2A - R2, uyfu - R3) for their feedback. We are pleased that they found our work addresses a clear weakness (R3) and an important problem (R2) with compelling motivation (R2), and our method is novel (R2), well-motivated (R2), geometrically appropriate (R1), clear and easy to follow (R1, R2, R3), and easy to understand and potentially adopt (R1). It is also noted that we achieve consistent improvements across strong VLA backbones in simulation and real-world experiments (R1, R2, R3) with useful pose-error and ground-truth analyses (R1). We address here the comments and will incorporate all feedback in revision.

Review uyfu

Q1: What are the benefits of self-estimating calibration, vs. using off-the-shelf foundation models to produce those?

Thank you for this thoughtful question! First, in real-world deployments, test-time camera extrinsics are typically unavailable or invalidated due to camera bumps, remounting, or dynamic camera motion, so training-time calibration generally cannot be relied upon at test time. Moreover, CamVLA†\text{CamVLA}^\dagger (52.3%) in our ablations represents the Oracle upper bound of our framework using ground-truth extrinsics. The minimal gap between calibration-free CamVLA (51.4%) and CamVLA†\text{CamVLA}^\dagger (52.3%) confirms that our self-predicted extrinsics are highly accurate (1.41∘1.41^\circ error) and effectively replace GT calibration at test time, while outperforming the base-frame π0\pi_0 baseline (33.2%) by +18.2%.

To compare against methods that are explicitly "told where the camera is", we evaluate AnyCamVLA and OC-VLA using ground-truth camera extrinsics at test time alongside calibration-free π0\pi_0 and CamVLA:

MethodCalibration-Free?Success Rate (%)
π0\pi_0Yes33.2
AnyCamVLANo46.3
OC-VLANo49.4
CamVLA (Ours)Yes51.4

These results and geometric considerations highlight three clear conclusions regarding self-estimating calibration:

(1) Practicality for Real-World Deployment: Unlike AnyCamVLA and OC-VLA, which are not calibration-free and fail to apply when cameras experience real-world mounting disturbances or motion, CamVLA operates completely calibration-free at test time while outperforming all evaluated baselines (51.4%).

(2) Implicit Camera-Aware Geometric Understanding: Even when AnyCamVLA (46.3%) and OC-VLA (49.4%) are privileged with ground-truth test-time extrinsics, they still underperform calibration-free CamVLA (51.4%). Jointly predicting hand-eye extrinsics forces the shared VLA backbone to implicitly understand camera geometry, yielding stronger manipulation performance than purely relying on external coordinate transformations.

(3) Inapplicability of Off-the-Shelf Foundation Models: General geometric foundation models like VGGT predict relative camera-to-camera poses with unknown scale rather than camera-to-robot-base extrinsics, so their predicted extrinsics cannot be directly used in practical calibration-free test scenarios where pre-calibrated reference views are unavailable. Meanwhile, baselines that rely on external extrinsics (AnyCamVLA and OC-VLA) already underperform calibration-free CamVLA (51.4%) even with ground-truth calibration, making an external foundation-model pose estimator unnecessary.

AuthorsOpenReview ↗

Rebuttal for Reviewer uyfu (2/2)

Q2: Were there any studies showing the impact of camera miscalibration during training, and how that influences results? I imagine if the method is robust to inaccurate labels it could lead to some meaningful benefits over using those same inaccurate labels at test time.

Thank you for this insightful question!

(1) Robustness to extrinsic rotation inaccuracies: Our robustness analysis in Figure 6 evaluates CamVLA's tolerance to inaccurate extrinsic rotations. When rotation errors ranging from 0∘0^\circ to 45∘45^\circ are injected during execution, CamVLA suffers minimal degradation under small errors (64.0% at 0∘0^\circ, 63.3% at 1∘1^\circ, and 58.7% at 5∘5^\circ) and consistently outperforms the noise-free π0\pi_0 baseline (36.0%) even under rotation errors of up to 12∘12^\circ. Crucially, our Geometric Head achieves a mean rotation error of only 1.41∘1.41^\circ, falling well within this 12∘12^\circ tolerance regime with negligible performance degradation.

(2) Invariance to translation miscalibration and tolerance to real-world label noise: As derived in Appendix Sec. B, our relative-action formulation cancels the hand-eye translation vector analytically, making both camera-centric delta action conversion during training and policy execution completely independent of translation miscalibration. Together with our real-world experiments (Table 2), which already rely on imperfect physical hand-eye calibration yet consistently outperform baselines across all 0∘–15∘0^\circ\text{--}15^\circ shifts (79.0% vs. 63.3% for π0\pi_0 and 80.7% vs. 64.7% for GR00T N1.7 at 0∘0^\circ, and 29.3% vs. 16.0% and 33.0% vs. 14.7% at 15∘15^\circ), these results confirm CamVLA's robustness to moderate label inaccuracies.

(3) Benefits over directly using inaccurate calibration at test time: Rather than directly consuming inaccurate or drifted calibration labels at test time, CamVLA re-estimates extrinsics online from live visual inputs (1.41∘1.41^\circ error) while gaining stronger camera-aware policy representations through joint training. Consequently, CamVLA delivers clear advantages over test-time reliance on external extrinsics, outperforming even baselines privileged with ground-truth test-time calibration (51.4% vs. 49.4% for OC-VLA and 46.3% for AnyCamVLA).

AuthorsOpenReview ↗

Kind Reminder of the Discussion Period Ending Soon

Dear Reviewer uyfu,

We hope you are doing well.

As the discussion period will end on Nov 18, 11.59pm AoE, we would like to know whether our response has addressed your concerns, and we would really appreciate it if you could take some time to read our responses and provide any further feedback that you have. Your feedback is important to us, and we would like to have the opportunity to discuss them with you before the discussion period ends.

Thank you again for helping us, we believe our paper has been standing much stronger after incorporating your constructive suggestions.

Best regards,

Reviewer_PF2AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

To address the sensitivity of vision-language-action models to camera viewpoint shifts, this paper introduces CAMVLA, a novel architecture that predicts end-effector delta actions in the camera frame rather than the robot base frame. A lightweight MLP simultaneously estimates the camera-to-base extrinsics from a single RGB image, and the predicted rotation is used to transform the actions into the base frame. Ground-truth extrinsics are used only as supervision during training, eliminating the need for explicit camera calibration at deployment. On RLBench with unseen viewpoints, CAMVLA improves the success rate of π0\pi_0 from 33.2% to 51.4% and that of GR00T N1.7 from 28.4% to 38.4%. Real-world experiments on a Franka robot further demonstrate improved robustness to camera offsets of up to 15∘15^\circ.

Critical strengths

I find the motivation of this paper compelling. Viewpoint robustness is a real and practically important problem for deploying VLAs. I also agree with the authors that a standard VLA observes the scene in the camera frame but predicts actions in the robot base frame, which leaves a transformation gap that the network must bridge implicitly. Disentangling these two is a well-motivated idea.

The experimental results also show consistent improvements when the method is applied to two strong VLA baselines, π0\pi_0 and GR00T N1.7, in both simulation and real-world experiments.

Critical weaknesses

I will combine the weaknesses and questions together.

  1. This method relies on dense, calibrated multi-view data. It requires calibrated demonstrations from many viewpoints (13 in simulation, 5 in the real world, 100 demos each), and evaluation only interpolates within the training range. This conflicts with the authors’ own claim that multi-view data is “prohibitively expensive and hard to scale”, and the single-view setting motivating Figure 1 is never evaluated. It would be important to see results on viewpoints outside the training range, such as different camera heights, pitch angles, or distances. Moreover, with this data, the π0\pi_0 baseline already interpolates reasonably well (36.5% on adjacent seen views vs. 33.2% unseen, Table 6), and most of CamVLA’s gain already appears at seen views (54.4%). This suggests the improvement stems from better multi-view training rather than viewpoint generalization. Finally, π0.5\pi_{0.5} has been shown to generalize considerably better than π0\pi_0 and has been available for over a year. Could a stronger backbone such as π0.5\pi_{0.5}, without the proposed decoupling, already generalize to viewpoints outside the training range as well as or better than CamVLA?

  2. The pose regressor has limited generalization. Regressing camera pose with a 3-layer MLP from a handful of fixed training poses is prone to memorizing those poses. Table 3 supports this: the real-world error grows from 1.35 cm / 2.49$^\circ$ at 0$^\circ$ to 27.16 cm / 9.39$^\circ$ at 15$^\circ$, and the authors themselves note such overfitting for the VLM-feature variant (Table 9). In Figure 6, the rotation noise is resampled at every planning step, which likely averages out in closed loop, whereas real prediction errors at unseen viewpoints are more likely systematic biases. How does CamVLA perform under a static rotation bias of comparable magnitude?

  3. The real-world tasks are relatively easy and lack view-robust baselines. The real-world tasks are coarse tabletop tasks, and the perturbations are limited to horizontal rotations of at most 15° around the base, far from the hand-held or remounted cameras that motivate the paper; the hand-held setting is shown only qualitatively (Figure 14). In addition, no prior view-robust method (e.g., OC-VLA, AnyCamVLA, or novel-view augmentation) is compared.

These weaknesses and questions constitute my main concerns regarding this paper.

Questions for authors

See weaknesses above.

6 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer PF2A (1/3)

We thank the reviewers (fNjd - R1, PF2A - R2, uyfu - R3) for their feedback. We are pleased that they found our work addresses a clear weakness (R3) and an important problem (R2) with compelling motivation (R2), and our method is novel (R2), well-motivated (R2), geometrically appropriate (R1), clear and easy to follow (R1, R2, R3), and easy to understand and potentially adopt (R1). It is also noted that we achieve consistent improvements across strong VLA backbones in simulation and real-world experiments (R1, R2, R3) with useful pose-error and ground-truth analyses (R1). We address here the comments and will incorporate all feedback in revision.

Review PF2A

W1: This method relies on dense, calibrated multi-view data.

Thank you for raising these important points!

(1) Multi-view data and seen/unseen performance: To achieve cross-viewpoint generalization, multi-view training data is unavoidable, as no existing method can generalize across wide viewpoint shifts without multi-view data. Crucially, all reported results in the paper are trained on the same multi-viewpoint dataset, and our method achieves substantially better performance under the identical data budget. In fact, while π0\pi_0 achieves 65.3%65.3\% when trained on a single view (Fig. 1), pooling multi-view data degrades π0\pi_0 to 36.5%36.5\% on seen views and 33.2%33.2\% on unseen views (Table 6), because mapping divergent visual flows from multiple cameras to identical base-frame actions causes severe cross-view conflict. Standard VLAs thus struggle to effectively leverage multi-view data without massive scale, whereas CamVLA eliminates this coordinate conflict, improving both seen (54.4%54.4\%) and unseen (51.4%51.4\%) performance while halving the relative seen-to-unseen drop (4.1%4.1\% vs. 8.9%8.9\%). Furthermore, Table 5 validates that under identical sparser viewpoint configurations with less training data, CamVLA consistently achieves better cross-viewpoint performance.

(2) Viewpoints outside the training range: We evaluate π0\pi_0 and CamVLA under unseen camera height (±10 cm\pm 10\text{ cm}), camera distance (±10 cm\pm 10\text{ cm}), and camera roll (±10∘\pm 10^\circ) shifts:

ModelCamera Height (±10 cm\pm 10\text{ cm})Camera Distance (±10 cm\pm 10\text{ cm})Camera Roll (±10∘\pm 10^\circ)
π0\pi_025.226.822.8
CamVLA (Ours)39.541.236.4

We also evaluate viewpoint extrapolation beyond the training range (training on [0∘,30∘][0^\circ, 30^\circ] and testing at 45∘45^\circ and 60∘60^\circ):

Model45∘45^\circ60∘60^\circ
π0\pi_016.89.2
CamVLA (Ours)22.514.2

Across both tables, CamVLA consistently outperforms π0\pi_0, demonstrating that our camera-centric decoupling is not restricted to in-range interpolation, but also generalizes effectively to out-of-distribution viewpoints outside the training range.

(3) Comparison with a stronger backbone (π0.5\pi_{0.5}): We first evaluate π0.5\pi_{0.5} under the single-view training setting (0∘0^\circ only):

Model0∘0^\circ5∘5^\circ10∘10^\circ15∘15^\circ
π0\pi_065.323.09.36.3
π0.5\pi_{0.5}67.320.35.03.0

Under single-view training, although π0.5\pi_{0.5} outperforms π0\pi_0 on the training view (67.3%67.3\% vs. 65.3%65.3\%), it similarly collapses from 67.3%67.3\% to 3.0%3.0\% as the viewpoint shifts from 0∘0^\circ to 15∘15^\circ. We further evaluate π0.5\pi_{0.5} and CamVLA under the multi-view unseen-viewpoint setting:

ModelSlide BlockPush ButtonsTake UmbrellaClose LaptopLamp OffPut KnifeMean
π0\pi_018.351.532.357.029.810.033.2
π0.5\pi_{0.5}37.341.725.354.336.318.735.6
CamVLA (Ours)44.572.339.269.058.025.351.4

Under multi-view training, while π0.5\pi_{0.5} improves over π0\pi_0 (35.6%35.6\% vs. 33.2%33.2\%), π0.5\pi_{0.5} without decoupling still suffers from base-frame geometric entanglement and performs substantially worse than CamVLA (35.6%35.6\% vs. 51.4%51.4\%).

AuthorsOpenReview ↗

Rebuttal for Reviewer PF2A (2/3)

W2: The pose regressor has limited generalization. Regressing camera pose with a 3-layer MLP from a handful of fixed training poses is prone to memorizing those poses. Table 3 supports this: the real-world error grows from 1.35 cm / 2.49 at 0 to 27.16 cm / 9.39 at 15, and the authors themselves note such overfitting for the VLM-feature variant (Table 9). In Figure 6, the rotation noise is resampled at every planning step, which likely averages out in closed loop, whereas real prediction errors at unseen viewpoints are more likely systematic biases. How does CamVLA perform under a static rotation bias of comparable magnitude?

Thank you for your careful analysis and constructive suggestion!

(1) Pose regressor generalization: Memorizing discrete training labels (15∘15^\circ intervals) would yield ≥5∘\ge 5^\circ test error on unseen intermediate viewpoints, whereas our Geometric Head achieves 1.41∘1.41^\circ rotation error (3.5×3.5\times lower), demonstrating continuous geometric estimation rather than pose memorization. Zero-shot evaluation of the Geometric Head on 3 unseen RLBench scenes (cup, hanger, tap) yields 1.9∘1.9^\circ pose error (vs. 1.41∘1.41^\circ in-scene), confirming it relies on robot arm geometry rather than scene memorization. For Table 3 at 15∘15^\circ offset, our relative-action formulation cancels translation error (Supp. Sec. B), leaving a 9.39∘9.39^\circ rotation error within controller tolerance (≤12∘\le 12^\circ).

(2) Static rotation bias: To compare systematic bias with per-step resampled noise in Fig. 6, we inject a static (episode-constant) rotation bias throughout each episode:

Noise Type0∘0^\circ5∘5^\circ10∘10^\circ15∘15^\circ20∘20^\circ
Per-Step Resampled Noise (Fig. 6)64.058.742.032.021.7
Static Systematic Bias64.058.242.631.120.5

Static systematic bias yields comparable performance to per-step resampled noise across all noise levels, demonstrating that CamVLA remains robust to systematic rotation biases rather than relying on step-wise noise averaging.

AuthorsOpenReview ↗

Rebuttal for Reviewer PF2A (3/3)

W3: The real-world tasks are relatively easy and lack view-robust baselines. The real-world tasks are coarse tabletop tasks, and the perturbations are limited to horizontal rotations of at most 15° around the base, far from the hand-held or remounted cameras that motivate the paper; the hand-held setting is shown only qualitatively (Figure 14). In addition, no prior view-robust method (e.g., OC-VLA, AnyCamVLA, or novel-view augmentation) is compared.

Thank you for this valuable suggestion!

(1) Moving camera evaluation: To quantitatively assess hand-held and remounted cameras under continuous dynamic motions, we evaluate performance across camera angular speeds of 5∘/s5^\circ/\text{s}, 10∘/s10^\circ/\text{s}, and 15∘/s15^\circ/\text{s}:

Camera Angular Speed5∘/s5^\circ/\text{s}10∘/s10^\circ/\text{s}15∘/s15^\circ/\text{s}
π0\pi_028.621.414.2
CamVLA (Ours)43.234.825.6

Across all angular speeds, CamVLA consistently outperforms π0\pi_0, demonstrating that our camera-centric decoupling effectively generalizes to hand-held and remounted camera setups.

(2) Comparison with view-robust baselines: We compare CamVLA with viewpoint augmentation, AnyCamVLA, and OC-VLA across unseen viewpoints:

MethodCalibration-Free?Success Rate (%)
π0\pi_0Yes33.2
π0\pi_0 + Viewpoint AugmentationYes38.1
AnyCamVLANo46.3
OC-VLANo49.4
CamVLA (Ours)Yes51.4

Crucially, both OC-VLA and AnyCamVLA require ground-truth camera extrinsics (hand-eye matrices) at test time: OC-VLA relies on the ground-truth hand-eye matrix to transform camera-frame actions to the base frame, while AnyCamVLA conditions a feed-forward novel view synthesis (NVS) model on test camera extrinsics (tracked via ArUco markers in deployment) to virtually re-render test-time camera observations into the canonical training viewpoint. Furthermore, purely relying on externally provided extrinsics for coordinate transformation (OC-VLA, 49.4%) prevents the policy backbone from learning implicit camera-aware geometric representations acquired through our joint end-to-end training. In contrast, CamVLA is entirely calibration-free at test time and achieves 51.4%, consistently outperforming viewpoint augmentation (38.1%), AnyCamVLA (46.3%), and OC-VLA (49.4%).

Reviewer_fNjdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CamVLA predicts camera-frame end-effector increments and a camera-to-base transform, then rotates the increments into the robot frame. Extrinsics supervise training but are not provided at deployment, which uses monocular RGB, proprioception, and language. Experiments adapt pi0 and GR00T N1.7 in simulation and on a Franka robot.

Critical strengths
  1. The factorization is simple and geometrically appropriate for the stated relative-action convention. The derivation and rotation-only ablation explain why transform translation is unnecessary for execution, making the method easy to understand and potentially adopt.
  2. Gains appear on two backbones under matched multiview data and training budgets, including physical experiments. RLBench success rises from 33.2% to 51.4% for pi0. Pose-error measurements and comparisons with ground-truth extrinsics provide useful support beyond success rates.
Critical weaknesses
  1. Quantitative evidence primarily concerns local viewpoint changes. Simulation tests intermediate angles between training views, and real-world evaluation uses offsets up to 15 degrees around training cameras. This supports local robustness but does not establish the broader remounting claim. Moving-camera demonstrations provide qualitative evidence, not a quantified test of broader shifts.
  2. Table 7 lacks a base-frame policy with the same auxiliary pose head and supervision. Consequently, it does not separate the benefit of camera-frame actions from additional geometric supervision. This is particularly relevant because gains also occur at the canonical real-world view.
Questions for authors
  1. How does a base-frame policy with identical auxiliary pose supervision, optionally conditioned on the predicted rotation, perform?
  2. Are there quantitative results for camera height, radius, roll, or viewpoints outside the training range?
  3. Does the geometric head use Image Encoder or VLM Backbone features? Section 4.1 and Appendix C leave the attachment point unclear.
4 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer fNjd

We thank the reviewers (fNjd - R1, PF2A - R2, uyfu - R3) for their feedback. We are pleased that they found our work addresses a clear weakness (R3) and an important problem (R2) with compelling motivation (R2), and our method is novel (R2), well-motivated (R2), geometrically appropriate (R1), clear and easy to follow (R1, R2, R3), and easy to understand and potentially adopt (R1). It is also noted that we achieve consistent improvements across strong VLA backbones in simulation and real-world experiments (R1, R2, R3) with useful pose-error and ground-truth analyses (R1). We address here the comments and will incorporate all feedback in revision.

Review fNjd

Q1: How does a base-frame policy with identical auxiliary pose supervision, optionally conditioned on the predicted rotation, perform?

Thanks for your suggestion! We conduct an ablation study by extending Table 7 with two base-frame variants: (1) π0\pi_0 + Aux Pose Sup., which adds the identical Geometric Head and Lext\mathcal{L}_{ext} supervision to π0\pi_0, and (2) π0\pi_0 + Aux Pose + Rot. Cond., which further conditions the base-frame Action Head on the predicted rotation RtR_t.

Model VariantState InputAction OutputSuccess Rate (%)
π0\pi_0BaseBase33.2
π0\pi_0 + Aux Pose Sup.BaseBase34.5
π0\pi_0 + Aux Pose + Rot. Cond.BaseBase35.9
CamVLA (Ours)BaseCamera51.4

Adding the identical Geometric Head and Lext\mathcal{L}_{ext} supervision brings only a 1.3%1.3\% gain (34.5%34.5\% vs. 33.2%33.2\%), and further conditioning on the predicted rotation RtR_t reaches 35.9%35.9\%, as the network has to implicitly approximate 3D rotations. In contrast, predicting camera-frame actions with the deterministic transformation yields a +16.9%+16.9\% improvement (51.4%51.4\% vs. 34.5%34.5\%), clearly separating the benefit of camera-frame action parameterization from auxiliary geometric supervision.

Q2: Are there quantitative results for camera height, radius, roll, or viewpoints outside the training range?

Thank you for this helpful question! First, we evaluate π0\pi_0 and CamVLA under unseen camera height (±10 cm\pm 10\text{ cm}), camera radius (±10 cm\pm 10\text{ cm}), and camera roll (±10∘\pm 10^\circ) perturbations:

ModelCamera Height (±10 cm\pm 10\text{ cm})Camera Radius (±10 cm\pm 10\text{ cm})Camera Roll (±10∘\pm 10^\circ)
π0\pi_025.226.822.8
CamVLA (Ours)39.541.236.4
CamVLA†\text{CamVLA}^\dagger (GT extr.)41.843.038.2

Second, to evaluate viewpoints outside the training range, we train models on viewpoints within [0∘,30∘][0^\circ, 30^\circ] and test extrapolation at 45∘45^\circ and 60∘60^\circ:

Model45∘45^\circ60∘60^\circ
π0\pi_016.89.2
CamVLA (Ours)22.514.2
CamVLA†\text{CamVLA}^\dagger (GT extr.)27.419.6

Across both tables, CamVLA consistently outperforms π0\pi_0 and closely approaches CamVLA†\text{CamVLA}^\dagger (using ground-truth camera extrinsics), confirming that performance drops under extreme OOD shifts are primarily driven by visual appearance shifts rather than hand-eye estimation errors.

Q3: Does the geometric head use Image Encoder or VLM Backbone features? Section 4.1 and Appendix C leave the attachment point unclear.

Thank you for pointing this out! The Geometric Head uses visual features extracted from the Image Encoder by default, as ablated in Appendix C (Table 9).

Thank you for the additional experiments and clarifications. The auxiliary-pose controls largely address my concern about the contribution of camera-frame action prediction, and the clarification regarding Image Encoder features resolves my implementation question.

I have two remaining clarification requests:

  1. The third camera perturbation is described as roll in your response to me and pitch in your response to PF2A, with identical results. Could you confirm which rotation was evaluated and the corresponding axis?
  2. The extrapolation results are encouraging. However, replacing predicted extrinsics with ground-truth extrinsics improves success by 4.9 and 5.4 percentage points, suggesting that pose-estimation errors remain relevant under extrapolation. Could you clarify the basis for attributing the degradation primarily to visual appearance shifts, or qualify this interpretation?
AuthorsOpenReview ↗

Rebuttal for Reviewer fNjd

We thank the reviewer for the encouraging feedback and thoughtful follow-up.

Q1: The third camera perturbation is described as roll in your response to me and pitch in your response to PF2A, with identical results. Could you confirm which rotation was evaluated and the corresponding axis?

The evaluated perturbation was camera roll, and the corresponding axis is the camera's optical viewing axis (local camera ZZ-axis). We apologize for the typo and have corrected it.

Q2: The extrapolation results are encouraging. However, replacing predicted extrinsics with ground-truth extrinsics improves success by 4.9 and 5.4 percentage points, suggesting that pose-estimation errors remain relevant under extrapolation. Could you clarify the basis for attributing the degradation primarily to visual appearance shifts, or qualify this interpretation?

By decoupling camera-frame control from camera geometry, CamVLA consistently outperforms the baseline π0\pi_0 under extrapolation (22.5% vs. 16.8% at 45∘45^\circ; 14.2% vs. 9.2% at 60∘60^\circ).

The basis for this attribution is that the performance drop under extreme viewpoints is primarily driven by visual perspective shifts:

  1. In Table 5, sparser training views cause even the Oracle with ground-truth extrinsics (CamVLA†\text{CamVLA}^\dagger) to drop from 52.3% to 26.3% due to visual coverage differences.
  2. Under extrapolation, as the viewpoint moves further from 45∘45^\circ to 60∘60^\circ, Oracle CamVLA†\text{CamVLA}^\dagger drops by 7.8% (27.4% →\to 19.6%), whereas the gap between predicted and GT extrinsics remains nearly constant (4.9% vs. 5.4%, a mere 0.5% difference).

This confirms that the degradation as viewpoints move further out-of-distribution is primarily driven by visual perspective shifts. Additionally, our relative-action formulation analytically cancels translation errors (Supp. Sec. B), preventing growing translation drift under extreme viewpoints from affecting execution.

The additional controls and evaluations have substantially addressed my original concerns. I have no further questions and maintain my rating of 3 (Weak acceptance).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.