acceptodds
Under review as a conference paper at ICLR 2027

CoRe-VLA: Preserving Cross-View Coordination in VLAs under Camera Shifts

Abstract

VLAs combine pretrained vision-language representations with action generation to enable language-guided control across diverse tasks, becoming a mainstream paradigm in embodied intelligence. However, multiple studies have reported VLA's substantial declines in task success under camera shifts, revealing a key vulnerability that limits reliable deployment. To address this vulnerability, existing methods collect paired observations of the same scene from different viewpoints to fine-tune the VLA or train visual adaptation modules. Unfortunately, they require additional data collection and VLA training costs. In this paper, we first identify cross-view coordination breakdown under external camera shifts: the robot may rely too heavily on wrist-view cues and consequently execute subtasks in the wrong order when losing global view. Motivated by this, we propose CoRe-VLA, a plug-and-play framework requiring neither additional multi-view data collection nor VLA fine-tuning, which can incorporate with exsiting VLAs. It reconstructs a scene point cloud and renders the observation from the VLA's training viewpoint to restore cross-view coordination. In CoRe-VLA, Render-to-Camera (R2C) Restoration reduces rendering-induced visual degradation, while Execution-Trajectory-Conditioned Alignment (ETCA) reduces robot idle time and mitigates motion conflicts during asynchronous execution. Experiments on 5 real-robot tasks, LIBERO-100 and LIBERO-Plus demonstrate CoRe-VLA substantially improves task success across mainstream VLAs under camera shifts. For example, CoRe-VLA raises PI0.5's success rate from 13.3% to 83.3% at a 1.6m camera shift in real-robot environment. %

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Embodied intelligence, 3D point cloud, Vision-language-action model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pkwTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks why vision-language-action policies lose most of their competence when the external camera is moved, and what can be recovered without touching the policy. A diagnostic on five real tabletop tasks with three mainstream policies separates two failure types, failing to accomplish any subtask and executing a locally correct action that contradicts the global task state. The second dominates as the shift grows, which the paper names cross-view coordination breakdown and attributes to the policy leaning on wrist-view cues once the global view stops being informative. The method back-projects a shifted aligned RGB-D observation into a coloured point cloud, reprojects it to the training-time external-camera pose using a known relative transform and intrinsics, and repairs the resulting texture, colour and sharpness loss with a lightweight NAFNet trained on degraded and original frame pairs synthesized from existing demonstration videos. A second component, ETCA, treats each arriving action chunk as a trajectory-matching problem, aligns candidate prefixes against the joint states recorded during inference, and resumes at the best match, rejecting the chunk when the match is weak. Experiments cover the five real tasks across a range of external-camera displacements, all of the LIBERO tasks, the LIBERO-Plus camera subset, and a timing study of ETCA.

Critical strengths
  1. The diagnosis earns the method. At the largest displacement the policies mostly do not fail to grasp. They execute a locally correct action that contradicts the global task state, which accounts for 76.7% of π0.5\pi_{0.5} trials and dominates for all three policies. Framing viewpoint robustness as coordination rather than perception is what makes a training-free wrapper plausible, and it predicts where the wrapper should help.
  2. The intervention is measured where it matters, with a frozen policy, three backbones, real hardware at four displacements, the full simulated suite, and the LIBERO-Plus camera subset where π0.5\pi_{0.5} rises from 72.0% to 89.1%. ETCA also cuts mean completion time from 64.5 s to 43.7 s with no success loss, so the latency objection is answered rather than deferred.
Critical weaknesses
  1. The wrapper consumes a quantity the paper never accounts for. Equation (1) takes the relative rigid transform between the shifted and the training-time external camera, together with the training intrinsics, as known. The word "known" appears once, and no procedure is given for obtaining either quantity once someone has moved the camera. In the real-robot study the transform is known by construction, because the experimenters position the camera themselves at sampled locations and re-aim it at the tabletop centre, and the LIBERO-Plus study restores the view with simulator-provided depth and camera parameters. Every reported number therefore comes from a camera pose that is known exactly, and in the simulation from depth free of sensor error. That is the easiest setting for the paper's economy argument, and the only acknowledgement of the dependence is one limitation sentence about depth accuracy. A sweep over injected pose error of a few centimetres and a few degrees, and over realistic depth noise, would show how much of the gain survives a calibrated camera. Without it I cannot tell whether CoRe-VLA is a deployable wrapper or a demonstration that, given the ground-truth training pose, reprojection plus restoration recovers a great deal of success.
  2. Roughly half the gain at the largest displacement comes from the component the paper documents least. At 1.6 m, reprojection without R2C already lifts π0.5\pi_{0.5} from 13.3% to 50.0% and the full pipeline reaches 83.3%, so R2C supplies 33.3 of the 70 points. At the nominal pose it also recovers most of the damage reprojection itself causes, with 96.7% for the raw image, 83.3% without R2C and 93.3% with it. Yet R2C's evidence is one qualitative comparison of restored crops, with no PSNR, SSIM or LPIPS on held-out pairs and no training recipe anywhere, no frame count, width, optimiser or schedule. The training pairs are projective warps with blacked-out holes, which the appendix concedes only approximate re-rendering artifacts, whereas at inference the artifacts also include colour drift and blur. Two cheap controls would settle the mechanism. Applying R2C to the raw shifted image without reprojection would exclude photometric re-normalisation toward the training distribution, and reprojecting to a deliberately wrong pose would show that the training viewpoint specifically is what matters. As written, the claim that restoring the nominal view preserves cross-view coordination is one of several readings these experiments allow.
Questions for authors
  1. How are the training-time intrinsics and the relative camera transform obtained in deployment, and how accurate must they be? A sweep over injected pose error of 2 to 5 cm and 2 to 5 degrees and over depth noise would show how much gain survives calibration error.
  2. Does R2C applied directly to the shifted image, with no reprojection, recover part of the 1.6 m gap? I want to separate geometry from photometric re-normalisation.
  3. Who labelled the Type-1 and Type-2 outcomes, were they blind to condition, and what agreement was reached? Also, which nominal success rate is correct, 98.0% in Section 3 or 96.7% in Figure 2 and Table 2?
  4. Please give the R2C recipe (frame count, demonstrations, NAFNet variant, optimiser, schedule) and PSNR/SSIM on held-out pairs, and whether the five tasks and three backbones share one model.
8 comments
AuthorsOpenReview ↗

Response to Reviewer pkwT Part 1

Update (Nov 15): We have updated Table R11 with the results for calibration errors, depth noise, and their combination, and added the corresponding analysis below.

Update (Nov 16): We have incorporated the camera-calibration procedure and parameter sources to Section C, and included the calibration-error and depth-noise experiments and the analysis to Appendix N of the revised submission.

We sincerely thank you for your time and constructive feedback. We provide detailed responses to each of your concerns below.

W1: The wrapper consumes a quantity the paper never accounts for. Equation (1) takes the relative rigid transform between the shifted and the training-time external camera, together with the training intrinsics, as known. The word "known" appears once, and no procedure is given for obtaining either quantity once someone has moved the camera. In the real-robot study the transform is known by construction, because the experimenters position the camera themselves at sampled locations and re-aim it at the tabletop centre, and the LIBERO-Plus study restores the view with simulator-provided depth and camera parameters. Every reported number therefore comes from a camera pose that is known exactly, and in the simulation from depth free of sensor error. That is the easiest setting for the paper's economy argument, and the only acknowledgement of the dependence is one limitation sentence about depth accuracy. A sweep over injected pose error of a few centimetres and a few degrees, and over realistic depth noise, would show how much of the gain survives a calibrated camera. Without it I cannot tell whether CoRe-VLA is a deployable wrapper or a demonstration that, given the ground-truth training pose, reprojection plus restoration recovers a great deal of success.

Q1: How are the training-time intrinsics and the relative camera transform obtained in deployment, and how accurate must they be? A sweep over injected pose error of 2 to 5 cm and 2 to 5 degrees and over depth noise would show how much gain survives calibration error.

Reply to R1 & Q1 Part 1:

Thank you for highlighting the limitations. We: 1. clarify the procedure for obtaining camera parameters and demonstrate that CoRe-VLA is a practically deployable, plug-and-play method; 2. evaluate the method’s robustness under imperfect inputs by injecting camera translation and rotation errors and depth noise.

Clarification for obtaining camera prarameters. In our real-robot experiments and practical deployment, the required camera poses are obtained using a small printed calibration board, whose material cost is limited to printing. In our real-robot experiments, we use an 18 cm × 13.5 cm board placed in the scene Fig\. 1. The nominal and shifted camera poses are calculated relative to this calibration board, allowing their relative transform to be computed. The required camera intrinsics can be obtained from the training dataset’s calibration records. These settings enable CoRe-VLA to be deployed directly in existing VLA systems.

Table R11. Robustness to additional calibration errors and depth noise. Depth-noise levels denote standard deviations relative to the depth value.

SettingAdditional translation error cmcmAdditional rotation error °°Additional depth noiseT1 success % ↑T2 success % ↑T3 success % ↑T4 success % ↑T5 success % ↑Overall success % ↑
Raw shifted input———0.033.30.016.716.713.3
CoRe-VLA (no additional error)00050.0100.083.3100.083.383.3
Translation error20066.783.383.3100.083.383.3
Translation error50066.783.383.3100.083.383.3
Rotation error02050.083.383.3100.083.380.0
Rotation error05066.7100.083.3100.083.386.7
Depth noise001%50.0100.083.383.383.380.0
Depth noise002%50.0100.083.383.366.776.7
Combined errors552%50.0100.083.383.383.380.0
AuthorsOpenReview ↗

Response to Reviewer pkwT Part 2

Update (Nov 15): We have updated Table R11 with the results for calibration errors, depth noise, and their combination, and added the corresponding analysis below.

Update (Nov 16): We have incorporated details of R2C to the method and implementation to Section 4.2, and include the quantitative evaluation and ablation experiments in Section 5.5 and Appendix O of the revised submission.

Reply to R1 & Q1 Part 2:

Robustness experiment of Calibration Errors and Depth Noise. To specify the required accuracy of camera parameters and depth measurements, we conduct experiments to assess CoRe-VLA’s robustness during task execution under injected translation errors, rotation errors, and depth noise. In Table R11, each condition covers five real-robot tasks, two layouts per task, and three randomly selected errors at a 1.6 m displacement. Depth-noise levels denote standard deviations relative to the depth value. “No additional error” retains the existing calibration and sensor errors.

The results show that CoRe-VLA effectively improves task success even in the presence of calibration errors, depth noise, and their combination, increasing the success rate from 13.3% to 76.7%–86.7%. Compared to the 83.3% success rate achieved when no errors were introduced, there is no significant difference. These results support CoRe-VLA’s robustness to imperfect calibration and depth measurements.

W2: Roughly half the gain at the largest displacement comes from the component the paper documents least. At 1.6 m, reprojection without R2C already lifts from 13.3% to 50.0% and the full pipeline reaches 83.3%, so R2C supplies 33.3 of the 70 points. At the nominal pose it also recovers most of the damage reprojection itself causes, with 96.7% for the raw image, 83.3% without R2C and 93.3% with it. Yet R2C's evidence is one qualitative comparison of restored crops, with no PSNR, SSIM or LPIPS on held-out pairs and no training recipe anywhere, no frame count, width, optimiser or schedule. The training pairs are projective warps with blacked-out holes, which the appendix concedes only approximate re-rendering artifacts, whereas at inference the artifacts also include colour drift and blur. Two cheap controls would settle the mechanism. Applying R2C to the raw shifted image without reprojection would exclude photometric re-normalisation toward the training distribution, and reprojecting to a deliberately wrong pose would show that the training viewpoint specifically is what matters. As written, the claim that restoring the nominal view preserves cross-view coordination is one of several readings these experiments allow.

Q2: Does R2C applied directly to the shifted image, with no reprojection, recover part of the 1.6 m gap? I want to separate geometry from photometric re-normalisation.

Q4: Please give the R2C recipe (frame count, demonstrations, NAFNet variant, optimiser, schedule) and PSNR/SSIM on held-out pairs, and whether the five tasks and three backbones share one model.

Reply to W2 & Q2 & Q4 Part 1:

Thank you for highlighting the limitations. We: 1. clarify the implementation and training details of R2C; 2. collect image pairs of the same scenes and evaluate R2C using PSNR, SSIM, and LPIPS; 3. add ablation experiments to isolate the contributions of R2C and viewpoint transformation. We will add details of R2C to the method and implementation sections, and include the quantitative evaluation and ablation experiments in the experimental section.

Implementation and training details of R2C. R2C learns to restore degraded inputs using the corresponding original demonstration images as supervision. A lightweight NAFNet is trained with the perceptual, color, and Charbonnier losses defined in Eqs. 22–33, while keeping the VLA frozen. Corruption consists of projective warping, short gaps near object contours, and sparse clusters of missing pixels, which are sequantially applied. Uncovered and missing pixels are set to black. This procedure approximates rendering artifacts without requiring additional multi-view data collection. In our real-world experiment, a single real-robot R2C checkpoint is shared across all five tasks and π₀.₅, OpenVLA-OFT, and GR00T, without task-specific or backbone-specific retraining. The detailed settings are provided in the Table R12.

Table R12. Training settings of R2C. A single R2C checkpoint is shared across all five real-robot tasks and three VLA backbones.

SettingValue
Training / validation split90% / 10%
Training crop size256 × 256 pixels
OptimizerAdamW
Initial learning rate1 × 10⁻⁴
Batch size16
Training steps50,000
Learning-rate scheduleCosine decay
AuthorsOpenReview ↗

Response to Reviewer pkwT Part 3

Update (Nov 15): We have updated Tables R13 and R14 with the quantitative R2C evaluation and the ablation results for viewpoint transformation and image restoration, and added the corresponding analysis below.

Reply to W2 & Q2 & Q4 Part 2:

Table R13. Image restoration quality with and without R2C at different camera displacements.

Displacement mmMethodPSNR dBdB ↑SSIM ↑LPIPS ↓
0.0Reprojection only35.30.880.07
0.0Reprojection + R2C40.50.970.03
0.8Reprojection only19.90.630.26
0.8Reprojection + R2C23.60.770.18
1.6Reprojection only17.10.600.30
1.6Reprojection + R2C22.10.750.24

Table R14. Ablation of viewpoint transformation and R2C. Task success rates at the camera displacement of 1.6m are compared to assess the contributions of geometric alignment and image restoration.

SettingT1 success ↑T2 success ↑T3 success ↑T4 success ↑T5 success ↑Overall success ↑
Raw shifted input0.033.30.016.716.713.3
R2C directly applied to shifted input16.733.30.00.016.713.3
Reprojection to an incorrect viewpoint + R2C16.733.366.766.750.046.7
Full CoRe-VLA50.0100.083.3100.083.383.3

Quantitative evaluation of R2C. Table R13 compares image quality before and after R2C restoration. For this quantitative evaluation, we collected 225 image pairs from the scenes used in our real-robot experiments. Each pair contains a shifted-view RGB observation and a real nominal-view reference image captured with the same scene layout and robot state. “Reprojection only” transforms the shifted observation into the nominal viewpoint; “Reprojection + R2C” additionally applies the trained restoration network. Both outputs are compared with the same reference image using PSNR, SSIM, and LPIPS, averaged separately at each displacement. The evaluation covers all five tasks, and the same R2C checkpoint is used throughout. These pairs are excluded from R2C training and checkpoint selection.

The results show that R2C consistently improves all three image-quality metrics across the evaluated camera displacements. Compared with reprojection alone, R2C increases PSNR by 3.7–5.2 dB and SSIM by 0.09–0.15, while reducing LPIPS by 0.04–0.08. At the largest displacement of 1.6 m, PSNR improves from 17.1 to 22.1 dB, SSIM increases from 0.60 to 0.75, and LPIPS decreases from 0.30 to 0.24. These improvements show that the training data we construct enable R2C to effectively improve the quality of images rendered from point clouds across different camera displacements in real-world settings.

Additional ablation experiments. Table R14 presents the four input conditions. T1–T5 and overall task success rates. All conditions use the frozen π₀.₅ backbone at a 1.6 m camera displacement, covering five tasks and two layouts per task. The first condition directly supplies the shifted image to the VLA. The second applies R2C without changing the viewpoint. The third reprojects the observation to a deliberately incorrect viewpoint before applying R2C, using three randomly selected target viewpoints per layout. These viewpoints follow the planned 0.4–1.6 m sampling range. The final condition follows the full CoRe-VLA pipeline, reprojecting to the nominal viewpoint before restoration. The R2C checkpoint and execution settings remain unchanged across the applicable conditions.

Table R14 shows that applying R2C directly to shifted inputs leaves overall success unchanged at 13.3%, indicating that image restoration without viewpoint alignment can not recover task performance. Reprojection to an incorrect viewpoint followed by R2C achieves 46.7% success. Although success rate improves in this setting, it is because the incorrect target viewpoint is probably closer to the training viewpoint than the original 1.6 m displacement. Nevertheless, its success rate remains 36.6 percentage points lower than that achieved by reprojecting to the training camera pose, highlighting the central importance of aligning the place of the viewpoint in our method.

Together, these two experiments support the complementary roles of geometric alignment and image restoration: reprojection aligns the observation with the nominal viewpoint, while R2C improves the quality of the reprojected image. The observed performance gains therefore cannot be explained by photometric re-normalization alone.

Reviewer_pemrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the problem of action generation of VLAs under camera shifts. It proposes CoRe-VLA, a framework that can be used in existing VLA pipelines without additional multi-view data or fine-tuning. The key mechanisms include 3D point cloud reconstruction of the scene and re-rendering to restore the view, and a trajectory alignment procedure called ETCA for final asynchronous control. The method is evaluated on the LIBERO-100 and -Plus benchmarks under camera shift scenarios and demonstrates improved performance over baselines.

Critical strengths
  • The paper has a clear motivation for restoring views for a shifted camera using 3D point cloud reconstruction, RGB image re-rendering, and style restoration. The design of the asynchronous control to address the latency issue is a good design choice.
  • The complete system seems to be working properly on the tested environment on LIBERO-100 and -Plus, with solid numbers of improvement over the \pi baseline (Table 3 and Figure 6).
Critical weaknesses
  • The cross-view coordination breakdown in Section 3 is a bit underwhelming as a systematic analysis. It is somewhat expected that when the model was trained in the normal view but tested in the shifted view, the performance will degrade: the well-studied domain shift problem. There is no meaningful difference in failure type 1 across different camera displacements. The information provided by the experiment in Figure 2 is pretty minimal, and it cannot directly lead to the motivation in lines 206-209 that 'the robot may rely too heavily on wrist-view cues' or the need to 'restoring the nominal external view'. I think some more fine-grained control experiments (and with more metrics or analysis tools) might be designed to really introduce such an evidence-based motivation.
  • Fundamentally, I'm not fully convinced that the solution to the problem of shifted views in VLAs is to estimate a 3D point cloud (a challenging and open research question itself) and then sample the training view and render a synthetic RGB image from sparse 3D points (another challenging and open research question). It just seems to be solving a few more difficult tasks, especially in the wild environments where the shifted views will happen more often.
Questions for authors
  1. If we don't use 3D point cloud reconstruction, but directly train a diffusion model to restore the shifted view back to the normal view, will this work? Given the limited shift intervals in the current setup, it seems the diffusion model might only need to learn a small number of novel views.
  2. The motivation, solution, and evaluation are all from the same camera shift setup. Can this method generalize to in-the-wild use cases with diverse camera changes?
9 comments
AuthorsOpenReview ↗

Response to Reviewer pemr Part 1

Update (Nov 16): We have incorporated these experiments and the corresponding discussion into Section 3 of the revised submission.

We sincerely thank you for your time and constructive feedback. We provide detailed responses to each of your concerns below.

W1: The cross-view coordination breakdown in Section 3 is a bit underwhelming as a systematic analysis. It is somewhat expected that when the model was trained in the normal view but tested in the shifted view, the performance will degrade: the well-studied domain shift problem. There is no meaningful difference in failure type 1 across different camera displacements. The information provided by the experiment in Figure 2 is pretty minimal, and it cannot directly lead to the motivation in lines 206-209 that 'the robot may rely too heavily on wrist-view cues' or the need to 'restoring the nominal external view'. I think some more fine-grained control experiments (and with more metrics or analysis tools) might be designed to really introduce such an evidence-based motivation.

Reply to W1 Part 1:

Thank you for highlighting the limitations of our original analysis. We have conducted two complementary experiments to provide more fine-grained evidence for our motivation:

  1. We subdivide the original two failure types into six fine-grained categories: wrong target selection, wrong subtask order, skipped prerequisites, incorrect placement relations, local execution failures, and stalling/wandering/jitter. The results show that camera shifts mainly disrupt the selection and coordination of task-appropriate actions.
  2. We conduct a paired-layout experiment with fixed local inputs: by holding the wrist-view image fixed and changing only task-relevant layouts outside the wrist camera’s field of view, we directly show that robots rely too heavily on wrist-view cues under external-camera shifts.

We have incorporated these experiments and the corresponding discussion into Section 3 of the revised submission.

Fine-grained failure analysis. We subdivide the original failure types into six behavioral categories: local execution failure and stall/wander/jitter under Type 1; and wrong target, wrong subtask order, skipped prerequisite, and wrong placement relation under Type 2. We analyze π₀.₅, OpenVLA-OFT, and GR00T on five real-robot tasks, with 30 trials per model at each displacement of 0, 0.4, 0.8, 1.2, and 1.6 m.

Table R4. Fine-grained outcome counts under external-camera shifts. Each model–displacement setting includes 30 trials. Type 1 and Type 2 failures are further divided into six categories.

ModelDisplacement mmTotal trialsSuccessType 1Type 2Wrong target (Type 2)Wrong subtask order (Type 2)Skipped prerequisite (Type 2)Wrong placement (Type 2)Local execution failure (Type 1)Stall / wander / jitter (Type 1)
π₀.₅0.0302910000010
π₀.₅0.4302046212131
π₀.₅0.83015312424221
π₀.₅1.2307320646421
π₀.₅1.6304323656621
OpenVLA-OFT0.0302712100110
OpenVLA-OFT0.43014313434221
OpenVLA-OFT0.8309516535332
OpenVLA-OFT1.2308418545431
OpenVLA-OFT1.6302523666532
GR00T0.0302541001031
GR00T0.43011613434242
GR00T0.8308319636421
GR00T1.2303324666621
GR00T1.6302424666631

Table R4 and Fig. R1

https://openreview.net/attachment?id=97cAXmQ1Zk&name=supplementary_material

show the distribution of fine-grained failure categories and how their proportions change with camera displacement. Wrong target selection and skipped prerequisites exhibit the largest increases, each rising by 15.6 percentage points relative to the normal view. Wrong subtask order and wrong placement relation increase by 12.8 and 11.4 percentage points, respectively. The larger increases concern decisions such as which object to manipulate, whether prerequisites are satisfied and which subtask to perform next. These decisions require relating local actions to the global task state. For example, grasping the wrong object can involve successful local manipulation while violating the task requirement. Together with the smaller increases in execution failures and stall/wander/jitter, this pattern suggests that camera shifts increasingly disrupt the selection and coordination of task-appropriate actions. This motivates us to conduct further experiments to test whether robots rely too heavily on wrist-view cues and fail to respond to changes in the global scene.

AuthorsOpenReview ↗

Response to Reviewer pemr Part 2

Update (Nov 16): We have incorporated paired-layout experiment and the corresponding discussion into Section 3 and Appendix I of the revised submission.

Reply to W1 Part 2:

Paired-layout experiment. In the experiment, the robot state, instruction, and wrist-view image are kept fixed within each pair, while task-relevant information outside the wrist view is changed so that the correct next decision differs. The changes involve the cup location in T1, basket location in T2, lid state in T3, basket position relative to the cup in T4, and the identity of the object nearest to the basket in T5. For example, T5 requires placing the object closest to the basket into it. We preserve the wrist-visible objects but move the basket outside the wrist view, changing which object is closest. Correct behavior therefore requires selecting different objects based on the external view.For each model and displacement, we evaluate two layout pairs per task: 10 pairs, comprising 20 individual layouts. We report two metrics: ii Single-layout accuracy: the number of successful task executions divided by 20. iiii Paired-layout accuracy: the number of layout pairs in which the task is successfully completed in both layouts divided by 10. Paired-layout accuracy explicitly requires correct responses to both global configurations despite identical local inputs.

Table R5. Results of the paired-layout experiment under external-camera shifts. Each setting includes 10 layout pairs (20 individual executions). A pair is counted as correct only when both layouts are executed correctly.

ModelDisplacement mmLayout pairs PPCorrect single-layout executionsPairs with both layouts correct
π₀.₅0.0102010
π₀.₅0.410188
π₀.₅0.810166
π₀.₅1.210123
π₀.₅1.61091
OpenVLA-OFT0.010189
OpenVLA-OFT0.410124
OpenVLA-OFT0.81092
OpenVLA-OFT1.21070
OpenVLA-OFT1.61070
GR00T0.0102010
GR00T0.410188
GR00T0.810145
GR00T1.21082
GR00T1.61071

As shown in Table R5 and Fig. R2

https://openreview.net/attachment?id=97cAXmQ1Zk&name=supplementary_material

, under the normal external view, paired-layout accuracy is 100%, 90%, and 100% for π₀.₅, OpenVLA-OFT, and GR00T, respectively, showing that these models can generally respond correctly to the global layout changes. At 0.8 m, these rates fall to 60%, 20%, and 50%. At 1.6 m, only 1/10, 0/10, and 1/10 pairs receive correct responses in both layouts. Single-layout accuracy remains 45%, 35%, and 35%, respectively. It reveals that individual correct responses still occur, but reliable responses across both configurations become rare. With wrist inputs held fixed within each pair, this trend demonstrates reduced ability to adapt decisions to global scene changes as the external camera shifts.

Fig. R3

https://openreview.net/attachment?id=97cAXmQ1Zk&name=supplementary_material

presents a representative example: without camera displacement, the robot selects appropriate actions based on whether the basket lid outside the wrist camera’s field of view is open or closed. When the external camera is shifted, the robot fails to integrate information from the external view and simply grasps the tape measure visible in the wrist view. This illustrates a typical cross-view coordination breakdown.

Together, the two analyses connect specific failure patterns with a controlled test of global-information use. Global-constraint violations increase disproportionately, while correct responses to global layout changes deteriorate despite unchanged local inputs. These findings support insufficient use of global information under external-camera shifts and are consistent with the hypothesis that robots rely too heavily on wrist-view cues. They therefore provide an evidence-based motivation for restoring the normal external view, so that the coordination between the wrist and external views can be recovered.

AuthorsOpenReview ↗

Response to Reviewer pemr Part 3

Update (Nov 16): We have incorporated the experiment about diffusion model into Appendix K of the revised submission.

W2: Fundamentally, I'm not fully convinced that the solution to the problem of shifted views in VLAs is to estimate a 3D point cloud (a challenging and open research question itself) and then sample the training view and render a synthetic RGB image from sparse 3D points (another challenging and open research question). It just seems to be solving a few more difficult tasks, especially in the wild environments where the shifted views will happen more often.

Q1: If we don't use 3D point cloud reconstruction, but directly train a diffusion model to restore the shifted view back to the normal view, will this work? Given the limited shift intervals in the current setup, it seems the diffusion model might only need to learn a small number of novel views.

Reply to W2 & Q1 Part 1:

Thank you for raising this question. We first clarify an important a point of understanding: directly training a task-specific diffusion model to restore shifted views requires additional paired-view data, which can be difficult and costly to collect in real-world settings. CoRe-VLA instead aims to provide a “camera-shift robustness patch” for existing VLAs, in which substantial time and effort have already been invested, thereby enhancing their capabilities (improvements are illustrated in Table 2 of the original submission) without collecting any data. Consequently, our approach is compelled to adopt a technical path that appears more challenging, which is dictated by how we have framed the problem and positioned our paper. Nevertheless, we greatly appreciate your suggestion. Applying diffusion to viewpoint transformation is indeed a viable approach, so we have added corresponding experiments. The detailed experimental setup, results, and analysis are presented below.

To train the diffusion model, we collected 225 image pairs from the scenes used in our real-robot experiments. Each pair contains a shifted-view RGB observation and a real nominal-view reference image captured with the same scene layout and robot state. A fixed set of 25 image pairs is used for validation and testing, while the remaining 200 pairs are used for training. We choose VIVID [1] as our diffusion model because it is a recent state-of-the-art method for novel view synthesis. Its publicly available code and pretrained checkpoints allow us to evaluate its performance when trained with different amounts of additional data. We use additional-data budgets of 0, 50, 100, and 200 image pairs, randomly sampled from the training set. For nonzero budgets, we fine-tune the model for 8,196 steps and keep all other training settings at their default values.

[1] Elata, Noam, et al. "Novel view synthesis with pixel-space diffusion models." IEEE Conference on Computer Vision and Pattern Recognition. 2025.

Table R6. Task success and Type 2 failure rates with different additional-data budgets. The data budgets apply to diffusion fine-tuning; CoRe-VLA requires no additional training-data collection.

Displacement mmAdditional training pairs (Diffusion)Diffusion success ↑Diffusion Type 2 failure ↓CoRe-VLA success ↑CoRe-VLA Type 2 failure ↓
0.0093.3%0.0%93.3%0.0%
0.05093.3%3.3%93.3%0.0%
0.010096.7%0.0%93.3%0.0%
0.020093.3%0.0%93.3%0.0%
0.8013.3%80.0%93.3%6.7%
0.85020.0%66.7%93.3%6.7%
0.810036.7%50.0%93.3%6.7%
0.820076.7%10.0%93.3%6.7%
1.603.3%86.7%83.3%13.3%
1.65023.3%73.3%83.3%13.3%
1.610046.7%46.7%83.3%13.3%
1.620046.7%43.3%83.3%13.3%

Table R7. Inference latency comparison using the π₀.₅ backbone.

MethodInference latency msms ↓
π₀.₅244
Diffusion + π₀.₅2,514
CoRe-VLA + π₀.₅478

Table R8. Image restoration quality at different camera displacements. Higher PSNR and SSIM and lower LPIPS indicate better restoration quality. VIVID is trained using 200 additional data pairs.

Displacement mmMethodPSNR dBdB ↑SSIM ↑LPIPS ↓
0.0VIVID41.00.980.01
0.0CoRe-VLA40.60.970.03
0.8VIVID21.10.760.22
0.8CoRe-VLA23.70.770.20
1.6VIVID19.50.700.33
1.6CoRe-VLA22.50.750.24
Reviewer_dhs4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoRe-VLA improves viewpoint robustness of VLA policies by reprojecting calibrated RGB-D observations into the training camera view, restoring rendering artifacts with a NAFNet-based R2C module, and preserving the wrist stream. R2C learns from synthetic degradations of existing demonstrations. ETCA aligns incoming action chunks with the measured trajectory executed during inference to support asynchronous control. Evaluation covers three backbones, five real-robot tasks, LIBERO-100, and LIBERO-Plus Camera. The real-robot policies are first fine-tuned on 100 nominal-view demonstrations per task, then frozen when CoRe-VLA is applied.

Critical strengths
  1. The method shows large robustness gains across policies and benchmarks. At a 1.6 m camera displacement, success rises from 13.3% to 83.3% for π0.5, 6.7% to 60.0% for OpenVLA-OFT, and 6.7% to 80.0% for GR00T under matched conditions (Table 2). LIBERO-100 and LIBERO-Plus provide additional support.
  2. This paper privides useful evidence for individual components. At 1.6 m, π0.5 success improves from 13.3% with raw observations to 50.0% with geometry alone and 83.3% with R2C (Fig. 7). On nominal-view trials successful under all four configurations, ETCA reduces CoRe-VLA completion time from 64.5 to 43.7 seconds and idle time from 39.1 to 13.8 seconds (Table 4).
Critical weaknesses
  1. Missing implementation details limit reproducibility. The paper leaves out the real-camera calibration procedure, R2C training settings and corruption ranges, and whether restoration weights are shared across tasks and benchmarks. ETCA's matching weights, candidate range, thresholds, and blending settings are also underspecified. These omissions make the pipeline and its adaptation cost difficult to assess.
  2. ETCA's alignment benefit is not isolated. Table 4 compares ETCA with synchronous execution but includes no simpler asynchronous alignment baseline. The size of the common-success subset, timing variability, and conflict/rejection statistics are also unreported. ETCA's specific benefit and robustness to variable latency remain unclear.
Questions for authors

please see the weakness part.

6 comments
AuthorsOpenReview ↗

Response to Reviewer dhs4 Part 1

Update (Nov 16): We have added the missing implementation details to the method and implementation sections of the revised submission.

We sincerely thank you for your time and constructive feedback. We provide detailed responses to each of your concerns below.

W1: Missing implementation details limit reproducibility. The paper leaves out the real-camera calibration procedure, R2C training settings and corruption ranges, and whether restoration weights are shared across tasks and benchmarks. ETCA's matching weights, candidate range, thresholds, and blending settings are also underspecified. These omissions make the pipeline and its adaptation cost difficult to assess.

Reply to W1:

Thank you for highlighting the limitations. We have released our code at [https://anonymous.4open.science/r/CoRe-VLA-97EF/ ]. We have also added the missing implementation details to the method and implementation sections of the revised submission. Below, we describe CoRe-VLA’s implementation details point by point.

The pipine of CoRe-VLA. The pipeline consists of the R2C training stage and the inference stage. R2C is trained offline to restore degraded images. The inference stage begins with camera calibration to obtain the parameters required for reprojection. RGB-D observations are then reprojected into the nominal external view, restored by R2C, and passed to the frozen VLA. ETCA aligns each predicted action chunk with the measured robot trajectory before execution.

Implementation and training settings of R2C. R2C learns to restore degraded inputs using the corresponding original demonstration images as supervision. A lightweight NAFNet is trained with the perceptual, color, and Charbonnier losses defined in Eqs. (2)–(3), while keeping the VLA frozen. Corruption consists of projective warping, short gaps near object contours, and sparse clusters of missing pixels, which are sequantially applied. Uncovered and missing pixels are set to black. This procedure approximates rendering artifacts without requiring additional multi-view data collection. The detailed settings are provided in the table below.

Table R1. Training settings of R2C.

SettingValue
Training / validation split90% / 10%
Training crop size256 × 256 pixels
OptimizerAdamW
Initial learning rate1 × 10⁻⁴
Batch size16
Training steps50,000
Learning-rate scheduleCosine decay

Within each evaluation setting, the same R2C checkpoint is shared across tasks and all the VLA backbones. Separate checkpoints are used for the real-robot setting, LIBERO, and LIBERO-Plus. Thus, R2C does not require separate training for each task or backbone within a setting.

Camera calibration details. In our real-robot experiments and practical deployment, camera poses are estimated using an 18 cm × 13.5 cm printed calibration board placed in the scene (Fig. 1). The nominal and shifted camera poses are estimated relative to the same calibration board, allowing their relative transform to be computed. The board’s material cost is limited to printing. We will provide the board template and calibration instructions with the code to make this procedure straightforward to reproduce.

Implementation details of ETCA. ETCA uses measured six-joint trajectories in radians. Its reference index counts completed control actions since observation acquisition. Candidate action prefixes are linearly resampled to the measured trajectory length and scored using mean path L2 distance, endpoint L2 distance, displacement-direction disagreement, and deviation from the reference index. The table below specifies the specific implementation.

Table R2. Implementation settings of ETCA.

SettingValue
Matching weights: path / endpoint / direction / reference-index deviation1.0 / 1.5 / 0.05 / 0.003
Absolute acceptance thresholdsPath distance ≤0.25 rad; endpoint distance ≤0.35 rad
Candidate rangeReference index ±12 actions
Nominal action / servo frequency15 Hz / 500 Hz
Joint-target blendingUp to three actions using cubic smoothstep; new-target weights of 0.15, 0.5, and 0.85 for a three-action transition

The lowest-scoring eligible index is selected and checked for ambiguity. Expired, threshold-failing, or ambiguous chunks trigger a fresh observation and inference. When unused actions remain in the previous chunk, up to three new joint targets are blended with the corresponding old targets using the weights specified above. Gripper commands are not blended and follow the selected new-chunk actions.

For new tasks already supported by the VLA, CoRe-VLA can be readily applied to new scenes. The VLA and ETCA requires no further training. R2C needs to be retrained only when moving to a substantially different scene, such as from LIBERO simulation to the real world.

AuthorsOpenReview ↗

Response to Reviewer dhs4 Part 2

Update (Nov 15): We have updated Table R3 with the experimental results and added the corresponding analysis below.

Update (Nov 16): We have added Table R3 with corresponding analysis to the experimental section of the revised submission.

W2: ETCA's alignment benefit is not isolated. Table 4 compares ETCA with synchronous execution but includes no simpler asynchronous alignment baseline. The size of the common-success subset, timing variability, and conflict/rejection statistics are also unreported. ETCA's specific benefit and robustness to variable latency remain unclear.

Reply to W2:

Thank you for highlighting the limitations. We have added the simple asynchronous alignment baseline (Async) to Table 4 to isolate ETCA’s alignment benefit under matched execution conditions. The expanded table also provides the common-success subset size, timing variability, and conflict/rejection statistics. Furthermore, we have extended the evaluation to OpenVLA-OFT and GR00T to assess ETCA’s robustness under different inference-latency.

Table R3. Comparison of synchronous execution, simple asynchronous alignment, and ETCA across three VLA backbones with raw and restored inputs. Latency and latency variability (standard deviation) are reported in ms. Conflicts and rejections are average counts per task run. Dashes indicate not applicable.

MethodLatency(ms)Latency variability (new)Success rateCompletion time(s)Idle time(s)Conflicts (new)Rejections (new)
π₀.₅2010.9696.7%45.219.9——
π₀.₅ + Async (new)1980.9890.0%36.99.223—
π₀.₅ + ETCA2020.9896.7%32.58.3022
π₀.₅ + CoRe-VLA w/o ETCA4350.9893.3%63.036.2——
π₀.₅ + CoRe-VLA w/o ETCA + Async (new)4380.9386.7%51.315.734—
π₀.₅ + CoRe-VLA4380.9893.3%41.012.1035
OpenVLA-OFT57687.9690.0%77.434.0——
OpenVLA-OFT + Async (new)58185.2590.0%59.519.536—
OpenVLA-OFT + ETCA56487.2590.0%44.315.4034
OpenVLA-OFT + CoRe-VLA w/o ETCA80085.5586.7%109.352.2——
OpenVLA-OFT + CoRe-VLA w/o ETCA + Async (new)81383.8786.7%71.834.046—
OpenVLA-OFT + CoRe-VLA81487.2590.0%47.417.5050
GR00T15145.0683.3%46.410.2——
GR00T + Async (new)15745.9480.0%36.28.939—
GR00T + ETCA15948.5386.7%34.38.8034
GR00T + CoRe-VLA w/o ETCA36248.9483.3%64.722.6——
GR00T + CoRe-VLA w/o ETCA + Async (new)34150.2383.3%49.110.151—
GR00T + CoRe-VLA36444.1286.7%44.99.6051

The expanded Table R3 presents the overall experimental framework. For each backbone, we compare Sync, Async, and ETCA with both raw and restored inputs. Async simply selects fixed starting index using the number of control steps elapsed since observation acquisition. Latency variability reports the standard deviation of end-to-end inference latency. Confliction* reports the average number of action-chunk handoff conflicts per task run. Rejection* reports the average number of candidate chunks rejected by ETCA per task run. The common-success subset size used for the timing comparisons is 22.

As shown in Table R3, compared with synchronous execution, ETCA effectively reduces both completion and idle times, achieving reductions of 26.1%–56.6% and 13.7%–66.6%, respectively, across the six backbone/input combinations. Compared with the simple asynchronous baseline, the primary advantage of ETCA is that it prevents all conflicts through a rejection strategy. By facilitating more coherent action sequences, it enables faster task execution reducingcompletiontimeby52˙reducing completion time by 5\.2%–34\.0% compared to simple asynchronous baseline. Furthermore, the VLA backbones exhibit substantial differences in latency and latency variability, with mean latencies of 151–814 ms and standard deviations of 0.98–87.25 ms. ETCA consistently performs well across these backbones, supporting its robustness to variable latency.

* A handoff conflict is counted when the Euclidean distance between the raw six-joint targets of the old chunk’s last executed action and the new chunk’s first executed action exceeds 0.05 rad.

* ETCA-rejected chunks are not executed and therefore are counted separately as rejections. Their reinference waits remain included in Idle.

AuthorsOpenReview ↗

Update: Experiment Results and Analysis of Table R3

Thank you again for your feedback. We have now completed the additional ETCA evaluation, including the Async baseline, latency variability, and conflict/rejection statistics (Table R3). We have now edited that original response to include the completed results and corresponding analysis for your review. We would appreciate any further feedback on these results.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions and Additional Evidence

We sincerely thank the reviewers and Area Chair for their time and constructive feedback. We completed additional experiments, updated individual responses, and incorporated results and implementation details into the revised manuscript, marking substantive revisions in blue. Below, we summarize the main concerns and revisions.

  1. Evidence for cross-view coordination breakdown — Reviewer pemr (W1). Reviewer pemr questioned whether the original failure analysis sufficiently supported camera coordination breakdown under camera shifts. We added six-category failure analysis and a controlled paired-layout experiment. The analysis shows that shifts primarily disrupt task-level decisions. With identical initial wrist inputs but different global layouts, paired-layout success drops from 90–100% at the nominal view to 0–10% at 1.6 m. These results support reduced use of global information under camera shifts. Revised submission: Section 3 and Appendix I; rebuttal Tables R4–R5.

  2. Comparison with diffusion-based restoration — Reviewer pemr (W2, Q1). Reviewer pemr questioned the motivation for point-cloud reprojection and suggested restoring the nominal view with diffusion. We clarify that CoRe-VLA’s technical choices follow its design goal: improving existing, frozen VLAs without collecting additional multi-view training data. This differs fundamentally from diffusion-based methods requiring additional paired-view training data. We also evaluated a state-of-the-art diffusion model, VIVID, with different adaptation budgets: at 1.6 m, CoRe-VLA achieves 83.3% success versus 46.7% for VIVID adapted with 200 pairs, with lower inference latency using the same frozen π₀.₅ backbone. These results support our design in the tested limited-data setting. Revised submission: Appendix K; rebuttal Tables R6–R8.

  3. Contributions of viewpoint alignment and R2C — Reviewer pkwT (W2, Q2, Q4). Reviewer pkwT asked whether appearance normalization explained the gains and requested quantitative R2C evidence. We added R2C-only and incorrect-viewpoint controls. These show that R2C alone cannot recover task performance under camera shifts, while aligning observations to the nominal viewpoint is crucial to CoRe-VLA’s gains. We also evaluated restoration quality on 225 real image pairs excluded from R2C training and checkpoint selection. R2C improves PSNR, SSIM, and LPIPS across all tested displacements, including a PSNR increase from 17.1 to 22.1 dB at 1.6 m. These results support the complementary roles of correct viewpoint alignment and image restoration. Revised submission: Section 5.5 and Appendix O; rebuttal Tables R13–R14.

  4. ETCA’s benefit beyond asynchronous execution — Reviewer dhs4 (W2). Reviewer dhs4 noted that synchronous comparisons alone did not isolate ETCA’s alignment benefit and requested timing variability and conflict/rejection statistics. We added a simple asynchronous baseline and evaluated all three backbones with raw and restored inputs, reporting completion time, latency variability, and conflict/rejection counts. Compared with this baseline, ETCA aligns trajectories and rejects inconsistent chunks to avoid handoff conflicts, enabling smoother action transitions and reducing completion time by 5.2–34.0%. Revised submission: Section 5.5 and Appendix J; rebuttal Table R3.

  5. Robustness under broader deployment conditions — Reviewers pemr (Q2) and pkwT (W1, Q1). Reviewer pemr asked about diverse camera changes and environmental disturbances, while Reviewer pkwT questioned dependence on accurate calibration and depth. We added tests of camera roll, combined translation and roll, lighting changes, clutter, RGB noise, combined disturbances, and injected calibration errors and depth noise. CoRe-VLA maintains substantial gains under diverse camera changes, combined environmental disturbances, and imperfect calibration and depth measurements, supporting its robustness across the tested deployment conditions. Revised submission: Section 5.2 and Appendices L–N; rebuttal Tables R9–R11.

  6. Implementation details and annotation transparency — Reviewers dhs4 (W1) and pkwT (Q3, Q4). To address reproducibility concerns, we specified the R2C architecture, training and corruption settings, checkpoint sharing, calibration procedure and ETCA matching and execution settings. We clarified that one R2C checkpoint serves all five real-robot tasks and three backbones. Following pkwT’s annotation question, we documented the independent labeling procedure and agreement, and corrected the nominal success rate to 96.7% (29/30). Revised submission: Sections 3–4 and Appendices C–E and I.

We thank the reviewers for the suggestions motivating these additions and welcome further feedback on the new evidence and revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.