- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks why vision-language-action policies lose most of their competence when the external camera is moved, and what can be recovered without touching the policy. A diagnostic on five real tabletop tasks with three mainstream policies separates two failure types, failing to accomplish any subtask and executing a locally correct action that contradicts the global task state. The second dominates as the shift grows, which the paper names cross-view coordination breakdown and attributes to the policy leaning on wrist-view cues once the global view stops being informative. The method back-projects a shifted aligned RGB-D observation into a coloured point cloud, reprojects it to the training-time external-camera pose using a known relative transform and intrinsics, and repairs the resulting texture, colour and sharpness loss with a lightweight NAFNet trained on degraded and original frame pairs synthesized from existing demonstration videos. A second component, ETCA, treats each arriving action chunk as a trajectory-matching problem, aligns candidate prefixes against the joint states recorded during inference, and resumes at the best match, rejecting the chunk when the match is weak. Experiments cover the five real tasks across a range of external-camera displacements, all of the LIBERO tasks, the LIBERO-Plus camera subset, and a timing study of ETCA.
Critical strengths
- The diagnosis earns the method. At the largest displacement the policies mostly do not fail to grasp. They execute a locally correct action that contradicts the global task state, which accounts for 76.7% of trials and dominates for all three policies. Framing viewpoint robustness as coordination rather than perception is what makes a training-free wrapper plausible, and it predicts where the wrapper should help.
- The intervention is measured where it matters, with a frozen policy, three backbones, real hardware at four displacements, the full simulated suite, and the LIBERO-Plus camera subset where rises from 72.0% to 89.1%. ETCA also cuts mean completion time from 64.5 s to 43.7 s with no success loss, so the latency objection is answered rather than deferred.
Critical weaknesses
- The wrapper consumes a quantity the paper never accounts for. Equation (1) takes the relative rigid transform between the shifted and the training-time external camera, together with the training intrinsics, as known. The word "known" appears once, and no procedure is given for obtaining either quantity once someone has moved the camera. In the real-robot study the transform is known by construction, because the experimenters position the camera themselves at sampled locations and re-aim it at the tabletop centre, and the LIBERO-Plus study restores the view with simulator-provided depth and camera parameters. Every reported number therefore comes from a camera pose that is known exactly, and in the simulation from depth free of sensor error. That is the easiest setting for the paper's economy argument, and the only acknowledgement of the dependence is one limitation sentence about depth accuracy. A sweep over injected pose error of a few centimetres and a few degrees, and over realistic depth noise, would show how much of the gain survives a calibrated camera. Without it I cannot tell whether CoRe-VLA is a deployable wrapper or a demonstration that, given the ground-truth training pose, reprojection plus restoration recovers a great deal of success.
- Roughly half the gain at the largest displacement comes from the component the paper documents least. At 1.6 m, reprojection without R2C already lifts from 13.3% to 50.0% and the full pipeline reaches 83.3%, so R2C supplies 33.3 of the 70 points. At the nominal pose it also recovers most of the damage reprojection itself causes, with 96.7% for the raw image, 83.3% without R2C and 93.3% with it. Yet R2C's evidence is one qualitative comparison of restored crops, with no PSNR, SSIM or LPIPS on held-out pairs and no training recipe anywhere, no frame count, width, optimiser or schedule. The training pairs are projective warps with blacked-out holes, which the appendix concedes only approximate re-rendering artifacts, whereas at inference the artifacts also include colour drift and blur. Two cheap controls would settle the mechanism. Applying R2C to the raw shifted image without reprojection would exclude photometric re-normalisation toward the training distribution, and reprojecting to a deliberately wrong pose would show that the training viewpoint specifically is what matters. As written, the claim that restoring the nominal view preserves cross-view coordination is one of several readings these experiments allow.
Questions for authors
- How are the training-time intrinsics and the relative camera transform obtained in deployment, and how accurate must they be? A sweep over injected pose error of 2 to 5 cm and 2 to 5 degrees and over depth noise would show how much gain survives calibration error.
- Does R2C applied directly to the shifted image, with no reprojection, recover part of the 1.6 m gap? I want to separate geometry from photometric re-normalisation.
- Who labelled the Type-1 and Type-2 outcomes, were they blind to condition, and what agreement was reached? Also, which nominal success rate is correct, 98.0% in Section 3 or 96.7% in Figure 2 and Table 2?
- Please give the R2C recipe (frame count, demonstrations, NAFNet variant, optimiser, schedule) and PSNR/SSIM on held-out pairs, and whether the five tasks and three backbones share one model.
8 comments
Response to Reviewer pkwT Part 2
Update (Nov 15): We have updated Table R11 with the results for calibration errors, depth noise, and their combination, and added the corresponding analysis below.
Update (Nov 16): We have incorporated details of R2C to the method and implementation to Section 4.2, and include the quantitative evaluation and ablation experiments in Section 5.5 and Appendix O of the revised submission.
Reply to R1 & Q1 Part 2:
Robustness experiment of Calibration Errors and Depth Noise. To specify the required accuracy of camera parameters and depth measurements, we conduct experiments to assess CoRe-VLA’s robustness during task execution under injected translation errors, rotation errors, and depth noise. In Table R11, each condition covers five real-robot tasks, two layouts per task, and three randomly selected errors at a 1.6 m displacement. Depth-noise levels denote standard deviations relative to the depth value. “No additional error” retains the existing calibration and sensor errors.
The results show that CoRe-VLA effectively improves task success even in the presence of calibration errors, depth noise, and their combination, increasing the success rate from 13.3% to 76.7%–86.7%. Compared to the 83.3% success rate achieved when no errors were introduced, there is no significant difference. These results support CoRe-VLA’s robustness to imperfect calibration and depth measurements.
W2: Roughly half the gain at the largest displacement comes from the component the paper documents least. At 1.6 m, reprojection without R2C already lifts from 13.3% to 50.0% and the full pipeline reaches 83.3%, so R2C supplies 33.3 of the 70 points. At the nominal pose it also recovers most of the damage reprojection itself causes, with 96.7% for the raw image, 83.3% without R2C and 93.3% with it. Yet R2C's evidence is one qualitative comparison of restored crops, with no PSNR, SSIM or LPIPS on held-out pairs and no training recipe anywhere, no frame count, width, optimiser or schedule. The training pairs are projective warps with blacked-out holes, which the appendix concedes only approximate re-rendering artifacts, whereas at inference the artifacts also include colour drift and blur. Two cheap controls would settle the mechanism. Applying R2C to the raw shifted image without reprojection would exclude photometric re-normalisation toward the training distribution, and reprojecting to a deliberately wrong pose would show that the training viewpoint specifically is what matters. As written, the claim that restoring the nominal view preserves cross-view coordination is one of several readings these experiments allow.
Q2: Does R2C applied directly to the shifted image, with no reprojection, recover part of the 1.6 m gap? I want to separate geometry from photometric re-normalisation.
Q4: Please give the R2C recipe (frame count, demonstrations, NAFNet variant, optimiser, schedule) and PSNR/SSIM on held-out pairs, and whether the five tasks and three backbones share one model.
Reply to W2 & Q2 & Q4 Part 1:
Thank you for highlighting the limitations. We: 1. clarify the implementation and training details of R2C; 2. collect image pairs of the same scenes and evaluate R2C using PSNR, SSIM, and LPIPS; 3. add ablation experiments to isolate the contributions of R2C and viewpoint transformation. We will add details of R2C to the method and implementation sections, and include the quantitative evaluation and ablation experiments in the experimental section.
Implementation and training details of R2C. R2C learns to restore degraded inputs using the corresponding original demonstration images as supervision. A lightweight NAFNet is trained with the perceptual, color, and Charbonnier losses defined in Eqs. –, while keeping the VLA frozen. Corruption consists of projective warping, short gaps near object contours, and sparse clusters of missing pixels, which are sequantially applied. Uncovered and missing pixels are set to black. This procedure approximates rendering artifacts without requiring additional multi-view data collection. In our real-world experiment, a single real-robot R2C checkpoint is shared across all five tasks and π₀.₅, OpenVLA-OFT, and GR00T, without task-specific or backbone-specific retraining. The detailed settings are provided in the Table R12.
Table R12. Training settings of R2C. A single R2C checkpoint is shared across all five real-robot tasks and three VLA backbones.
| Setting | Value |
|---|---|
| Training / validation split | 90% / 10% |
| Training crop size | 256 × 256 pixels |
| Optimizer | AdamW |
| Initial learning rate | 1 × 10⁻⁴ |
| Batch size | 16 |
| Training steps | 50,000 |
| Learning-rate schedule | Cosine decay |
Response to Reviewer pkwT Part 3
Update (Nov 15): We have updated Tables R13 and R14 with the quantitative R2C evaluation and the ablation results for viewpoint transformation and image restoration, and added the corresponding analysis below.
Reply to W2 & Q2 & Q4 Part 2:
Table R13. Image restoration quality with and without R2C at different camera displacements.
| Displacement | Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|
| 0.0 | Reprojection only | 35.3 | 0.88 | 0.07 |
| 0.0 | Reprojection + R2C | 40.5 | 0.97 | 0.03 |
| 0.8 | Reprojection only | 19.9 | 0.63 | 0.26 |
| 0.8 | Reprojection + R2C | 23.6 | 0.77 | 0.18 |
| 1.6 | Reprojection only | 17.1 | 0.60 | 0.30 |
| 1.6 | Reprojection + R2C | 22.1 | 0.75 | 0.24 |
Table R14. Ablation of viewpoint transformation and R2C. Task success rates at the camera displacement of 1.6m are compared to assess the contributions of geometric alignment and image restoration.
| Setting | T1 success ↑ | T2 success ↑ | T3 success ↑ | T4 success ↑ | T5 success ↑ | Overall success ↑ |
|---|---|---|---|---|---|---|
| Raw shifted input | 0.0 | 33.3 | 0.0 | 16.7 | 16.7 | 13.3 |
| R2C directly applied to shifted input | 16.7 | 33.3 | 0.0 | 0.0 | 16.7 | 13.3 |
| Reprojection to an incorrect viewpoint + R2C | 16.7 | 33.3 | 66.7 | 66.7 | 50.0 | 46.7 |
| Full CoRe-VLA | 50.0 | 100.0 | 83.3 | 100.0 | 83.3 | 83.3 |
Quantitative evaluation of R2C. Table R13 compares image quality before and after R2C restoration. For this quantitative evaluation, we collected 225 image pairs from the scenes used in our real-robot experiments. Each pair contains a shifted-view RGB observation and a real nominal-view reference image captured with the same scene layout and robot state. “Reprojection only” transforms the shifted observation into the nominal viewpoint; “Reprojection + R2C” additionally applies the trained restoration network. Both outputs are compared with the same reference image using PSNR, SSIM, and LPIPS, averaged separately at each displacement. The evaluation covers all five tasks, and the same R2C checkpoint is used throughout. These pairs are excluded from R2C training and checkpoint selection.
The results show that R2C consistently improves all three image-quality metrics across the evaluated camera displacements. Compared with reprojection alone, R2C increases PSNR by 3.7–5.2 dB and SSIM by 0.09–0.15, while reducing LPIPS by 0.04–0.08. At the largest displacement of 1.6 m, PSNR improves from 17.1 to 22.1 dB, SSIM increases from 0.60 to 0.75, and LPIPS decreases from 0.30 to 0.24. These improvements show that the training data we construct enable R2C to effectively improve the quality of images rendered from point clouds across different camera displacements in real-world settings.
Additional ablation experiments. Table R14 presents the four input conditions. T1–T5 and overall task success rates. All conditions use the frozen π₀.₅ backbone at a 1.6 m camera displacement, covering five tasks and two layouts per task. The first condition directly supplies the shifted image to the VLA. The second applies R2C without changing the viewpoint. The third reprojects the observation to a deliberately incorrect viewpoint before applying R2C, using three randomly selected target viewpoints per layout. These viewpoints follow the planned 0.4–1.6 m sampling range. The final condition follows the full CoRe-VLA pipeline, reprojecting to the nominal viewpoint before restoration. The R2C checkpoint and execution settings remain unchanged across the applicable conditions.
Table R14 shows that applying R2C directly to shifted inputs leaves overall success unchanged at 13.3%, indicating that image restoration without viewpoint alignment can not recover task performance. Reprojection to an incorrect viewpoint followed by R2C achieves 46.7% success. Although success rate improves in this setting, it is because the incorrect target viewpoint is probably closer to the training viewpoint than the original 1.6 m displacement. Nevertheless, its success rate remains 36.6 percentage points lower than that achieved by reprojecting to the training camera pose, highlighting the central importance of aligning the place of the viewpoint in our method.
Together, these two experiments support the complementary roles of geometric alignment and image restoration: reprojection aligns the observation with the nominal viewpoint, while R2C improves the quality of the reprojected image. The observed performance gains therefore cannot be explained by photometric re-normalization alone.
Response to Reviewer pkwT Part 1
Update (Nov 15): We have updated Table R11 with the results for calibration errors, depth noise, and their combination, and added the corresponding analysis below.
Update (Nov 16): We have incorporated the camera-calibration procedure and parameter sources to Section C, and included the calibration-error and depth-noise experiments and the analysis to Appendix N of the revised submission.
We sincerely thank you for your time and constructive feedback. We provide detailed responses to each of your concerns below.
W1: The wrapper consumes a quantity the paper never accounts for. Equation (1) takes the relative rigid transform between the shifted and the training-time external camera, together with the training intrinsics, as known. The word "known" appears once, and no procedure is given for obtaining either quantity once someone has moved the camera. In the real-robot study the transform is known by construction, because the experimenters position the camera themselves at sampled locations and re-aim it at the tabletop centre, and the LIBERO-Plus study restores the view with simulator-provided depth and camera parameters. Every reported number therefore comes from a camera pose that is known exactly, and in the simulation from depth free of sensor error. That is the easiest setting for the paper's economy argument, and the only acknowledgement of the dependence is one limitation sentence about depth accuracy. A sweep over injected pose error of a few centimetres and a few degrees, and over realistic depth noise, would show how much of the gain survives a calibrated camera. Without it I cannot tell whether CoRe-VLA is a deployable wrapper or a demonstration that, given the ground-truth training pose, reprojection plus restoration recovers a great deal of success.
Q1: How are the training-time intrinsics and the relative camera transform obtained in deployment, and how accurate must they be? A sweep over injected pose error of 2 to 5 cm and 2 to 5 degrees and over depth noise would show how much gain survives calibration error.
Reply to R1 & Q1 Part 1:
Thank you for highlighting the limitations. We: 1. clarify the procedure for obtaining camera parameters and demonstrate that CoRe-VLA is a practically deployable, plug-and-play method; 2. evaluate the method’s robustness under imperfect inputs by injecting camera translation and rotation errors and depth noise.
Clarification for obtaining camera prarameters. In our real-robot experiments and practical deployment, the required camera poses are obtained using a small printed calibration board, whose material cost is limited to printing. In our real-robot experiments, we use an 18 cm × 13.5 cm board placed in the scene Fig\. 1. The nominal and shifted camera poses are calculated relative to this calibration board, allowing their relative transform to be computed. The required camera intrinsics can be obtained from the training dataset’s calibration records. These settings enable CoRe-VLA to be deployed directly in existing VLA systems.
Table R11. Robustness to additional calibration errors and depth noise. Depth-noise levels denote standard deviations relative to the depth value.