- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies viewpoint robustness of VLA in settings with both a third-person and a fixed wrist camera. The authors begin with a motivating analysis showing that VISTA-style learned viewpoint synthesis increases the policy's attention share on wrist images by ~6 percentage points on LIBERO, suggesting that viewpoint augmentation encourages the policy to rely more on the stable wrist view. Building on this observation, they question whether faithful novel-view synthesis is actually necessary, given the wrist camera's unchanged perspective. They instead propose SHARP, which applies random 2D homography warps to third-person images during LoRA finetuning while leaving wrist images, proprioceptive state, language instructions, and action labels untouched. Evaluated on 12 RLBench tasks and 40 LIBERO tasks, under third-person azimuth shifts, SHARP yields improvements over no-augmentation baselines and matches or slightly lags VISTA / VISTA-FT / GenSplat in success rate.
Critical strengths
-
The paper's strongest contribution is the clean experimental story: first show via attention analysis that learned viewpoint synthesis already pushes the policy toward the wrist camera, then show that a non-learned 2D warp that explicitly corrupts the third-person view achieves nearly the same robustness. The cost comparison in Fig. 4 (0.015 s vs. 7 s per augmentation, 0.08 GB vs. 6+ GB peak memory) drives home the practical consequence.
-
Beyond the headline numbers, the paper examines (a) performance across three qualitatively different VLA backbones (Tab. 1), (b) every component of the homography and its strength (Figs. 5–6), (c) direct comparison against 10 alternative augmentation strategies (Tab. 2), and (d) camera-use diagnostics via attention share, wrist-only policy, and input-intervention at test time (Fig. 7).
Critical weaknesses
-
The evaluation regime is substantially narrower than the paper's framing suggests. The problem is introduced as general "viewpoint robustness" with complementary third-person + wrist cameras, but all core experiments vary a single azimuth degree of freedom in two tabletop simulators, with wrist camera fully fixed. The method's central tool (homography) models viewpoint shifts that preserve a scene plane; robustness to arbitrary 3D camera translation, changes in wrist mounting, occlusions that asymmetrically affect only the third-person view, or real-world domain shift is essentially uncharacterized. This limits confidence that the observed gains will transfer to the multi-DOF, non-planar, real-world setups where viewpoint robustness is most urgently needed.
-
Algorithmic novelty is thin and several close baselines are missing. SHARP's technical core (apply random homography warps during training) has been a standard augmentation in geometry-aware vision for two decades. Its novelty is therefore contextual: applying it specifically to the third-person stream of a multi-camera VLA while preserving wrist and action labels. That contextual novelty is real, but the paper does not help the reader disentangle whether the benefit comes from the homography parameterization per se, or from broader geometric variation. Classical geometric-augmentation baselines (e.g., affine + perspective crop sweep, depth-free plane sweep, or elastic warps) that would isolate this question are absent from Tab. 2. The closest comparison is random crop, which SHARP beats by a solid but not overwhelming 4.8 points on average; understanding the source of this gap would substantially strengthen the contribution.
Questions for authors
- The paper evaluates almost exclusively on third-person azimuth sweeps. Could the authors comment on, or ideally add a small ablation on, the behavior of SHARP when multiple viewpoint parameters change simultaneously (e.g., a combined azimuth, elevation, distance, and roll shift), and when the third-person camera is translated (not just rotated) relative to the scene plane—i.e., conditions where the homography model (which assumes a single reference plane) is most likely to break?
- Missing stronger geometric-augmentation baselines. Tab. 2 includes crop, resize, flips, masking, etc., but not parameterically richer geometric baselines such as affine-only warps, random perspective crops with aspect-ratio variation, or warps derived from an assumed scene plane with added jitter (a direct ablation of the virtual-camera parameterization). Would the authors consider adding one or two such baselines, or alternatively a quantitative similarity analysis (e.g., distribution of induced optical-flow magnitudes) that situates SHARP among the alternative augmentations?
- Wrist-camera variation and real-robot experiments. The paper's Discussion correctly flags that wrist-camera stability is critical to the story. Do the authors have any preliminary results (even on a single task) when the wrist camera's extrinsic or intrinsic parameters also change, or any commitment to including a real-robot evaluation (e.g., one or two manipulation tasks on a physical arm) in the camera-ready version?
4 comments
2/2 Response to Reviewer zP1P
The paper's Discussion correctly flags that wrist-camera stability is critical to the story. Do the authors have any preliminary results (even on a single task) when the wrist camera's extrinsic or intrinsic parameters also change?
Thank you for highlighting this important point. We tilted the wrist camera by ±10° at test time, jointly with third-person viewpoint shifts (Appendix C.1.4, Figure 15). The following table compares the original wrist-camera orientation with the average over both wrist tilts. Both columns average the same 40 LIBERO tasks and five third-person azimuths (0°, ±45°, and ±75°), using one training seed.
| Method | Original wrist orientation (%) | Wrist tilt ±10° (%) |
|---|---|---|
| No augmentation | 71.9 | 59.3 |
| VISTA | 89.3 | 73.0 |
| GenSplat | 90.0 | 72.9 |
| SHARP | 89.4 | 73.3 |
Although wrist-camera shifts lower absolute performance for all methods, SHARP retains an average advantage of 14.0 percentage points over No augmentation and performs comparably to VISTA and GenSplat. These results show that SHARP’s gains persist even when the wrist-camera orientation also differs from training.
any commitment to including a real-robot evaluation (e.g., one or two manipulation tasks on a physical arm) in the camera-ready version?
We have now completed real-robot experiments on three manipulation tasks using a Franka Research 3 arm (Section 5.2.2, Table 2). We evaluate two shifted third-person viewpoints, with 15 trials per task, viewpoint, and method.
| Method | Successful trials | Success rate | Aug. time (s/image) | Peak aug. GPU memory (GB) |
|---|---|---|---|---|
| No augmentation | 53/90 | 58.9% | — | — |
| VISTA | 62/90 | 68.9% | 6.357 | 5.44 |
| SHARP | 64/90 | 71.1% | 0.01339 | 0.0824 |
SHARP improves average success by 12.2 percentage points over No augmentation and achieves success comparable to VISTA. The reference measurements additionally show approximately 475× faster augmentation and 66× lower peak GPU memory than VISTA. These results support SHARP’s effectiveness on the evaluated real-robot tasks alongside its low augmentation cost.
We are happy to address any further questions or concerns and would welcome additional feedback or suggestions for improving our work. Thank you.
Thank you for new experiments and clarifications. The multi-parameter viewpoint and translation ablation is valuable. My remaining minor concern is that the real-robot evaluation is limited to only three tasks and two viewpoint offsets, and wrist-camera perturbations are still restricted to moderate ±10° tilts. More extensive real-robot variation would still be needed to fully confirm transfer to messy physical setups.
Overall, these additions largely resolve my original weaknesses. I keep my current rating as Weak Accept.
Follow-up response to Reviewer zP1P
Thank you for your careful review and constructive feedback. We are glad that the additional experiments and clarifications have addressed your original concerns. Your suggestions helped us strengthen the evaluation and clarify the scope of our findings.
We agree that broader real-robot evaluation and testing under larger wrist-camera perturbations are important directions for future work. Thank you again for your time and thoughtful discussion.
1/2 Response to Reviewer zP1P
Thank you for your thoughtful review. We appreciate your recognition of our experimental findings and SHARP’s computational efficiency. Please see our general response to all reviewers for an overview of our manuscript updates. Below, we address your questions and concerns in detail.
Thank you for this helpful suggestion. We added evaluations of both camera translation and simultaneous changes in multiple viewpoint parameters using the existing checkpoints (Appendix C.1.3, Tables 12–14). For translation, we move the third-person camera horizontally or vertically while keeping its orientation fixed. For combined shifts, we evaluate four combinations of azimuth, elevation, distance, and roll.
The following results average all 40 LIBERO tasks and the four shifted viewpoints in each category, excluding the source view. We use one training seed per method (Table 12).
SHARP retains gains of 11.0 and 16.1 percentage points over No augmentation under translations and combined shifts, respectively. These results show that SHARP is effective beyond azimuth shifts, including under camera translations and combined viewpoint shifts.
Thank you for your insightful suggestion. We added a six-degree-of-freedom affine baseline combining rotation, shear, scale, aspect-ratio variation, and translation. We compared affine augmentation and random crop, a strong baseline reported in Table 3, against SHARP after matching both to SHARP's mean induced flow magnitude (Appendix E.3).
We evaluate 40 LIBERO tasks at ±45° and ±75° (Section 5.3, Table 19). Success rates below are means and standard errors across three training seeds, after averaging tasks and viewpoints within each seed.
SHARP retains a 2.5-point advantage over crop and a 3.8-point advantage over affine, with both differences positive in all three seeds. These results suggest that homography-based transformations, which approximate viewpoint changes, offer benefits beyond simply making augmentation stronger.