acceptodds
Under review as a conference paper at ICLR 2027

Rethinking High-Fidelity View Synthesis: Simple Homography Augmentation for Viewpoint-Robust VLAs

Abstract

Vision-Language-Action (VLA) models perform strongly across diverse language-conditioned manipulation tasks but remain sensitive to camera viewpoint changes. Recent 3D-aware augmentation methods address this challenge by generating observations from novel viewpoints through learned view synthesis or 3D reconstruction. However, generating these views is computationally expensive and requires additional pretrained models. To address these limitations, we propose SHARP (Simple Homography Augmentation for Robust Policies), a lightweight view-augmentation method that improves VLA robustness to unseen third-person camera viewpoints. For VLAs that combine third-person and wrist-camera observations, SHARP applies randomly sampled homographies to third-person images during policy finetuning while preserving the paired wrist observations and action labels. These inexpensive 2D transformations enable online augmentation of demonstrations from a single third-person viewpoint, without additional pretrained synthesis or reconstruction models. Compared with policies finetuned on the same data without augmentation, SHARP improves average success rates across all evaluated viewpoints by 11.8 percentage points on RLBench and 16.4 percentage points on LIBERO. It achieves success rates competitive with the evaluated view-synthesis methods at substantially lower augmentation cost. Comparisons with alternative 2D augmentations highlight the benefit of geometric augmentation for viewpoint robustness. Attention analyses show a shift toward wrist observations, while camera-input analyses suggest that SHARP still benefits from third-person information under certain viewpoint shifts. These results suggest that lightweight geometric augmentation can improve viewpoint robustness in VLAs with stable wrist-camera inputs, even without faithfully reconstructing novel views.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action models, robotic manipulation, viewpoint robustness, data augmentation, homography transformation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zP1POpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies viewpoint robustness of VLA in settings with both a third-person and a fixed wrist camera. The authors begin with a motivating analysis showing that VISTA-style learned viewpoint synthesis increases the policy's attention share on wrist images by ~6 percentage points on LIBERO, suggesting that viewpoint augmentation encourages the policy to rely more on the stable wrist view. Building on this observation, they question whether faithful novel-view synthesis is actually necessary, given the wrist camera's unchanged perspective. They instead propose SHARP, which applies random 2D homography warps to third-person images during LoRA finetuning while leaving wrist images, proprioceptive state, language instructions, and action labels untouched. Evaluated on 12 RLBench tasks and 40 LIBERO tasks, under third-person azimuth shifts, SHARP yields improvements over no-augmentation baselines and matches or slightly lags VISTA / VISTA-FT / GenSplat in success rate.

Critical strengths
  1. The paper's strongest contribution is the clean experimental story: first show via attention analysis that learned viewpoint synthesis already pushes the policy toward the wrist camera, then show that a non-learned 2D warp that explicitly corrupts the third-person view achieves nearly the same robustness. The cost comparison in Fig. 4 (0.015 s vs. 7 s per augmentation, 0.08 GB vs. 6+ GB peak memory) drives home the practical consequence.

  2. Beyond the headline numbers, the paper examines (a) performance across three qualitatively different VLA backbones (Tab. 1), (b) every component of the homography and its strength (Figs. 5–6), (c) direct comparison against 10 alternative augmentation strategies (Tab. 2), and (d) camera-use diagnostics via attention share, wrist-only policy, and input-intervention at test time (Fig. 7).

Critical weaknesses
  1. The evaluation regime is substantially narrower than the paper's framing suggests. The problem is introduced as general "viewpoint robustness" with complementary third-person + wrist cameras, but all core experiments vary a single azimuth degree of freedom in two tabletop simulators, with wrist camera fully fixed. The method's central tool (homography) models viewpoint shifts that preserve a scene plane; robustness to arbitrary 3D camera translation, changes in wrist mounting, occlusions that asymmetrically affect only the third-person view, or real-world domain shift is essentially uncharacterized. This limits confidence that the observed gains will transfer to the multi-DOF, non-planar, real-world setups where viewpoint robustness is most urgently needed.

  2. Algorithmic novelty is thin and several close baselines are missing. SHARP's technical core (apply random homography warps during training) has been a standard augmentation in geometry-aware vision for two decades. Its novelty is therefore contextual: applying it specifically to the third-person stream of a multi-camera VLA while preserving wrist and action labels. That contextual novelty is real, but the paper does not help the reader disentangle whether the benefit comes from the homography parameterization per se, or from broader geometric variation. Classical geometric-augmentation baselines (e.g., affine + perspective crop sweep, depth-free plane sweep, or elastic warps) that would isolate this question are absent from Tab. 2. The closest comparison is random crop, which SHARP beats by a solid but not overwhelming 4.8 points on average; understanding the source of this gap would substantially strengthen the contribution.

Questions for authors
  1. The paper evaluates almost exclusively on third-person azimuth sweeps. Could the authors comment on, or ideally add a small ablation on, the behavior of SHARP when multiple viewpoint parameters change simultaneously (e.g., a combined azimuth, elevation, distance, and roll shift), and when the third-person camera is translated (not just rotated) relative to the scene plane—i.e., conditions where the homography model (which assumes a single reference plane) is most likely to break?
  2. Missing stronger geometric-augmentation baselines. Tab. 2 includes crop, resize, flips, masking, etc., but not parameterically richer geometric baselines such as affine-only warps, random perspective crops with aspect-ratio variation, or warps derived from an assumed scene plane with added jitter (a direct ablation of the virtual-camera parameterization). Would the authors consider adding one or two such baselines, or alternatively a quantitative similarity analysis (e.g., distribution of induced optical-flow magnitudes) that situates SHARP among the alternative augmentations?
  3. Wrist-camera variation and real-robot experiments. The paper's Discussion correctly flags that wrist-camera stability is critical to the story. Do the authors have any preliminary results (even on a single task) when the wrist camera's extrinsic or intrinsic parameters also change, or any commitment to including a real-robot evaluation (e.g., one or two manipulation tasks on a physical arm) in the camera-ready version?
4 comments
AuthorsOpenReview ↗

1/2 Response to Reviewer zP1P

Thank you for your thoughtful review. We appreciate your recognition of our experimental findings and SHARP’s computational efficiency. Please see our general response to all reviewers for an overview of our manuscript updates. Below, we address your questions and concerns in detail.

The paper evaluates almost exclusively on third-person azimuth sweeps. Could the authors comment on, or ideally add a small ablation on, the behavior of SHARP when multiple viewpoint parameters change simultaneously (e.g., a combined azimuth, elevation, distance, and roll shift), and when the third-person camera is translated (not just rotated) relative to the scene plane—i.e., conditions where the homography model (which assumes a single reference plane) is most likely to break?

Thank you for this helpful suggestion. We added evaluations of both camera translation and simultaneous changes in multiple viewpoint parameters using the existing checkpoints (Appendix C.1.3, Tables 12–14). For translation, we move the third-person camera horizontally or vertically while keeping its orientation fixed. For combined shifts, we evaluate four combinations of azimuth, elevation, distance, and roll.

The following results average all 40 LIBERO tasks and the four shifted viewpoints in each category, excluding the source view. We use one training seed per method (Table 12).

MethodTranslation (%)Combined shifts (%)
No augmentation79.773.6
VISTA88.287.4
GenSplat85.486.7
SHARP90.789.7

SHARP retains gains of 11.0 and 16.1 percentage points over No augmentation under translations and combined shifts, respectively. These results show that SHARP is effective beyond azimuth shifts, including under camera translations and combined viewpoint shifts.

Missing stronger geometric-augmentation baselines. Tab. 2 includes crop, resize, flips, masking, etc., but not parameterically richer geometric baselines such as affine-only warps, random perspective crops with aspect-ratio variation, or warps derived from an assumed scene plane with added jitter (a direct ablation of the virtual-camera parameterization). Would the authors consider adding one or two such baselines, or alternatively a quantitative similarity analysis (e.g., distribution of induced optical-flow magnitudes) that situates SHARP among the alternative augmentations?

Thank you for your insightful suggestion. We added a six-degree-of-freedom affine baseline combining rotation, shear, scale, aspect-ratio variation, and translation. We compared affine augmentation and random crop, a strong baseline reported in Table 3, against SHARP after matching both to SHARP's mean induced flow magnitude (Appendix E.3).

We evaluate 40 LIBERO tasks at ±45° and ±75° (Section 5.3, Table 19). Success rates below are means and standard errors across three training seeds, after averaging tasks and viewpoints within each seed.

AugmentationMean displacement (px)Success rate (%)
Random crop40.585.4 ± 0.3
Affine40.284.1 ± 0.5
SHARP40.487.9 ± 0.1

SHARP retains a 2.5-point advantage over crop and a 3.8-point advantage over affine, with both differences positive in all three seeds. These results suggest that homography-based transformations, which approximate viewpoint changes, offer benefits beyond simply making augmentation stronger.

AuthorsOpenReview ↗

2/2 Response to Reviewer zP1P

The paper's Discussion correctly flags that wrist-camera stability is critical to the story. Do the authors have any preliminary results (even on a single task) when the wrist camera's extrinsic or intrinsic parameters also change?

Thank you for highlighting this important point. We tilted the wrist camera by ±10° at test time, jointly with third-person viewpoint shifts (Appendix C.1.4, Figure 15). The following table compares the original wrist-camera orientation with the average over both wrist tilts. Both columns average the same 40 LIBERO tasks and five third-person azimuths (0°, ±45°, and ±75°), using one training seed.

MethodOriginal wrist orientation (%)Wrist tilt ±10° (%)
No augmentation71.959.3
VISTA89.373.0
GenSplat90.072.9
SHARP89.473.3

Although wrist-camera shifts lower absolute performance for all methods, SHARP retains an average advantage of 14.0 percentage points over No augmentation and performs comparably to VISTA and GenSplat. These results show that SHARP’s gains persist even when the wrist-camera orientation also differs from training.

any commitment to including a real-robot evaluation (e.g., one or two manipulation tasks on a physical arm) in the camera-ready version?

We have now completed real-robot experiments on three manipulation tasks using a Franka Research 3 arm (Section 5.2.2, Table 2). We evaluate two shifted third-person viewpoints, with 15 trials per task, viewpoint, and method.

MethodSuccessful trialsSuccess rateAug. time (s/image)Peak aug. GPU memory (GB)
No augmentation53/9058.9%——
VISTA62/9068.9%6.3575.44
SHARP64/9071.1%0.013390.0824

SHARP improves average success by 12.2 percentage points over No augmentation and achieves success comparable to VISTA. The reference measurements additionally show approximately 475× faster augmentation and 66× lower peak GPU memory than VISTA. These results support SHARP’s effectiveness on the evaluated real-robot tasks alongside its low augmentation cost.

We are happy to address any further questions or concerns and would welcome additional feedback or suggestions for improving our work. Thank you.

Thank you for new experiments and clarifications. The multi-parameter viewpoint and translation ablation is valuable. My remaining minor concern is that the real-robot evaluation is limited to only three tasks and two viewpoint offsets, and wrist-camera perturbations are still restricted to moderate ±10° tilts. More extensive real-robot variation would still be needed to fully confirm transfer to messy physical setups.

Overall, these additions largely resolve my original weaknesses. I keep my current rating as Weak Accept.

AuthorsOpenReview ↗

Follow-up response to Reviewer zP1P

Thank you for your careful review and constructive feedback. We are glad that the additional experiments and clarifications have addressed your original concerns. Your suggestions helped us strengthen the evaluation and clarify the scope of our findings.

We agree that broader real-robot evaluation and testing under larger wrist-camera perturbations are important directions for future work. Thank you again for your time and thoughtful discussion.

Reviewer_kyHGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the sharp performance drop of Vision-Language-Action (VLA) policies when the third-person camera viewpoint differs from training. Prior fixes require costly learned view synthesis. Observing that a VISTA-finetuned policy increases its wrist-image attention share, the authors hypothesize that faithful 3D view synthesis is unnecessary when the wrist camera stays stable. They propose SHARP, which applies online 2D homographies to the third-person image only, keeping the wrist image and actions unchanged, and fine-tunes policies with LoRA.

On RLBench and LIBERO, evaluated over a wide range of viewpoint changes, SHARP substantially improves average success over no augmentation and is competitive with VISTA/GenSplat, while being far cheaper in per-image generation time and GPU memory. It also beats other 2D augmentations, and ablations show pitch, yaw, and translation matter most. Without a wrist camera, SHARP underperforms the baseline. It also helps in one-shot adaptation.

Critical strengths
  • Consistent large gains across two benchmarks and three backbones, with 3 seeds and standard errors for main results.
  • Clear efficiency advantage: 0.0154 s / 0.082 GB per image vs. 7.14 s / 5.45 GB (VISTA) and 0.053 s / 6.22 GB (GenSplat), measured on the same GPU and frames.
  • Broad ablations on augmentation type, homography components, intensity, online vs. fixed augmentation, camera elevation/distance, and wrist-only / third-person-only controls.
  • Transparent reporting of drops at the source view, GenSplat's information advantage, and failure without a wrist camera.
  • Transfers to one-shot adaptation, improving over DART by +5.4 to +8.2 pp.
Critical weaknesses
  • Limited technical novelty. Homography augmentation is a standard technique. The only new ingredients are the wrist-attention observation and applying the warp to the third-person image only. The paper does not show why this is more than a well-tuned standard augmentation, and the geometric explanation is itself undercut by the masking/crop results below.
  • Likely to lose relevance with stronger models. The largest gains occur on weak backbones (X-VLA 7.6→72.4), suggesting SHARP patches a lack of viewpoint diversity in training data. Foundation VLAs pretrained on multi-view or large-scale data may close much of this gap, and this scenario is not tested.
  • Causal claim not isolated. Non-geometric random masking (82.4%) and crop (83.4%) come close to SHARP (88.2%) in Table 2. Much of the benefit may be generic regularization that increases wrist reliance; there is no perturbation-strength-matched control.
  • Degradation at the source viewpoint. At 0°, SHARP is below no augmentation (RLBench 57.1% vs. 62.5%; LIBERO Long 84.6% vs. 92.7%). Mixing in unaugmented data is left as future work.
  • Hard dependence on a wrist camera. Without it, SHARP is worse than the baseline (18.9% vs. 29.0%), so the viewpoint-robust VLA claim is limited to a specific sensor setup.
  • Weak statistics for the mechanism analysis. Table 2, component ablations, and intensity sweeps use a single seed, though many effects are only a few points.
  • No real-world deployment results. The paper motivates SHARP by efficiency and practicality, yet evaluation is only in simulation with exact pivot rotations. A single-plane homography cannot model parallax, real lighting, lens distortion, or imperfect pivots, so transfer to physical robots is unsupported. This is a central gap rather than a side limitation.
Questions for authors
  1. Can you run a control with matched perturbation strength (e.g., fraction of changed pixels or image similarity), with multiple seeds, to separate geometric-specific effects from generic perturbation effects, given masking and crop are close?
  2. What is technically new beyond applying standard homography augmentation to the third-person view? Please show evidence, beyond the single attention-share observation, that the wrist/third-person asymmetry is the key factor.
  3. Do the gains persist on stronger backbones, such as VLAs pretrained on multi-view or large-scale data? If the benefit shrinks as the backbone improves, what is SHARP's long-term value?
  4. Can you provide real-robot results showing SHARP's gains hold under real lighting, lens distortion, and parallax? Without them, I cannot assess the practicality claims.
  5. Does mixing a fraction of unaugmented source-view demos recover the 0° performance (and the no-wrist-camera case) while keeping robustness?
7 comments
AuthorsOpenReview ↗

1/3 Response to Reviewer kyHG

Thank you for your thoughtful review. We are encouraged that you recognize SHARP’s consistent gains and computational efficiency, as well as our extensive ablations. Please see our general response to all reviewers for an overview of our manuscript updates. Below, we address your questions and concerns in detail.

Can you run a control with matched perturbation strength (e.g., fraction of changed pixels or image similarity), with multiple seeds, to separate geometric-specific effects from generic perturbation effects, given masking and crop are close?

Thank you for your insightful question. To address this question, we added strength-matched comparisons with random crop, affine transformations, and random masking. We calibrated crop and affine transformations to match SHARP’s mean induced pixel displacement, and masking to match its mean pixel L1 change.

All methods use the same source demonstrations, initial policy checkpoint, and finetuning budget. We evaluate on all 40 LIBERO tasks at ±45° and ±75°, averaging success over tasks and viewpoints within each seed and reporting the mean and standard error across three training seeds (Section 5.3, Table 19; Appendix E.3):

AugmentationSuccess rate (%)
No augmentation66.0 ± 0.2
Random crop85.4 ± 0.3
Affine84.1 ± 0.5
Random masking84.0 ± 0.1
SHARP87.9 ± 0.1

The matched controls also substantially improve viewpoint robustness over No augmentation. SHARP nevertheless outperforms random crop, affine augmentation, and random masking by 2.5, 3.8, and 3.9 percentage points, respectively. Its advantage thus persists after matching the measures of average perturbation strength, supporting a benefit of SHARP’s projective augmentation over these controls.

What is technically new beyond applying standard homography augmentation to the third-person view? Please show evidence, beyond the single attention-share observation, that the wrist/third-person asymmetry is the key factor.

Thank you for bringing up this crucial point. We agree that homography augmentation itself is an established technique. As clarified in the third contribution in Section 1 (Introduction), our analyses suggest that third-person perturbations encourage control grounded in stable wrist observations, while geometric variations that roughly mimic viewpoint changes improve viewpoint robustness. This interpretation helps explain why SHARP achieves robustness comparable to the evaluated view-synthesis methods without requiring faithful reconstruction of novel views, at substantially lower augmentation cost (Section 5.2.1, Figure 5).

Beyond attention, we test camera reliance by blacking out the third-person image at inference while retaining current wrist observations. The table reports mean success across 40 LIBERO tasks at the source viewpoint (Section 5.4, Figure 10; Appendix E.2, Table 18):

MethodNormal input (%)Third-person blackout (%)Drop (percentage points)
No augmentation96.9554.0042.95
VISTA93.2083.459.75
GenSplat94.2078.1016.10
SHARP94.1085.258.85

SHARP, VISTA, and GenSplat retain higher success under blackout and show smaller drops than No augmentation. Together with increased wrist-image attention (Figure 9), this supports greater wrist reliance across all three augmentation methods.

SHARP also outperforms a wrist-only policy at small to moderate viewpoint shifts, and blacking out or freezing its third-person input reduces success (Figures 9–10), suggesting that third-person information remains useful. Conversely, without wrist inputs, SHARP's robustness advantage disappears, and VISTA-FT and GenSplat also suffer substantial performance drops (Appendix D.4, Figure 18 left). Together, these results support the complementary roles of stable wrist observations and useful third-person information in SHARP's robustness gains.

AuthorsOpenReview ↗

2/3 Response to Reviewer kyHG

Do the gains persist on stronger backbones, such as VLAs pretrained on multi-view or large-scale data? If the benefit shrinks as the backbone improves, what is SHARP's long-term value?

Thank you for your insightful questions. Our evaluation already includes models with large-scale, diverse robot pretraining. The π₀.₅ paper describes training across multiple robots and environments, including approximately 100 homes [1]. OpenVLA, the base model for OpenVLA-OFT, is pretrained on 970,000 robot trajectories [2], including BridgeData V2 with varied camera poses and workspace configurations [3].

SHARP improves success across all three evaluated backbones, including a 21.9 percentage-point gain on π₀.₅, which has the highest No augmentation performance in our evaluation (Table 1). All entries below average the same 40 tasks and four shifted viewpoints, ±45° and ±75°:

BackboneNo augmentation (%)SHARP (%)
X-VLA7.672.4
OpenVLA-OFT45.275.5
π₀.₅66.087.9

These results show that SHARP's benefit is not confined to the backbone with the weakest baseline.

We agree that stronger pretraining may reduce SHARP's gains. However, it does not guarantee viewpoint robustness in downstream tasks, as Table 1 shows. SHARP helps close this gap with robustness comparable to the evaluated view-synthesis methods at substantially lower augmentation cost. It therefore remains a useful, low-cost complement to pretraining wherever viewpoint sensitivity persists.

Can you provide real-robot results showing SHARP's gains hold under real lighting, lens distortion, and parallax? Without them, I cannot assess the practicality claims.

Thank you for bringing this up. We added real-robot experiments on three tasks using a Franka Research 3 arm, evaluating two shifted third-person viewpoints with 15 trials per task, viewpoint, and method (Section 5.2.2, Table 2):

MethodSuccessful trialsSuccess rateAug. time (s/image)Peak aug. GPU memory (GB)
No augmentation53/9058.9%——
VISTA62/9068.9%6.3575.44
SHARP64/9071.1%0.013390.0824

SHARP improves average success by 12.2 percentage points over No augmentation and achieves success comparable to VISTA. The reference measurements additionally show approximately 475× faster augmentation and 66× lower peak GPU memory than VISTA. These experiments use real camera observations, providing evidence that SHARP’s gains under viewpoint shifts extend to the lighting and imaging conditions of our evaluated real-robot setup.

Does mixing a fraction of unaugmented source-view demos recover the 0° performance (and the no-wrist-camera case) while keeping robustness?

Thank you for pointing this out. We added mixing experiments both with and without wrist observations to examine the trade-off.

With wrist observations, mixing partially recovers 0° performance while retaining most of SHARP's robustness (Appendix C.1.1, Figure 12). Here, shifted-view success averages the 12 viewpoints at ±15° to ±90°. On LIBERO, mixing 25% unaugmented images raises 0° success from 94.5% to 95.9% (No augmentation: 96.8%), while shifted-view success changes from 88.6% to 87.4%. On RLBench, 12.5% mixing raises 0° success from 57.1% to 60.5% (No augmentation: 62.5%), with shifted-view success nearly unchanged, from 42.0% to 41.8%.

Without wrist observations, mixing reduces SHARP's performance gap to No augmentation but does not fully close it (Appendix D.4, Figure 18 right). At 50% mixing, 0° success rises from 69.3% to 84.0% (No augmentation: 89.6%). It also improves mean success over the seven evaluated viewpoints (0°–90° in 15° increments) from 18.9% to 26.0%, although this remains below No augmentation (29.0%).

AuthorsOpenReview ↗

3/3 Response to Reviewer kyHG

Hard dependence on a wrist camera. Without it, SHARP is worse than the baseline (18.9% vs. 29.0%), so the viewpoint-robust VLA claim is limited to a specific sensor setup.

Thank you for raising this point. Our claim specifically concerns robustness to third-person viewpoint changes in VLAs equipped with both third-person and wrist cameras, as stated in Section 1 (Introduction). We acknowledge that, without wrist observations, SHARP performs worse than No augmentation (18.9% vs. 29.0%; Appendix D.4, Figure 18 left). VISTA-FT and GenSplat also suffer substantial performance drops without wrist observations, indicating that this sensitivity is shared among the evaluated viewpoint-augmentation methods.

In practice, third-person and wrist cameras are commonly used together in robot-learning setups such as DROID [4] and ALOHA [5]. This makes the camera configuration studied in our work practically relevant. Within this configuration, SHARP provides a low-cost approach to improving robustness to third-person viewpoint changes.

Table 2, component ablations, and intensity sweeps use a single seed, though many effects are only a few points.

Thank you for raising this concern. Our main π₀.₅ comparisons on RLBench and LIBERO (Section 5.2.1) used three training seeds in the original submission. In this revision, we have added a three-seed strength-matched augmentation comparison (Table 19) and expanded the homography component-removal ablations from one to three training seeds (Figure 7; Appendix D.2). In the strength-matched comparison, SHARP retains a 2.5–3.9 percentage-point advantage over crop, affine transformations, and masking. These experiments provide statistical support for our central claim that homography augmentation improves viewpoint robustness.

Moreover, our existing three-seed experiments show little variation in aggregate LIBERO performance: SHARP achieves 89.25%, 88.99%, and 88.86% success averaged over 40 tasks and 13 viewpoints, with a standard deviation of 0.20 percentage points. This observed stability provides useful context for interpreting the remaining single-seed augmentation-strength sweeps (Figure 8; Appendix D.3).

We hope these additional experiments and clarifications address your concerns. We would welcome further feedback on any points that remain unclear. Thank you.

References:

[1] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π₀.₅: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025.

[2] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024.

[3] Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning, pp. 1723–1736. PMLR, 2023.

[4] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024.

[5] Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems, 2023.

Reviewer_eGnVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SHARP, a lightweight homography-based augmentation method for improving viewpoint robustness in Vision-Language-Action (VLA) models. Instead of relying on expensive 3D reconstruction or novel-view synthesis, SHARP applies geometric transformations to third-person observations during finetuning while preserving wrist-camera inputs and action labels. Experiments on RLBench and LIBERO demonstrate substantial improvements in viewpoint robustness with significantly lower computational cost.

Critical strengths

S1.The proposed method is simple, computationally efficient, and easy to implement.

S2.The paper provides comprehensive experiments across multiple VLA backbones and benchmarks, demonstrating consistent improvements in viewpoint robustness.

Critical weaknesses

W1.Limited technical contribution. The proposed method primarily applies conventional homography-based augmentation to VLA training, without introducing substantial methodological advances beyond existing geometric augmentation techniques. While the empirical improvements are encouraging, the approach largely demonstrates the effectiveness of an established technique in a specific setting. The overall contribution appears incremental and does not provide sufficiently strong new technical insights to meet the expected contribution bar for ICLR.

W2.Limited generalizability. The effectiveness of the proposed approach heavily depends on stable wrist-camera observations, and the evaluation is limited to simulation environments. This restricts the general applicability of the findings.

Questions for authors

See weakness

4 comments
AuthorsOpenReview ↗

1/2 Response to Reviewer eGnV

Thank you for your thoughtful review. We appreciate your acknowledgment of SHARP’s simple design, low computational cost, and ease of implementation. Please see our general response to all reviewers for an overview of our manuscript updates. Below, we address your concerns in detail.

The overall contribution appears incremental and does not provide sufficiently strong new technical insights

We agree that homography augmentation itself is an established technique. Our central insight is that simple geometric augmentation can improve viewpoint robustness without faithfully reconstructing novel views when complementary observations still provide useful information for the task. We believe that this finding, together with the analyses explaining it and identifying its limits, contributes new empirical knowledge with practical value. We clarify these three aspects below.

First, we provide a negative result about the presumed need for faithful novel-view reconstruction. Despite using simple 2D warps that do not faithfully reconstruct novel views (Figure 11), SHARP achieves robustness comparable to the evaluated view-synthesis methods at substantially lower augmentation cost (Section 5.2.1, Figure 5). This demonstrates that faithful reconstruction is not necessary to achieve competitive viewpoint robustness in the evaluated VLA settings with complementary third-person and wrist observations.

Second, our analyses provide insight into mechanisms shared across methods. They suggest two complementary effects: (i) broadening the training-data distribution through image variations that mimic viewpoint changes, and (ii) changing how the policy uses complementary observations. Increased wrist-image attention and smaller performance drops under source-view third-person blackout suggest greater wrist reliance across SHARP, VISTA, and GenSplat. SHARP's advantage over a wrist-only policy at small to moderate viewpoint shifts, together with its sensitivity to third-person input interventions, indicates that useful third-person information is retained (Section 5.4, Figures 9–10). Strength-matched comparisons against crop, affine augmentation, and masking further support the value of geometric variation beyond perturbation magnitude alone (Table 19; Appendix E.3). Together, these findings help explain why a geometric approximation of viewpoint changes can suffice without faithful reconstruction.

Third, we characterize the conditions and limits of SHARP's effectiveness. On LIBERO, its gains persist on average under small wrist-camera tilts but disappear without wrist observations (Appendices C.1.4 and D.4). These results suggest that lightweight geometric augmentation can suffice when complementary observations retain useful task information, providing a practical basis for understanding when such augmentation can be effective. We discuss these results in detail below.

We have revised Sections 1 and 6 to clarify why simple geometric augmentation can improve viewpoint robustness without faithful novel-view reconstruction, and when it can be effective.

The effectiveness of the proposed approach heavily depends on stable wrist-camera observations

Thank you for raising this concern. To address it, we separately evaluated SHARP’s robustness to wrist-camera viewpoint shifts and its dependence on the availability of wrist observations.

To directly address wrist-camera stability, we rotated the wrist camera by ±10° at evaluation, without retraining (Appendix C.1.4, Figure 15). Both columns below average success over 40 LIBERO tasks and five third-person azimuths (0°, ±45°, and ±75°), using one training seed. The tilt column additionally averages both wrist tilts.

MethodOriginal wrist orientation (%)Wrist tilt ±10° (%)
No augmentation71.959.3
VISTA89.373.0
GenSplat90.072.9
SHARP89.473.3

Although wrist-camera tilt lowers absolute performance for all methods, SHARP retains a 14.0 percentage-point advantage over No augmentation and performs comparably to VISTA and GenSplat. These results show that SHARP's average gains persist even when the wrist-camera viewpoint differs from training.

We acknowledge that SHARP’s dependence on wrist observations remains a limitation: without wrist observations during finetuning and evaluation, SHARP does not improve over No augmentation (Appendix D.4, Figure 18 left). VISTA-FT and GenSplat also achieve substantially lower success without wrist observations than their two-camera counterparts, indicating that this sensitivity is shared among the evaluated viewpoint-augmentation methods. In practice, third-person and wrist cameras are widely used together in robot-learning setups such as DROID [1] and ALOHA [2]. This motivates our focus on viewpoint robustness in this practically relevant camera configuration.

AuthorsOpenReview ↗

2/2 Response to Reviewer eGnV

The evaluation is limited to simulation environments.

Thank you for raising this concern. We added experiments on three tasks using a Franka Research 3 arm (Section 5.2.2, Table 2). We evaluated two shifted third-person viewpoints, with 15 trials per task, viewpoint, and method.

MethodSuccessful trialsSuccess rateAug. time (s/image)Peak aug. GPU memory (GB)
No augmentation53/9058.9%——
VISTA62/9068.9%6.3575.44
SHARP64/9071.1%0.013390.0824

SHARP improves average success by 12.2 percentage points over No augmentation and achieves success comparable to VISTA. The reference cost measurements show approximately 475× faster augmentation and 66× lower peak GPU memory than VISTA. These results extend our evaluation beyond simulation and support SHARP’s effectiveness on the evaluated real-robot tasks.

We are happy to address any further questions or concerns and would welcome additional feedback or suggestions for improving our work. Thank you.

References:

[1] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024.

[2] Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems, 2023.

AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer eGnV

Dear Reviewer eGnV,

Thank you again for your constructive feedback. We have revised the manuscript and responded to your comments, including additional experiments and analyses.

As the discussion period ends on November 18 (AoE), we would greatly appreciate it if you could share your thoughts on whether our responses and revisions address your concerns. If any questions remain, we would be happy to provide further clarification.

Thank you for your time and consideration.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewer eGnV,

Thank you very much for your time and effort in reviewing our paper and for your constructive feedback.

We would like to kindly follow up on our previous message. We have responded to your concerns and made a minor update to our response to clarify our contribution, particularly the central insight and the analyses supporting it.

As the author-reviewer discussion period ends on November 18 (AoE), we would greatly appreciate it if you could review our updated response and let us know if any concerns remain. We would be happy to provide further clarification if needed.

Thank you again for your valuable time and consideration.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their thorough feedback and constructive suggestions.

Main concerns addressed

Clarification of contributions — eGnV, kyHG, zP1P

We have clarified our three main contributions in Sections 1 and 6:

  1. Competitive viewpoint robustness without faithful novel-view reconstruction. In VLAs with third-person and wrist cameras, SHARP achieves robustness competitive with the evaluated view-synthesis methods at substantially lower augmentation cost. This provides a negative result on the presumed necessity of faithful reconstruction, a central goal of prior view-synthesis methods.

  2. Insights into how viewpoint augmentation improves robustness. Our analyses suggest that SHARP and the other evaluated viewpoint-augmentation methods improve robustness through two mechanisms: (i) broadening the training-data distribution with image variations that mimic viewpoint changes, and (ii) changing how the policy uses complementary observations.

  3. Conditions under which lightweight geometric augmentation can suffice. On LIBERO, SHARP’s gains persist on average under small wrist-camera tilts but disappear without wrist observations, suggesting that lightweight geometric augmentation can suffice when useful wrist observations remain available.

Real-robot validation — eGnV, kyHG, zP1P

Across three real-robot tasks and two shifted viewpoints, SHARP achieves 71.1% success, versus 58.9% for No augmentation and 68.9% for VISTA (Table 2). These results demonstrate SHARP's effectiveness beyond simulation, achieving success comparable to VISTA at substantially lower augmentation cost in the evaluated real-robot setting.

Augmentation controls — kyHG, zP1P

Across three training seeds, SHARP outperforms strength-matched crop, affine, and masking controls by 2.5–3.9 percentage points (Table 19). These results suggest that homography-based geometric variation offers benefits beyond simply increasing perturbation strength.

Wrist-camera stability — eGnV, kyHG, zP1P

We evaluated wrist-camera orientation changes together with third-person viewpoint shifts to test SHARP's dependence on a stable wrist view. SHARP retains an average advantage of 14.0 percentage points over No augmentation (Appendix C.1.4), suggesting that its gains can persist even when the wrist-camera viewpoint also changes.

Summary of paper updates

Main text:

  • Clarified the empirical finding, the roles of geometric variation and camera use, and the conditions under which lightweight augmentation can suffice (Sections 1 and 6).
  • Added real-robot results and augmentation costs (Section 5.2.2).
  • Added strength-matched augmentation comparisons and extended component ablations to three training seeds (Section 5.3).
  • Expanded blackout comparisons across methods to examine camera reliance beyond attention (Section 5.4).
  • Clarified training and evaluation settings, including seed counts (Section 5.1).

Appendix:

  • Added real-robot setup and cost details (Appendices A.4–A.5 and C.2).
  • Added source-image mixing, camera translations and combined shifts, and wrist-camera variation (Appendices C.1.1, C.1.3, and C.1.4).
  • Added detailed component-ablation results, no-wrist mixing, blackout comparisons, and strength-matched control protocols (Appendices D.2, D.4, and E.2–E.3).

Changes in the revised manuscript are marked in red to help readers locate the updates. The additional results strengthen the evidence for SHARP as a low-cost approach to viewpoint robustness in VLAs with third-person and wrist cameras.

We appreciate the reviewers' constructive suggestions and the time and attention of both the reviewers and area chair. We hope the revisions and detailed responses address the concerns raised, and we would be happy to provide further clarification or discuss any remaining questions.

AuthorsOpenReview ↗

Summary of the discussion

We thank the reviewers and Area Chair for their feedback and discussion. Reviewer zP1P stated that the additions largely resolved the original weaknesses, while noting the limited real-robot and wrist-perturbation coverage. Reviewer kyHG acknowledged the method’s effectiveness and the stronger experimental evidence, while maintaining concerns about novelty. We also responded to Reviewer eGnV’s concerns about contribution, wrist dependence, and real-world evaluation.

The revisions address the main experimental questions:

  • Real-robot validation and broader viewpoint coverage. Across three real-robot tasks and two shifted viewpoints, SHARP achieves 71.1% success, compared with 58.9% without augmentation and 68.9% for VISTA. Additional LIBERO tests show gains under camera translations and combined viewpoint shifts.
  • Strength-matched augmentation comparisons. Across three training seeds, SHARP exceeds strength-matched crop, affine, and masking controls by 2.5–3.9 percentage points, supporting a benefit beyond perturbation strength alone.
  • Wrist-camera dependence. On LIBERO, gains persist on average under wrist-camera tilts, but disappear without wrist observations, where performance falls below the no-augmentation baseline.

Details are provided in our General Response and reviewer-specific replies.

On novelty and significance, which all three reviewers raised, our position rests on three points:

  1. Challenging the presumed need for high-fidelity novel-view synthesis. Prior view-synthesis approaches aim to faithfully reconstruct novel views. We show that simple geometric augmentation can achieve competitive viewpoint robustness in the evaluated VLAs with third-person and wrist cameras. Our proposed method, SHARP, demonstrates this at substantially lower augmentation cost.
  2. Insights into how viewpoint augmentation improves robustness with complementary observations. Our analyses across SHARP, VISTA, and GenSplat suggest two complementary effects: broadening the training distribution through image variations that mimic viewpoint changes, and changing how the policy uses complementary observations. These shared effects help explain why a geometric approximation can suffice without faithful reconstruction.
  3. Testing the conditions and limits of simple geometric augmentation. Evaluations under camera translations, combined shifts, and real-robot conditions broaden the tested range. On LIBERO, gains persist on average under small wrist-camera tilts but disappear without wrist observations, identifying the importance of useful complementary inputs.

We thank the reviewers and Area Chair again for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.