acceptodds
Under review as a conference paper at ICLR 2027

Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse

Abstract

Vision-language-action (VLA) models adapt pretrained vision-language models (VLMs) for closed-loop robot control, transferring their perceptual and semantic capabilities to action prediction. Despite strong in-distribution performance, however, VLAs often degrade under deployment shifts. Adversarial training (AT) offers a model-adaptive approach to robustness without explicitly anticipating individual shifts, but its effect on natural distribution-shift generalization in multi-view VLAs remains unclear. We study this question using a multi-view VLA directly adapted from a pretrained VLM and evaluate generalization across seven LIBERO-Plus shift axes. Direct AT substantially improves Camera Viewpoint and Sensor Noise, the two shifts affecting only the third-person view, yet produces mixed or negative effects on other shifts. Controlled view interventions reveal a surprising failure mode that we term view collapse: Direct AT can shift cross-view reliance so strongly that the policy becomes dominated by the wrist view. This exposes a robustness shortcut: apparent robustness to a shifted view can arise from reduced use of that view rather than more robust perception of it. This motivates a distinction between robust perception, extracting reliable information under within-view shifts, and robust fusion, adapting reliance across views according to their reliability. To reduce fixed view reliance, we use a simple View Swap intervention and then re-evaluate AT. With View Swap, AT further improves Camera Viewpoint, Sensor Noise, and Robot Initial State, while its effects remain mixed on other shifts. Our results show that multi-view robustness requires separating improved perception from changes in cross-view reliance, and that AT provides selective rather than generic distribution-shift benefits.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Adversarial Training, Distribution Shift, Out-of-Distribution Generalization, Multimodal Robustness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aXJWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether adversarial training (AT) can improve the generalization of multi-view Vision-Language-Action (VLA) models under natural distribution shifts, revealing a failure mode termed "view collapse": direct adversarial training can cause the policy to primarily rely on the wrist view. Consequently, while the model achieves high success rates when the third-person camera undergoes distribution shifts, this does not demonstrate that the model can robustly utilize information from that camera. The authors validate this phenomenon through experiments including view blacking, progressive blurring, training dynamics analysis, and altering the attacked view, and subsequently explore SFT initialization and View Swap to mitigate fixed view reliance. Experiments covering seven categories of distribution shifts in LIBERO-Plus across various model configurations show that View Swap alleviates fixed view reliance, whereas further adding adversarial training yields selective rather than universal gains.

Critical strengths

The core finding carries practical significance and is supported by compelling interventional evidence. On LIBERO-Object, Direct AT maintains a 98.8% success rate even after removing the third-person view, but its success rate drops to 0% when the wrist view is removed. Progressive blurring experiments support the same asymmetry in reliance, and altering the attacked view reverses the direction of policy reliance (Table 10). These results convincingly demonstrate that performance gains under a specific view's shift may stem from simply evading that view, highlighting a crucial pitfall in evaluating the robustness of multi-view policies.

The inclusion of matched-condition comparisons among SFT/AT/Swap/AT+Swap, View Dropout baselines, perturbation budget and norm ablations, as well as experiments across different backbones and action decoders, enhances the credibility of the study. The authors also report performance drops under certain shifts, along with distinct results observed in the robot-pretrained 0 model, maintaining reasonable boundaries for the applicability of their conclusions rather than merely emphasizing optimal configurations.

Critical weaknesses

The distinction between "robust perception" and "robust fusion" lacks sufficient empirical validation. Figure 3 demonstrates that the model copes better with degradation when either view is removed or blurred, but it does not directly clarify how the policy adjusts its reliance when both views retain useful information while their relative reliability changes. Thus, it remains unclear whether the additional gains from adding AT on top of View Swap stem from improved perception, improved fusion, or other training effects. Although the authors acknowledge this limitation in Section 6, it still constrains the mechanistic insights provided by this core conceptual distinction. Furthermore, detailed view-intervention experiments are primarily concentrated on LIBERO-Object, whereas the distribution shift results are aggregated across four task suites.

Questions for authors
  1. Could you evaluate view reliance under conditions where both views still contain useful information but their relative reliability is altered, covering representative tasks beyond LIBERO-Object?
  2. How many independent training runs were performed for the main comparisons? What is the number of rollouts evaluated per task suite and per shift category?
  3. All reported AT configurations utilize single-step Fast-AT. Does view collapse still occur when training with multi-step PGD? How does it compare to training with random perturbations under matched conditions?
4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer aXJW (1/N)

Rebuttal to Reviewer aXJW

We sincerely appreciate your recognition that (1) the core finding carries practical significance with compelling interventional evidence, and (2) the matched comparisons and ablations enhance the credibility of the study.

Below, we address the raised concerns in detail.

[W1, Q1: Robust Perception vs. Robust Fusion Beyond LIBERO-Object]

We added view interventions under Camera/Noise shifts and joint-degradation experiments, providing behavioral evidence consistent with contributions from both robust fusion and perception. We also extended Fig. 3 to all four LIBERO suites.

1. Camera and Noise Shifts Reveal Different Sources of Improvement

Input conditionCamera SwapCamera AT+SwapNoise SwapNoise AT+Swap
Shifted third-person only2.31.79.98.4
Unshifted wrist only82.784.482.286.3
Both views77.984.185.788.3

LIBERO-Plus success rates (%), four-suite average, seed 42.

  • Camera: Evidence consistent with more robust fusion. Adding the shifted view reduces Swap success by 4.8 pp but AT+Swap by only 0.3 pp, suggesting reduced interference rather than improved shifted-view perception.
  • Noise: Gains largely reflect improved wrist-only performance. AT+Swap improves wrist-only success by +4.1 pp versus +2.6 pp with both views. Both models nevertheless benefit from retaining the noisy third-person view.

2. Joint Degradation Reveals Improved Robustness and Asymmetric View Reliance

We additionally evaluated different combinations of third-person and wrist-view degradation on LIBERO-Object, including cases where neither view remains intact.

Input conditionSFTSwapAT+Swap
Clean (0, 0)100.0100.099.8
Third blur (6, 0)38.298.499.6
Wrist blur (0, 6)64.290.692.2
Both blurred (6, 6)0.03.827.8
Both blurred (8, 6)0.05.042.4
Both blurred (6, 8)0.00.03.6
Wrist blur σ=6; third removed0.010.058.4
Wrist blur σ=8; third removed0.00.016.4

LIBERO-Object success rates (%), 500 episodes per condition. Pairs denote (third-person blur σ, wrist blur σ); removed views are blacked out.

  • Finding 1: AT+Swap improves robustness when both views are degraded. It substantially outperforms Swap under (6,6) and (8,6), where neither view is intact.
  • Finding 2: The improvement depends on relative view reliability. AT+Swap achieves 42.4% under (8,6), but only 3.6% under (6,8), indicating that robust performance remains strongly dependent on the wrist view.
  • Finding 3: The gains extend beyond fallback to an intact view. With the third-person view removed, AT+Swap still achieves 58.4% using a blurred wrist view, versus 10.0% for Swap. This is consistent with improved robustness to degraded visual inputs, although it does not isolate the internal perceptual mechanism.

The four-suite averages also show gains under joint blur, although smaller: 3.0% versus 8.4% at (6,6) and 3.2% versus 12.0% at (8,6) (Table 4).

3. View Interventions Extended to All Four Suites (Revised Fig. 3)

We expanded Fig. 3 from LIBERO-Object to four-suite averages and added suite-wise results in the Appendix. The results confirm that Swap-based models remain functional when either view degrades, whereas Direct AT exhibits wrist-dominant reliance across all four suites. AT+Swap does not consistently outperform Swap, particularly under wrist blur. The extent of mitigation varies across suites.

Additional Control: Random-Noise Training

Matched random-noise controls do not reproduce AT's Camera gains (see our response to CunZ, W1-1), indicating that these improvements are not explained by noise augmentation alone.

Summary of W1/Q1

The results provide behavioral evidence for reduced interference from unreliable views and improved robustness to degraded inputs, while showing that these effects remain shift- and task-dependent. They do not establish the internal mechanisms.

Paper: Secs. 5.2–5.3; revised Fig. 3; Appendix E.3 (Fig. 5); Appendix G (Table 18)

[Q2: Training Runs and Rollouts]

We clarified the training-run counts and evaluation protocol in the Appendix.

  • Training runs: One run per configuration, except for Qwen3.5-0.8B-OFT Swap and AT+Swap (3 continuation seeds each).
  • Clean LIBERO: 500 rollouts per suite (10 tasks × 50 rollouts).
  • LIBERO-Plus: One rollout per shift variant (10,030 total per model).
    • Per suite: Spatial 2,402; Object 2,518; Goal 2,591; LIBERO-10 2,519.
    • Per shift: Camera 1,599; Robot 1,550; Language 1,537; Light 1,142; Background 1,076; Noise 1,601; Layout 1,525.

The Appendix provides the complete suite-by-shift breakdown.

AuthorsOpenReview ↗

Rebuttal to Reviewer aXJW (2/2)

[Q3: Does View Collapse Persist Under Multi-Step PGD?]

We added matched random-noise and two-step PGD controls, confirming that view collapse is not reproduced by random perturbations but also occurs under multi-step AT.

1. Matched Random Noise Does Not Reproduce AT's Effects

Training (30k updates)CameraNoiseWrist only
Direct Rand.21.845.816.3
Direct AT88.486.879.4

Success rates (%), four-suite averages; ε=1/255 with matched initialization and training schedule. Camera/Noise: LIBERO-Plus; wrist only: clean LIBERO.

  • Finding: Random perturbations reproduce neither AT's Camera/Noise gains nor its extreme wrist-dominant reliance.

2. View Collapse Persists Under Two-Step PGD

Training (10k updates)CleanThird onlyWrist only
Direct AT (1-step)99.20.0100.0
Direct AT (2-step PGD)74.40.074.2

LIBERO-Object success rates (%); ε=1/255. Two-step PGD uses a random start and step size α=ε.

  • Finding: Both methods fail completely with only the third-person view while retaining success with the wrist view alone. Thus, view collapse is not specific to single-step AT, although the two-step checkpoint has lower clean performance.

Summary of Q3

These experiments establish that the observed collapse is not explained by matched random noise and can persist under multi-step adversarial optimization.

--

Summary of the rebuttal

Overall, our additional experiments strengthen the evidence that AT's generalization gains are shift-dependent and must be interpreted alongside view reliance. Four-suite view interventions confirm wrist-dominant collapse under Direct AT and its mitigation by View Swap. Degraded-view experiments provide behavioral evidence consistent with contributions from robust perception and robust fusion. Matched random-noise and two-step PGD controls further show that view collapse is not reproduced by random noise and can persist under multi-step AT.

We hope these additional results and clarifications address your concerns and strengthen the conclusions of our work.

Thanks for the thorough rebuttal and helpful new experiments on multi-suite interventions and PGD controls. Overall, I remain positive about the view-collapse insights and will maintain my original score.

Dear Reviewer,

Thank you again for reviewing our rebuttal and additional experiments. We are pleased that you remain positive about our view-collapse insights, and we truly appreciate your continued support toward the acceptance of our work.

Thank you once again for your time and consideration.

Best regards,
The Authors

Reviewer_JpYkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether adversarial training (AT) helps multi-view VLAs generalize to natural distribution shifts. Using a Qwen3.5-0.8B model adapted directly to LIBERO, the authors find that AT gives large gains on the camera-viewpoint and sensor-noise shifts of LIBERO-Plus. View interventions, however, show that this is largely because the policy stops using the third-person view, which the authors call "view collapse". They propose View Swap, which randomly replaces one view with that of another sample, and show that AT on top of View Swap gives more consistent gains across several model configurations.

Critical strengths
  1. The view-collapse finding is interesting and supported by the black-out/blur interventions and the attack-placement experiment. It is a useful warning for anyone evaluating the robustness of multi-view policies.
  2. View Swap is very simple and works well (65.3 → 77.0 on LIBERO-Plus), and it outperforms a matched View Dropout baseline.
Critical weaknesses
  1. For an analysis paper, the explanation of why AT leads to collapse stays largely at the level of intuition. The analysis and the observations are suggestive, but there is no formal or theoretical analysis. View reliance is also judged mostly qualitatively from black-out/blur success rates. In addition, the key interventions are done only on LIBERO-Object, where the wrist view alone is already informative. I am not sure the conclusions hold for tasks that rely more on global information.
  2. The experimental setup makes it hard to attribute the effects to AT. The AT used is very weak (single step, ε = 1/255), and there is no random-noise control of the same magnitude, so the gains could simply come from noise augmentation. Moreover, the SFT baseline in Table 3 (~13.75k steps) is better than the 30k-step SFT in Table 1. This is surprising and not explained.
Questions for authors
  1. Could you add a random-noise control with the same ε and schedule?
  2. Why is the shorter SFT run better than the 30k-step one?
  3. Do the view interventions give the same picture on more task suites and benchmarks?
  4. Can you give some theoretical or toy-model explanation of when AT suppresses a view and why simply View Swap is useful?
  5. Why View Swap noticeably hurts the language shift, and neither View Swap nor AT helps for π0 (Table 4)?
7 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer JpYk (1/N)

Rebuttal to Reviewer JpYk

We sincerely appreciate your recognition that (1) the view-collapse finding is interesting and supported by black-out/blur and attack-placement experiments, and (2) View Swap is simple and works well (65.3 → 76.6), outperforming a matched View Dropout baseline.

Below, we address the concerns one by one.

[W1-1, Q4: Why Does AT Suppress a View, and Why Does View Swap Help?]

We added a formal two-view analysis (Sec. 4.2; Appendix D) showing that AT can favor single-view reliance even when both views are informative, whereas View Swap encourages contributions from both. We further support this explanation with attack-target interventions.

1. Why can AT suppress an informative view?

Consider a binary target y∈{−1,+1}y\in\{-1,+1\} and two informative views xg=ayx^g=ay and xw=byx^w=by, where 0<ϵ<a<b0<\epsilon<a<b. A linear predictor combines them as y^=wgxg+wwxw\hat y=w_gx^g+w_wx^w. Let A=awgA=aw_g and B=bwwB=bw_w denote their predictive contributions.

Under independent ℓ∞\ell_\infty-bounded perturbations of each view, the exact adversarial squared loss is

LAT=(∣A+B−1∣+ϵ(∣A∣a+∣B∣b))2.\mathcal{L}_{\mathrm{AT}}=\left(|A+B-1|+\epsilon\left(\frac{|A|}{a}+\frac{|B|}{b}\right)\right)^2.

Although both views perfectly predict yy, the first view has a higher adversarial sensitivity cost per unit of predictive contribution (ϵ/a>ϵ/b\epsilon/a>\epsilon/b). Consequently, the unique AT optimum is (A,B)=(0,1)(A,B)=(0,1), relying exclusively on the second view. In contrast, the clean minimum-norm solution uses both views.

Thus, AT can reduce adversarial sensitivity by suppressing an informative but more costly view, rather than robustifying both.

2. Why does View Swap counteract this tendency?

View Swap replaces one randomly selected view with an independently sampled view while keeping the original target. Since the replaced view may contain conflicting information, relying exclusively on either view becomes costly. In this toy model, the expected loss on swapped examples is

LSwap=A2+B2−A−B+1,\mathcal{L}_{\mathrm{Swap}}=A^2+B^2-A-B+1,

whose unique optimum is (A,B)=(1/2,1/2)(A,B)=(1/2,1/2). The same optimum holds when swapped examples are mixed with clean examples at any nonzero swap probability.

Unlike AT, View Swap favors contributions from both views rather than exclusive reliance on one, counteracting the tendency illustrated above.

3. Experimental support

Our attack-target interventions show that changing the attacked view reverses the observed reliance: third-person-only AT favors the wrist view, whereas wrist-only AT favors the third-person view. These results are consistent with AT suppressing reliance on a view when its adversarial sensitivity is costly (Sec. 4.2; Appendix C).

Summary of W1-1/Q4

The toy model demonstrates how AT can favor view collapse and why View Swap can counteract this tendency. It illustrates a possible mechanism rather than establishing the precise optimization dynamics of deep VLAs.

Paper: Sec. 4.2; Appendices C–D.

[W1-2: View Reliance Is Quantified Through Controlled Interventions]

Our black-out and graded-blur experiments quantitatively measure functional view reliance through changes in task success.

  • Black-out: We measure the success-rate drop when either view is removed (500 episodes per condition and suite).
  • Graded blur: We vary blur strength to measure how performance changes as each view degrades.
  • Scope: These interventions provide quantitative behavioral evidence of view dependence, rather than a direct measurement of internal attention or feature use.

Paper: Sec. 4.2; Figs. 1, 3.

[W1-3, Q3: View Collapse Across All Four LIBERO Suites]

Black-out and blur confirm Direct AT's view collapse across all suites and its mitigation by View Swap.

SuiteCleanThird onlyWrist onlyThird blur σ=8Wrist blur σ=8
Spatial96.60.091.297.40.0
Object98.40.088.498.230.2
Goal93.60.073.293.68.6
LIBERO-1083.20.065.081.20.0
Average93.00.079.492.69.7
  • Direct AT: Third-only success is 0% in every suite, versus 79.4% wrist-only on average. Blur shows the same asymmetry.

The following comparison tests whether third-person failure is inherent or training-induced.

TrainingThird onlyWrist only
Direct AT0.079.4
Swap68.085.9
AT+Swap66.788.4
Third-only SFT86.9—
Wrist-only SFT—93.8

Success rates (%), four-suite averages; single-view SFT: 30k updates.

  • Mitigation: Swap and AT+Swap recover third-only success (68.0%, 66.7%) while retaining wrist-only success.
  • Not information-limited: Third-only SFT reaches 86.9%, indicating the third-person view itself is informative.

Scope: Findings extend to LIBERO-10, but not yet beyond LIBERO. Paper: Sec. 4.2; Figs. 1, 3; Table 2; App. E.

AuthorsOpenReview ↗

Rebuttal to Reviewer JpYk (2/2)

[W2-1, Q1: Are the Findings Specific to Weak AT or Random Noise?]

We added matched random-noise and multi-step PGD experiments, showing that (1) random noise reproduces neither AT's generalization gains nor view collapse, and (2) view collapse also occurs with two-step PGD.

1. Matched Random Noise Does Not Reproduce AT's Effects

We replaced the adversarial gradient step with uniform random noise under the same ε=1/255 budget, initialization, and 30k-update training schedule.

TrainingPlus OverallCameraNoiseWrist only
SFT61.931.350.423.6
Direct Rand.58.321.845.816.3
Direct AT69.688.486.879.4

Success rates (%), averaged across four LIBERO suites. Wrist only refers to clean LIBERO with the third-person view blacked out.

  • Finding: Random noise does not reproduce AT's Camera/Noise gains or wrist-dominant view collapse. Direct Rand. remains close to SFT, whereas Direct AT achieves large Camera/Noise gains and retains 79.4% success using only the wrist view.

2. View Collapse Also Occurs with Multi-Step PGD

We additionally evaluated Direct AT with two-step PGD on LIBERO-Object, comparing checkpoints after 10k optimizer updates.

Training (10k updates)CleanThird onlyWrist only
Direct AT (1-step)99.20.0100.0
Direct AT (2-step PGD)74.40.074.2

LIBERO-Object success rates (%), 500 episodes per condition.

  • Finding: View collapse is not unique to single-step AT. Two-step PGD also exhibits extreme wrist-dominant reliance, although its clean success rate is lower at this checkpoint.
  • Perturbation strength: Increasing ε from 1/255 to 2/255 for AT+Swap reduces LIBERO-Plus success by 4.4 pp, indicating that stronger perturbations do not necessarily improve generalization.

Summary of W2-1/Q1

Together, these controls show that the observed view collapse is not explained by matched random noise and can also arise under multi-step adversarial training.

Paper: Sec. 4.2; Appendices E–F.

[W2-2, Q2: Why Does the Apparently Shorter SFT Perform Better?]

We identified an optimizer-update counting error and confirmed that the corrected comparison preserves our conclusions.

  • Cause: The original 30k-step SFT actually received 3,750 optimizer updates due to gradient accumulation. The Sec. 5 SFT continued from this checkpoint for another 10k updates (13,750 total).
  • Correction: A new SFT baseline trained for 30k updates achieves 96.0% clean / 61.9% LIBERO-Plus, versus 95.6% clean / 65.3% LIBERO-Plus for the 13,750-update SFT. Direct AT still achieves 69.6% LIBERO-Plus under the matched 30k-update budget.
  • Sec. 5: All continuation methods share the same SFT initialization and matched 10k-update training budgets.

Paper: Secs. 4.1, 5.3; Table 1; Appendix A.

[Q5: Why Does View Swap Hurt Language Shifts, and Does AT Help π0?]

View Swap's benefits are shift-dependent, and our corrected π0 experiments show that AT also improves generalization in a robot-pretrained VLA.

  • Language shift: View Swap reduces Language success on 0.8B-OFT (71.0 → 69.7%) and 2B-OFT (84.0 → 73.9%). Since View Swap targets cross-view reliance rather than language grounding, improvements on visual shifts do not necessarily extend to language shifts. These shift-dependent effects are reported in Tables 5 and 7.
  • π0 correction: We changed action-expert training from LoRA to full fine-tuning, consistent with the other model configurations, and reran the experiments.
π0 TrainingOverallCameraRobotNoise
Swap42.917.45.416.9
AT+Swap45.622.97.418.2

LIBERO-Plus success rates (%), averaged over four suites.

  • Finding: AT improves π0 generalization across these shifts. AT+Swap improves over Swap by +2.7 pp overall, +5.5 pp on Camera, +2.0 pp on Robot, and +1.3 pp on Noise.

Thus, the corrected π0 results demonstrate positive AT gains, although their magnitude remains model- and shift-dependent.

Paper: Secs. 5.3–5.4; Tables 5, 7.


Summary of the rebuttal

Overall, our additional analysis and experiments strengthen the central finding that AT can induce view collapse even when both views are informative. The formal example explains how this behavior can arise, quantitative interventions confirm it across four LIBERO suites, and matched controls distinguish it from random-noise effects. View Swap mitigates the failure mode, while the observed generalization gains remain shift- and model-dependent.

We hope these additional results and revisions address your concerns and clarify the scope and significance of our findings.

AuthorsOpenReview ↗

Gentle Reminder: Discussion Deadline Approaching (Nov 18 AoE)

Dear Reviewer,

As the discussion period ends in just over one day (Nov 18, AoE), we would appreciate it if you could review our rebuttal and let us know whether your concerns have been addressed.

We are happy to provide further clarification if needed.

Thank you for your time.

Best regards, The Authors

Reviewer_CunZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates whether adversarial training (AT) improves distribution-shift generalization in multi-view VLAs, evaluated on LIBERO-Plus with VLMs adapted directly to robot control. It shows that Direct AT’s gains on Camera and Noise shifts come from “view collapse,” where the policy relies almost entirely on the wrist view. It then proposes View Swap to remove this fixed reliance, and reports that AT on top of View Swap gives further but selective gains across four VLM-to-VLA configurations.

Critical strengths
  1. The view-collapse finding is supported by converging interventions and the attack-placement experiment, and it carries a general lesson: robustness to a shift affecting one camera cannot be interpreted without checking whether that camera is still used.

  2. View Swap is a zero-cost intervention that substantially improves LIBERO-Plus performance across all VLM-to-VLA configurations, which has practical value even if the AT claims do not hold up.

Critical weaknesses

Weakness 1: The experiments do not isolate the contribution of adversarial perturbations.

The paper compares AT with training without perturbations, but it does not include a matched random-noise baseline. With ε = 1/255 and a uniform random start, some gains could come from adding pixel noise rather than from the adversarial gradient step. Comparing AT with random noise of the same magnitude, as well as with suitable standard data augmentations, would isolate AT’s contribution.

The view-collapse diagnosis is valuable on its own, but the paper’s claims about AT depend on showing that adversarial perturbations add value beyond View Swap. The incremental Total gains from adding AT to View Swap are +2.7 pp for the main model and +0.4 pp for both Qwen3.5-2B and π0, with no further improvement on Camera or Noise for π0, the only robot-pretrained VLA tested. The paper reports no multi-seed estimates or uncertainty intervals, so the reliability of these small increments is difficult to assess; the SFT Total also differs between Sec. 4 and Sec. 5 (62.7% versus 65.3%), which is not a direct variance estimate given the different settings, but makes uncertainty reporting particularly important.

Weakness 2: The paper does not directly test robust perception under the shifts used to support its claims.

Section 4 argues that Direct AT’s gains on Camera and Noise may result from ignoring the third-person view. The paper then treats AT+Swap gains on those same shifts as evidence of robustness. However, View Swap trains the policy to handle an unreliable view, and Camera and Noise affect only the third-person view. The policy could therefore gain by relying on the unshifted wrist view, without becoming more robust to the shifted third-person input.

The paper reports view-ablation and blur interventions for AT+Swap, but does not evaluate Camera or Noise shifts with the wrist view removed. That evaluation would help assess whether the gains persist when the policy must use the affected view. The perception/fusion distinction is central to the paper’s argument, but the paper does not directly apply it to explain whether AT improves perception of the shifted view or changes the policy’s reliance on it.

Questions for authors

As Weakness

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer CunZ (1/N)

We sincerely appreciate your recognition that (1) the view-collapse finding carries a general lesson for evaluating multi-view robustness, and (2) View Swap is a zero-cost intervention with practical value across all VLM-to-VLA configurations.

[W1-1: Is AT's gain due to the adversarial step or pixel noise?]

We added matched random-noise controls and found that (1) random noise reproduces neither Direct AT's view collapse nor its Camera/Noise gains, and (2) AT+Swap outperforms Rand.+Swap in overall and Camera success across all five model configurations.

  • Control: Replace adversarial perturbations with uniform random noise within the same ε-ball (ε = 1/255), keeping all other training settings fixed.
  • Evaluation: Four LIBERO suites; success rates (%).

1. Direct Training from the Pretrained VLM

LIBERO-Plus generalization:

TrainingOverallCameraRobotNoiseLayout
SFT61.931.349.450.474.0
Rand.58.321.844.645.875.8
Direct AT69.688.436.986.868.6
  • Finding: Random noise does not reproduce Direct AT's large Camera/Noise gains. Direct AT improves overall generalization over SFT, whereas Rand. does not; its benefits remain shift-dependent.

View reliance under black-out (clean LIBERO, four-suite average):

TrainingThird onlyWrist only
SFT1.123.6
Rand.0.316.3
Direct AT0.079.4
  • Finding: Random noise does not reproduce Direct AT's extreme wrist-dominant view collapse. Unlike SFT and Rand., Direct AT maintains high success with only the wrist view, but fails completely with only the third-person view.

Paper: Secs. 4.1–4.2; Table 1; Fig. 1; Appendix E, Table 15.

2. SFT-Initialized Continuation Across Five Model Configurations

ModelTrainingOverallCameraRobotNoiseLayout
0.8B-OFTSwap76.677.760.283.277.4
Rand.+Swap76.976.961.681.678.0
AT+Swap77.682.663.684.879.4
PaliGemma-OFTSwap39.931.333.430.755.2
Rand.+Swap41.330.835.232.257.2
AT+Swap53.674.739.168.665.2
0.8B-PISwap67.351.858.979.673.4
Rand.+Swap66.455.358.372.874.2
AT+Swap71.568.864.882.778.0
2B-OFTSwap77.781.255.185.381.6
Rand.+Swap75.872.559.983.977.2
AT+Swap78.185.057.690.179.2
π0Swap42.917.45.416.967.7
Rand.+Swap43.719.64.915.668.1
AT+Swap45.622.97.418.269.7

0.8B-OFT Swap and AT+Swap are three-seed means; the other entries are single runs. Comparisons use matched settings within each model.

  • Finding: AT provides benefits beyond random noise across all five model configurations. Rand.+Swap remains close to Swap, whereas AT+Swap consistently achieves higher Overall and Camera success than Rand.+Swap. The magnitude of improvement varies across models and shifts.

Paper: Secs. 5.3–5.4; Tables 5, 7.

Summary of W1-1

Matched random noise reproduces neither Direct AT's view collapse and Camera/Noise gains nor AT+Swap's advantages over Rand.+Swap across model configurations.

AuthorsOpenReview ↗

Rebuttal to Reviewer CunZ (2/N)

[W1-2: Small increments, statistical uncertainty, and cross-model consistency]

We re-evaluated AT across models and training seeds and found that (1) Camera/Robot/Noise improvements are consistent across all five configurations, with substantial gains for some models, and (2) Camera improvements are reproducible across seeds.

  • Scope of our claim: Our main contribution is not a uniform improvement in overall LIBERO-Plus success, but an analysis of how AT affects distribution-shift generalization, including view collapse and its mitigation.

1. Improvements across model configurations

We compared AT+Swap against Swap across five configurations, including corrected experiments for robot-pretrained π0.

AT+Swap − Swap (pp)OverallCameraRobotNoise
0.8B-OFT (3 seeds)+1.0+4.9+3.4+1.6
PaliGemma-OFT+13.7+43.4+5.7+37.9
0.8B-PI+4.2+17.0+5.9+3.1
2B-OFT+0.4+3.8+2.5+4.8
π0 (corrected)+2.7+5.5+2.0+1.3
  • Finding: AT improves Camera, Robot, and Noise across all five configurations, with substantial gains in some models. Although the overall gain is small for 0.8B-OFT, PaliGemma achieves +13.7 pp overall (+43.4 Camera, +37.9 Noise), and 0.8B-PI achieves +4.2 pp overall (+17.0 Camera).
  • π0 correction: We changed π0's action-expert training from LoRA to full fine-tuning, matching the training configuration used for the other models (VLM LoRA + full action-expert training), and reran the experiments. AT+Swap now improves over Swap by +2.7 pp overall and +5.5 pp on Camera, compared with the previously reported +0.4 pp overall.

2. Reliability across training seeds (0.8B-OFT)

The Camera improvement is consistent across three training seeds, despite smaller and more variable overall gains. We evaluated three independent continuations from the same SFT checkpoint. AT+Swap improves mean Camera success by +4.9 pp (82.6 ± 1.2% vs. 77.7 ± 0.4%), with non-overlapping run ranges (81.8–84.1% vs. 77.3–78.0%). Overall success improves by +1.0 pp on average (77.6 ± 1.8% vs. 76.6 ± 0.8%), but the run ranges overlap.

Paper: Secs. 5.3–5.4; Tables 5, 7; Appendix A.2.

Summary of W1-2

AT+Swap improves Camera, Robot, and Noise across all five model configurations, while three-seed evaluations confirm consistent Camera gains despite smaller and more variable overall improvements.

[W1-3: Why do the SFT results differ between Secs. 4 and 5?]

We identified and corrected an optimizer-update counting error and confirmed that our main findings remain unchanged under matched training budgets.

  • Cause: Due to gradient accumulation, the original Sec. 4 SFT received 3,750 rather than 30,000 optimizer updates. The Sec. 5 SFT continued from this checkpoint for another 10k updates (13,750 total), explaining the different results (62.7% vs. 65.3%).
  • Correction: We retrained Sec. 4 SFT for 30k updates, matching Direct AT. The corrected SFT achieves 96.0% clean / 61.9% LIBERO-Plus, compared with 95.6% clean / 65.3% LIBERO-Plus for the 13,750-update SFT. Direct AT still improves LIBERO-Plus success to 69.6% and exhibits view collapse.
  • Sec. 5: All methods share the same SFT checkpoint and matched 10k-update continuations, so their comparisons remain valid.

Paper: Secs. 4.1–4.2; Table 1; Fig. 1; Appendix A.

AuthorsOpenReview ↗

Rebuttal to Reviewer CunZ (3/3)

[W2: Robust perception vs. robust fusion]

Our additional view interventions suggest that (1) Camera gains are consistent with more robust fusion rather than improved perception of the shifted view, while (2) degraded-view results support a contribution from robust perception beyond clean-view fallback.

1. Camera Gains Are Consistent with Robust Fusion Rather Than Improved Shifted-View Perception

We blacked out either view under Camera/Noise shifts across four LIBERO suites (seed 42; success rates, %).

Input conditionCamera SwapCamera AT+SwapNoise SwapNoise AT+Swap
Shifted third-person only2.31.79.98.4
Unshifted wrist only82.784.482.286.3
Both views77.984.185.788.3
  • Camera: Evidence consistent with more robust fusion, not improved shifted-view perception. Adding the shifted view reduces Swap success by 4.8 pp but AT+Swap by only 0.3 pp. Neither improves success using the shifted view alone.
  • Noise: Gains largely reflect improved wrist-only performance. AT+Swap improves wrist-only success by +4.1 pp, compared with +2.6 pp using both views.

Paper: Secs. 5.2–5.3; Table 6.

2. Degraded-View Results Support Robust Perception Beyond Cross-View Fusion

We evaluated blurred inputs on LIBERO-Object (seed 42; success rates, %).

Input conditionSwapAT+Swap
Wrist blur σ=6; third-person blacked out10.058.4
Wrist blur σ=8; third-person blacked out0.016.4
Both blurred (6, 6)3.827.8
Both blurred (8, 6)5.042.4
  • Evidence for robust perception beyond cross-view fusion. AT+Swap substantially outperforms Swap even when only a degraded wrist view is available. This supports improved processing of degraded visual inputs rather than merely falling back to an intact view, although the internal mechanism remains unidentified.

Paper: Sec. 5.2; Appendix G, Table 18.

Summary of W2

These results suggest that AT can contribute to both robust perception and robust fusion, with their relative contributions depending on the type of visual degradation.


Summary of the rebuttal

The additional experiments strengthen our central conclusion. AT can improve distribution-shift generalization, but these gains are selective and must be interpreted alongside how the model uses its input views. Matched-noise controls isolate AT-specific effects, cross-model and multi-seed evaluations establish their consistency and limitations, and view interventions clarify the distinct roles of robust perception and robust fusion.

We hope these new results and revisions address your concerns and clarify the contributions of our work.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response by Authors

General Response by Authors

We thank the reviewers for their valuable feedback. We are encouraged that all reviewers found the view-collapse finding valuable and well supported:

  • CunZ: "carries a general lesson"
  • JpYk: "a useful warning for anyone evaluating the robustness of multi-view policies"
  • aXJW: "supported by compelling interventional evidence"

We have conducted additional experiments and substantially revised the manuscript to address the concerns. The key updates are summarized below.

Summary of New Results

TopicKey Updates
Is it AT or just noise? (CunZ W1; JpYk W2/Q1; aXJW Q3)We added matched random-noise controls and confirmed that noise alone cannot explain AT's view collapse or Camera/Noise gains. AT+Swap also outperforms Rand.+Swap in both overall performance and Camera across all four additional model configurations, with model-dependent gains. (Tables 1, 5, 7, 15)
Why does view collapse occur? (JpYk W1/Q4)We added a formal analysis of a two-view linear model, showing mathematically how AT can favor single-view reliance while View Swap counteracts it. Attack-placement interventions show that changing the attacked view reverses the observed reliance pattern. Matched single-view SFT experiments further confirm that either view alone can support successful task execution. (Secs. 4.2, 5.1; Appendices C–D)
Beyond LIBERO-Object? (JpYk W1/Q3; aXJW W1/Q1)We extended quantitative black-out and graded-blur interventions to all four LIBERO suites and confirmed wrist-dominant view collapse under Direct AT across all suites. (Figs. 1, 3, 5; Appendix E.3)
Perception vs. fusion? (CunZ W2; aXJW W1/Q1)We conducted shift-specific view ablations and simultaneous-view degradation experiments, providing behavioral evidence for improved robustness to degraded inputs and reduced interference from shifted views. Camera gains are consistent with reduced interference, while degraded-view-only experiments support improved robust perception. (Secs. 5.2–5.3; Tables 4, 6; Appendix G)
Are the gains reliable? (CunZ W1; aXJW Q2)We evaluated five model configurations and found consistent improvements in Camera, Robot Initial State, and Sensor Noise from AT+Swap over Swap. AT+Swap also outperforms Rand.+Swap in Overall and Camera across all five configurations, although the gains vary in magnitude. For 0.8B-OFT, the Camera improvement is also consistent across three continuation seeds (+4.9 pp). (Tables 5, 7)

Other Updates

TopicKey Updates
SFT training budget? (CunZ W1; JpYk W2/Q2)We identified and corrected an optimizer-update counting error in the SFT baseline and reran the budget-matched comparison, confirming that the main findings remain valid. Direct AT retains its Camera/Noise gains and view collapse under matched 30k-update training budgets. (Table 1; Fig. 1; Appendix A)
Does AT help π0? (CunZ W1; JpYk Q5)We corrected π0's training configuration and confirmed that AT provides benefits for this robot-pretrained model. AT+Swap improves over Swap by +2.7 pp overall and +5.5 pp on Camera. (Sec. 5.4; Table 7)
Does view collapse persist with multi-step AT? (aXJW Q3)We evaluated two-step PGD adversarial training and confirmed that view collapse is not specific to single-step AT. The same wrist-dominant reliance appears on LIBERO-Object. (Appendix E)
Evaluation protocol? (CunZ W1; aXJW Q2)We clarified training-run counts, seed variability, and evaluation protocols. Each configuration uses a single training run, except for 0.8B-OFT Swap and AT+Swap, which use three continuation seeds each. LIBERO-Plus evaluation covers 10,030 variants across four suites (Table 5; Appendix A.2)

Detailed responses to each reviewer's concerns are provided below.

AuthorsOpenReview ↗

Gentle Reminder: Author–Reviewer Discussion Deadline Approaching

Dear Reviewers,

Thank you again for your valuable feedback.

As the discussion period will end in approximately two days (November 18, AoE), we would greatly appreciate it if you could review our rebuttal and let us know whether our responses have addressed your concerns.

We would be happy to provide any further clarification if needed.

Thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.