acceptodds
Under review as a conference paper at ICLR 2027

Uncovering and Mitigating Positional Blind Spots in Vision-Language-Action Models

Abstract

Recent Vision-Language-Action (VLA) models achieve promising performance in robotic manipulation, typically measured by success rates aggregated over predefined object configurations, an evaluation that implicitly assumes spatially uniform competence across the workspace. We show that this assumption does not hold: even with the instruction and every other scene factor held fixed, merely relocating a task-irrelevant distractor can sharply raise the failure probability within localized, spatially coherent regions, which we term Positional Blind Spots (PBS). In this paper, we propose a two-stage black-box framework to uncover and mitigate PBS. During the uncovering stage, we grid the workspace and apply a one-sided log-likelihood-ratio test to localize PBS cells with significantly elevated risk. During the mitigation stage, we fine-tune the policy via LoRA on demonstrations collected from these PBS regions, improving competence there while largely preserving performance across the rest of the workspace. We evaluate our framework on five state-of-the-art VLA policies across two benchmarks, and find that PBS are pervasive and spatially concentrated in all of them, with failure rates up to 0.58. Our search strategy achieves an average F1-score of 0.678, outperforming random search and adaptive sampling baselines by 0.268 and 0.178, respectively. Guided by the discovered regions, targeted fine-tuning reduces the overall failure rate by 40.00%–85.19%.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action (VLA) Models, Embodied AI, Robot Learning, Robotic Manipulation, Robustness and Reliability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ePRAOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Positional Blind Spots (PBS), a targeted-data collection framework for improving the robustness of VLAs to task-irrelevant distractors. PBS follows a two-stage pipeline: (i) discretize the state-space into cells for which to place the task-irrelevant distractor, roll-out the policy many times in each cell, average performance in each cell, then select the top K cells based on a LLR score comparing failure rate relative to the rest of the workspace. The K cells represent the PBS, where the authors then (ii) collect demos in the PBS, fine-tune with LoRA, and re-evaluate, showing improved performance. They test on different VLA backbones and conduct this study in LIBERO and VLA-Arena.

Critical strengths

To me, the main strength of the paper is the controlled study characterizing VLA policy sensitivity to a distractor's position in the workspace. By holding all other variables constant, the authors demonstrate that VLAs are particularly sensitive to specific locations in the environment, rather than the presence of the distractor at any random region. This matters because it shows that average success rates can hide localized failure modes, which is useful for policy evaluation.

Critical weaknesses

There are two critical weakness that I can find.

  1. Impractical: performing this exhaustive grid-sweep is prohibitively expensive in the real-world. For example, in their experiments, they require 980 policy executions to determine the seven PBS regions, which is incredibly time-consuming for evaluation. Moreover, while simulation provides a test-bed and allows for reproducible experiments, there are no real-world evaluation, so its difficult to assess how the method performs in practice.

  2. Missing baselines: a strong baseline that the authors do not compare to is doing un-targeted data collection with distractors placed at random, with an equal demo budget This is much more aligned with how practitioners would perform fine-tuning in real life, and is a critical baseline to compare to. As it stands now, there is no observed benefit purely due to the search process of PBS relative to a baseline without it.

Questions for authors
  1. Can you add the baseline method I mentioned above: collect the same number of demos as the PBS method, but with the distractor placed uniformly at random within the environment, and fine-tune with identical LoRA parameters? Again, this is how practitioners would perform teleoperation for fine-tuning, and without clear benefits over this baseline, its hard to argue for the expensive data collection scheme to identify PBS in the first place.
Reviewer_dVYPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates positional blind spots in VLA models, where changing the position of a task-irrelevant distractor can substantially affect robotic manipulation success rates. The authors propose a two-stage black-box framework to identify and mitigate these vulnerabilities. In the first stage, the workspace is partitioned into spatial grids, and a log-likelihood-ratio test is applied to detect regions with elevated failure probabilities. In the second stage, additional demonstrations are collected from the identified regions and used to fine-tune the VLA policy through LoRA. The framework is evaluated on five VLA models across LIBERO and VLA-Arena, with experiments examining positional vulnerability, localization accuracy, and mitigation effectiveness.

Critical strengths
  1. The paper investigates spatially localized failures in VLA models. By isolating distractor position while controlling other scene factors, the study examines a specific source of reliability degradation that may be obscured by aggregate task success rates. The proposed PBS characterization provides a way to analyze spatial variation in model performance.
  2. The paper evaluates both vulnerability detection and mitigation across multiple VLA policies. The experiments include five VLA models and two manipulation benchmarks, comparing blind-spot localization against alternative sampling strategies. The authors also examine the effects of search granularity, mitigation rounds, and cross-benchmark transfer.
Critical weaknesses
  1. The proposed framework mainly combines spatial grid sampling, statistical likelihood-ratio testing, and LoRA-based fine-tuning, all of which are established techniques. Although their application to positional vulnerabilities in VLA models provides a different perspective, the underlying methodology is relatively straightforward. The paper should better clarify its technical novelty beyond combining existing approaches.
  2. While the paper compares different PBS detection strategies, it lacks direct comparisons between PBS-guided fine-tuning and alternative data collection strategies, such as random sampling, uncertainty-guided sampling, or uniform workspace coverage. Without experiments using equal demonstration budgets, it is difficult to determine whether the improvements arise specifically from PBS localization or simply from additional fine-tuning data.
  3. The experiments focus on simulated manipulation benchmarks where only the position of a single task-irrelevant distractor changes. Real-world environments involve multiple objects, varying camera viewpoints, lighting conditions, and dynamic scene configurations. It remains unclear whether the identified positional vulnerabilities and mitigation strategy generalize to these more complex conditions. Additional evaluations in realistic environments would strengthen the conclusions.
  4. The proposed method relies on uniform spatial discretization and fixed-budget sampling, but its effectiveness may depend heavily on grid resolution, spatial distribution of failures, and the number of selected cells. Although the paper examines grid granularity, this analysis is restricted to one model and benchmark. A more comprehensive sensitivity analysis across different policies, spatial configurations, and sampling budgets is needed.
  5. The paper demonstrates that relocating task-irrelevant objects can induce spatially concentrated failures, but provides limited insight into why these failures occur. Potential factors include visual attention biases, occlusion, spatial representation limitations, and training data distribution. Without further investigation, it is difficult to distinguish the underlying causes of PBS or determine whether the proposed mitigation addresses these causes rather than memorizing specific configurations.
Questions for authors

please see the above comments.

Reviewer_aVN1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper demonstrates that VLAs have positional blind spots. The paper then focuses on approaches towards mitigating these PBS'es, by search and then fine-tuning, with these approaches reducing PBS'es significantly. Overall, the proposed fixes result in increased performance and appeared generalization.

Critical strengths
  • The authors demonstrate that VLAs do indeed have many PBS'es
  • The authors demonstrate an approach for detecting and mitigating PBS'es that achieves decent performance boosts
  • The paper is rigorous in formalizations
Critical weaknesses
  • It's unclear whether the number of PBS'es is simple a symptom of the current VLA scale, and whether they would hold at larger scale. This is my biggest concern, and I strongly believe more rigorous benchmarking is needed to settle it.
  • The PBS fix may also not scale to very complex scenes
  • Fine-tuning on the exact scene likely results in overfitting, so the results aren't quite grounded or fair
  • The general approach only applies to VLAs, so its general application space is limited.
  • The proposed approach is somewhat incremental as understanding distractor and layout robustness has already been studied in VLAs.
  • It's unclear whether the results generalize to real-world rather than simulation
Questions for authors

Are there any ways to see whether the proposed PBS fixing approach would scale? Is there any evidence that PBS'es aren't simply a symptom of the current models being used? Can we compare to baselines also fine-tuned on comparable in-distribution data, so experiments are more fair?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.