acceptodds
Withdrawn from ICLR 2027

BrainVLA: Towards VLA-driven High-level Embodied Planning for Motor Brain-Computer Interfaces

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Intracortical Brain-Computer Interfaces (iBCIs) restore motor function for paralyzed patients by decoding neural signals into control commands. However, current decoding models lack environmental awareness and require users to continuously control spatial trajectories, causing cognitive fatigue and compounding control errors. To address this limitation, we introduce BrainVLA, a hybrid-intelligence framework that integrates iBCIs with pre-trained Vision-Language-Action (VLA) models by leveraging neural signals as high-level planning priors. BrainVLA extracts intent embeddings from neural activity during motor preparation and projects them into the VLA model's semantic space. This integration establishes a control hierarchy in which neural signals dictate high-level goals, leaving the VLA model to leverage visual context for fine-grained action generation. Experiments on the maze reach task demonstrate that BrainVLA achieves a 31.6% higher success rate than classical neural decoders on average. It outperforms the VLA model by 17.7% in unseen scenarios, highlighting its impressive generalizability. Our approach establishes a new embodied iBCI paradigm where robotic agents harness cognitive abilities from biological systems.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Brain-Computer Interfaces, Neural Decoding, Embodied Artificial Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uiK9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

BrainVLA encodes preparatory neural activity into intent representations and combines them with visual and language inputs for VLA action generation. A neural encoder is first trained using keypoint and target directions, followed by VLA adaptation with action and spatial losses. Experiments use MC Maze macaque recordings and corresponding physical-robot scenes. The method achieves 80.2% success on 96 unseen-condition trials, a 17.7-percentage-point improvement over π0.

Critical strengths

Combining neural intent decoding with a visual action model is a worthwhile research direction. The paper evaluates the resulting system on a physical robot rather than only reporting offline decoding accuracy. Improvements over π0 in obstacle and distractor settings are also consistent with the motivation for providing path information.

The target-matched shuffle in Appendix D is particularly informative. Exchanging neural signals between trials with the same target but different layouts substantially increases keypoint error, indicating information beyond target direction. The paper also includes shuffled-intent, text-representation, and ground-truth-direction controls, and reports success counts and confidence intervals for the robot experiments.

Critical weaknesses

The experiments do not yet establish the advantage of the proposed latent intent representation over a simpler neural interface. The π0 comparison shows that additional neural information helps. However, writing decoded directions as text changes both the information supplied and its numerical encoding, while the ground-truth-direction control eliminates decoding error. Feeding continuously projected Stage-1 direction predictions into the same VLA would provide a more direct comparison and help distinguish representation gains from interface effects.

The model-selection procedure for the generalization evaluation needs clarification. Appendix C states that training uses 54 conditions, whereas validation and test trials are sampled from all 108 conditions. Validation is used to select the neural encoder and neural decoding baselines. It is therefore necessary to establish whether the conditions described as unseen have already influenced model selection. Moreover, robot execution driven by recorded neural signals does not establish improvements in online neural closed-loop control or cognitive load.

There is an explicit inconsistency in the spatial-loss ablation. Section 4.3 reports 58.1% and 43.8% success on single- and multi-goal obstacle trials, pooled at 50.8%. Appendix Table 6 reports 54.8%, 34.4%, and 44.4% for the same ablation. The main-text values are identical to those of the text-representation ablation and should be checked.

Finally, the Wilson intervals in Table 5 quantify each model's success rate, not uncertainty in the model differences. Since the principal comparison uses the same 96 trials, a paired analysis would better characterize the improvement over π0.

Questions for authors
  1. Has a continuous projection of Stage-1 predicted directions been compared with the six intent tokens under the same VLA training budget?
  2. Did unseen-condition validation labels influence encoder or baseline selection? Please specify the selection criterion and recording-session split.
  3. What are the correct success counts for the no-spatial-loss ablation? Please also report a paired confidence interval for the BrainVLA–π0 success-rate difference.
Reviewer_Yt2rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BrainVLA, a framework that integrates iBCIs with pretrained VLA models for high-level embodied planning. Instead of continuously decoding low-level motor commands, BrainVLA extracts high-level spatial intent from preparatory neural activity, representing planned reaching trajectories through intermediate waypoints and target directions. These neural intent tokens are incorporated into a pretrained π0\pi_{0} model through cross-modal alignment and spatial grounding, enabling robot action generation guided by both neural intent and visual observations.

Critical strengths
  1. The paper explores a novel direction by integrating neural intent decoding with pretrained VLA models, enabling high-level planning from neural signals rather than continuous low-level motor control.

  2. The proposed method demonstrates improvements over traditional neural decoders on maze-reaching tasks. The real-robot experiments and ablations further validate the framework design.

Critical weaknesses
  1. The experiments are limited to 2D maze-reaching tasks. It is unclear whether the proposed neural intent tokens would still work well for more complex 3D manipulation tasks.

  2. The paper claims to reduce user cognitive burden, but provides no direct experimental evidence to support this claim.

  3. The comparison with fine-tuned π0\pi_{0} is not entirely fair, as BrainVLA has access to additional neural information. A baseline using the same neural inputs would help better demonstrate the performance improvement.

Questions for authors

Please refer to the weaknesses mentioned above.

Reviewer_c7hgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrainVLA conditions the pretrained π₀ vision-language-action model on neural activity recorded during motor preparation. The neural encoder outputs six intent tokens, supervised by two trajectory-related directions, one target direction, and barrier presence. During evaluation, prerecorded neural signals acquired from one monkey are replayed, and a physical robot performs the mapped MC Maze reaching task, which further validates the effectiveness of neural conditioning.

Critical strengths
  • Using neural signals for high-level intent guidance while allowing the VLA model to perform vision-guided, fine-grained actions provides a well-motivated and insightful division of responsibilities.

  • Extracting path-related information solely from preparatory neural activity offers an alternative to brain–computer control methods that rely on continuous kinematic decoding.

  • On a physical robotic arm, the method achieves an 80.2% success rate under unseen conditions, outperforming π₀ by 17.7 percentage points, with gains concentrated in obstacle scenarios.

  • Ablation experiments, including target-matched neural shuffling, support the contribution of informative neural signals and the role of decoding quality in task performance.

Critical weaknesses
  1. A matched baseline that feeds the predicted directions into the VLA as continuous vectors is missing. The existing text-based control changes both the information content and the encoding scheme, making it difficult to assess the distinctive value of the latent neural tokens themselves.
  2. The validation set covers all 108 conditions and is used for model selection. Here, “unseen” appears to mean exclusion from parameter training, rather than exclusion from the entire model-development process. The authors should clarify whether validation labels from these conditions influenced checkpoint or hyperparameter selection, and how this affects the generalization claims.
  3. The experiments use only offline neural recordings from a single monkey and are restricted to a two-dimensional maze-reaching task. This setup cannot evaluate how users adjust their intentions in response to robot feedback, nor does it directly establish whether the framework reduces cognitive burden.
  4. The current task restricts intent to a few spatial directions, while the textual instructions remain largely fixed. It is therefore unclear whether this setting adequately demonstrates the value of high-level intent planning and language-conditioned control. Would performance remain comparable when using only visual and neural inputs, without text prompts? Alternatively, would a vision–action policy suffice as the low-level controller? In that case, neural intent would amount to a trivial conditioning signal.
Questions for authors

I would like to better understand the essential role of the VLA model in this framework. If it were replaced with a vision–action policy, a behavior tree, or a planning program, it seems that comparable results might still be obtained. My understanding is that the framework essentially consists of an intent classifier followed by a low-level execution policy. Is this interpretation correct? This distinction substantially affects my assessment of the paper: if the method is simply an intent classifier coupled with a low-level execution policy, its methodological distinction from prior related work, including the papers cited above, seems unclear.

Another important reason for this question is that, although the framework may reduce the control burden on users, low-level policies such as π₀ still require task-specific fine-tuning. This introduces additional learning costs for downstream policies, which should also be considered when assessing the benefits of the proposed framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.