acceptodds
Under review as a conference paper at ICLR 2027

Brain-Conditioned Action Policies for Neural Motor Decoding

Abstract

Motor brain-computer interfaces (BCIs) aim to decode motor intention, enabling people with paralysis to control external devices. Neural motor decoding typically learns task-specific mappings from neural activity to kinematics, yet remains constrained by scarce paired neural–action data. We propose BrainVLA, a framework that enables neural motor decoding by drawing on a pretrained vision-language-action (VLA) model through language-mediated alignment. BrainVLA mitigates reliance on scarce paired neural–action data by leveraging VLA policies. We first construct VLA-compatible datasets including paired neural activity, action signals, language instructions, and rendered visual observations. Then, we adapt the OpenVLA-OFT policy to the target action spaces through LoRA fine-tuning. To establish an effective interface through which neural activity can convey motor intention to adapted VLA policies and guide action generation, we train a neural encoder via neural–language alignment, using language representations as semantic targets to capture latent motor intent from neural activity. The resulting neural representations serve as an endogenous intention signal to guide VLA policies to generate executable actions, while visual observations provide complementary information about the evolving task state. BrainVLA is evaluated on two neural motor datasets with different action dimensionalities using causal rollout decoding. It outperforms the evaluated baselines in cross-session decoding and task success rate, while demonstrating high training data efficiency. These results establish a route for neural motor decoding to draw on large-scale robotic priors through brain-conditioned VLA policies.

Primary area:
applications to neuroscience & cognitive science
Keywords:
neural decoding, motor decoding, BCI, VLA

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_zhKnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a methodology, BrainVLA, for neural motor decoding using a pretrained visuomotor policy. Neural representations are aligned with language representations and replace the task instruction during inference. The policy then uses these representations, visual observations, and the rcurrent state to generate actions. Experiments are conducted on two datasets. The idea is interesting, and the results are promising. However, the baseline comparisons need better justification, and the role of language alignment is not sufficiently clear.

Critical strengths

a) The problem addressed and the method introduced are interesting. While the methods used in individual components are well established, I believe applying them to neural motor decoding seems like a useful contribution.

b) The evaluation, while limited to two datasets, is well directed and considers criteria such as trajectory accuracy and task success, which seem appropriate. The ablations also provide some useful insights into how the proposed components improve performance.

Critical weaknesses

a) The baseline comparisons need better justification. The stronger baselines, such as NoMAD and NDT2, are inspired/adapted versions, as the authors explain in Appendix C.2. Furthermore, BrainVLA receives vision and state information, whereas these baselines receive only neural input. Can the authors explain whether the improvements come from the proposed method or the additional inputs? The same-input MLP comparison in Table 2 is useful, but does not fully resolve this concern.

b) Appendix B.3 averages frozen token embeddings, which loses word-order information. This matters in D2, where exchanging which finger group should flex and which should extend changes the intended action. It is unclear whether the averaged representations still distinguish these instructions. If the action loss can distinguish them, then why do we need this particular language-alignment step?

Questions for authors

Refer to the weaknesses.

3 comments

Thank you for your thoughtful feedback and suggestions. We are excited to read that "the problem addressed and the method introduced are interesting" and “the evaluation is appropriate”.
Weaknesses:

W a.1: Baseline reproduction

Thank you for this important suggestion. To address the concern about baseline fidelity, we reproduced the baselines using the official NoMAD implementation and publicly released pretrained NDT2 checkpoints. The results are shown below:

Method             D1 R²        D1 SR       D2 R²        D2 SR
--------------------------------------------------------------------------
NDT2             0.35±0.19    0.33±0.13   0.23±0.14    0.25±0.14
NoMAD           -0.06±0.13    0.19±0.05   0.12±0.05    0.21±0.09

These results replace the previous NoMAD-inspired and NDT2-style values in Table 1, providing a more faithful comparison with our BrainVLA.

We updated Table 1 and Appendix C (highlighted in blue). We appreciate the reviewer's suggestion, which has helped us more rigorous baseline comparison.

W a.2: Distinguishing the contributions of the method and modalities.

Thank you for this thoughtful question. To further distinguish the contributions of method or additional inputs, we provide complementary evaluations of input modalities, neural encoder architectures, and action decoder architectures.

(1) Modality ablation. The results are shown in Table 3 (BrainVLA’s D2 R² has been updated following our response to reviewer zhKn’s weakness (b)).

Modality (Submission Table 3)   D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------------------
w/o vision                     0.27    0.29   -0.04    0.03
w/o proprio                    0.35    0.39    0.45    0.99
BrainVLA                       0.65    0.77    0.48    1.00

Removing vision and proprio weaken the performance, which shows evidences that additional inputs contribute to BrainVLA’s performance.

(2) Alternative neural encoder. We additionally evaluated LSTM and TCN neural encoders, connected to the same frozen, task-adapted VLA under matched input conditions and training budgets. The results are shown below (BrainVLA’s D2 R² has been updated following our response to reviewer zhKn’s weakness (b)).

Neural encoder             D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------
LSTM                       0.62    0.70    0.44    1.00
TCN                        0.56    0.63    0.43    1.00
BrainVLA(POYO-inspired)    0.65    0.77    0.48    1.00

These encoders achieve comparable performance, with the POYO-inspired encoder consistently performing better. This indicates that BrainVLA’s effectiveness is not tied to a specific neural encoder, highlighting the flexibility of the neural-to-VLA framework.

(3) Alternative action decoders. We also retained the original neural encoder but replaced the VLA policy with MLP or transformer action decoders under matched input conditions and training budgets. The results are shown below.

Action decoder               D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------------------
MLP (Submission Table2)      0.14    0.31    0.18    0.22
Transformer (Rebuttal)       0.25    0.41    0.20    0.35
BrainVLA                     0.65    0.77    0.48    1.00

Both alternative decoders perform substantially worse than BrainVLA under matched input conditions, supporting the benefit of using the adapted VLA policy.

Overall, these experiments show that both additional input modalities and model architecture contribute to BrainVLA's performance.

We have appended these results to Appendix D.4 and D.5 (highlighted in blue). We appreciate the the reviewer's suggestion, which has helped us better distinguish the contributions of method and input modalities.

W b: Word-order information in language embedding

Thank you for raising this important concern. We agree that mean pooling of raw frozen token embeddings loses word-order information.

(1) Contextual language representations. We replaced “frozen token embeddings” with “contextual hidden states” from the 32-layer transformer, which incorporates positional information through RoPE. All other settings remain. The results are shown below:

BrainVLA (Language embedding)         D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------------------
Raw token embedding (Submission)      0.65    0.77    0.46    1.00
Contextual hidden state (Rebuttal)     0.65    0.77    0.48    1.00

Using “contextual hidden state” preserves overall performance and yields a modest improvement on D2 R² from 0.46 to 0.48.

We also report task-level results for both representations:

                          Raw token embedding     Contextual hidden state
D2 Tasks                     D2 R²   D2 SR             D2 R²   D2 SR 
--------------------------------------------------------------------------
Extend index & flex MRS      0.46    1.00               0.48   1.00
Flex index & Extend MRS      0.45    1.00               0.49   1.00
Extend/flex index            0.47    1.00               0.47   1.00
Extend/flex MRS              0.47    1.00               0.48   1.00
Extend/flex Both             0.48    1.00               0.48   1.00
              Raw token embedding        Contextual hidden state
D1 Tasks         D1 R²   D1 SR               D1 R²   D1 SR 
--------------------------------------------------------------------------
Reach            0.52    0.95                0.57     0.95
Carry            0.63    0.94                0.58     0.94
Orient           0.40    0.59                0.47     0.60
Orient2          0.51    0.60                0.52     0.61
Shape            0.68    0.71                0.66     0.70
Grasp            0.67    0.90                0.62     0.89
Release          0.53    0.57                0.55     0.57

Notably, contextual representations improve R² on the two D2 tasks “Extend index & flex MRS” and “Flex index & Extend MRS”, which contains exchange words but with different intention. This finding are consistent with the benefit of word-order, as highlighted by the reviewer. Performance remains largely comparable on other tasks.

(2) Language alignment remain beneficial. Action loss supervises movements, while language alignment provides complementary semantic regularization. As shown in Table 2, removing alignment reduces performance, supporting its benefit beyond action supervision.

Modality (Submission Table2)   D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------------------
w/o language alignment         0.56    0.68    0.41    0.87
BrainVLA                       0.65    0.77    0.48    1.00

We revised Table 13-14 in Appendix B.3, and the D2 R² results in Table 1 (highlighted in blue).

We appreciate the reviewer's insightful comment, which has helped us address the word-order limitation and better clarify the role of neural-language alignment.

AuthorsOpenReview ↗

Dear Reviewer, we are looking forward to hearing from you during the discussion

Dear Reviewer,

Thank you for taking the time to review our manuscript and for your meaningful feedback. We have carefully addressed your concerns and provided additional clarifications and experimental results.

With the discussion period concluding in less than two days, we would greatly appreciate it if you could take a look at our responses and let us know whether we have adequately addressed your concerns. Doing so would give us sufficient opportunity to address any follow-up questions or concerns you might have.

We really appreciate your feedback and look forward to improving our work based on your suggestions.

Reviewer_GvQmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes BrainVLA, a framework that conditions a pretrained vision-language-action policy on neural activity. It first adapts OpenVLA-OFT with LoRA to reconstructed motor tasks using rendered observations, proprioception, and task-specific language instructions. A neural encoder is then trained with action supervision and a neural-language alignment loss, and its output replaces the task instruction as a neural soft prompt. The method is evaluated on human recordings paired with 7-DoF robotic-arm actions and primate recordings paired with 2-DoF finger actions. Evaluation uses held-out sessions and causal MuJoCo rollouts that start from the rendered initial observation of each trial and are driven by recorded neural activity. The paper reports action R² and task success against neural decoding baselines, together with data-efficiency experiments and ablations.

Critical strengths
  1. The paper offers a fairly direct way to connect neural recordings to an existing robot policy by feeding neural representations into the positions normally used for the language instruction. The approach is demonstrated on two different action spaces. Evaluation uses rollouts in which the rendered scene evolves from the model's own predictions, allowing prediction errors to affect subsequent policy inputs. The held-out-session R² values of 0.65 on D1 and 0.46 on D2, together with the task-level results, make this a useful proof of concept.
  2. The retrained component ablations support the main training choices. Removing neural-language alignment lowers R² from 0.65 to 0.56 on D1 and from 0.46 to 0.41 on D2, while skipping task-specific VLA adaptation lowers it to 0.39 and 0.34. Each variant is retrained and evaluated with the same splits and rollout protocol, so these comparisons indicate that alignment and task-specific adaptation both help within the proposed framework.
Critical weaknesses
  1. The main comparison does not separate the benefit of neural conditioning from access to visual task information. The rendered scenes contain task objects and target markers, while the neural decoding baselines in Table 1 do not receive these observations. Removing neural input reduces D1 success from 77% to 25%, while D2 retains 87% success despite R² falling from 0.46 to 0.17. These results show that neural input contributes, but they are interventions on a frozen model. No policy is retrained using vision and proprioception alone. Without that control, it remains unclear how much neural conditioning adds over a policy trained with the same scene information, which limits the interpretation of the large gains in Table 1.
  2. The baseline implementations limit what Table 1 can establish. Appendix C.2 describes the NoMAD entry as a causal GRU with maximum mean discrepancy alignment and the NDT2 entry as a compact causal Transformer trained from scratch. It calls them NoMAD-inspired and NDT2-style implementations, but Table 1 uses the original method names. The appendix also states that the neural baselines are trained from scratch for at most 30 epochs. These comparisons therefore provide limited evidence of an advantage over the original NoMAD and NDT2 models or pretrained neural decoders. As the authors acknowledge in Section 4.5, the VLA ablation also does not separate pretrained initialization from model capacity. It supports the usefulness of the adapted policy as a whole, while the specific contribution of robotic pretraining remains unresolved.
Questions for authors
  1. Are results available for the same VLA backbone trained with vision and proprioception, but without neural input or task instructions? This would help assess the contribution of neural conditioning beyond the task information available in the scene.
  2. Are results available from original implementations or pretrained neural decoders under the same evaluation protocol? Appendix C.2 describes substantially modified NoMAD and NDT2 implementations, so such results would help interpret the gaps in Table 1.
  3. In Figures 4A and 10, were both language-conditioned VLA adaptation and neural-encoder training restricted to the same training subset? Please also clarify whether the fractions were sampled by session, trial, or time window. This affects whether the curves demonstrate reduced requirements for paired neural-action data throughout training.
4 comments

Thank you for your insightful comments and questions, and for pointing out that our work is the “useful proof of concept that offers a fairly direct way to connect neural recordings to an existing robot policy ”.
Weaknesses:

W1: Isolating the contribution of neural conditioning

Thank you for this insightful comment. Our original version contains:

language+vision+proprioception (adaptation) → neural+vision+proprioception (inference)

To further isolate the contribution of neural conditioning, we conducted additional experiments:

Vision+proprioception (adaptation) → vision+proprioception (inference)

This experiment further evaluated neural conditioning on the resulting vision+proprioception adapted policy. Please see our reply to Question 1.

W2: Baseline problem

Thank you for raising this important point. We agree that our previous NoMAD-inspired and NDT2-style baselines were not faithful reproductions. We revise the reproduction, please see our reply to Question 2.

Questions:

Q1: Result of fine-tuned with vision+proprioception

Thank you for this insightful question. We fine-tuned the same VLA backbone using only vision and proprioception, without language instructions, rather than merely removing neural input at inference.

We then trained a neural encoder without neural-language alignment, while keeping the vision+proprioception-adapted VLA frozen. This isolates the contribution of neural conditioning under the same VLA adaptation setting. The results are shown below:

Condition                                D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------------------
Vision+proprio (adaptation)             -0.09    0.52    0.002   0.00
Neural+Vision+proprio (inference)        0.30    0.63    0.05    0.004

Introducing neural conditioning improves the performance, which supports the contribution of neural conditioning beyond the task information available in the scene.

We added these results to Appendix D.9 (highlighted in blue). We appreciate the reviewer's suggestion, which has helped us more rigorously isolate the contribution of neural conditioning.

Q2: Baseline reproduction

Thank you for this important suggestion. To address the concern about baseline fidelity, we reproduced the baselines using the official NoMAD implementation and publicly released pretrained NDT2 checkpoints. The results are shown below:

Method             D1 R²        D1 SR       D2 R²        D2 SR
--------------------------------------------------------------------------
NDT2             0.35±0.19    0.33±0.13   0.23±0.14    0.25±0.14
NoMAD           -0.06±0.13    0.19±0.05   0.12±0.05    0.21±0.09

These results replace the previous NoMAD-inspired and NDT2-style values in Table 1, providing a more faithful comparison with our BrainVLA.

We updated Table 1 and Appendix C (highlighted in blue). We appreciate the reviewer's suggestion, which has helped us more rigorous baseline comparison.

Q3: Data fraction restrictions on both LoRA adaptation and neural encoder training stages

Thank you for highlighting this distinction. In our original Figure 4, data fractions applied only to neural encoder training, while VLA adaptation used the full held-in training set.

We now additionally restrict both LoRA adaptation and neural encoder training to the same fraction, keeping validation and test sets fixed. The results are shown below (100% D2 R² has been updated following our response to reviewer zhKn’s weakness (b)):

Fraction         D1 R²   D1 SR     D2 R²   D2 SR
----------------------------------------------------
1%               0.13     0.18     0.19    0.55
10%              0.36     0.47     0.42    1.00
25%              0.49     0.58     0.45    1.00
50%              0.59     0.67     0.46    1.00
100%             0.65     0.77     0.48    1.00

The updated results show a similar trend of improving performance with increasing data fractions, with modest reductions consistent with less data for VLA adaptation. Notably, BrainVLA continues to outperform the baselines, supporting its data efficiency.

We updated Fig.4(A) and Fig.10, Section 4.4, and Appendix D.7 (highlighted in blue). We appreciate the reviewer's suggestion, which has helped us provide a more rigorous evaluation of data efficiency.

AuthorsOpenReview ↗

Dear Reviewer, we are looking forward to hearing from you during the discussion

Dear Reviewer,

Thank you for taking the time to review our manuscript and for your meaningful feedback. We have carefully addressed your concerns and provided additional clarifications and experimental results.

With the discussion period concluding in less than two days, we would greatly appreciate it if you could take a look at our responses and let us know whether we have adequately addressed your concerns. If so, we would be grateful if you consider updating your assessment accordingly.

We really appreciate your feedback and look forward to improving our work based on your suggestions.

Thank you for the extra experiments and clarifications. They address my concerns and strengthen the paper, so I’m raising my score to 3.

Dear Reviewer,

Thank you sincerely for your insightful feedback and positive reassessment. Your suggestions have helped us significantly strengthen our work. We greatly appreciate your time and consideration.

Reviewer_NqNSOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes BrainVLA, a new framework for motor brain-computer interfaces. The authors claim that conventional neural-action data is not enough for VLA tasks. To this end, they use a pre-trained OpenVLA as backend and utilize its pre-trained abilities. In the first stage, they use the neural-action data to record the episodes (vision data and language captions) and tune the LoRA on these data. In the second stage, they fix the tuned OpenVLA and train the neural encoder to align neural signals with language embeddings. The authors demonstate their method on the consturcted D1 and D2 dataset and provide the ablation study on the BrainVLA.

Critical strengths
  1. The motivation makes sense to me. Given the limited neural-action data, the authors resort to a pre-trained VLA model. By aligning neural signals with language embeddings, it's efficient to adapt neural signals to action sequences.
  2. The overall experiments can support their claim. The ablation studies (Table 2, Table 3, and Figure 4) on two datasets also provide some insight into the relationship between neural signals and action sequences.
Critical weaknesses
  1. I think this work can enhance its novelty by differentiating it from previous works. SRRL [1] and SimKD [2] both proposed aligning student features with the teacher task head. By analogy, the neural encoder is the student, and the OpenVLA backend is the teacher task head. Also, BrainLLM [3] translated neural signals into language embeddings.

[1] KNOWLEDGE DISTILLATION VIA SOFTMAX REGRESSION REPRESENTATION LEARNING. ICLR 2021

[2] Knowledge Distillation with the Reused Teacher Classifier. CVPR 2022

[3] BrainLLM: Generative Language Decoding from Brain Recordings. Nature Communications Biology

  1. I have some suggestions regarding the experiments. Please refer to the Question Sections.
Questions for authors
  1. In Table 1, BrainVLA received neural activity, vision, and proprioception, while other comparison baselines do not receive such multimodal input. Could the authors evaluate representative alternative encoders, such as LSTM and POYO-style encoders, connected to the same frozen, task-adapted VLA under matched input conditions and training budgets? This would help distinguish the contribution of the proposed neural interface from the benefits of the shared VLA policy and additional modalities.

  2. In Figure 4, does the same restriction apply to both LoRA adaptation and neural encoder training?

  3. I have a suggestion regarding the language embeddings. We can assign each instruction category (pinch, reach) a randomly initialized vector that remains fixed during training, and use it instead of the language embedding as the alignment target. During evaluation on the held-out session, the neural encoder would generate representations from neural activity as usual, without access to category labels or the fixed target vectors. This would help assess whether language embeddings offer benefits beyond simply distinguishing instruction categories.

5 comments

Thank you for your thoughtful feedback. We are excited to read that “the motivation makes sense to you and the overall experiments can support the claim”.
Weaknesses:

W1: Clarify difference from precedent

Thank you for the constructive suggestion. SRRL and SimKD use pretrained teacher models to guide student models in image classification, whereas our BrainVLA applies action priors to visuomotor action generation. Unlike BrainLLM, which uses neural representations for LLM-based text generation, our BrainVLA uses neural representations to condition a task-adapted VLA policy to realize motor decoding alongside visual feedback and proprioception.

Our contribution lies not in representation alignment alone, but in establishing a neural interface to pretrained VLA policies, leveraging action priors to mitigate reliance on paired neural-action data.

We added these claims in the Introduction and Related Work (highlighted in blue).

Questions:

Q1: Evaluation of alternative neural encoders

Thank you for this insightful question. We would like to clarify that BrainVLA already uses a POYO-inspired neural encoder. We additionally evaluated LSTM and TCN neural encoders, connected to the same frozen, task-adapted VLA under matched input conditions and training budgets. The results are shown below (BrainVLA’s D2 R² has been updated following our response to reviewer zhKn’s weakness (b)).

Neural encoder             D1 R²   D1 SR   D2 R²   D2 SR
--------------------------------------------------------------
LSTM                       0.62    0.70    0.44    1.00
TCN                        0.56    0.63    0.43    1.00
BrainVLA(POYO-inspired)    0.65    0.77    0.48    1.00

These encoders achieve comparable performance, with the POYO-inspired encoder consistently performing better. This indicates that BrainVLA’s effectiveness is not tied to a specific neural encoder, highlighting the flexibility of the neural-to-VLA framework.

We appreciate the reviewer's suggestion, which has helped us better clarify the role of neural encoder architecture and the general applicability of our framework. We have added these results to Appendix D.4 (highlighted in blue).

Q2: Data fraction restrictions on both LoRA adaptation and neural encoder training stages

Thank you for highlighting this distinction. In our original Figure 4, data fractions applied only to neural encoder training, while VLA adaptation used the full held-in training set.

We now additionally restrict both LoRA adaptation and neural encoder training to the same fraction, keeping validation and test sets fixed. The results are shown below (100% D2 R² has been updated following our response to reviewer zhKn’s weakness (b)):

Fraction         D1 R²   D1 SR     D2 R²   D2 SR
----------------------------------------------------
1%               0.13     0.18     0.19    0.55
10%              0.36     0.47     0.42    1.00
25%              0.49     0.58     0.45    1.00
50%              0.59     0.67     0.46    1.00
100%             0.65     0.77     0.48    1.00

The updated results show a similar trend of improving performance with increasing data fractions, with modest reductions consistent with less data for VLA adaptation. Notably, BrainVLA continues to outperform the baselines, supporting its data efficiency.

We updated Fig.4(A) and Fig.10, Section 4.4, and Appendix D.7 (highlighted in blue). We appreciate the reviewer's suggestion, which has helped us provide a more rigorous evaluation of data efficiency.

Q3: Evaluation of language embeddings beyond category supervision

Thank you for this valuable suggestion. To test whether pretrained language embeddings offer benefits beyond category labels, we replaced them with fixed, randomly initialized vectors, one per instruction category, while keeping all other settings unchanged. The results are shown below (BrainVLA’s D2 R² has been updated following our response to reviewer zhKn’s weakness (b)):

Condition                   D1 R²   D1 SR   D2 R²   D2 SR
----------------------------------------------------------------
Random category alignment   0.57    0.68    0.43    0.89
BrainVLA                    0.65    0.77    0.48    1.00

Random category vector alignment reduces both R² and SR on D1 and D2, indicating that pretrained language embeddings provide more informative alignment targets than category vectors, beyond category-level supervision alone.

We added this result to Appendix D.8 (highlighted in blue). We appreciate the reviewer's constructive suggestion, which helped us better clarify the contribution of neural-language alignment.

AuthorsOpenReview ↗

Dear Reviewer, we are looking forward to hearing from you during the discussion

Dear Reviewer,

Thank you for taking the time to review our manuscript and for your meaningful feedback. We have carefully addressed your concerns and provided additional clarifications and experimental results.

With the discussion period concluding in less than two days, we would greatly appreciate it if you could take a look at our responses and let us know whether we have adequately addressed your concerns. Doing so would give us sufficient opportunity to address any follow-up questions or concerns you might have.

We really appreciate your feedback and look forward to improving our work based on your suggestions.

I have no further questions. I would like to raise the score to 4. Thanks for your discussion.

Comments on the paper

1 comment

We thank the reviewers for their thoughtful and constructive feedback. Their reviews highlight BrainVLA as a novel neural motor decoding framework, providing an instructive way to use neural recordings to condition pretrained robot policies. They also appreciated its competitive decoding performance, data efficiency under limited paired neural-action data, and the supporting experiments.

Based on their feedback, we have clarified our contributions and conducted additional experiments to strengthen the paper:

  • evaluation of alternative neural encoder and decoder; this better distinguishes the contributions of model architecture and input modalities.
  • adjusted the scope of the data fraction in data efficiency experiments; further supporting the data efficiency claim.
  • evaluated the language embedding’s function beyond the category supervision.
  • added a vision+proprioception adaptation baseline; isolating the contribution of neural conditioning.
  • baseline reproduction: NoMAD and NDT2.
  • new language embedding with word-order information; exactly improving the performance on D2 tasks.

We are grateful to the reviewers and Area Chair for their time and valuable suggestions. We believe these revisions have substantially strengthened our work and addressed the concerns. We sincerely hope that, with these amendments, the paper will be accepted.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.