acceptodds
Under review as a conference paper at ICLR 2027

Behavior-Aligned Neural Supervision for Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models learn embodied representations primarily from externally observable signals such as vision, language, and action trajectories. However, external observations may not fully capture behaviorally relevant structure and temporal dynamics during physical interaction. Biological organisms and embodied agents both operate through continuous perception–action loops under related physical constraints, motivating us to ask whether neural activity recorded during human interaction can provide complementary supervision for embodied learning. To answer this, we investigate EEG as a training-time supervisory signal for adapting pretrained embodied models. We first synchronously collect egocentric video and 128-channel EEG during natural manipulation tasks. We then introduce EEG-guided representation learning, with EEG used only during training and removed at inference. Using OpenVLA as the primary backbone, we evaluate the approach across multiple manipulation benchmarks and out-of-distribution settings involving visual, semantic, and execution shifts, and further validate its transfer to on LIBERO. EEG-guided training improves task success over the corresponding robot-only baselines across these settings and backbones. These gains depend on aligned EEG–behavior correspondence, as they are reduced when the alignment is disrupted or EEG supervision is ablated. Behavioral and representation analyses further associate the improvements with more effective task progression and completion, with the strongest representational changes occurring in late, action-proximal layers. These results suggest that behavior-aligned human neural dynamics can provide transferable training-time supervision for embodied representation learning without introducing an additional modality at deployment.

Primary area:
applications to robotics, autonomy, planning
Keywords:
EEG, vision-language-action models, neural supervision, representation learning, multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_yhW8OpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether EEG recorded while people do manipulation tasks can be used as an extra training signal for VLA models. The authors record 128-channel EEG and egocentric video from two participants on 12 tasks. During LoRA fine-tuning, the egocentric frames and instructions also pass through the backbone, and a small MLP head on the hidden state at the final prompt token predicts the EEG of the next 1-s window (MSE plus InfoNCE). The head is dropped at test time. Against robot-only fine-tuning, the reported gains are +0.3 to +6.0 points, for OpenVLA on four LIBERO suites and three RL4VLA OOD settings, and for π0 on LIBERO. On LIBERO-Long, shuffling the EEG targets brings performance back to the robot-only level. The data are not released, so the paper offers no new dataset contribution. All robot experiments are in simulation; there are no real-robot experiments.

Critical strengths

1. A new and interesting question.

The paper asks whether neural signals recorded while people act can serve as an extra training signal for robot policies (p.1–2). This question is new and interesting, and it is the main reason I think the work deserves attention.

Critical weaknesses

1. The data are too small to be a core contribution, and the paper does not say clearly how much it uses.

The paper lists the synchronized EEG–behavior data paradigm as its first contribution (p.2), but the data come from two participants on 12 tasks. The paper never clearly states how much data is used for training. The main text says Fig. 2(B) summarizes the data of both participants, but App. A.1 notes that it shows only a 689-episode subset. The 1,224 episodes and 6,291 one-second windows (about 1.75 hours of execution-phase EEG) are given only for the decoding analysis (App. A.1, A.4), and only the supplementary code shows that training uses these same episodes. Neither the size of the training set nor the amount of data per participant is reported.

What this field lacks most is data at scale. Non-invasive EEG is much easier to collect than invasive recordings, so adding participants is feasible. Yet the paper mentions no plan to expand the data and does not commit to releasing it: the Reproducibility Statement (p.10) covers only code, and the supplementary README says the data were left out for size and privacy reasons. With two participants, the results cannot support a general claim about human neural dynamics.

2. All robot results are in simulation.

Every robot experiment runs in simulation (LIBERO and RL4VLA), while the human data come from the real world. Simulated success rates can overstate real-world performance and vary from run to run, and the gains here are small: 8 of the 11 comparisons in Tables 1–2 improve by only 0.3–1.6 points. If ± is the SD over three seeds (as for the main run in the supplement), only OpenVLA on LIBERO-Long, Vision OOD and Execution OOD reach p < 0.05. Without real-robot results, the paper says little about embodied intelligence in the physical world, which is the setting its own discussion stresses (App. C).

Overall. The motivation is positive and worth exploring, but two participants and simulation-only results cannot support the paper's core contribution. I would encourage the authors to bring this work to a related ICLR workshop first, scale up the data, and use the workshop to learn from researchers in neighboring communities which data are most needed, before resubmitting. I believe this would lead to a much more solid paper.

Questions for authors

Tags: [C] = clarification; [A] = additional analysis or experiment, feasible on the existing pipeline. The questions are ordered by how much the answers matter for my assessment. Suggestions for future work are listed at the end and do not affect my rating.

Q1 [C] How much data is used, and how was it collected? (Weakness 1)

  • How much data is used for training in total (episodes, hours and windows), and how much comes from each participant? Please report this in the main text.
  • How do the two source datasets (App. A.1; "v1" and "ego128" in the supplied code) relate to the two participants, their sessions and their recording days? App. A.1 states that each participant completed all 12 tasks, yet five of the 12 tasks have episodes only from "v1" (prepare_waveform.py, l.24–37).
  • Were tasks recorded in blocks or interleaved? How were EEG and video synchronized, and how accurately? Fig. 2(C) plots time since a "video-estimated onset".

→ The data are presented as the first contribution, so these basic numbers belong in the main text. A blocked design without session-level control would also weaken the decoding evidence in Fig. 4.

Q2 [C] Data release, scaling and ethics. (Weakness 1)

  • Will the synchronized EEG–video data be released, and under what access conditions?
  • Do you plan to record more participants and to test on a held-out participant?
  • Did an ethics board approve the study? The Ethics Statement (p.10) mentions informed consent, de-identification and minimal risk, but no ethics-board approval.

→ A firm commitment to release and scale up the data would partly address my main concern.

Q3 [C + A] Real-robot evidence. (Weakness 2)

  • Do you have any real-robot results?
  • If not, which parts of your claim do you expect to carry over to a physical robot, and why? The human data are recorded in the real world, but all robot tests are in simulation.

→ Even a small real-robot study would make the claim about embodied learning much more convincing. Without any real-robot experiments, the evidence stays at the level of simulated benchmarks, which is not enough for the paper’s claims about embodied intelligence in the physical world (Fig. 1).

Q4 [C + A] Number of runs, selection and statistics.

  • How many runs underlie each table entry? For example, 54.30 and 53.50 (Table 5) and 90.17 (Table 2) cannot be means of three 500-episode runs.
  • Were the default loss weights chosen on the LIBERO-Long evaluation rollouts (App. B.3)? Was the robot-only baseline tuned at all?
  • Please report paired per-initial-state tests (McNemar or paired bootstrap) for all comparisons in Tables 1–3, ideally with ≥5 seeds and with hyperparameters chosen on held-out initial states.

→ Significant paired gains on settings not used for tuning would strengthen the paper. Continued non-significance for the small gains (8 of the 11 comparisons in Tables 1–2) would confirm my reading that most gains are within run-to-run variation.

Q5 [A + C] Starting point and training setup.

On LIBERO-Spatial and Object, both arms score below what the OpenVLA paper reports for fine-tuned OpenVLA (Spatial: 79.53 / 80.93 vs. 84.7; Object: 66.93 / 68.53 vs. 88.4; OpenVLA paper, Table 12). That paper also evaluates 500 trials per suite on LIBERO's original initial states.

  • What do the released suite-specific OpenVLA checkpoints score under your evaluation pipeline with zero additional training steps?
  • Which robot-image preprocessing was used in training? The supplied prepare_waveform.py center-crops robot images to 80% of their area (l.183–196), while train.py rejects such a cache (l.39–40). A train–test crop mismatch could explain the lower scores.
  • Which checkpoint does each arm use? train.py records "best full EEG validation MSE; R1 final step" (l.392), while the README merges step 2,000.

→ If the released checkpoints match or beat EEG-guided training on most suites, the gain may only reflect reduced damage from continued fine-tuning.

Q6 [A] Is the gain specific to aligned EEG?

On LIBERO-Long (and ideally one more setting), how does aligned EEG compare with each of the following?

  • EEG shifted in time within the same episode;
  • episode-mean EEG;
  • InfoNCE with same-episode windows allowed as negatives (the supplied code excludes them, sequence_contrast.py l.20);
  • a non-neural auxiliary target on the same frames (e.g., random targets with matched statistics, or task and phase labels).

Also, on held-out episodes, does the EEG head predict EEG better than a per-task-mean baseline? The README notes that the current validation split uses random windows that may share episodes.

→ If aligned EEG clearly beats all of these, the "human neural dynamics" interpretation would be much better supported. If episode-mean EEG or a non-neural target matches it, the gain is more likely a generic auxiliary-task effect.

Q7 [C] Analyses in Figs. 5–6.

  • Which rollouts and which pairing underlie Fig. 5A? Why do the per-task nets sum to +65 rather than the +49 implied by Table 3 (851 vs. 802 successes), and what does n = 577 count?
  • How is "hidden-state change (%)" in Fig. 6 computed, what are the n = 6 samples, and how does it compare with a null comparison (e.g., two robot-only seeds)?

→ I need this before giving weight to the behavior- and representation-level claims.

Q8 [C] π0 and RL4VLA protocols.

The main text twice points to the appendix for backbone-specific details (p.5), but the appendix covers only OpenVLA on LIBERO. Please give the initialization, training data, number of steps and per-seed results for both. For π0, also give the representation S(·) and the LoRA placement; for RL4VLA, the sub-settings and the reason for the 72.99 ± 0.00 on Semantic OOD.

→ Without these, I cannot assess the cross-backbone and OOD claims.

Suggestions for future work (beyond the rebuttal; not affecting my rating).

Record more participants and evaluate on held-out participants. Test whether human data from tasks that match the robot tasks help more than data from unrelated tasks. Validate the approach on a real robot.

10 comments
AuthorsOpenReview ↗

Response to Reviewer yhW8 (1/7)

We appreciate the reviewer’s recognition of the novelty of our research question and the potential value of the synchronized EEG–egocentric video data. To our knowledge, this is the first study to investigate whether real human EEG can improve VLA manipulation performance through training-time supervision. In response to your comments, we have made five main additions and revisions to strengthen the evidence and clarify the reporting:

  1. Real-world robot experiments. We added two tasks on a physical PIPER robot using π0\pi_0, comparing the starting checkpoint, Robot-only LoRA, and EEG-guided LoRA to assess effectiveness in physical environments. Figure 7 presents the results, with implementation details in the appendix.
  2. Additional EEG controls. We added time-shifted EEG, episode-mean EEG, same-episode negatives, and non-neural auxiliary targets on LIBERO-Long and LIBERO-Goal, alongside EEG-head evaluation on held-out episodes, to further investigate the source of the gains.
  3. More seeds and statistical checks. We expanded the main comparisons to six training seeds and added significance tests. We also checked the raw logs, corrected aggregation errors, and clarified the runs and counting procedure underlying Figure 5.
  4. Starting-checkpoint performance and training settings. We evaluated the released OpenVLA checkpoints with zero additional training and clarified the simulator settings, fixed-step evaluation, loss-weight selection, and Robot-only configuration.
  5. Clearer data documentation. We clarified the recording batches, data usage, and execution-phase selection, and added our dataset expansion and public-release plans. We address your specific questions below:

yhW8 -W1:\color{red}{\text{yhW8 -W1:}} The data are too small to be a core contribution, and the paper does not say clearly how much it uses.

R- yhW8-W1:\color{blue}{\text{R- yhW8-W1:}} Thank you for the suggestion. Our dataset was limited at submission. Since then, we have recruited additional participants and expanded collection to desks, workbenches, and dining tables, accumulating over 10 hours of recordings, including the data used in this paper. We plan to release the data publicly in December 2026 and will state this plan in the manuscript. As stated in Contribution 1, our primary contribution is a paradigm integrating synchronized EEG recording with embodied experiments, rather than dataset scale. Our initial EEG–egocentric behavior recordings provide preliminary evidence that EEG auxiliary supervision can benefit embodied models; a larger dataset will be the focus of our subsequent release.

We initially prioritized repeated, longitudinal recordings from two participants under stable conditions to investigate whether brain signals can improve embodied model performance. This approach is consistent with prior brain-tuning research: for example, Moussa et al. (ICLR 2025) fine-tuned speech models separately using individual participants’ fMRI recordings and improved downstream semantic task performance. Our design aims to reduce additional variability while exploring useful neural information. The revised manuscript will specify the training data volume, each participant’s contribution, and the relationship between the illustrated subset and the data actually used, while acknowledging the limitations of evidence from only two participants.

yhW8 -W2:\color{red}{\text{yhW8 -W2:}} All robot results are in simulation.

R - yhW8-W2:\color{blue}{\text{R - yhW8-W2:}} Thank you for the suggestion. We agree that real-world robot experiments are important for assessing effectiveness in physical environments. We therefore added two tasks: placing a plate onto a dish-drying rack and placing a pen into a pen holder.

We first fine-tuned π0 on 50 real-world teleoperation demonstrations to obtain the baseline checkpoint. Starting from this checkpoint, we then performed LoRA fine-tuning using robot data alone (Robot-only) or with auxiliary EEG supervision (EEG-guided). The success rates are:

MethodPlate onto drying rack SR (%)Pen into pen holder SR (%)
π0 baseline (fine-tuned on 50 demonstrations)66.740.0
Robot-only LoRA80.053.3
EEG-guided LoRA93.373.3

EEG-guided training outperforms both the baseline and Robot-only on both tasks, providing additional evidence of effectiveness beyond simulation. We have added Figure 7 to the main text and provided implementation details in the appendix.

We also increased the number of random seeds in the simulation experiments and conducted statistical significance tests. Most comparisons show statistically significant improvements. We report small or nonsignificant gains and discuss EEG signal characteristics as possible contributing factors. The revised manuscript clarifies the scope and limitations of these benefits and avoids directly extrapolating simulation results to real-world performance.

AuthorsOpenReview ↗

Response to Reviewer yhW8 (2/7)

Q1 [C] How much data is used, and how was it collected?

yhW8-Q1.1:\color{red}{\text{yhW8-Q1.1:}} How much data is used for training in total (episodes, hours and windows), and how much comes from each participant? Please report this in the main text.

R - yhW8-Q1.1:\color{blue}{\text{R - yhW8-Q1.1:}} We summarize the total and per-participant data (episodes, hours, and EEG windows), along with the training/validation split, in the table below. These details are now clarified in Section 3.1 and included in a new table in the Appendix.

Subset / SplitEpisodes1-s WindowsDuration (Hours)
Participant 17493,538~0.98 h
Participant 24752,753~0.76 h
Total Dataset1,2246,291~1.75 h
**VLA Training **—5,032~1.40 h
**VLA Validation **—1,259~0.35 h

yhW8-Q1.2:\color{red}{\text{yhW8-Q1.2:}} How do the two source datasets (App. A.1; "v1" and "ego128" in the supplied code) relate to the two participants, their sessions and their recording days? App. A.1 states that each participant completed all 12 tasks, yet five of the 12 tasks have episodes only from "v1" (prepare_waveform.py, l.24–37).

R- yhW8-Q1.2:\color{blue}{\text{R- yhW8-Q1.2:}} In the supplied code, “v1” and “ego128” denote two recording batches collected from the same two participants on different days, containing 919 and 305 episodes, respectively. The five tasks appearing only in “v1” therefore reflect differences in task coverage between batches, rather than separate participant datasets.

yhW8-Q1.3:\color{red}{\text{yhW8-Q1.3:}} Were tasks recorded in blocks or interleaved? How were EEG and video synchronized, and how accurately? Fig. 2(C) plots time since a "video-estimated onset".

R- yhW8-Q1.3:\color{blue}{\text{R- yhW8-Q1.3:}} Tasks were randomly interleaved during recording to reduce confounding between task identity and recording time. We have clarified this in the main text.

Video frames (30 Hz) are timestamped directly on the acquisition PC master clock (t_pc), while EEG (1000 Hz) is linearly mapped onto t_pc via ParallelBox hardware triggers: hardware trigger alignment achieves ~0.48 ms median residual, while end-to-end mapping to the shared PC reception clock has a median USB/software jitter residual of ~66 ms (~2 video frames).

In Fig. 2(C), "video-estimated onset" serves ** as the display origin for this illustrative plot**, not for dataset synchronization or training: to skip ~7 s of pre-movement rest and highlight active execution-phase EEG. Model training directly uses the clock-synchronized video–EEG windows without manual onset estimation.

yhW8-Q1.4:\color{red}{\text{yhW8-Q1.4:}} The data are presented as the first contribution, so these basic numbers belong in the main text. A blocked design without session-level control would also weaken the decoding evidence in Fig. 4.

R- yhW8-Q1.4:\color{blue}{\text{R- yhW8-Q1.4:}} Thank you for this suggestion. Clarifying these dataset statistics and experimental details greatly improves clarity and prevents misunderstandings; we have updated both Section 3.1 and Appendix A.1 accordingly. Moreover, because tasks were randomly shuffled within each recording block during data collection, the decoding evaluation in Fig. 4 is free from block-level temporal proximity bias.

**Q2 [C] Data release, scaling and ethics. **

yhW8-Q 2.1:\color{red}{\text{yhW8-Q 2.1:}} Will the synchronized EEG–video data be released, and under what access conditions?

R- yhW8-Q 2.1:\color{blue}{\text{R- yhW8-Q 2.1:}} Yes. At the end of December, we will publicly release an expanded embodied EEG–behavior dataset covering additional newly collected scenarios, which will include the full synchronized EEG–video dataset from this paper. The data will be openly accessible to everyone without restrictions, and we have updated our Reproducibility Statement accordingly.

yhW8-Q2.2:\color{red}{\text{yhW8-Q2.2:}} Do you plan to record more participants and to test on a held-out participant?

R- yhW8-Q 2.2:\color{blue}{\text{R- yhW8-Q 2.2:}} Thank you for the suggestion. Since submission, we have continued data collection, recruited additional participants, and expanded to more settings, including desks and workbenches. We have accumulated over 10 hours of recordings, and collection is ongoing.

At this stage, we prioritized repeated, longitudinal recordings from two participants under stable conditions to control additional variability arising from individual differences and investigate whether brain-signal supervision can improve embodied model performance. Dense sampling of a small participant cohort also has precedents, such as the core language-decoding experiments in Tang et al. (Nature Neuroscience, 2023). Our current results provide preliminary evidence of effectiveness. With the expanded dataset, we also hope to assess the consistency of these benefits through held-out-participant evaluation.

AuthorsOpenReview ↗

Response to Reviewer yhW8 (3/7)

yhW8-Q2.3:\color{red}{\text{yhW8-Q2.3:}} Did an ethics board approve the study? The Ethics Statement (p.10) mentions informed consent, de-identification and minimal risk, but no ethics-board approval.

R- yhW8-Q2.3:\color{blue}{\text{R- yhW8-Q2.3:}} Yes, the study has been approved by our institutional ethics board. The specific details were omitted during submission to preserve double-blind anonymity. We will include the full ethics approval details in the camera-ready Ethics Statement and alongside the official open-source dataset release.

yhW8-Q3 Real-robot evidence.

yhW8-Q3.1:\color{red}{\text{yhW8-Q3.1:}} Do you have any real-robot results?

R- yhW8-Q3.1:\color{blue}{\text{R- yhW8-Q3.1:}} Yes. We added evaluations on a physical PIPER robot using the π0\pi_0 backbone for two tabletop manipulation tasks.

We first fine-tuned π0\pi_0 on 50 real-world teleoperation demonstrations. From this checkpoint, we performed further LoRA fine-tuning using robot data alone (Robot-only) or with EEG supervision (EEG-guided). Inference remains EEG-free.

Real-world taskπ0\pi_0 baseline SR (%)Robot-only LoRA SR (%)EEG-guided LoRA SR (%)
Place a plate onto a dish-drying rack66.780.093.3
Place a pen into a pen holder40.053.373.3

EEG-guided training improves success rates over Robot-only by 13.3 and 20.0 percentage points, respectively, providing evidence of benefits beyond simulation.

We have added Figure 7 to the main text and described the experimental setup and implementation details in the appendix.

yhW8-Q4 Number of runs, selection and statistics.

yhW8-Q4.1:\color{red}{\text{yhW8-Q4.1:}} How many runs underlie each table entry? For example, 54.30 and 53.50 (Table 5) and 90.17 (Table 2) cannot be means of three 500-episode runs.

R- yhW8-Q4.1:\color{blue}{\text{R- yhW8-Q4.1:}} The submitted LIBERO results were intended to summarize three training seeds, with 500 evaluation episodes per suite per seed. We rechecked the original summaries against the raw episode-level JSON logs and identified the following reporting errors:

EntryOriginally reportedCause of reporting errorOriginal three-seed raw countsCorrected original three-seed summary (mean ± sample SD)
Table 5 BrainTokenize54.30 ± 1.40Accidentally pooled six seeds rather than three, including three additional runs: 268/500, 273/500, and 270/500284/500 (56.80%), 263/500 (52.60%), 271/500 (54.20%)54.53 ± 2.12
Table 5 LaBraM53.50 ± 1.00Rounded 53.4667 ± 1.0263 to one decimal place before zero-padding263/500 (52.60%), 266/500 (53.20%), 273/500 (54.60%)53.47 ± 1.03
Table 2 Object (Robot-only)90.17 ± 0.37Used a stale summary containing an incomplete run (409/456), together with 90.60% and 90.20%, and population SD453/500 (90.60%), 451/500 (90.20%), 437/500 (87.40%)89.40 ± 1.74

These corrections document the original three-seed results. Separately, we have expanded the main comparisons in Tables 1 and 2 to six training seeds; their revised entries report the corresponding six-seed means and sample SDs, superseding the original three-seed summaries.

We will explicitly state the number of seeds and evaluation episodes for each experiment and consistently define “±” as the sample SD across training seeds.

yhW8-Q4.2:\color{red}{\text{yhW8-Q4.2:}} Were the default loss weights chosen on the LIBERO-Long evaluation rollouts (App. B.3)? Was the robot-only baseline tuned at all?

R- yhW8-Q4.2:\color{blue}{\text{R- yhW8-Q4.2:}} We would like to clarify how the default loss weights were determined and how the Robot-only baseline was configured.

1. Loss weights. The default weights were not selected using LIBERO-Long evaluation results. They were fixed beforehand based on the batch ratio and loss scales of the robot and EEG branches, and kept unchanged across LIBERO and RL4VLA, and OpenVLA and π0\pi_0. The ablations in Appendix B.3 were conducted afterward to examine sensitivity to these weights.

2. Robot-only baseline. We did not perform a separate hyperparameter search for Robot-only. Both Robot-only and EEG-guided were initialized from the same official suite-specific fine-tuned checkpoints and used identical robot-policy training settings. Robot-only omitted the EEG branch.

Reviewer_hUwKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores EEG as supervision for adapting VLA models. Human egocentric images and instructions are used to predict synchronized EEG, while a robot branch learns action prediction through shared LoRA parameters. EEG is removed at inference. Experiments evaluate OpenVLA on LIBERO and RL4VLA, with additional results using π0.

Critical strengths
  1. The paper explores an interesting supervision source without changing deployment inputs. Human EEG provides an auxiliary training signal, while the adapted robot policy operates without neural recordings.

  2. The results provide encouraging initial evidence. OpenVLA improves across all evaluated settings, with a 6.01 percentage point gain under execution shifts. Shuffling EEG correspondence removes the improvement on LIBERO Long, supporting the importance of behavioral alignment.

Critical weaknesses
  1. The specific benefit of EEG remains unclear. Shuffling demonstrates that correspondence matters, but does not establish an advantage over auxiliary targets derived from the same human videos, such as motion or task progression. These comparisons are important to justify the additional effort of collecting EEG.

  2. The evidence for broad transfer is preliminary. Recordings involve only two participants, several improvements are modest relative to reported variation, and the main alignment controls are limited to LIBERO Long. The results do not yet establish robustness across participants or consistently strong gains across backbones.

Questions for authors
  1. How does EEG supervision compare with predicting motion, visual features, or task progression from the same human recordings under a matched training budget?

  2. Can the improvements be reproduced using each participant’s recordings separately? What additional evidence distinguishes neural contributions from residual movement or physiological artifacts after preprocessing?

  3. Could the authors report the number of independent training runs and paired confidence intervals, particularly for the smaller π0 improvements?

4 comments
AuthorsOpenReview ↗

Response to Reviewer hUwK (1/4)

We appreciate the reviewer’s recognition of the value of exploring EEG as a training-time supervisory signal without changing deployment inputs, and of the encouraging initial results supporting the importance of behavioral alignment. In response to your comments, we have made six main additions and clarifications:

  1. Real-world robot experiments. We added two tasks on a physical PIPER robot using π0: placing a plate onto a dish-drying rack and placing a pen into a pen holder. We compare the starting checkpoint, Robot-only LoRA, and EEG-guided LoRA, with EEG-free inference. Figure 7 presents the results, with implementation details in the appendix.

  2. Additional auxiliary-target controls. We added optical-flow statistics, task/phase progress, and matched random targets on LIBERO-Long and LIBERO-Goal, together with frozen CLIP ViT-B/16 feature supervision on LIBERO-Long, to directly compare EEG with alternative auxiliary objectives.

  3. Alignment controls beyond LIBERO-Long. We extended the controls to LIBERO-Goal, including temporally shifted EEG, episode-mean EEG, and same-episode negatives, to examine the roles of temporal correspondence and contrastive sampling.

  4. More training runs and paired uncertainty estimates. We expanded the main LIBERO comparisons to six training seeds and added paired statistical tests and nominal 95% confidence intervals. We distinguish positive mean gains from statistical support and identify the selected-seed RL4VLA results as exploratory.

  5. Quantitative artifact-attenuation analysis. We added before-and-after ICA analyses on non-interpolated good channels, with an appendix figure showing approximately 90–97% reductions in HF/LF power ratios in representative trials. This provides quantitative evidence supplementing the preprocessing description.

  6. Clearer participant coverage and validation plans. We clarified the rationale for longitudinal recordings from two participants and the limits of the current pooled-data evidence. We also documented ongoing collection from additional participants and environments, exceeding 10 hours of recordings, alongside public-release and participant-specific validation plans.

We address your specific concerns and questions below.

hUwK-W1:\color{red}{\text{hUwK-W1:}} The specific benefit of EEG remains unclear. Shuffling demonstrates that correspondence matters, but does not establish an advantage over auxiliary targets derived from the same human videos, such as motion or task progression. These comparisons are important to justify the additional effort of collecting EEG.

R- hUwK-W1:\color{blue}{\text{R- hUwK-W1:}} Thank you for this suggestion. We agree that shuffling alone establishes the importance of correspondence, but does not demonstrate an advantage over video-derived auxiliary targets. We therefore added controls using task/phase progress, optical-flow statistics, and pretrained visual features.

The visual-feature control uses frozen CLIP ViT-B/16 features as auxiliary targets with a cosine loss. It was trained for 2,000 steps on LIBERO-Long using three seeds, with 500 evaluation episodes per seed. Results are mean ± sample SD:

Auxiliary supervisionLIBERO-Long (%)LIBERO-Goal (%)
Robot-only53.47 ± 0.4277.87 ± 0.12
Aligned EEG56.73 ± 0.7079.40 ± 1.04
Task/phase progress52.20 ± 1.4074.27 ± 1.72
Optical-flow statistics53.00 ± 2.4276.07 ± 0.95
Frozen CLIP features (cosine loss)52.60 ± 2.3176.47 ± 2.25

In the current summaries, aligned EEG achieves higher mean success rates than these tested auxiliary targets. The additional CLIP control examines whether pretrained visual representations can substitute for EEG supervision, providing a more direct comparison for assessing the value of collecting EEG.

We have added these controls and their implementation details to the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer hUwK (2/4)

hUwK-W2:\color{red}{\text{hUwK-W2:}} The evidence for broad transfer is preliminary. Recordings involve only two participants, several improvements are modest relative to reported variation, and the main alignment controls are limited to LIBERO Long. The results do not yet establish robustness across participants or consistently strong gains across backbones.

R- hUwK-W2:\color{blue}{\text{R- hUwK-W2:}} Thank you for this assessment. We agree that evidence for broad transfer remains preliminary. We clarify the participant coverage, reliability of the gains, and expanded controls below.

1. Participant coverage, recording design, and dataset expansion.

Contribution and scope. Our dataset was limited at submission. As stated in Contribution 1, our primary contribution is a paradigm integrating synchronized EEG recording with embodied experiments, rather than dataset scale. Our initial EEG–egocentric behavior recordings provide preliminary evidence that EEG auxiliary supervision can benefit embodied models. Since submission, we have recruited additional participants and expanded collection to desks, workbenches, and dining tables, accumulating over 10 hours of recordings, including the data used in this paper. Collection is ongoing, and we plan to publicly release the synchronized data with supporting documentation.

Recording design. We initially prioritized repeated, longitudinal recordings from two participants under stable conditions to reduce additional variability arising from individual differences and investigate whether brain-signal supervision can improve embodied model performance. This approach is consistent with prior brain-tuning research: for example, Moussa et al. (ICLR 2025) fine-tuned speech models separately using individual participants’ fMRI recordings and improved downstream semantic task performance. Dense sampling of a small participant cohort also has precedents, such as the core language-decoding experiments in Tang et al. (Nature Neuroscience, 2023). Our study follows a similar approach, with cross-participant generalization outside its primary scope.

2. Cross-backbone results and paired statistical evidence.

We expanded the main LIBERO comparisons to six training seeds and computed raw two-sided McNemar tests using matched training seeds, tasks, and evaluation initial states. RL4VLA uses 64 episodes per sub-setting.

Model / BenchmarkSettingRobot-onlyEEG-guidedGain (pp)McNemar p-valueNominal significance
OpenVLA / LIBEROSpatial79.370%80.800%+1.4300.040*
OpenVLA / LIBEROObject66.770%68.770%+2.0000.030*
OpenVLA / LIBEROGoal76.100%77.900%+1.8000.032*
OpenVLA / LIBEROLong53.667%55.833%+2.1670.052ns
π0 / LIBEROSpatial94.330%94.930%+0.6000.225ns
π0 / LIBEROObject92.800%94.230%+1.4300.016*
π0 / LIBEROGoal92.770%94.230%+1.4700.008**
π0 / LIBEROLong76.830%78.670%+1.8300.032*
OpenVLA / RL4VLAVisual OOD94.115%96.198%+2.083<0.001***
OpenVLA / RL4VLASemantic OOD71.801%72.805%+1.0040.006**
OpenVLA / RL4VLAExecution OOD33.941%40.625%+6.684<0.001***

Note: * p<0.05; ** p<0.01; *** p<0.001; ns denotes p≥0.05. Stars are based on the original, unrounded p-values.

OpenVLA on LIBERO-Long and π0 on LIBERO-Spatial show positive mean gains but do not reach nominal significance. The remaining reported comparisons have raw p-values below 0.05, strengthening the preliminary evidence for transfer across the evaluated settings.

3. Controls beyond LIBERO-Long.

We extended the controls to LIBERO-Goal, including temporally shifted EEG, episode-mean EEG, same-episode negatives, and non-neural targets such as task/phase progress, optical-flow statistics, and matched random targets. These controls use three seeds × 500 episodes per suite per seed, separately from the expanded main comparisons.

These additions strengthen the evidence within the evaluated settings.

AuthorsOpenReview ↗

Response to Reviewer hUwK (3/4)

hUwK-Q1:\color{red}{\text{hUwK-Q1:}} How does EEG supervision compare with predicting motion, visual features, or task progression from the same human recordings under a matched training budget?

R- hUwK-Q1:\color{blue}{\text{R- hUwK-Q1:}} As shown in R-W1, we compare EEG supervision with motion and task/phase-progress targets derived from the same human recordings, using the same VLA backbone, LoRA configuration, and 2,000-step fine-tuning budget. Each setting uses three seeds and 500 evaluation episodes per suite per seed. EEG-guided achieves 56.73% and 79.40% on LIBERO-Long and LIBERO-Goal, respectively, compared with 53.00% and 76.07% for optical-flow supervision and 52.20% and 74.27% for task/phase-progress supervision. Fixed random targets also fail to improve over Robot-only.

Under the same 2,000-step training budget, the additional frozen CLIP ViT-B/16 feature-prediction control with cosine loss achieves a mean success rate of 52.60% on LIBERO-Long. Together with the optical-flow, task-progress, and random-target controls, these results show that the tested alternative auxiliary objectives do not reproduce the gains from aligned EEG. This provides direct empirical support for the added supervisory value of behavior-aligned EEG beyond simply introducing an auxiliary training objective.

hUwK-Q2.1:\color{red}{\text{hUwK-Q2.1:}} Can the improvements be reproduced using each participant’s recordings separately?

R- hUwK-Q2.1:\color{blue}{\text{R- hUwK-Q2.1:}} As discussed in R-hUwK-W2, we initially prioritized repeated, longitudinal recordings under stable conditions to reduce variability arising from individual differences and investigate whether brain-signal supervision can improve embodied model performance. This design has precedents in participant-specific brain-tuning (Moussa et al., ICLR 2025) and dense sampling of small cohorts for neural decoding (Tang et al., Nature Neuroscience, 2023). Our study follows a similar approach, with cross-participant reproducibility outside its primary scope.

Since submission, we have expanded data collection to additional participants and environments, accumulating over 10 hours of recordings. Using this expanded dataset, we plan to evaluate participant-specific training with controlled data quantities and generalization to held-out participants. We will explicitly acknowledge this limitation in the main text.

hUwK-Q2.2:\color{red}{\text{hUwK-Q2.2:}} What additional evidence distinguishes neural contributions from residual movement or physiological artifacts after preprocessing?

R- hUwK-Q2.2:\color{blue}{\text{R- hUwK-Q2.2:}} We thank the reviewer for raising this concern. We added before-and-after ICA analyses to quantify artifact attenuation. Our pipeline excludes bad channels from ICA fitting and removes muscle-like components identified by their high-frequency spectral characteristics. The analysis uses non-interpolated good channels to avoid attributing interpolation-related smoothing to artifact removal.

In representative trials, the HF/LF power ratio decreases by approximately 90–97% after ICA cleaning, while lower-frequency activity remains visible. These results provide quantitative evidence of substantial attenuation of high-frequency activity consistent with muscle contamination, strengthening the preprocessing evidence. The corresponding figure has been added to the appendix.

AuthorsOpenReview ↗

Response to Reviewer hUwK (4/4)

hUwK-Q3:\color{red}{\text{hUwK-Q3:}} Could the authors report the number of independent training runs and paired confidence intervals, particularly for the smaller π0 improvements?

R- hUwK-Q3:\color{blue}{\text{R- hUwK-Q3:}} As discussed in R-hUwK-W2 (“2. Cross-backbone results and paired statistical evidence”), we expanded the main comparisons to six training runs per method per suite, yielding six matched Robot-only/EEG-guided pairs. We conducted paired statistical tests and have now added nominal 95% confidence intervals for the success-rate gains:

Model / BenchmarkSettingRobot-onlyEEG-guidedGain (pp)Gain 95% CI (pp)McNemar p-valueNominal significance
OpenVLA / LIBEROSpatial79.367%80.800%+1.433[0.100, 2.767]0.040058*
OpenVLA / LIBEROObject66.767%68.767%+2.000[0.233, 3.767]0.029800*
OpenVLA / LIBEROGoal76.100%77.900%+1.800[0.200, 3.400]0.031796*
OpenVLA / LIBEROLong53.667%55.833%+2.167[0.000, 4.333]0.052404ns
π0 / LIBEROSpatial94.333%94.933%+0.600[−0.300, 1.533]0.224546ns
π0 / LIBEROObject92.800%94.233%+1.433[0.300, 2.567]0.016387*
π0 / LIBEROGoal92.767%94.233%+1.467[0.400, 2.533]0.007540**
π0 / LIBEROLong76.833%78.667%+1.833[0.200, 3.467]0.031767*
OpenVLA / RL4VLAVisual OOD94.115%96.198%+2.083[1.428, 2.726]0.00001876***
OpenVLA / RL4VLASemantic OOD71.801%72.805%+1.004[0.639, 1.427]0.006106**
OpenVLA / RL4VLAExecution OOD33.941%40.625%+6.684[5.000, 8.715]0.00000110***

Note: * p<0.05; ** p<0.01; *** p<0.001; ns denotes p≥0.05.

The reported intervals for OpenVLA on LIBERO-Long and π0 on LIBERO-Spatial include zero, while the other intervals exclude zero under the respective nominal analyses. These results provide empirical support for the method’s effectiveness in the evaluated settings.

Reviewer_Zz7nOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether human EEG recorded during manipulation can serve as extra training supervision for vision-language-action models. The authors record egocentric video and 128-channel EEG from 2 participants doing 12 everyday tasks. During LoRA fine-tuning of OpenVLA or π0, a second branch takes a human egocentric frame and the instruction and predicts the next 1 s of EEG, with an MSE loss and an InfoNCE loss. EEG is not used at test time. The method is evaluated on four LIBERO suites and on three RL4VLA OOD settings. It reports gains of about 0.3 to 6 points over a robot-only LoRA baseline. A shuffled-EEG control and a run without InfoNCE both lose most of the gain. Layer-wise and rollout analyses are given to explain the effect.

Critical strengths

The question is new. Using real human brain signals as training-time supervision for a robot policy, and removing them at test time, has not been tried for VLAs as far as I know. The synchronized EEG and egocentric video data could be useful to others if it is released with clear documentation.

Critical weaknesses

The paper gives no real reason why this should work, and the experiments cannot show that the gain comes from neural information. The human tasks (grapes, drawers, plates in a real room) are different from the robot tasks (LIBERO and ManiSkill in simulation). The paper never explains what information in scalp EEG, recorded while a person does task A, could help a robot arm do task B. The motivation rests on a loose analogy that both humans and robots “operate under physical constraints”. EEG recorded during free hand and head movement is also heavily affected by head motion, eye movement and muscle artifacts, and the paper only reports removing muscle components. Task decoding from this EEG is about 14% against 8.3% chance, so the signal is weak. A model that learns to predict this EEG from an egocentric frame may simply learn camera motion. The shuffle control only shuffles within each task, so it shows that frame-level timing matters, but it does not show that the brain signal matters. The layer analysis is also not evidence of a mechanism, because LoRA is only applied to layers 24 to 31, so larger changes in the last layers are expected by design. This weakness is in the premise of the work, not in a missing table.

The reported gains are too small to trust under the way the experiments were run. All hyperparameters (LoRA module, depth, rank, learning rate, batch size, head size, window, two loss weights) were chosen on LIBERO-Long test rollouts, and the default wins in every one of the nine sweeps. Many nearby settings with EEG score below the robot-only baseline of 53.47, for example 51.0, 52.0, 52.3, 52.4 and 53.2. So the +3.26 gain is within the range produced by tuning alone. The baseline itself is also weak. The robot-only LIBERO-Object result is 66.9, while the public OpenVLA checkpoint that the authors start from reports about 88.4. The π0 baselines are also several points below public numbers. Most gains in Table 1 and Table 2 are 0.3 to 1.6 points with standard deviations of about 1 to 2. Taken together, I do not think the paper shows that EEG supervision improves the policy.

Questions for authors
  1. In Figure 5A the net paired flips sum to +65 episodes, and the robot-only failure count is given as n = 577. With 3 runs of 500 episodes, Table 1 implies +49 episodes and about 698 robot-only failures. Which runs does Figure 5 use, and why do the numbers differ?

  2. Why is the robot-only baseline far below the released OpenVLA and π0 checkpoints, especially on LIBERO-Object? What is the success rate of the starting checkpoint with no further training?

  3. Was any held-out split used to select hyperparameters, or were all choices made on the same test rollouts that are reported?

  4. Semantic OOD in Table 1 reports 72.99 ± 0.00. How many runs is this, and how is the number computed from 64 episodes?

  5. The appendix mentions two “source datasets” and a protocol with additional phases. Where does this data come from, and was any of it collected for a different study?

22 comments
AuthorsOpenReview ↗

Response to Reviewer Zz7n (1/6)

We appreciate the reviewer’s recognition of our research question’s novelty and the potential value of the synchronized EEG–egocentric video data. In response, we have made six main additions and clarifications:

  1. Real-world validation. We added two PIPER tasks using π0, comparing the starting checkpoint, Robot-only, and EEG-guided models with EEG-free inference (Figure 7).

  2. Clearer rationale and scope. We clarified the Brain-tuning motivation, human–robot task relationship, and descriptive scope of the decoding and representation analyses.

  3. Additional controls. We added shifted and episode-mean EEG, motion/progress targets, random targets, and frozen CLIP features, alongside quantitative ICA artifact-attenuation analyses.

  4. Baseline and protocol checks. We added zero-step checkpoint evaluations and clarified simulator versions, preprocessing, fixed-step evaluation, and hyperparameter selection.

  5. Expanded statistical reporting. We increased the main LIBERO comparisons to six seeds, added paired tests, and clarified aggregation and Figure 5 counts. Selected-seed RL4VLA results are identified as exploratory.

  6. Data documentation and release. We clarified data provenance and ethics approval, and documented expanded collection exceeding 10 hours and plans for public release with supporting documentation.

We address your specific questions below.

Zz7n-W1.1:\color{red}{\text{Zz7n-W1.1:}} The paper gives no real reason why this should work, and the experiments cannot show that the gain comes from neural information.

R-Zz7n-W1.1:\color{blue}{\text{R-Zz7n-W1.1:}} Thank you for raising this concern. Below, we clarify the rationale for our approach and discuss the potential source of the gains using both the original and newly added controls.

Why might EEG supervision help? Our working hypothesis is that EEG recorded during manipulation contains behavior-related temporal structure that can provide a useful learning constraint beyond the robot action objective. The human and robot branches share the same trainable LoRA parameters. Therefore, predicting behavior-aligned EEG from human egocentric observations shapes representations also used for robot action prediction. This provides a concrete pathway for EEG supervision to transfer to the robot policy without requiring EEG at inference. Our experiments test whether this supervision produces practical benefits.

Are the gains merely an auxiliary-task effect? Building on this control, we added experiments examining EEG temporal structure, visual motion information, and non-neural auxiliary targets. On LIBERO-Long and LIBERO-Goal, we use the same human video frames paired with temporally shifted EEG, episode-mean EEG, task/phase progress targets, optical-flow statistics, or fixed random targets. Each setting is evaluated over three seeds × 500 episodes per seed; the table reports mean success rates.

Training settingLIBERO-Long (%)LIBERO-Goal (%)
Robot-only53.4777.87
Aligned EEG56.7379.40
EEG shifted within episode (±2 s)54.0777.00
Episode-mean EEG55.2777.47
Task/phase progress targets52.2074.27
Optical-flow statistics53.0076.07
Fixed random targets matched to EEG mean and variance53.4777.20

Aligned EEG achieves higher mean success rates than these controls on both suites. The non-neural targets do not improve mean performance over Robot-only, while disrupting EEG alignment or removing within-episode temporal variation reduces the gains. These results support a contribution from the content and temporal correspondence of EEG supervision that cannot be explained solely by adding an auxiliary objective.

Together, these controls analyses provide additional evidence that behavior-aligned EEG offers useful training-time supervision. We have included the corresponding experimental settings, results, and discussion in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Zz7n (2/6)

Zz7n-W1.2:\color{red}{\text{Zz7n-W1.2:}} The human tasks (grapes, drawers, plates in a real room) are different from the robot tasks (LIBERO and ManiSkill in simulation). The paper never explains what information in scalp EEG, recorded while a person does task A, could help a robot arm do task B. The motivation rests on a loose analogy that both humans and robots “operate under physical constraints”.

R-Zz7n-W1.2:\color{blue}{\text{R-Zz7n-W1.2:}} Thank you for identifying this gap in our motivation. We clarify the data design and methodological rationale below.

1. Human data capture natural everyday manipulation, without requiring one-to-one pairing with robot tasks.

We collected diverse natural manipulation tasks in everyday settings rather than restricting them to counterparts of LIBERO tasks. Human and robot data are sampled independently during training. Although they differ in objects, environments, and goals, they may share interaction processes such as approaching, grasping, transporting, making contact, and completing an interaction. Our hypothesis is that EEG temporal structure associated with these behavioral state changes can, through joint optimization of shared parameters, provide constraints on robot representation learning beyond specific task identities.

2. Establishing the utility of EEG supervision does not require identifying a specific neural component beforehand.

Previous studies show that EEG can reflect processes such as motor-related activity and error monitoring under their respective experimental paradigms (Pfurtscheller & Lopes da Silva, 1999; Gehring et al., 1993). These findings motivate using EEG, but do not establish which neural component contributes to the gains during natural manipulation. We take a data-driven approach, using controlled experiments to test whether behavior-aligned EEG provides useful supervision. Evaluating its overall utility and identifying the contributions of specific neural components are distinct questions; the former does not require resolving the latter first.

Our approach follows the Brain-tuning principle discussed in Section 2. For example, Moussa et al. (2025) fine-tuned speech models using fMRI recorded while participants listened to natural stories and improved downstream semantic performance on other datasets. This demonstrates that representations shaped by neural supervision can benefit tasks beyond the original recording context. We extend this principle to embodied VLA models to test whether EEG supervision can improve robot policies. Isolating specific neural components and establishing their individual contributions constitute a separate research question beyond the focus of this paper.

Zz7n-W1.3:\color{red}{\text{Zz7n-W1.3:}} EEG recorded during free hand and head movement is also heavily affected by head motion, eye movement and muscle artifacts, and the paper only reports removing muscle components.

R-Zz7n-W1.3:\color{blue}{\text{R-Zz7n-W1.3:}} Thank you for raising this important concern. Appendix A.2 reports our preprocessing pipeline, including 50/100 Hz notch filtering, 1–100 Hz band-pass filtering, bad-channel interpolation, and Infomax ICA with 64 independent components. During ICA-based cleaning, we also removed components identified as ocular, muscular, or suspected head-motion artifacts to reduce non-neural contamination. The original description emphasized muscle-artifact removal without sufficiently explaining the treatment of other artifacts; we will clarify this in the revised manuscript.

Additionally, the newly added optical-flow-statistics control did not reproduce the gains from EEG supervision on either LIBERO-Long or LIBERO-Goal, supporting that the gains cannot be explained solely by the tested visual motion statistics. Our preprocessing reduces artifact contamination, while the additional control further examines motion information as an alternative explanation, providing complementary evidence for the robustness of our results.

Zz7n-W1.4:\color{red}{\text{Zz7n-W1.4:}} Task decoding from this EEG is about 14% against 8.3% chance, so the signal is weak.

R-Zz7n-W1.4:\color{blue}{\text{R-Zz7n-W1.4:}} Thank you for your comment. Task decoding is an auxiliary diagnostic of above-chance task-related information, not a comprehensive measure of EEG’s supervisory value. EEG can reflect multiple aspects of behavior, including actions, sequence positions, and chunk structure (Kikumoto & Mayr, 2018), which task-category labels do not fully capture.

In our study, the utility of EEG supervision is primarily evaluated through downstream robot performance and corresponding controlled experiments; task-category decoding serves only as an auxiliary diagnostic.

AuthorsOpenReview ↗

Response to Reviewer Zz7n (3/6)

Zz7n-W1.5:\color{red}{\text{Zz7n-W1.5:}} A model that learns to predict this EEG from an egocentric frame may simply learn camera motion. The shuffle control only shuffles within each task, so it shows that frame-level timing matters, but it does not show that the brain signal matters.

R-Zz7n-W1.5:\color{blue}{\text{R-Zz7n-W1.5:}} To further test whether the model merely learns visual motion information, we added an optical-flow-statistics auxiliary-target control using the same human video frames. This control achieved success rates of 53.00% and 76.07% on LIBERO-Long and LIBERO-Goal, respectively, compared with 53.47% and 77.87% for Robot-only and 56.73% and 79.40% for aligned EEG.

These results support that the benefits of EEG supervision extend beyond the visual motion statistics tested.

Zz7n-W1.6:\color{red}{\text{Zz7n-W1.6:}} The layer analysis is also not evidence of a mechanism, because LoRA is only applied to layers 24 to 31, so larger changes in the last layers are expected by design. This weakness is in the premise of the work, not in a missing table.

R-Zz7n-W1.6:\color{blue}{\text{R-Zz7n-W1.6:}} Thank you for highlighting this limitation. Figure 6 compares hidden states between EEG-guided and Robot-only models with identical initialization and LoRA configurations. We have clarified its descriptive scope in the main text.

Our rationale is to use neural supervision to shape model representations; it does not rest on this layer-wise pattern. We evaluate its utility through downstream robot performance and aligned, shuffled/shifted EEG, and non-neural controls. These results support EEG supervision in the tested settings, while its precise neural basis and transfer mechanism remain questions for future work.

Zz7n-W2.1:\color{red}{\text{Zz7n-W2.1:}} The reported gains are too small to trust under the way the experiments were run. All hyperparameters (LoRA module, depth, rank, learning rate, batch size, head size, window, two loss weights) were chosen on LIBERO-Long test rollouts, and the default wins in every one of the nine sweeps. Many nearby settings with EEG score below the robot-only baseline of 53.47, for example 51.0, 52.0, 52.3, 52.4 and 53.2. So the +3.26 gain is within the range produced by tuning alone.

R-Zz7n-W2.1:\color{blue}{\text{R-Zz7n-W2.1:}} Thank you for raising concerns about hyperparameter selection and the reliability of the results. We clarify that the default configuration was determined before these ablation experiments. The parameter sweeps on LIBERO-Long assessed sensitivity rather than selected the defaults based on test results. In particular, the loss weights were preset based on the loss scales of the two branches.

Our evidence also extends beyond LIBERO-Long: with configurations fixed and no retuning for individual evaluation settings, we observed gains on other LIBERO suites, RL4VLA, and across backbones. EEG-Shuffle and non-neural auxiliary-target controls under matched configurations further examined the role of supervision content.

Reviewer_FvrAOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper uses synchronized EEG and egocentric video collected during human manipulation tasks to guide visual representation learning. The experiment results show that integrating the learned representations into a vision-language-action model improves success rates in most settings on LIBERO and RL4VLA.

Critical strengths

This paper uses human EEG data to guide robot representation learning, which is a novel approach. The shuffle control is well-designed, and the main results show significant gains across most settings, supporting the method’s effectiveness.

Critical weaknesses

The study includes only two participants (see Appendix A.1), making generalization across individuals unclear. Moreover, for these gains from 79.53 ± 1.03 to 80.93 ± 1.42 on LIBERO for the Spatial setting and also the Object setting, where the gain from 66.93 ± 2.34 to 68.53 ± 1.7, the gains are smaller than the standard deviation, so the claim of “consistent improvements” could be qualified.

Questions for authors
  1. Please analyze why the gains vary substantially across different settings.
  2. As shown in Appendix A.1, each participant completes all 12 tasks. How do the human tasks correspond to the LIBERO tasks?
  3. Since the paper uses EEG only during training and removes it at inference, whether changing the participants will result in different representations?
6 comments
AuthorsOpenReview ↗

Response to Reviewer FvrA (1/3)

We appreciate the reviewer’s recognition of the novelty of using human EEG to guide robot representation learning, the design of the shuffle control, and the encouraging initial results. In response to your comments, we have strengthened the experimental evidence and clarified the interpretation and scope of our findings through six main additions and revisions:

  1. Real-world robot experiments. We added two physical PIPER tasks using π0, comparing the starting checkpoint, Robot-only LoRA, and EEG-guided LoRA with EEG-free inference. Figure 7 presents the results, with implementation details in the appendix.

  2. More training runs and paired statistical evidence. We expanded the main LIBERO comparisons to six training seeds and added paired tests and nominal confidence intervals. We distinguish positive mean gains from statistical significance and clarify the interpretation of “consistent improvements.” The selected-seed RL4VLA results are identified as exploratory.

  3. Clearer human–robot task correspondence. We clarified that human and robot tasks are not paired one-to-one. Their data are sampled independently, and the two branches jointly optimize shared LoRA parameters. We also clarified the connection to Brain-tuning as a rationale for cross-task supervision.

  4. Analysis of variation across settings. We expanded the discussion of task demands, baseline headroom, and backbone architecture, supported by the behavioral examples in Figure 5. We distinguish these plausible explanations from experimentally established mechanisms.

  5. Participant coverage and validation plans. We clarified the rationale for longitudinal recordings from two participants and acknowledged that the current results do not establish cross-participant generalization or participant-invariant representations. We also documented ongoing collection from additional participants and environments, exceeding 10 hours of recordings, together with public-release and participant-specific validation plans.

We address your specific concerns and questions below.

FvrA -W1:\color{red}{\text{FvrA -W1:}} The study includes only two participants (see Appendix A.1), making generalization across individuals unclear.

R-FvrA -W1:\color{blue}{\text{R-FvrA -W1:}} Thank you for highlighting this limitation. We acknowledge that the current recordings from two participants do not establish cross-participant generalization.

Contribution and scope. Our dataset was limited at submission. As stated in Contribution 1, our primary contribution is a paradigm integrating synchronized EEG recording with embodied experiments, rather than dataset scale. Our initial EEG–egocentric behavior recordings provide preliminary evidence that EEG auxiliary supervision can benefit embodied models. Since submission, we have recruited additional participants and expanded collection to desks, workbenches, and dining tables, accumulating over 10 hours of recordings, including the data used in this paper. Collection is ongoing, and we plan to publicly release the synchronized data with supporting documentation.

Recording design. We initially prioritized repeated, longitudinal recordings from two participants under stable conditions to reduce additional variability arising from individual differences and investigate whether brain-signal supervision can improve embodied model performance. This approach is consistent with prior brain-tuning research: for example, Moussa et al. (ICLR 2025) fine-tuned speech models separately using individual participants’ fMRI recordings and improved downstream semantic task performance. Dense sampling of a small participant cohort also has precedents, such as the core language-decoding experiments in Tang et al. (Nature Neuroscience, 2023). Our study follows a similar approach, with cross-participant generalization outside its primary scope. We will explicitly clarify this limitation in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer FvrA (2/3)

FvrA -W2:\color{red}{\text{FvrA -W2:}} Moreover, for these gains from 79.53 ± 1.03 to 80.93 ± 1.42 on LIBERO for the Spatial setting and also the Object setting, where the gain from 66.93 ± 2.34 to 68.53 ± 1.7, the gains are smaller than the standard deviation, so the claim of “consistent improvements” could be qualified.

R-FvrA -W2:\color{blue}{\text{R-FvrA -W2:}} Thank you for highlighting the need to qualify this statement. We expanded the main LIBERO comparisons to six training seeds and computed raw two-sided McNemar tests using matched training seeds, tasks, and evaluation initial states.

Model / BenchmarkSettingRobot-onlyEEG-guidedGain (pp)McNemar p-valueNominal significance
OpenVLA / LIBEROSpatial79.370%80.800%+1.4300.040*
OpenVLA / LIBEROObject66.770%68.770%+2.0000.030*
OpenVLA / LIBEROGoal76.100%77.900%+1.8000.032*
OpenVLA / LIBEROLong53.667%55.833%+2.1670.052ns
π0 / LIBEROSpatial94.330%94.930%+0.6000.225ns
π0 / LIBEROObject92.800%94.230%+1.4300.016*
π0 / LIBEROGoal92.770%94.230%+1.4700.008**
π0 / LIBEROLong76.830%78.670%+1.8300.032*
OpenVLA / RL4VLAVisual OOD94.115%96.198%+2.083<0.001***
OpenVLA / RL4VLASemantic OOD71.801%72.805%+1.0040.006**
OpenVLA / RL4VLAExecution OOD33.941%40.625%+6.684<0.001***

Note: * p<0.05; ** p<0.01; *** p<0.001; ns denotes p≥0.05. Stars are based on the original, unrounded p-values.

OpenVLA on LIBERO-Long and π0 on LIBERO-Spatial show positive mean gains but do not reach nominal significance. The remaining reported comparisons have raw p-values below 0.05, strengthening the preliminary evidence for transfer across the evaluated settings.

We have added these experiments and statistical analyses to the manuscript and clarified that “consistent improvements” refers to the positive direction of the reported mean gains, rather than uniformly significant improvements across every setting or robust gains across all training runs.

FvrA -Q1:\color{red}{\text{FvrA -Q1:}} Please analyze why the gains vary substantially across different settings.

R-FvrA -Q1:\color{blue}{\text{R-FvrA -Q1:}} Thank you for suggesting a closer analysis of the variation across settings. The observed differences may reflect task demands, baseline headroom, and backbone architecture. We will expand the discussion of these factors in the revised manuscript.

1. Task demands and behavioral alignment. One plausible explanation is that the variation in gains across settings is not random, but reflects how closely each task's primary failure mode aligns with the information captured by EEG. Due to volume conduction, EEG has low spatial resolution and carries limited detail about static object identity or exact coordinates (Nunez & Srinivasan, 2006); consequently, on single-stage localization and semantic tasks (LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and RL4VLA Semantic OOD), EEG provides limited extra signal and yields modest gains. By contrast, EEG offers high temporal resolution suited to tracking sub-second to second-scale action-phase transitions and sensorimotor state changes, such as 8–30 Hz Mu/Beta ERD/ERS (Pfurtscheller & Lopes da Silva, 1999) and movement-related slow potentials (Shibasaki & Hallett, 2006). Predicting the upcoming 1s of EEG helps the policy track sub-goal progression and avoid stalling mid-task.

2. Behavioral evidence. Figure 5 supports this interpretation descriptively: recovered LIBERO-Long rollouts include cases where EEG-guided policies complete tasks that Robot-only policies leave unfinished or stall during execution. Some tasks requiring object identification and spatial target binding show smaller or negative gains. The larger mean improvement under Execution OOD in the RL4VLA analysis is also compatible with benefits related to execution, although it does not establish a specific neural mechanism.

3. Baseline headroom, architecture. Baseline performance and backbone architecture may also contribute. In particular, π0 has stronger baselines across the evaluated LIBERO suites, leaving less absolute headroom in near-saturated settings.

AuthorsOpenReview ↗

Response to Reviewer FvrA (3/3)

FvrA -Q2:\color{red}{\text{FvrA -Q2:}} As shown in Appendix A.1, each participant completes all 12 tasks. How do the human tasks correspond to the LIBERO tasks?

R-FvrA -Q2:\color{blue}{\text{R-FvrA -Q2:}} Thank you for identifying this gap in our motivation. We clarify the data design below.

1. Human data capture natural everyday manipulation, without requiring one-to-one pairing with robot tasks.

We collected diverse natural manipulation tasks in everyday settings rather than restricting them to counterparts of LIBERO tasks. Human and robot data are sampled independently during training. Although they differ in objects, environments, and goals, they may share interaction processes such as approaching, grasping, transporting, making contact, and completing an interaction. Our hypothesis is that EEG temporal structure associated with these behavioral state changes can, through joint optimization of shared parameters, provide constraints on robot representation learning beyond specific task identities.

  1. Shared basic actions & cross-task transfer (Brain-tuning):

Although specific objects and scene layouts differ, the 12 everyday tasks cover the same basic manipulation steps needed in LIBERO—such as reaching, grasping, moving, placing, and interacting with containers. This follows the brain-tuning idea discussed in Sec. 2 (Schwartz et al., 2019; Moussa et al., 2025; Moussa & Toneva, 2025), where fine-tuning language or speech models on brain recordings from one dataset helps them generalize to entirely different downstream tasks. Consistent with R1, co-training a VLA with everyday manipulation EEG teaches general action-transition timing rather than scene-specific object identity, allowing one human dataset to transfer across diverse robot tasks without task-specific recollection. We will make this clear in the revised manuscript.

FvrA -Q3:\color{red}{\text{FvrA -Q3:}} Since the paper uses EEG only during training and removes it at inference, whether changing the participants will result in different representations?

R-FvrA -Q3:\color{blue}{\text{R-FvrA -Q3:}} Thank you for raising this important question. Below, we clarify our recording design, the scope of the current evidence, and our plans.

Recording design. We initially prioritized repeated, longitudinal recordings from two participants under stable conditions to reduce additional variability arising from individual differences and investigate whether brain-signal supervision can improve embodied model performance. This approach is consistent with prior brain-tuning research: for example, Moussa et al. (ICLR 2025) fine-tuned speech models separately using individual participants’ fMRI recordings and improved downstream semantic task performance. Dense sampling of a small participant cohort also has precedents, such as the core language-decoding experiments in Tang et al. (Nature Neuroscience, 2023). Our study follows a similar approach, with cross-participant generalization outside its primary scope.

Contribution and scope. Since submission, we have recruited additional participants and expanded collection to desks, workbenches, and dining tables, accumulating over 10 hours of recordings, including the data used in this paper. Collection is ongoing, and we plan to publicly release the synchronized data with supporting documentation. Using this expanded dataset, we plan to evaluate participant-specific training with controlled data quantities and generalization to held-out participants. We will explicitly acknowledge this limitation in the main text.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Responses to All Reviewers and the AC

We sincerely thank all reviewers (yhW8\color{royalblue}{\text{yhW8}}, Zz7n\color{purple}{\text{Zz7n}}, hUwK\color{green}{\text{hUwK}}, FvrA\color{brown}{\text{FvrA}}) and the Area Chair for their constructive feedback. We appreciate the reviewers' recognition of the novelty of using human EEG as training-time supervision for VLA models (yhW8\color{royalblue}{\text{yhW8}}, Zz7n\color{purple}{\text{Zz7n}}, FvrA\color{brown}{\text{FvrA}}), the practical advantage of EEG-free inference (hUwK\color{green}{\text{hUwK}}), and the encouraging initial results and EEG alignment controls (hUwK\color{green}{\text{hUwK}}, FvrA\color{brown}{\text{FvrA}}). Building on these recognized strengths, we have conducted additional experiments, strengthened statistical analyses, and clarified our methodology to address the remaining concerns.

The key updates are summarized below:

1. Real-World Robot Validation (Reviewer yhW8\color{royalblue}{\text{yhW8}}). We added two manipulation tasks on a physical PIPER robot using π0. EEG-guided training improves success rates over Robot-only by 13.3 and 20.0 percentage points, respectively, while requiring no EEG at inference.

2. EEG-Specific Supervision and Additional Controls (Reviewers Zz7n\color{purple}{\text{Zz7n}}, hUwK\color{green}{\text{hUwK}}, yhW8\color{royalblue}{\text{yhW8}}). We introduced new controls using optical flow, task/phase progress, pretrained CLIP features, random targets, temporally shifted EEG, and episode-mean EEG. The results support the benefits of behavior-aligned EEG beyond the tested non-neural auxiliary targets. We also added quantitative ICA analyses to assess artifact attenuation.

3. Statistical Reliability and Generalization (Reviewers Zz7n\color{purple}{\text{Zz7n}}, hUwK\color{green}{\text{hUwK}}, FvrA\color{brown}{\text{FvrA}}, yhW8\color{royalblue}{\text{yhW8}}). We expanded the main LIBERO evaluations to six training seeds and added paired statistical tests and nominal 95% confidence intervals. The updated results provide stronger evidence across OpenVLA, π0, and RL4VLA, while distinguishing statistically supported gains from smaller, nonsignificant improvements.

4. Experimental Protocol and Reproducibility (Reviewers Zz7n\color{purple}{\text{Zz7n}}, yhW8\color{royalblue}{\text{yhW8}}). We added zero-step checkpoint evaluations, clarified simulator-version differences and hyperparameter selection, corrected reporting inconsistencies, and documented EEG data collection, synchronization, and training protocols.

5. Dataset Expansion and Limitations (Reviewers FvrA\color{brown}{\text{FvrA}}, hUwK\color{green}{\text{hUwK}}, yhW8\color{royalblue}{\text{yhW8}}, Zz7n\color{purple}{\text{Zz7n}}). We expanded EEG data collection to additional participants and environments, accumulating over 10 hours of recordings, with a public release planned. We also clarified the limitations regarding cross-participant generalization and the identification of neural-specific mechanisms.


Overall, these additions strengthen the evidence that behavior-aligned EEG can provide useful training-time supervision for VLA models across simulated and physical manipulation settings, without modifying deployment inputs.

Dear Reviewers,

As the AC–reviewer discussion phase is about to begin, please highlight any disagreements or inconsistencies across the reviews, as well as any concerns that remain unresolved after the author rebuttal.

Please share your perspectives on these points and engage with one another to help us reach a well-supported assessment of the paper.

Thank you for your time and contributions.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.