- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a framework to decode continuous motion from EEG. It is seperated into two steps: 1. decode the discrete motion primities, 2. a geometric refinement stage to turn discrete sequences to continous motion trajectory. Experiments on three different datasets validate the effectiveness of it.
Critical strengths
- Discrete codebook decoding is insightful, I see it is the first method to achieve a positive on FBM dataset.
- Paper is written clearly, figure is easy to understand.
- Ablation is sufficient.
Critical weaknesses
- Why ablation study uses different subject combination? Table 2 and Table 3 is using 4 and 2 subjects? It is confusing.
- Figure 6 shows the data has already started clustering quite well, does this suggest that the codebook is actually learning stronger spatiotemporal correlations—such as those associated with sessions—rather than specific action patterns?
Questions for authors
- Use \citep{xx} in your latex rather than \cite{xx}, ICLR template is different.
- I see table 4, the codebook size is not big compared to LaBraM, so I want to see the visualization of all learned codebook unit, not t-SNE, but the decoding results, such as Figure 1's motion, since it is the prototype of a motion, can it be done?
4 comments
Response to Reviewer PjxJ(2/2)
Q4. Given the small codebook, can the decoded movements of individual codebook entries be visualized?
First, LaBraM and NeuroAction model different signals and cover different scopes, so their codebook sizes are not directly comparable. LaBraM [1] targets large-scale EEG representation learning across multiple datasets, and its codebook represents EEG. Our codebook instead learns motion primitives from each dataset’s trajectories. Given the relatively limited scope of our movement tasks and training data, the key question is whether codebook capacity suits the motion representation and its decodability from EEG, rather than whether a larger codebook is inherently better.
Table 4 and Appendix D.1 evaluate this choice through codebook-capacity ablations. Among the tested capacities, FBM, TM, and FR achieve their best mean performance with K = 32, 16, and 12, respectively. Further increasing codebook size reduces performance, supporting the use of smaller codebooks for the current tasks.
We have added decoded outputs of eight representative FBM codebook entries in Appendix D.4, “Decoded Motion-Code Visualization.” Each code is individually supplied to the motion decoder to generate a local sequence of four frames, with the second frame displayed as the representative pose.
These poses show the outputs obtained by decoding individual codes in isolation; they do not represent unique, fixed motion templates. During trajectory reconstruction, the motion decoder applies temporal convolutions and upsampling to the complete primitive sequence, so the output associated with a code also depends on neighboring primitives. Consequently, even when the central code remains unchanged, different preceding and following codes may produce different local trajectories.
References
[1] Jiang, W.-B., Zhao, L.-M., & Lu, B.-L. (2024). Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI. International Conference on Learning Representations. https://arxiv.org/abs/2405.18765
Reply
-
I don't recognize the anwer to Q1, I understand that the collecting quality has large influence on the results, but it is not the reason that "we omit part of subjects when we doing ablation study". In contrast, you could list all subjects results (some in appendix) to show that generally your model is good. Overall you give more results, this is good.
-
Q2 and Q3 has been set.
-
I appreciate the new figure give in appendix D.4, but I am not asking why your codebook size is small. I fully know that LaBraM's codebook can not be visualized, but your codebook can be visualized, so I just wonder what's the visualization results. Just curious about the probablity that "codebook learns prototype of motion in EEG".
Overall, I decide to maintain my score.
Response to Reply
Thank you for your constructive feedback.
Regarding Q1, our main comparisons cover all included subjects, with individual results already reported in Appendix D.6 (Tables 12, 14, and 15). Given the training cost of numerous ablation configurations, we selected subsets spanning different performance levels and listed their subject IDs. Similar designs appear in fMRI and EEG research: MindEye2 uses ten sessions from Subject 1 for its Section 3.5 ablations [2], while CAST uses four subjects spanning high to very low EEG-to-iEEG reconstruction performance [3].
Thank you again for confirming that Q2 and Q3 have been resolved, and for your positive comments on the additional results and the motion-code visualization in Appendix D.4. We appreciate your time and constructive feedback.
References
[2] Scotti, P. S., et al. (2024). MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 44038–44059. https://proceedings.mlr.press/v235/scotti24a.html
[3] Pham, T.-D., & Tran, X.-T. (2026). Cross-Subject Intracranial EEG Reconstruction from Scalp Recordings Using Multi-Scale Cross-Attention Transformers. arXiv preprint arXiv:2605.18897. https://arxiv.org/abs/2605.18897
Response to Reviewer PjxJ(1/2)
We thank you for the thoughtful assessment and constructive feedback. We address the comments and questions below.
Q1. Why do Tables 2 and 3 use different subject subsets?
Tables 2 and 3 serve different experimental purposes, and their subject subsets were originally selected independently, resulting in different subject combinations and sample sizes.
Table 2 presents component ablations examining the roles of alignment, E2M, and staged training, using four subjects per dataset spanning different performance levels. Table 3 presents EEG input controls using temporal reversal, random shuffling, and zero input to examine the model's dependence on EEG information and its temporal structure; the original experiments included only two subjects per dataset. The former examines how individual model components contribute to decoding, whereas the latter examines whether the model effectively uses the input signals. For input controls, perturbation effects are easier to observe in recordings containing clearer and more stable movement-related information. EEG recording quality can vary across subjects because of acquisition conditions and artifacts. We therefore initially selected two higher-performing subjects whose recordings were considered more likely to retain stable decodable information.
To improve the representativeness of Table 3, we added two lower-performing subjects per dataset, expanding each subset to four subjects. The additional experiments retain the original data splits, preprocessing, and training settings, and the results are averaged together with those of the original subjects. The revised subsets are FBM S03/S04/S05/S10, FR S05/S07/S13/S15, and TM S02/S05/S07/S08.
The expanded results continue to support the original conclusions: standard EEG achieves better mean PCC, R², and RMSE than reversed, shuffled, or zero inputs across all three datasets. Mean PCC with standard input is 0.510, 0.674, and 0.517 on FBM, FR, and TM, respectively, falling to 0.010, 0.000, and 0.003 with zero input. Temporal shuffling reduces PCC to 0.083 on FBM and 0.004 on TM. FR retains a PCC of 0.511, but its R² decreases from 0.235 to −0.162 and RMSE increases from 0.234 to 0.331. Thus, including lower-performing subjects preserves the evidence that decoding depends on informative input and its temporal structure, while retaining the original finding that FR preserves more predictive information after temporal shuffling. The results have been updated in Table 3. Section 3.3 explains the experimental purposes, and the captions of Tables 2 and 3 identify their respective subject subsets.
Q2. Could the clustering in Figure 6 reflect session-related correlations rather than specific movement patterns?
Each panel in Figure 6 contains data from only one held-out test session of one subject, so the clusters cannot arise from mixing multiple sessions. The two panels correspond to FBM S10 and FR S05, respectively. We will further clarify this in the figure caption.
The figure shows EEG-token representations colored by the codebook labels of their paired motion segments, rather than the codebook vectors themselves. The original visualization selects EEG tokens associated with the four most frequently used motion codes. It therefore provides qualitative evidence of motion-primitive-related organization in EEG representations within a single test session.
Q3. Parenthetical citations following the ICLR template
Thank you for pointing this out. We have checked the manuscript against the ICLR template's citation conventions and standardized the relevant parenthetical citations using \citep{...}.