acceptodds
Under review as a conference paper at ICLR 2027

Learning Residual Kinematic Corrections for Continuous Neural Decoding

Abstract

Directly improving continuous three-dimensional kinematic decoding from non-invasive EEG remains challenging because of its low signal-to-noise ratio, variability, and non-stationarity. Deep learning architectures such as convolutional neural network–long short-term memory (CNN–LSTM) models can capture spatial and temporal dynamics for continuous kinematic decoding; however, systematic residual errors persist in decoded trajectories. We propose a two-stage decoding framework that applies a residual error learning model to perform kinematic correction on the outputs of a CNN–LSTM decoder. The residual model is trained offline without direct EEG input and instead operates on predicted kinematic trajectories to optimize movement accuracy relative to target trajectories. This design enables targeted correction of systematic decoder errors while preserving the primary neural decoding pipeline. The proposed framework was evaluated offline using data from ten participants across ten sessions of online continuous 3D motor imagery, with feedback provided in both 2D and immersive virtual reality (VR) environments. Decoding performance was quantified using Pearson correlation coefficients () and root mean square error (RMSE) along the , , and axes. Compared with the CNN–LSTM applied alone, the CNN–LSTM with residual error correction improved the mean correlation from 0.5076 to 0.7182 () in 2D and from 0.6420 to 0.7783 () in VR, corresponding to relative improvements of 41.5% and 21.2%, respectively. Correspondingly, RMSE was reduced from 0.0564 to 0.0380 in 2D () and from 0.0528 to 0.0352 in VR (), representing relative reductions of 32.6% and 33.3%, respectively. These findings demonstrate that this scalable framework enhances continuous 3D MI decoding in BCIs by correcting kinematic errors using an offline residual model without requiring additional neural data, thereby advancing applications in neurorehabilitation, prosthetics, and virtual interaction.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Brain–computer interface (BCI), electroencephalography (EEG), motor imagery, three-dimensional kinematic decoding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EPhqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes a new method for motor-imagery-based BCI using an online baseline decoder and an offline correction (or residual) model. The authors gather data from 10 participants, with 5 sessions, each including runs where the participants moved and others where they imagined movement. For the latter, MI sessions, the target was each participant’s average velocity profile across the that movement session, separately for each target. The authors then demonstrate that this correction model generally outperforms the online model, training the decoder before each session, or training it within each session. They compare the models across 2D-screen feedback and 3D VR feedback trajectories and using correlation and RMSE.

This method could be a nice advance for noninvasive BCIs. The authors further run the experiments well, as far as I can tell, and their statistical analyses make sense (see the Strengths below). But the paper also has several weaknesses that need to be addressed for it to be considered for publication at ICLR.

Critical strengths
  • The Introduction motivates the study well.

  • The methods are well explained.

  • The authors gathered a decent cohort or a conference paper (10 participants with 10 sessions each).

  • The results are clearly in favor of the FRL.

  • The EEG collection seems serious (the system and the explanation). The Vive is not state of the art, but it seems good enough for their purposes and usage.

  • The ML procedure seems fine, as far as I can tell.

  • The authors’ Limitation section is honest and real. They should be lauded for that.

Critical weaknesses
  • The authors suggest offline analysis of the residual error is a good thing because it need not involve the user. However, they only tested their model with the stage-2 residual offline, so we do not know how well the correction 2-stage decoder works in practice—online and in real time. This is a major disadvantage for an ICLR paper.

    • To their credit, the authors mention it in the limitations, though a bit obliquely. The authors should state this more prominently in the Abstract and possibly in Fig. 1’s caption.

    • This method is further problematic because while the baseline model was trained on 100% assisted data, sessions S2-S5 were on partially assisted data. So, the test was pseudo-online replay only, also because the correction model was never used in real time.

    • In this regard, it is important to ask whether the windows that fed the offline decoder were strictly causal, not containing information from the future? It was not clear enough in the paper.

    • They tested the error model against neural data that was generated while the user was seeing feedback data that increasingly blended the decoder’s output (from the baseline model) with the target trajectory. It is unclear why the authors chose this protocol. They should motivate it and explain why it is not making their residual decoder’s job easier, especially because they do not test it online.

  • The lack of online results leads to another issue. Reaching movements are quite stereotyped, with the same speed profiles across trials. This is also true here (see Appendix E), especially because the target trajectories were here averaged for each participant. (It would have been different had the participants used MI to run actual BCIs and had then been tested on how well the results matched what they were attempting to achieve. Hence, the EEG from the imagined movement on every trial was compared against the same average trajectory. And the model is given information about the time steps and the decoder. If the variance is low, can we be sure that the correction model does not just memorize the average pattern (minus the decoder values) and override the decoder? This is possible given that the velocities are scaled to 0-0.3 but the decoder can scale up to values up to 1. Then, the network is trained to output values that are as close as possible to these stereotyped velocities, so it might be trained to do the above. If this interpretation is correct, a user in a closed-loop BCI setting would not be able to steer it. This could only be tested online and in real time.

    • This interpretation could explain why FRL tends to outperform the other measures in Fig. 2. Predicting the average curve might also explain why the FRL results are stable while the other measured move around much more.

    • We also see higher scores in Y and Z and less in X, which seems to be in line with the results in Appendix E, where the Y and Z curves might be more stereotypical than the X curve.

  • To test this, it would be good to do three things.

    • See the chosen scale (or “s”) values to see whether they are large.

    • The authors could also provide the correction model with shuffled results during training, to decorrelate it from the decoder and see what happens.

    • Finally, what happens if the residual decoder receives only time values? How much would that decrease the very high correlations in the Y direction and somewhat how ones in the Z direction?

  • FRL is FDG with residual training, so a fairer comparison to SAT and WSR would be to apply residual training on those two models too.

  • I found Fig. 1 more confusing than helpful. Better explanation in the caption and the text would help a lot.

  • It was hard for me to understand what the target was for the residual model. I finally figured it out, but it rook quite a bit of reading and re-reading. Also, please explain why you used decoder assistance for the feedback to the user. And why does the assistance stay at 40% of higher throughout? Has the model never gotten good enough to produce feedback on its own?

  • It is unclear, to me at least, whether this result important and non-incremental enough for the community to be featured in a top-tier conference like ICLR.

Questions for authors

The questions are included in the Weaknesses section.

4 comments

We appreciate your time and thank you for your careful reviews.

1.Response to first weakness part

Many studies on motor imagery BCIs evaluate their methods offline, including studies that use publicly available recordings. In our paper, we identify real-time evaluation of the residual corrector as future work. As stated in the paper, our work extends the experimental paradigm introduced in [1]. The authors of [1] collected the dataset, trained the CNN–LSTM decoder, and used it for online adaption of S2-S5. Given the early stage of 3D MI BCI research and, to our knowledge, the absence of another comparable ten-session dataset, this dataset provides a suitable basis for establishing our proposed FRL approach. We use their dataset and adapt their experimental paradigm to conduct a full offline evaluation of our proposed residual correction method in a pseudo-online setting across eight unseen sessions, a standard step before proceeding to online evaluation.

In the CNN–LSTM implementation, the decoder uses backward-looking input windows. For a prediction associated with time t, the model uses ERSP features from the current window and from earlier windows, all of which end at or before t. Temporal resizing is performed within each selected window. EEG filtering uses only current and past samples, and spectral features are computed from windows ending at their corresponding time points. So all decoding within a trial is causal and all test session are tested causally aligning to 8 different sessions.

Trajectory assistance is used to provide a degree of stability, gradually ease participants into learning to control the visual feedback and prevent erratic movement. The decoder used in S2–S5 does not consistently generate accurate trajectories toward the targets, as it does not achieve a high correlation with the target trajectory. Trajectory assistance therefore provides participants with positively biased feedback toward the targets.

The residual corrector only receives the decoder predictions and temporal context as input. Target trajectories are used only as training labels, and the residual corrector is evaluated on test sessions. Trajectory assistance does not directly provide target information to the residual corrector during inference. Moreover, to address the reviewer’s concern that trajectory assistance may facilitate residual correction, we hypothesised that higher assistance levels would be associated with larger FRL gains over FDG. We examined this prediction across the held-out sessions S2–S5. The table reports the percentage increase in mean Pearson correlation and the percentage reduction in RMSE relative to FDG (Figs. 2–3 in our paper).

SessionTrajectory assistance2D correlation increase2D RMSE reductionVR correlation increaseVR RMSE reduction
S265%42.8%32.9%18.2%34.1%
S360%39.7%33.3%24.7%33.5%
S450%45.5%34.4%22.2%32.6%
S540%38.3%30.0%20.1%33.2%

We have also provided the SAT+FRL and WSR + FRL results across sessions (See responses in the next part). Because assistance level is tied to session, these comparisons are descriptive. Across the tested sessions/trials, the results provide no consistent evidence that stronger trajectory assistance was associated with larger FRL gains.

[1] McShane, N., Korik, A., McCreadie, K., Bois, N.D., Charles, D. and Coyle, D., 2026. Embodied Virtual Reality Feedback Reshapes Neural Representations to Support Continuous Three-Dimensional Motor Imagery Decoding. arXiv preprint arXiv:2605.29677.

2.Response to the second weakness part

Our residual does not simply memorize an average pattern. The explanations are below:

1)We provide the mean scale values of each axis from our original optuna search:

FeedbackMean scale XMean scale YMean scale Z
2D0.64210.71050.6372
VR0.53060.61500.4478

We found that setting the scale values too low (below 0.1) reduced the performance of the residual corrector because the correction was very small relative to it. We therefore set the scale range to 0.1–1.0. To address the reviewer’s concern that large scale values could override the decoder’s output, we also report results from retraining the residual corrector for all participants and feedback conditions with the scale values for all axes fixed at 0.1 and, separately, at 0.01. FRL with these low scale values still showed a clear advantage over FDG, although the advantage was smaller than that of the original FRL.

FeedbackMetricMethodXYZMeanp
2DCorrFDG0.1209 ± 0.68850.7469 ± 0.32080.6551 ± 0.36780.5076 ± 0.3206—
2DCorrFRL(scale = 0.1)0.2666 ± 0.67230.9739 ± 0.04590.8772 ± 0.20140.7059 ± 0.24360.0001
2DRMSEFDG0.0557 ± 0.02900.0581 ± 0.02290.0555 ± 0.02330.0564 ± 0.0158—
2DRMSEFRL(scale = 0.1)0.0494 ± 0.02520.0339 ± 0.01670.0333 ± 0.01650.0389 ± 0.0099<0.0001
VRCorrFDG0.2492 ± 0.68540.8455 ± 0.22430.8316 ± 0.23660.6420 ± 0.2870—
VRCorrFRL(scale = 0.1)0.3856 ± 0.66860.9718 ± 0.04980.9677 ± 0.05250.7751 ± 0.2264<0.0001
VRRMSEFDG0.0533 ± 0.03180.0550 ± 0.02410.0502 ± 0.02280.0528 ± 0.0171—
VRRMSEFRL(scale = 0.1)0.0491 ± 0.02880.0330 ± 0.01770.0263 ± 0.01500.0361 ± 0.0125<0.0001
FeedbackMetricMethodXYZMeanp
2DCorrFDG0.1209 ± 0.68850.7469 ± 0.32080.6551 ± 0.36780.5076 ± 0.3206—
2DCorrFRL(scale = 0.01)0.1613 ± 0.67960.8022 ± 0.26360.7169 ± 0.31620.5601 ± 0.29060.0086
2DRMSEFDG0.0557 ± 0.02900.0581 ± 0.02290.0555 ± 0.02330.0564 ± 0.0158—
2DRMSEFRL(scale = 0.01)0.0532 ± 0.02790.0530 ± 0.02250.0508 ± 0.02230.0524 ± 0.0149<0.0001
VRCorrFDG0.2492 ± 0.68540.8455 ± 0.22430.8316 ± 0.23660.6420 ± 0.2870—
VRCorrFRL(scale = 0.01)0.2971 ± 0.67020.8743 ± 0.20010.8653 ± 0.20300.6789 ± 0.26860.0001
VRRMSEFDG0.0533 ± 0.03180.0550 ± 0.02410.0502 ± 0.02280.0528 ± 0.0171—
VRRMSEFRL(scale = 0.01)0.0511 ± 0.03060.0497 ± 0.02320.0449 ± 0.02210.0486 ± 0.0161<0.0001

2)We trained the residual model on shuffled decoded–target trajectory pairs, ensuring that each decoded trajectory was paired with a reference trajectory for a different target. The model trained on shuffled pairs showed a clear drop in performance compared with the original FRL. This suggests that correct pairing contributes to FRL’s performance and provides evidence against the explanation that FRL simply memorises a single average trajectory shared across targets.

FeedbackMetricMethodXYZMeanp
2DCorrFRL0.2865 ± 0.62640.9829 ± 0.01800.8850 ± 0.10310.7182 ± 0.21670.0015
2DCorrFRL(shuffled)0.2496 ± 0.68920.9135 ± 0.11690.8082 ± 0.24730.6571 ± 0.2629—
2DRMSEFRL0.0497 ± 0.02550.0325 ± 0.01640.0318 ± 0.01670.0380 ± 0.01010.0004
2DRMSEFRL(shuffled)0.0501 ± 0.02560.0415 ± 0.02060.0417 ± 0.01860.0444 ± 0.0113—
VRCorrFRL0.3819 ± 0.65950.9786 ± 0.03620.9744 ± 0.04190.7783 ± 0.22180.0059
VRCorrFRL(shuffled)0.3588 ± 0.66250.9153 ± 0.12320.9005 ± 0.16210.7249 ± 0.2411—
VRRMSEFRL0.0491 ± 0.02950.0316 ± 0.01780.0248 ± 0.01450.0352 ± 0.01330.0014
VRRMSEFRL(shuffled)0.0527 ± 0.02970.0407 ± 0.01910.0375 ± 0.01930.0436 ± 0.0140—

3)Compared with full FRL, time-only FRL showed a clear drop in performance along the Y and Z axes in both feedback conditions. In 2D, Pearson correlation decreased by 0.0643 along Y and 0.0701 along Z, with corresponding RMSE increases of 31.4% and 36.5%. In VR, Pearson correlation decreased by 0.0501 along Y and 0.0617 along Z, with corresponding RMSE increases of 26.6% and 48.8%. Please see our response to reviewer 3QZw for the full per-axis results.

The results of SAT + FRL and WSR + FRL are provided. +FRL significantly improves both SAT and WSR in every condition and metric. The hyperparameters setting follows those of FDG for each participant and feedback. Since WSR involves training and testing within the same session, we split each session’s data into an 80% training set and a 20% test set. The reported results compare mean performance of WSR and WSR+FRL on the test set.

FeedbackMetricMethodXYZMeanp
2DCorrSAT0.1647 ± 0.69450.7894 ± 0.27190.7149 ± 0.31800.5563 ± 0.3029—
2DCorr+FRL0.2967 ± 0.66400.9805 ± 0.02740.9045 ± 0.15670.7272 ± 0.2319<0.0001
2DRMSESAT0.0596 ± 0.03180.0597 ± 0.02710.0580 ± 0.02500.0591 ± 0.0191—
2DRMSE+FRL0.0499 ± 0.02650.0339 ± 0.01760.0315 ± 0.01720.0384 ± 0.0109<0.0001
VRCorrSAT0.3019 ± 0.67270.8245 ± 0.26470.8090 ± 0.26890.6451 ± 0.2977—
VRCorr+FRL0.4033 ± 0.65720.9771 ± 0.03670.9614 ± 0.07160.7806 ± 0.2217<0.0001
VRRMSESAT0.0546 ± 0.03410.0574 ± 0.02570.0515 ± 0.02390.0545 ± 0.0197—
VRRMSE+FRL0.0474 ± 0.03010.0314 ± 0.01920.0256 ± 0.01580.0348 ± 0.0145<0.0001
SessionTrajectory assistance2D correlation increase2D RMSE reductionVR correlation increaseVR RMSE reduction
S265%42.8%32.9%18.2%34.1%
S360%32.9%37.3%41.0%42.6%
S450%18.4%27.1%12.7%31.9%
S540%31.3%40.8%15.8%34.6%
FeedbackMetricMethodXYZMeanp
2DCorrWSR0.3787 ± 0.68080.8255 ± 0.25280.7776 ± 0.27550.6606 ± 0.3047—
2DCorr+FRL0.4589 ± 0.63680.9827 ± 0.02850.9342 ± 0.11420.7919 ± 0.2217<0.0001
2DRMSEWSR0.0500 ± 0.03550.0547 ± 0.03480.0536 ± 0.03320.0528 ± 0.0288—
2DRMSE+FRL0.0415 ± 0.02680.0290 ± 0.01750.0266 ± 0.01640.0324 ± 0.01360.0002
VRCorrWSR0.5342 ± 0.60890.8848 ± 0.17150.8740 ± 0.18370.7643 ± 0.2489—
VRCorr+FRL0.5973 ± 0.57830.9811 ± 0.03090.9711 ± 0.06680.8498 ± 0.1990<0.0001
VRRMSEWSR0.0426 ± 0.03230.0488 ± 0.02350.0436 ± 0.02340.0450 ± 0.0195—
VRRMSE+FRL0.0369 ± 0.02860.0260 ± 0.01970.0209 ± 0.01600.0279 ± 0.0163<0.0001
SessionTrajectory assistance2D correlation increase2D RMSE reductionVR correlation increaseVR RMSE reduction
S1100%21.6%31.9%9.4%34.4%
S265%18.9%37.9%19.9%42.3%
S360%10.1%30.4%7.9%35.7%
S450%17.7%41.4%9.7%37.6%
S540%33.7%48.2%9.8%39.1%

The target used to train the residual model is averaged executed trajectories in real runs, as written in the first paragraph in Section 2.1 in the paper. The residual action is added to decoded output and the parameters of residual model are updated using these targets with Equations (3) and (4).

We agree to add a paragraph describing the experimental paradigm below Fig. 1 and to provide a clearer and more detailed explanation of the figure in the main text. We will upload a revised manuscript incorporating these changes. The reviewer has raised several concerns about the experimental paradigm described in our paper. The paradigm serves as a setting for evaluating our method. The focus of our paper is the residual correction method itself. The proposed corrector operates on neural decoder outputs and improves performance across all baseline decoders we tested. We additionlly reproduce another experimental protocol described in [1] using the same publicly available WAY-EEG-GAL dataset [2] and extend their protocol to evaluate our residual learning method. Adding the residual corrector provides statistically significant improvements over the baselines in all reported comparisons in this paradigm. For rEEGNet, rDeepConvNet, and rShallowConvNet, adding FRL increases mean Pearson correlation by 0.0487, 0.0265, and 0.0691, respectively, with corresponding reductions in mean RMSE of 15.9%, 7.5%, and 28.1%. Please see our second part response to Reviewer e7aZ for the detailed comparisons.

[1] Jain, A. and Kumar, L., 2025. ESI-GAL: EEG source imaging-based trajectory estimation for grasp and lift task. Computers in Biology and Medicine, 186, p.109608. [2] Luciw, M.D., Jarocka, E. and Edin, B.B., 2014. Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and friction. Scientific data, 1(1), p.140047.

Reviewer_e7aZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper considers continuous 3D kinematic decoding from non-invasive EEG signals. It proposes a two-stage neural decoding architecture that augments an existing CNN–LSTM decoder with a neural network trained to predict residual errors and correct the resulting kinematic trajectories. The residual correction model operates on the predicted trajectories without requiring additional EEG measurements. The proposed approach is evaluated using data collected from ten participants performing motor imagery tasks in 2D and VR environments.

Critical strengths

The main strength of the paper is its experimental validation, demonstrating considerable improvements in decoding accuracy across both 2D and VR environments. The reported gains suggest that learning residual corrections can be an effective approach for improving EEG-based continuous kinematic decoding.

Critical weaknesses
  • My primary concern is the rather limited methodological contribution of the paper, particularly considering the standards of ICLR. The proposed approach essentially consists of augmenting an existing CNN–LSTM architecture with an additional neural network trained to correct its prediction errors. Residual learning and neural post-processing are well-established concepts, and their application in this setting does not appear to introduce a substantially new learning methodology or provide new insights into the underlying learning problem. The technical contribution appears rather incremental and primarily application-oriented.

  • The paper is primarily motivated by the specific application of EEG-based kinematic decoding, and its literature review focuses largely on biomedical and BCI-related works, with limited engagement with methodological developments from the broader machine learning community. Consequently, the paper does not sufficiently establish how the proposed approach advances the state of the art in machine learning beyond its particular application. Overall, I believe that its contributions would be better suited to a specialized biomedical engineering or neural engineering venue than to ICLR.

Questions for authors
  • Could the authors clarify what aspects of the proposed residual correction approach constitute a methodological contribution beyond applying a trainable neural network to correct the prediction errors of an existing decoder? What distinguishes the proposed method from established residual learning and neural post-processing approaches?
  • To what extent is the proposed residual correction framework specific to the considered CNN–LSTM architecture and EEG-based kinematic decoding task? Is there evidence that the approach provides benefits beyond this particular decoder and experimental setting?
4 comments

We appreciate your time and thank you for your careful reviews.

1.Response to the highlighted weaknesses and the first question

As discussed in the Introduction, residual learning has been widely used in vehicle modelling and robotic control to compensate for errors in baseline models and controllers. Its application to correcting kinematic decoding errors in BCI, however, remains unexplored in any paper or context. Here, we propose an alternative method designed for this BCI problem instead of directly decoding kinematics from EEG and investigate it in the challenging setting of continuous 3D MI kinematics decoding.

The learning target and purpose of the residual component differ across these settings. In robotic residual reinforcement learning [5-8], the residual policy learns additional control actions that improve task performance by maximising expected rewards. This does not necessarily involve regressing a known prediction error. In vehicle modelling [9-11], residual correction compensates for errors arising from an incomplete dynamic system. The noisy EEG makes accurate EEG-to-kinematics decoding challenging. Motivated by this challenge, our approach treats neural decoding outputs as a kinematic sequence and learns to compensate for prediction errors left by the EEG-to-kinematics mapping.

Specifically, the novelty of our approach lies in using the trajectory produced by any baseline decoder to directly estimate its kinematic errors, without feeding EEG signals into the residual model. Conventional post-processing simply applies a filter to smooth the predictions, without training an additional network to correct them [1-4]. We also identified several most related studies in the field of BCI kinematics decoding, which may include training a second model but none involve “residual learning”, and their methods and experimental paradigms are completely different from ours.

PaperTaskMethod
Lee et al. (2025)Continuous 2D cursor velocity controlA learned module downstream of the EEG decoder used the decoded velocity and cursor position to assign weights to predefined candidate goals. A pre-defined potential field converted those weights into an assistive velocity, which was blended with the decoded velocity.
Pulferer et al. (2024)Continuous 2D cursor position decoding and correction during attempted movementThe paradigm involved deliberately showing users erroneous or delayed feedback. A second model was trained to estimate the position error of the displayed feedback relative to the target from EEG error-related potentials.
Mondini et al. (2020)Continuous 2D hand kinematics decoding during executed movementPartial least squares regression was used to extract latent components from low-frequency EEG. A Kalman filter used these components as observations and incorporated a movement dynamics model to estimate 2D hand positions, velocities, and accelerations.
Mondini et al. (2024)Continuous 2D trajectory decoding and feedback error detection during attempted movementPLS regression was combined with an unscented Kalman filter to decode continuous trajectories. A separate EEG classifier was trained to detect erroneous feedback.
Hosseini and Shalchyan (2023)Continuous 2D hand trajectory decoding during executed reachingFor movements toward orthogonal targets, pre-movement EEG was used to classify the movement axis and select the corresponding Gaussian process regressor for continuous trajectory decoding.
FRL (ours)Continuous 3D upper-limb velocity decoding during motor imageryFRL directly predicts a 3D velocity residual from a baseline decoder’s output and temporal context. It learns the correction without a pre-defined component, deliberately generated erroneous feedback, or EEG error-related potentials. It does not assign weights to targets and classify them. None of the other studies above involve residual learning and none are evaluated on 3D MI data.

Furthermore, we have provided many additional experiments comparing the baselines with and without FRL in different experimental settings. Please see the response in the next part and also the response to other reviewers. All comparisons show that our residual corrector substantially improves baseline performance. Since ICLR accepts work on BCIs and our methods achieve SOTA results compared to the available literature, we believe that our paper makes a strong contribution to ICLR and will be of interest on both AI and BCI researchers.

[1] Wang, J., Bi, L., Fei, W. and Tian, K., 2022. EEG-based continuous hand movement decoding using improved center-out paradigm. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, pp.2845-2855.

[2] Lv, J., Li, Y. and Gu, Z., 2010. Decoding hand movement velocity from electroencephalogram signals during a drawing task. Biomedical engineering online, 9(1), p.64.

[3] Heger, D., Jäkel, R., Putze, F., Lösch, M. and Schultz, T., 2012, August. Filling a glass of water: continuously decoding the speed of 3D hand movements from EEG signals. In 2012 annual international conference of the IEEE engineering in medicine and biology society (pp. 4095-4098). IEEE.

[4] Korik, A., Sosnik, R., Siddique, N. and Coyle, D., 2019. Decoding imagined 3D arm movement trajectories from EEG to control two virtual arms—a pilot study. Frontiers in neurorobotics, 13, p.94.

[5] Silver, T., Allen, K., Tenenbaum, J. and Kaelbling, L., 2018. Residual policy learning. arXiv preprint arXiv:1812.06298.

[6] Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J.A., Solowjow, E. and Levine, S., 2019, May. Residual reinforcement learning for robot control. In 2019 international conference on robotics and automation (ICRA) (pp. 6023-6029). IEEE.

[7] Alakuijala, M., Dulac-Arnold, G., Mairal, J., Ponce, J. and Schmid, C., 2021. Residual reinforcement learning from demonstrations. arXiv preprint arXiv:2106.08050.

[8] Davchev, T., Luck, K.S., Burke, M., Meier, F., Schaal, S. and Ramamoorthy, S., 2022. Residual learning from demonstration: Adapting dmps for contact-rich manipulation. IEEE Robotics and Automation Letters, 7(2), pp.4488-4495.

[9] Jiang, S., Lin, W., Cao, Y., Wang, Y., Miao, J. and Luo, Q., 2021, September. Learning-based vehicle dynamics residual correction model for autonomous driving simulation. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (pp. 782-789). IEEE.

[10] Long, K., Sheng, Z., Shi, H., Li, X., Chen, S. and Ahn, S., 2025. Physical enhanced residual learning (PERL) framework for vehicle trajectory prediction. Communications in Transportation Research, 5, p.100166.

[11] Miao, J., Yan, R., Zhang, B., Wen, T., Li, J., Fu, Z., Jiang, K., Yang, M., Huang, J., Zhong, Z. and Yang, D., 2025. Residual learning towards high-fidelity vehicle dynamics modeling with transformer. IEEE Robotics and Automation Letters, 10(7), pp.7404-7411.

Lee, J.Y., Lee, S., Mishra, A., Yan, X., McMahan, B., Gaisford, B., Kobashigawa, C., Qu, M., Xie, C. and Kao, J.C., 2025. Brain–computer interface control with artificial intelligence copilots. Nature machine intelligence, 7(9), pp.1510-1523.

Pulferer, H.S., Kostoglou, K. and Müller-Putz, G.R., 2024. Improving non-invasive trajectory decoding via neural correlates of continuous erroneous feedback processing. Journal of Neural Engineering, 21(5), p.056010.

Mondini, V., Kobler, R.J., Sburlea, A.I. and Müller-Putz, G.R., 2020. Continuous low-frequency EEG decoding of arm movement for closed-loop, natural control of a robotic arm. Journal of Neural Engineering, 17(4), p.046031.

Mondini, V., Sburlea, A.I. and Müller-Putz, G.R., 2024. Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing. Scientific Reports, 14(1), p.4714.

Hosseini, S.M. and Shalchyan, V., 2023. State-based decoding of continuous hand movements using EEG signals. IEEE Access, 11, pp.42764-42778.

2.Response for the second question

In addition to the CNN–LSTM, we evaluate a CNN–Transformer that we designed and other architecture such as EEGNet and DeepConvNet used for regression [1]. All three baselines follow the same hyperparameter tuning, training, and evaluation protocol as the CNN–LSTM under the FDG setting. FRL increase Pearson correlation and reduced RMSE along all three axes for each architecture, including baselines with stronger and weaker overall performance than the CNN–LSTM. Improvements in the across-axis(xyz) averages are statistically significant for both metrics under both feedback conditions. Relative to each decoder’s baseline, FRL increase mean Pearson correlation by 0.1180/0.1298 for CNN–Transformer, 0.5886/0.6261 for EEGNet, and 0.6352/0.6497 for DeepConvNet, with corresponding mean RMSE reductions of 29.9%/28.8%, 55.1%/57.6%, and 67.6%/64.8%, respectively, with each pair given in 2D/VR order. Please see our response to Reviewer 3QZw for the detailed comparisons.

In addition, we reproduce the experimental protocol described in [1] using the same publicly available WAY-EEG-GAL motor execution dataset [2] and extend the protocol to evaluate our residual learning method. As shown in the tables below, adding the residual corrector also provides statistically significant improvements over the baseline in all reported comparisons on the motor execution dataset.

MetricMethodXYZMeanp
CorrrEEGNet0.6459 ± 0.09640.6499 ± 0.10230.6089 ± 0.08980.6349 ± 0.0917—
Corr+FRL0.6977 ± 0.09900.7029 ± 0.10310.6502 ± 0.10440.6836 ± 0.0983<0.0001
RMSErEEGNet0.0558 ± 0.01130.0604 ± 0.01180.0595 ± 0.00810.0585 ± 0.0090—
RMSE+FRL0.0475 ± 0.00760.0488 ± 0.00820.0513 ± 0.00550.0492 ± 0.0066<0.0001
MetricMethodXYZMeanp
CorrrDeepConvNet0.6273 ± 0.10540.6332 ± 0.11230.6005 ± 0.09920.6203 ± 0.1011—
Corr+FRL0.6543 ± 0.10440.6608 ± 0.10990.6252 ± 0.09170.6468 ± 0.0968<0.0001
RMSErDeepConvNet0.0538 ± 0.00990.0574 ± 0.01120.0572 ± 0.00880.0561 ± 0.0082—
RMSE+FRL0.0506 ± 0.00790.0520 ± 0.00890.0531 ± 0.00510.0519 ± 0.00670.0112
MetricMethodXYZMeanp
CorrrShallowConvNet0.5565 ± 0.09580.5612 ± 0.10050.5534 ± 0.08630.5570 ± 0.0882—
Corr+FRL0.6304 ± 0.10150.6361 ± 0.10490.6119 ± 0.09790.6261 ± 0.0955<0.0001
RMSErShallowConvNet0.0758 ± 0.01030.0844 ± 0.01170.0623 ± 0.00400.0741 ± 0.0072—
RMSE+FRL0.0524 ± 0.00750.0538 ± 0.00760.0539 ± 0.00440.0533 ± 0.0059<0.0001

[1] Jain, A. and Kumar, L., 2025. ESI-GAL: EEG source imaging-based trajectory estimation for grasp and lift task. Computers in Biology and Medicine, 186, p.109608. [2] Luciw, M.D., Jarocka, E. and Edin, B.B., 2014. Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and friction. Scientific data, 1(1), p.140047.

Reviewer_e7aZOpenReview ↗

Thank you for the reply

I thank the authors for their detailed response and additional experimental results. I must admit that my primary concern remains regarding the methodological contribution. The rebuttal further clarifies that the proposed approach essentially applies established residual learning principles to the specific problem of EEG-based kinematic decoding. While the reported improvements are potentially valuable for BCI applications, they do not establish a novel learning methodology or broader insights of significance to the machine learning community. I therefore maintain my original assessment, as I believe the work would be better suited to a specialized biomedical or neural engineering venue.

Reviewer_3QZwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper analyzes the problem of wrist velocity prediction based on EEG signals captured from a g.Nautilus system.

The authors propose a two-stage architecture, in which a CNN-LSTM BCI decoder is trained and frozen, followed by the training of an MLP for error correction based on residual learning.

Data was collected with 10 right-handed individuals. Each individual collected one session in a different day for a total of 10 days. Half of the sessions were collected with 2D visual feedback and the other half was collected with VR feedback. For each feedback and each user, a separate pair of BCI decoder and residual network was trained using a single session (with 5-fold validation for hyperparameter search). All the other sessions were used as test split.

The result was evaluated based on (1) the Pearson correlation between the decoded trajectory and the executed-movement average trajectory, measured with the Vive tracker; and (2) average RMSE across trials and dimensions with the same trajectory as reference. All results are further validated with a statistical analysis and the improvement was found significant.

Critical strengths

The paper is well presented, the proposed architecture outperforms the alternatives and statistical results are provided to support the advantage.

Critical weaknesses

The paper limits itself to a simplified problem of synchronous BCI. Although acknowledged by the "Limitations and Future Work" section, it restricts the relevance of the proposed solution.

The training and testing targets are located in the same positions, which disallows the comparison of the effect of the proposed approach on generalization of movement.

Lastly, the residual network adds complexity and representation power to the CNN-LSTM decoder, but all the comparisons keep the original CNN-LSTM decoder. It would be beneficial having an additional comparison validating whether the improvement comes from the residual learning approach or from the expressiveness of the overall model. At an extreme case, the residual network could be memorizing a time-table (given it receives the time tick as input) to correct the trajectory.

Not critical, but would be good for the paper:

  1. Provide the resulting hyperparameters found by optuna search
  2. Describe whether or not the 5-fold training was split with stratification of trajectory
Questions for authors

Given the architecture, the residual network could be memorizing a time-table. Can the authors provide per-axis results when the residual network receives (a) only the time variable as input, and (b) all inputs except time? If (a) approaches FRL’s performance, or (b) falls back toward FDG’s, the improvement would come mostly from learning the average trajectory over time rather than from correcting decoder errors.

6 comments

We appreciate your time and thank you for your careful reviews.

1.Response to Weaknesses

The reviewer suggest that this is a simplified synchronised BCI paradigm but very few paper have tackled 3D motor imagery decoding. The majority of the 3D decoding research to date focuses on 3D movement decoding and not motor imagery and to our knowledge, there is no comparable 3D MI dataset that has 10 sessions (8 with real time feedback involving either 2D or VR and thousands of trials per participant). Because of the challenges associated with this problem, cue-synchronised protocols are very commonly used in EEG-based continuous kinematics decoding [1–8].

Given the current SOTA and the challenges associated with decoding imagined 3D movements of a single limb in four directions from EEG, we have highlighted generalisation to unseen targets as an important direction for future work in the Limitations section. In the field of continuous kinematics decoding, the common evaluation approaches are to hold out a subset of trials [1][2][3][9] or to leave one participant out [3][10][11][12]. In our evaluation we perform 5 fold cross validations on two sessions to train the residual corrector and evaluate on completely unseen data on 8 sessions which is a rigorous test of generalization performance.

To optimize the residual corrector framework, five-fold cross-validation was performed at the trial-trajectory level using random shuffling. Four folds were used for training and the remaining fold for validation, rotating through all five folds. Each trial’s entire trajectory was assigned to one fold, keeping its time points together. No stratification by trajectory type or target direction was applied. The hyperparameters found by optuna search are provided. Please see the below where we give the hyperparameters chosen each participants at the first 2D and first VR session:

participant_feedbacklr (×10⁻⁵)hidden_dimbatch_sizex_scale_biasy_scale_biasz_scale_biasnum_layers
P1_2D1.81592561280.91180.84420.56703
P2_2D1.52952561280.93790.94520.97653
P3_2D1.3699512640.31330.86590.79552
P4_2D2.2057128640.22100.91500.63724
P5_2D1.2132128640.78130.46090.81013
P6_2D4.18772561280.66380.69210.35683
P7_2D9.57065121280.66190.78760.79292
P8_2D14.26712561280.46260.55130.35732
P9_2D2.2644256640.85130.42630.53802
P10_2D1.56445122560.61620.61590.54064

participant_feedbacklr (×10⁻⁵)hidden_dimbatch_sizex_scale_biasy_scale_biasz_scale_biasnum_layers
P1_VR1.43495121280.59330.65650.63444
P2_VR4.8908512640.60880.96640.20852
P3_VR1.7143256640.24480.32790.50692
P4_VR3.15691281280.48550.67310.32473
P5_VR6.46215121280.63150.33680.71082
P6_VR9.00261281280.79590.24110.28774
P7_VR2.66105121280.65420.96550.49463
P8_VR1.5858128640.27070.97440.67103
P9_VR1.1734256640.21020.40580.27163
P10_VR1.6534256640.81080.60280.36732

As the reviewer points out that all the comparisons keep the original CNN-LSTM decoder as reported in [14], to test if the improvement comes from residual learning, we evaluated the proposed FRL on three additional architectures, using a CNN–Transformer that we design and regression variants of EEGNet and DeepConvNet [13] that we reproduce. All three baselines follow the same hyperparameter tuning, training, and evaluation protocol as the CNN–LSTM under the FDG setting. FRL significantly increase correlation and reduce RMSE across all three architectures, regardless of whether the baseline before correction perform better or worse than the CNN–LSTM. We have also evaluated residual learning in another experimental paradigm, please see the second part of response to Reviewer e7aZ.

FeedbackMetricMethodXYZMeanp
2DCorrCNN-Transformer0.1630 ± 0.10860.8671 ± 0.04980.7410 ± 0.17280.5904 ± 0.0818—
2DCorr+FRL0.2639 ± 0.11320.9788 ± 0.01190.8827 ± 0.14230.7084 ± 0.0718<0.0001
2DRMSECNN-Transformer0.0632 ± 0.02260.0517 ± 0.01220.0504 ± 0.01060.0551 ± 0.0142—
2DRMSE+FRL0.0506 ± 0.00360.0330 ± 0.00460.0322 ± 0.00900.0386 ± 0.00440.0025
VRCorrCNN-Transformer0.3006 ± 0.10730.8369 ± 0.09420.8130 ± 0.14620.6502 ± 0.0995—
VRCorr+FRL0.3910 ± 0.08390.9758 ± 0.01260.9733 ± 0.02400.7800 ± 0.03090.0003
VRRMSECNN-Transformer0.0506 ± 0.00910.0494 ± 0.01280.0460 ± 0.00680.0486 ± 0.0065—
VRRMSE+FRL0.0482 ± 0.01040.0312 ± 0.00580.0243 ± 0.00840.0346 ± 0.0070<0.0001
FeedbackMetricMethodXYZMeanp
2DCorrEEGnet0.0176 ± 0.03390.1772 ± 0.11680.1516 ± 0.14190.1155 ± 0.0835—
2DCorr+FRL0.2531 ± 0.14620.9784 ± 0.01540.8808 ± 0.15760.7041 ± 0.0873<0.0001
2DRMSEEEGnet0.0727 ± 0.01730.0994 ± 0.01510.0905 ± 0.01560.0875 ± 0.0103—
2DRMSE+FRL0.0516 ± 0.00430.0336 ± 0.00450.0328 ± 0.00900.0393 ± 0.0046<0.0001
VRCorrEEGnet0.0466 ± 0.04570.1982 ± 0.09440.1975 ± 0.09020.1474 ± 0.0703—
VRCorr+FRL0.3675 ± 0.09890.9781 ± 0.01110.9748 ± 0.02330.7735 ± 0.0363<0.0001
VRRMSEEEGnet0.0707 ± 0.01600.0983 ± 0.01730.0924 ± 0.02090.0871 ± 0.0130—
VRRMSE+FRL0.0512 ± 0.00770.0338 ± 0.00380.0257 ± 0.00770.0369 ± 0.0047<0.0001
FeedbackMetricMethodXYZMeanp
2DCorrDeepConvnet0.0506 ± 0.03350.1376 ± 0.18280.0971 ± 0.16650.0951 ± 0.1219—
2DCorr+FRL0.3130 ± 0.13780.9849 ± 0.00960.8930 ± 0.13790.7303 ± 0.0761<0.0001
2DRMSEDeepConvnet0.1122 ± 0.03140.1311 ± 0.06340.1052 ± 0.02830.1162 ± 0.0320—
2DRMSE+FRL0.0490 ± 0.00440.0324 ± 0.00390.0313 ± 0.00890.0376 ± 0.0047<0.0001
VRCorrDeepConvnet0.0446 ± 0.03250.1858 ± 0.14570.1893 ± 0.15210.1399 ± 0.1024—
VRCorr+FRL0.4077 ± 0.07380.9828 ± 0.01230.9781 ± 0.01830.7896 ± 0.0272<0.0001
VRRMSEDeepConvnet0.0961 ± 0.03410.1145 ± 0.04880.0972 ± 0.02090.1026 ± 0.0283—
VRRMSE+FRL0.0497 ± 0.00650.0333 ± 0.00480.0252 ± 0.00810.0361 ± 0.0050<0.0001

[1] Kim, Y.J., Park, S.W., Yeom, H.G., Bang, M.S., Kim, J.S., Chung, C.K. and Kim, S., 2015. A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals. Biomedical engineering online, 14(1), p.81.

[2] Korik, A., Sosnik, R., Siddique, N. and Coyle, D., 2018. Decoding imagined 3D hand movement trajectories from EEG: evidence to support the use of mu, beta, and low gamma oscillations. Frontiers in neuroscience, 12, p.130.

[3] Jeong, J.H., Shim, K.H., Kim, D.J. and Lee, S.W., 2020. Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals. IEEE transactions on neural systems and rehabilitation engineering, 28(5), pp.1226-1238.

[4] Pei, D., Olikkal, P., Adali, T. and Vinjamuri, R., 2022. Reconstructing synergy-based hand grasp kinematics from electroencephalographic signals. Sensors, 22(14), p.5349.

[5] Hosseini, S.M. and Shalchyan, V., 2022. Continuous decoding of hand movement from EEG signals using phase-based connectivity features. Frontiers in human neuroscience, 16, p.901285.

[6] Wang, P., Li, Z., Gong, P., Zhou, Y., Chen, F. and Zhang, D., 2023. MTRT: Motion trajectory reconstruction transformer for EEG-based BCI decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, pp.2349-2358.

[7] Li, S., Tian, M., Xu, R., Cichocki, A. and Jin, J., 2024. Decoding continuous motion trajectories of upper limb from EEG signals based on feature selection and nonlinear methods. Journal of Neural Engineering, 21(6), p.066039.

[8] Kwon, B.H., Jeong, J.H. and Lee, S.W., 2026. E2T: EEG-to-Trajectory Transformer for Motor Imagery-Based Fully-DoF Motion Prediction. IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9] Antelis, J.M., Montesano, L., Ramos-Murguialday, A., Birbaumer, N. and Minguez, J., 2013. On the usage of linear regression models to reconstruct limb kinematics from low frequency EEG signals. PloS one, 8(4), p.e61976.

[10] Borra, D., Mondini, V., Magosso, E. and Müller-Putz, G.R., 2023. Decoding movement kinematics from EEG using an interpretable convolutional neural network. Computers in Biology and Medicine, 165, p.107323.

[11] Jain, A. and Kumar, L., 2023. Subject-independent trajectory prediction using pre-movement EEG during grasp and lift task. Biomedical Signal Processing and Control, 86, p.105160.

[12] Zhu, L., Jiang, P., Huang, A., Zhang, J. and Yuan, P., 2026. M3T-attention: a multi-level multi-scale temporal attention transformer for EEG hand movement trajectory decoding. Cognitive Neurodynamics, 20(1), p.33.

[13] Jain, A. and Kumar, L., 2025. ESI-GAL: EEG source imaging-based trajectory estimation for grasp and lift task. Computers in Biology and Medicine, 186, p.109608.

[14] McShane, N., Korik, A., McCreadie, K., Bois, N.D., Charles, D. and Coyle, D., 2026. Embodied Virtual Reality Feedback Reshapes Neural Representations to Support Continuous Three-Dimensional Motor Imagery Decoding. arXiv preprint arXiv:2605.29677.

Comments on the paper

2 comments

We once again thank the reviewers for their reviews and appreciate the time they spent on their responses.

During this rebuttal, we provided extensive additional experiments and evidence supporting our stated contribution: residual correction without direct EEG input can improve baseline decoding performance and robustness. The reviewer’s main concern is that the residual corrector learns an average, time-locked pattern. While we acknowledge that time contributes to the large improvement reported in the paper, we use the additional experiments below to argue that our residual corrector learns beyond the average pattern and improves on baseline performance, thereby supporting our stated contribution.

1.Although experiments show that time-only model improved upon the baseline, it still performed worse than full-input model (correlation from 0.7182 to 0.6440 in 2D, 0.7783 to 0.7231 in VR). We also compared model without time input against the baseline. The results showed that residual correction still significantly improved baseline decoding performance even without temporal information (correlation from 0.5076 to 0.5641 in 2D, 0.6420 to 0.6784).

2.We demonstrated that residual learning can be applied to baseline decoders beyond CNN–LSTM. All comparisons showed increased correlation and reduced RMSE. The residual corrector without time input also outperformed all additional baselines. Taking together with results from reason 1, these results demonstrated that the residual corrector uses information from the decoded outputs rather than simply memorizing a pattern.

3.We applied our method to another EEG kinematics decoding dataset WAY-EEG-GAL involving motor execution in a single-direction grasp-and-lift task. This dataset does not present the potential issue of learning an average pattern because it does not involve multiple targets. The following table compares the baseline with three residual input settings, time-only, without time, and full input, across three models in WAY-EEG-GAL. Each entry reports correlation/RMSE. The without-time setting achieves the highest performance for all three models. Full input yields a slightly lower performance. Both of them outperform than the baseline, but the time-only model performs worse than the baseline. Our experiments again showed that residual learning improved performance across all three baseline decoders, countering the claim that residual learning only learns an average pattern.

Each entry reports Pearson correlation / RMSE.

ModelBaselineTime-onlyWithout timeFull input
rEEGNet0.6349 / 0.058530.6309 / 0.052560.6923 / 0.048620.6836 / 0.04919
rDeepConvNet0.6203 / 0.056140.6023 / 0.054510.6605 / 0.050970.6468 / 0.05189
rShallowConvNet0.5570 / 0.074150.5438 / 0.056500.6267 / 0.053450.6261 / 0.05333

4.We conducted additional experiments with residual scales of 0.01, 0.03, 0.05, 0.07, and 0.1. Even at a scale of 0.01, equivalent to one-thirtieth of the data range, residual correction still improved baseline performance. This demonstrates that the improvements do not arise from overriding the decoded outputs and also counters the claim that the residual corrector merely learns an average template correction. These scale values specify the upper bounds on the corrections, rather than their actual magnitudes. The table below reports the mean absolute correction for each axis and feedback modality. All values are substantially smaller than the data range of 0.3, with a maximum value of about 0.040

Scale (2D)X-axisY-axisZ-axis
0.010.0080310.0094700.008798
0.030.0147690.0238230.020956
0.050.0171520.0324530.027701
0.070.0180460.0354510.030739
0.100.0182630.0380550.033399
Original0.0204730.0396680.035809
Scale (VR)X-axisY-axisZ-axis
0.010.0072600.0091460.009086
0.030.0122600.0226910.021737
0.050.0139280.0303110.027492
0.070.0142620.0339620.031001
0.100.0165120.0364120.033584
Original0.0178810.0379900.035512

5.We also trained the residual corrector on shuffled decoded–target pairs and evaluated it on correctly paired data. Performance was substantially lower than when the corrector was trained on correctly paired data (correlation from 0.7182 to 0.6571 in 2D, 0.7783 to 0.7249 in VR), demonstrating that the residual corrector requires information from the correct decoded–target pairing. Although training on shuffled pairs still improved performance over the baseline, we also showed that the shuffled decoded–target pairs themselves had high correlations, especially along the Y and Z axes. In 2D, cross-target shuffling yielded Y and Z axis correlations of 0.8289 and 0.6971, compared with 0.8354 and 0.7491 under correct pairing. In VR, the corresponding correlations were 0.8856 and 0.8794 under cross-target shuffling, compared with 0.9005 and 0.8949 under correct pairing. This indicates that the baseline decoder’s predictions already share similar temporal structure with different targets, reflecting an average target pattern; it does not demonstrate that our residual corrector learns an average template correction.

6.We also compared the model training with correct data and the model training with shuffled data in a broader experiment. Please see the Appendix P in the revised manuscript. The four curves show FRL trained and evaluated with the original target pairings (blue), FDG evaluated against the original targets (green), FRL trained on a fixed cross-target permutation in S1 and evaluated with that same permutation in S2–S5 (orange), and FDG evaluated with that permutation in all sessions (purple). To compute the classification accuracy, we compared each trial’s output trajectory with four target trajectories. For each target, we computed the RMSE jointly across all time points and axes, then predicted the target with the lowest RMSE. Accuracy was the percentage of trials in which the predicted target matched the original or permuted target label used for scoring. Averaged across 10 participants and four test sessions, normally trained FRL achieved higher mean XYZ correlation, lower mean XYZ RMSE, and higher joint XYZ classification accuracy than shuffled FRL in both 2D and VR. Accuracy was 29.30% versus 25.05% in 2D and 36.88% versus 23.98% in VR. In 2D, accuracy differences ranged from −6.45 percentage points for P8 to +13.38 points for P10, while session averages ranged from +2.70 points in S2 to +6.95 points in S4. In VR, the differences ranged from −3.32 points for P8 to +31.93 points for P2, while session averages ranged from +8.24 points in S4 to +17.70 points in S2. Normally trained FRL achieved higher mean classification accuracy in every test session and almost every participant, with gains varying across participants and sessions. These results support the role of correct input–target pairings (decoder information) during training in preserving distinctions between targets across sessions.

No prior work on BCI kinematic decoding has applied residual learning to correct the outputs of a baseline decoder. We believe our novelty contributes to both machine learning and BCI, and our experiments show that the residual corrector achieves SOTA performance in the settings evaluated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.