- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors investigate the question of whether brain activity from the primary motor cortex can be used to better train artificial motor controllers. Specifically when trainign an GRU or transformer to control a simulated arme that add a loss temr that pushes the nwtroks internal acvitiy to the low dimensional structure of the cortical activity. Afte rtrain, no cortical data is used and they compare against task-only training. There cortical informed approach generally gives the most accurate reaches on trained, held-out, and new targets, and it improves robustness to noisy sensory feedback. The aprroach also show low “trajectory tangling,” a known property of motor cortex, which the authors propose as the mechanism.
Critical strengths
The authors use a rigorous control design, with different controls isolating different benefits. Statistics are solid.
The tangling analysis is very interesting and is a nice link to what we know about motor cortex.
Critical weaknesses
The claims are a bit overstated, given the task is simple and limited -- a planar two-joint arm with 40 steps and 10cm reach. For example I recommend authors back off a bit from the claim "a novel class of neuro-interfaces” for robotics and embodied AI. I think it is a good simple example, but it is not clear that this will be a "class" of controllers that generalize to more complex and realistic motor-control scenerios.
Questions for authors
Are task-only controllers trained to convergence? Please show training curves, and add standard regularization baselines and noise-augmented training as comparators for the robustness result.
Does a direct low-tangling regularizer reproduce the gains? That would test whether tangling is the mechanism or merely correlated with it.
3 comments
Detailed responses (Part 2/2)
3. Standard regularization and noise-augmented training
Add standard regularization baselines and noise-augmented training as comparators for the robustness result.
Thank you for this suggestion. We added 3,300 GRU fits, testing six weight-decay strengths and five training-noise levels, with 100 seeds per setting and animal. These controls use the same architecture, matched initializations and target splits, and the same 700-update training budget. We evaluate them under the original sensory-noise protocol.
The results reveal an interesting accuracy–robustness tradeoff, which we have integrated as a new, additional finding in the revised version of the manuscript:
- Neuro-Dist has lower mean error under clean feedback and at the lowest positive test-noise amplitude than every tested conventional setting in all three animals.
- At higher noise, weight decay and noise augmentation can achieve lower error, with a cost to clean accuracy. For example, augmentation with training-noise SD 0.05 reduces held-out mean RMSE at test-noise SD 0.1 by 41% relative to Neuro-Dist, while increasing clean RMSE by 135%. These percentages compare equally weighted dataset means.
Thus, cortical supervision improves accuracy and noise tolerance without noise-augmented training, while conventional robustness training can provide additional advantages at high noise. This motivates a next experiment combining Neuro-Dist with noise augmentation or weight decay to test whether their benefits can be combined while preserving clean accuracy. We now discuss this next step explicitly in the main text; it remains a proposed experiment.
The methods and results are reported in Appendix C.1, “Conventional regularization and noise-augmented training,” and Table 4, with a summary in the main-text robustness section.
4. Direct low-tangling regularization and mechanism
Does a direct low-tangling regularizer reproduce the gains? That would test whether tangling is the mechanism or merely correlated with it.
We particularly appreciate this suggestion, which prompted an intervention that substantially strengthens the mechanistic investigation. We trained GRUs with a direct tangling penalty and, as an additional control, a temporal-smoothness penalty. Across three animals, 100 seeds, and 44 tangling or 40 smoothness strengths, this adds 25,200 fits. The penalties use the full GRU state and only the training reaches, without a neural teacher. Strengths are selected by training MSE and then fixed for evaluation.
Direct tangling regularization reproduces part of the accuracy benefit: it reduces median clean reaching MSE relative to task-only training by 9.3–19.2% across all nine animal–target comparisons, covering trained, student-held-out, and entirely new targets. Neuro-Dist achieves larger gains, with lower median MSE in eight of these nine comparisons and at every tested positive sensory-noise amplitude in all three animals. At the highest tested noise level, its median MSE is 41–63% lower than that of direct tangling regularization. Neuro-Dist also generally outperforms temporal-smoothness regularization.
These results provide additional support for the utility of low tangling, extending the original correlational evidence. They strengthen our hypothesis that reduced tangling contributes to Neuro-Dist's benefits as one facet of a broader transfer of computationally useful neural population structure. The direct penalty's gains and the additional benefits of cortical alignment therefore support both parts of our original interpretation: tangling helps, and additional population structure may also contribute.
We have integrated these new key findings into Appendix F, “Direct tangling and temporal-smoothness regularization,” and Table 5, and revised the Discussion (Section 5) accordingly. Figure 13 additionally illustrates how the GRU trajectories evolve during matched task-only and direct-tangling training.
We want to thank the reviewer again for suggestions that have led to these new experiments and additional important findings!
Thank you for your responses. I maintain my score and would be happy to see this paper accepted.
Detailed responses (Part 1/2)
Dear Reviewer 9Xdu,
Thank you for the insightful and constructive review. We are happy that the reviewer appreciates the rigorous control design, statistical analyses, and the link between trajectory tangling and motor-cortical population dynamics, which we also found very intriguing. The comments have helped us substantially strengthen both the experimental evaluation and the interpretation of our findings. In particular, they motivated new conventional-regularization and noise-augmentation baselines, as well as direct tangling and temporal-smoothness interventions, with very interesting new results. We have incorporated these results into the revised manuscript and supplement and respond to each point below.
1. Scope of the claims
We agree that the scope of the claims was overstated. Our study establishes a proof of concept in a controlled reaching task with a simulated planar arm; generalization to more complex motor behaviors, other embodiments, and physical robots clearly requires further investigation. The Discussion explicitly frames the study as a proof of concept and identifies these extensions as future research. To make this distinction equally clear in the abstract, we have replaced the “novel class of neuro-interfaces” formulation with:
This retains the broader motivation while making clear what the present experiments demonstrate. Within the reaching task, we have also broadened the range of controllers: neural supervision also improves a state-space controller and a PPO-trained policy (Appendices O–P).
2. Training convergence
We thank the reviewer for this important question. We have added a new Appendix H, "Convergence behavior and extended training," with:
Why a shared, finite budget. Our aim was to test whether neural supervision improves the acquisition of accurate control. Following studies that examine learning before convergence and under finite training budgets (Salimans & Kingma, 2016; Wu et al., 2021), and to avoid ceiling effects (Zhou et al., 2022), we use this paradigmatic reaching task for this comparison. In a simple task, prolonged training drives controllers toward very low error, leaving little room to reveal the teacher's contribution. We chose 700 updates as a common point after substantial learning: median task-only training error has fallen by more than 97% in all settings, yet residual error remains.
As intended by this design, task-only controllers are still improving at 700 updates. Training to 2,000 updates removes a further median 95.2–97.1% of the remaining error for GRU 128 and 77.2–85.4% for the Transformer. Even late in GRU 16 training, error keeps declining: in Monkey 1, comparing each run's median error over updates 8,000–8,950 with that over 9,000–10,000 gives median paired reductions of 14.3% (trained) and 13.4% (student-held-out targets). This is precisely the learning regime in which we ask whether neural supervision helps.
Neural supervision still helps late in training. In GRU 16 (Monkey 1), the strength selected by late-window training error alone (α = 0.025) lowers held-out error over updates 9,000–10,000, with a 10.6% median paired reduction (exact sign test; Holm-adjusted p = 0.0281 across 27 strengths). After 20,000 PPO cycles (Monkey 2, GRU 128), all 20 positive strengths yield lower median errors than PPO alone; at the training-selected strength, reductions are 44.6%, 26.0%, and 44.3% on trained, held-out, and new targets.
This proof of concept motivates testing neural supervision in harder tasks where substantial residual error persists under extended training budgets.