- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MaPIC, a framework for robust neural decoding that integrates learned neural evidence with structured physical motion priors. The approach combines a neural encoder that models observation likelihoods with a Bayesian dynamic model incorporating physical motion primitives, such as translation and rotation. The components are integrated through variational inference, where the KL divergence encourages consistency with the physical priors, while learned posterior uncertainties enable adaptive weighting of the different primitives. The proposed framework is designed to be causal and applicable to existing decoding architectures. The approach is evaluated on simulated dynamical systems and real rat hippocampal recordings, demonstrating improved decoding accuracy and robustness to missing neural observations compared with the considered baselines.
Critical strengths
- The central idea of incorporating explicit physical motion primitives into neural decoding is appealing. By combining learned neural evidence with structured dynamic priors, the proposed approach seeks to exploit complementary sources of information, particularly when the neural observations become unreliable.
-The paper evaluates the proposed approach on both simulated dynamical systems and real neural recordings, including scenarios where subsets of neurons are removed. The reported improvements in decoding accuracy, particularly in the presence of missing neural observations, are encouraging and suggest that the proposed approach may provide practical benefits for robust neural decoding.
Critical weaknesses
-
A central claim of the paper concerns the novelty of integrating learned neural decoding with physically motivated dynamic models. However, this general direction has been explored in previous works combining neural networks with model-based Bayesian inference and dynamical priors, e.g., "Exploring the Trade-off Between Deep-Learning and Explainable Models for Brain-Machine Interfaces" (NeurIPS 2024).
-
Although the experiments demonstrate improved performance when neural observations are artificially degraded through neuron dropout, this evaluation addresses only one aspect of the robustness challenges motivating the paper. In particular, it is not sufficiently clear what training data and supervision are required to learn the proposed model, whether it must be trained separately for each subject or recording session, and whether the learned physical priors can transfer across sessions or subjects. These questions are particularly important given the claimed practical advantages of the approach. Moreover, robustness to limited calibration data and non-stationarity is a central challenge in neural decoding, as emphasized by Karpowicz et al., "Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark" (NeurIPS 2024).
Questions for authors
-
Could the authors clarify what data and supervision are required to train the proposed framework? In particular, does learning the physical primitive model require labeled kinematic trajectories, and is the model trained separately for each subject or recording session? How is the proposed plug-and-play property realized in practice, and does applying MaPIC to a new subject or session require retraining or calibration?
-
Please elaborate on the methodological differences between MaPIC and existing approaches that combine learned neural decoding with dynamical models? In particular, what advantages arise specifically from the representation in terms of physical motion primitives and their inferred uncertainties, compared with more conventional learned or model-based Bayesian dynamics?
-
Are the neuron dropout patterns considered in the evaluation introduced exclusively at inference, or are similar dropout patterns also included during training? Moreover, to what extent are the reported improvements attributable to the structured physical primitives and their uncertainty-based adaptive weighting, rather than simply to the introduction of an additional dynamic prior?
-
Have the authors evaluated the proposed framework in cross-session or few-shot adaptation settings, where the available labeled data for a new recording session are limited? Such experiments, motivated by settings considered in the FALCON benchmark, would be particularly valuable for assessing whether the proposed physical priors indeed facilitate robust decoding under practical non-stationarity and limited supervision.
2 comments
Thank you for the detailed responses and additional experiments. The responses have adequately addressed my concerns.
Training with Frozen Decoders and the Role of Motion Correction Under Neuronal Dropout
We sincerely thank the reviewer for their helpful suggestions.
Q1: Training Data, Supervision, and the Plug-and-Play Property.
Response:
Learning the physical primitive model requires labeled kinematic trajectories, and the model is trained separately for each recording session. No labels are needed at inference. Plug-and-play is realized by keeping the base decoder frozen: MaPIC takes only the decoder's output and is trained on top of it, so the base decoder is never modified or retrained. For a new subject or session, the frozen base decoder requires no retraining for MaPIC, and only the MaPIC module is trained on that session's labeled trajectories.
The code is released and can be inspected at any time: https://anonymous.4open.science/r/MaPIC-F671/
Q2/W1: Methodological Differences from Existing Decoding–Dynamics Approaches.
Response:
The methodological difference is that existing approaches, including KalmanNet and the heteroscedastic Kalman filter of Cubillos et al. (NeurIPS 2024), adapt only the balance between neural evidence and the model's motion prediction, while MaPIC also corrects the motion prediction itself before fusion, along the physical motion primitives. This gives two advantages. First, when the dynamic model is inaccurate and neural observations are unreliable, MaPIC corrects the biased prediction that a model-based Bayesian filter must follow: error is 39–52% lower than a geometry-aware EKF under a loss of 80% of neurons (Table 5). Second, the inferred uncertainty of each primitive sets the fusion weight, and one module handles circular, linear and toroidal variables on three real datasets.
The work of Cubillos et al. is now cited, and we compare directly with KalmanNet and model-based Kalman filters on rat CA1; the row below shows Achilles, with all four rats in App. A.2, Table 6.
Mean ± s.d. over 9 paired runs. * p < 0.05 against MaPIC, paired Wilcoxon test.
Q3/W2: Dropout in Training and Attribution of the Improvements.
Response:
Similar dropout patterns are also included during training, and the improvements are attributable to the structured physical primitives and their uncertainty-based adaptive weighting, not to an additional dynamic prior alone. We evaluate two scenarios (App. A.1.2): persistent neuron dropout, where a random fraction of neurons is removed for the whole test session, and transient signal loss, where observations are degraded or removed within a short window. During training, random windows have random neurons dropped at random rates, with the same augmentation for all trained methods, and the test patterns are drawn independently.
On attribution (App. A.1.3–A.1.4): a filter with the same dynamic prior and the same adaptive observation covariance, but without our learned primitive correction, is matched by MaPIC when the prior is exact and is 39–52% worse when it is not. Removing the learned primitive correction roughly doubles the error under 80% dropout (13.72° to 29.46°), and replacing uncertainty-based adaptive weighting with a fixed weight significantly slows recovery after signal loss (4.56° to 5.81°); both p < 0.05 over 10 seeds.
Median angular error during a 2-s loss of 80% of neurons, mean ± s.d. over 10 seeds. * p < 0.05 against MaPIC, two-sided paired Wilcoxon test.
Q4: Cross-Session and Few-Shot Adaptation.
Response:
Cross-session and few-shot adaptation are not the focus of the current work, but we will address them in future work following the FALCON settings. Under limited supervision within a session, the physical priors already facilitate robust decoding: in simulation the MaPIC module is trained with two labeled trials (16 s), and on grid cells MaPIC trained on 20% of the data (9.28 cm) outperforms both baselines trained on all of it (10.16 and 9.40 cm). FALCON is now cited in Related Work.