- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+2)
Read review
Paper summary
The paper proposes a method for learning control-oriented locally linear method neural dynamics. For this, an encoder is employed to compress the neural observations into a (jointly) learned latent space. The (locally) linear latent dynamics are nonlinear manifold meaning that the state space matrices are functions of the (approximate/coarse) state but held fixed (and with that linear) over the prediction horizon. After training the encoder and the latent dynamics, a receding-horizon (linear) MPC can be used for closed-loop control The proposed method, coined ConFINE, is evaluated on the (ground-truth) dynamics of synthetic nonlinear manifolds, task-trained RNNs (not sure what the point is of learning the dynamics of another NN) and simulations of stroke recovery.
Critical strengths
- The claimed contribution of learning control-oriented but nonlinear embeddings for neural dynamics is worthwile and interesting.
- The results demonstrate a clear advantage of employing linear instead of nonlinear latent dynamics.
Critical weaknesses
- The presentation quality and clarity is lacking in parts. Examples include (i) the introduction does not sufficiently introduce the (non-expert) reader to and motivate the problem setting of learning neural dynamics, (ii) the lack of a methodology overview figures, (iii) core parts of the methodology that are crucial for understanding the paper are hidden in the appendix (particularly appendices E and F), including the generative model and what is used for, the locally linear terms, Algorithm 3, etc, (iv) how the local linearization is chosen (i.e., on which state the state space matrices are evaluated/conditioned on) and how this interacts with the training rollouts and the receding-horizon control remains unclear and confusing (see questions below).
- The validation is (largely) / entirely done on either toy reference dynamics (the nonlinear manifolds from Section 3.1) or on already trained, other neural networks functioning as plants (specifically learning the RNN dynamics in Sections 3.2 and 3.3). This seems curious and it is not clear what the point is of learning from already trained neural networks; wouldn't you prefer to train directly on the source dataset? The authors should validate their method on original datasets representative for neural dynamics (the focus of the paper) instead of (only) such synthetic data.
Questions for authors
- How is the state computed on which the state-space matrices are evaluated on? I.e., how does the model choose its locally linear dynamics? How is this taken into account during training: For each horizon contained in the dataset, does the training supervise a single linearization? How is this aspect handled during closed-loop control? Does the receding horizon MPC update the linearization (i.e., the state-space matrices) during receding-horizon control? Sometimes, the authors refer in this context to a nominal trajectory. Where do you get this from?
- Could you include an _Oracle MPC in Figure? So employing a (nonlinear) dynamics on the ground-truth dynamics? This would provide a good upper-bound for the performance we could expect.¨
- Are there any other nonlinear dynamical baselines that should/could be considered for learning the latent dynamics?
5 comments
W2. Simulations vs real data
It is not clear what the point is of learning from already trained neural networks; wouldn't you prefer to train directly on the source dataset? The authors should validate their method on original datasets representative for neural dynamics..
We have now validated our model on real neural recordings for modeling and prediction as a complementary validation. Specifically, we fitted ConFINE to real stimulation-driven neural recordings, consisting of cortical surface recordings (μECoG) acquired during optogenetic (light-based) stimulation in nonhuman primates [2]. We now provide these evaluations in Section 3.4, “Modeling in-vivo input-driven neural dynamics” of the manuscript. We find that our model is able to predict future neural activity in response to stimulation better than baselines. Since model-predictive control (MPC) relies on high-fidelity future prediction of neural responses to stimulation, the strong prediction of ConFINE serves as a prerequisite for future in-vivo control experiments in the brain.
We note that previously recorded neural data allow us to validate modeling and prediction, as in our new Section 3.4, but do not directly evaluate closed-loop control: each input selected by the controller affects the activity observed next, whereas recordings contain responses to a fixed stimulation sequence. Simulated neural circuits therefore complement this validation by allowing us to apply controller-selected inputs and measure their effects on activity and behavior. In particular, the grasping circuit fitted to recorded primate kinematics provides a biologically motivated setting for system identification and closed-loop testing, with access to ground-truth dynamics.
We agree that demonstrating closed-loop control in vivo would be the strongest validation. Such experiments, however, involve substantial experimental and safety considerations, making simulation a useful and necessary first step. As we note in the Introduction, closed-loop continuous control using a dynamical model has remained elusive at the level of neural populations in real brains: existing demonstrations are limited to the firing rate of single neurons using linear models [1]. Our Discussion identifies achieving such population-level control in real brains as a critical next step for the field of neuroscience. Our simulations establish feasibility toward this goal.
[1] Bolus, M. F., et al. (2021). State-space optimal feedback control of optogenetically driven neural activity. Journal of Neural Engineering, 18, 036006.
[2] Yazdan-Shahmorad, A., et al. (2018). Targeted cortical reorganization using optogenetics in non-human primates. eLife, 7, e31034.
Q1. Method clarifications
Thank you for the opportunity to clarify these points, we answer each one-by-one in detail below, with further references to the corresponding additions to the manuscript.
How is the state computed on which the state-space matrices are evaluated? How does the model choose its locally linear dynamics?
A Kalman filter recursively computes the state estimate from encoded neural observations and known inputs. A learned neural network maps the filtered mean to transition matrices and the predicted mean to observation matrices. These computations are now explicit in Sections 2.1–2.2 and illustrated in the new Appendix E architecture figure.
During training, does each prediction horizon supervise a single linearization?
No. Starting from the filtered state, each rollout step regenerates the transition matrices from the preceding predicted mean; observation matrices are evaluated at the resulting predicted mean. Prediction losses train the model end-to-end across the specified horizons. Section 2.3 now states this explicitly.
How is this handled during closed-loop control?
At each control step, we roll out candidate inputs from the filtered state, evaluate matrices along that nominal trajectory, and hold them fixed while solving a QP for updated inputs. This procedure is now described in Section 2.4, with references to the nominal-rollout equation, control objective, and Algorithm 3.
Does receding-horizon MPC update the state-space matrices?
Yes. Matrices are recomputed along the nominal trajectory between QP iterations. After applying the first optimized input, the controller replans from the updated filtered state at the next control step. The learned network parameters remain fixed. Section 2.4 now clarifies these updates.
Where does the nominal trajectory come from?
The nominal trajectory is a model rollout from the current filtered state under a candidate input sequence, initialized from the preceding control solution (zeros initially). Each QP’s updated inputs generate the next nominal trajectory. Section 2.4 now defines this trajectory and points directly to its equation; initialization and sequence shifting are detailed in Algorithm 3.
Q2. Oracle MPC comparison
Could you include an Oracle MPC in Figure?
Tables 1–2 already report oracle MPC performance using the true plant dynamics and state, and we agree that showing this reference in the figures makes the comparison clearer. We have added it to the manifold sweeps (Figs. A1-A3), reach-RNN evaluation plots (Figs. 4, A7–A9) and the new sample-efficiency and ablation experiments (Figs. A10–A11). We also added an “Oracle (true plant)” row to the stroke-recovery table (Table 3) and reference lines in Figs. A12–A13, using MPC through the true nonlinear lesioned circuit.
Q3. Nonlinear baselines
Are there any other nonlinear dynamical baselines that should/could be considered for learning the latent dynamics?
We have added nonlinear gated recurrent unit (GRU) RNN baselines with varying hidden-state sizes and training budgets as a representative nonlinear recurrent baseline to test whether ConFINE’s advantages persist against more expressive dynamics. We evaluate them on the reaching RNN plants (Section 3.2), where even with 8x as many training updates, GRUs have higher mean control costs than LL-ConFINE on all three plants at both input-penalty settings (Table 2). LL-ConFINE also matches or exceeds the best GRU configuration trained on 4x sequences, even with 8x updates (Fig. A10). GRU controllers require 190–352 ms per control step, versus <1 ms for ConFINE and <7 ms for LL-ConFINE on the same CPU, with GRU runtime dominated by nonlinear planning (Table 4). Additional comparisons appear in Figs. 4 and A7–A9. On the simulated stroke-recovery task (Section 3.3), LL-ConFINE achieves higher recovery than GRUs across all four lesions under behavioral supervision, including when GRUs receive 8x as many training updates (Table 3). We include the GRUs in the recovery and prediction analyses in Figs. A12–A21.
We also tested GRUs on the torus-rotation plant from the main text. With ConFINE’s training protocol and CEM planning, the GRU achieved a control cost of 3.43 versus 0.57 for ConFINE and 0.24 for the oracle. Larger hidden states (32 and 128 dimensions) did not improve performance. As such, these results are not included in the paper. We investigated two potential sources of poor generalization: training inputs are bounded by ±0.5 and change every step, whereas holding targets requires larger, sustained inputs (median steady-state magnitude: 2.6). On a slower-rotation plant, where most required inputs fall within the training range, GRU cost decreased to 1.80. Adding sustained-input training sequences reduced it further to 1.31, still above ConFINE’s 0.55 (table below).
| GRU training data | Torus-rotation plant | GRU, 1× updates | GRU, 8× updates | ConFINE | Oracle |
|---|---|---|---|---|---|
| Random inputs | Original | 3.46 | 3.43 | 0.57 | 0.24 |
| Random inputs | Slower rotation | 2.08 | 1.80 | 0.55 | 0.24 |
| Random + sustained inputs | Slower rotation | 1.93 | 1.31 | 0.55 | 0.24 |
Cost: distance to target over the last 10 steps (lower is better). 1× / 8×: ConFINE's training budget and 8 times it.
We additionally implemented a stochastic nonlinear state-space model [1][2] with a GRU filter, an MLP transition and readout, a 32-dimensional latent state, and CEM planning. On reaching RNN plants B and C, its control costs were 2.1–4.6x those of ConFINE, exceeding LDM’s costs at both input penalties (table below). Additional data and optimization did not close the gap: on plant C, even with 16x the training data and 8x the training updates, its cost remained 6.9 versus 3.6 for ConFINE. Likewise, these results are not included in the paper.
| Model | B, R = 10⁻² | B, R = 10⁻³ | C, R = 10⁻² | C, R = 10⁻³ |
|---|---|---|---|---|
| ConFINE | 2.09 ± 0.02 | 1.75 ± 0.02 | 3.57 ± 0.03 | 3.41 ± 0.04 |
| LDM | 2.53 ± 0.09 | 2.46 ± 0.09 | 4.22 ± 0.11 | 4.07 ± 0.12 |
| Stochastic SSM | 7.57 ± 0.17 | 7.99 ± 0.17 | 7.61 ± 0.12 | 7.48 ± 0.11 |
Closed-loop cost as in Table 2 (mean ± s.e., 4096 training sequences, lower is better).
We also implemented a Mamba baseline, trained with the same multi-step objective and evaluated with the same CEM planner as the GRUs. Under matched CPU benchmarking conditions, each control step took several seconds, compared with 190–352 ms for the GRUs and approximately 3 ms for ConFINE. We therefore did not pursue a broader Mamba evaluation: this implementation’s planning latency substantially exceeded the timing requirements of the neural feedback applications discussed in the paper.
[1] Krishnan, R. G., Shalit, U., and Sontag, D. Deep Kalman Filters. arXiv:1511.05121, 2015.
[2] Krishnan, R. G., Shalit, U., and Sontag, D. Structured Inference Networks for Nonlinear State Space Models. AAAI, 2017.
We appreciate the reviewer’s constructive feedback and hope the additional results and clarifications resolve the concerns raised. If so, we would be grateful if you would consider raising your score.
Thank you for the helpful review. We have worked to address each point carefully through additional experiments and clarifications, as detailed below.
W1. Presentation quality and clarity
Thank you for these suggestions. We have substantially improved the clarity of the presentation by expanding the introductory motivation (Section 1), adding overview figures (Fig 1A-B, Fig A22), and clarifying the model, inference, training, and control procedures (Sections 2.1–2.4). Changes are highlighted in red. We address each point below.
Neural population dynamics describe how activity across neurons evolves through circuit interactions and responses to external inputs. Learning these dynamics allows us to predict how candidate stimulation inputs affect future activity and select inputs toward neural or behavioral targets. Because these dynamics and input responses are unknown, they must be identified from recordings paired with stimulation. We now explain this motivation at the beginning of Section 1, with references to the identification and control diagrams in Fig. 1A–B.
We added an overview of open-loop identification and closed-loop control (Fig. 1A–B), alongside the graphical model (Fig. 1C). Fig. A22 additionally illustrates the locally linear architecture, including how state estimation supplies the states used for matrix generation.
For clarity, we have renamed Section 2.1 “Generative Model” and clarified these components and the generation of locally linear matrices. Sections 2.2–2.4 explain their use in inference, training, and control, supported by Figs. 1 and A22. In particular, Section 2.4 now explains the procedure underlying Algorithm 3. Only the detailed classical inference/control equations and algorithms – Kalman filtering and model predictive control – remain in the appendices, included for reference, with precise annotations added wherever they are invoked in the main text.
Section 2.1 now explicitly specifies the conditioning states: transition matrices are generated from the filtered mean, while observation matrices are generated from the predicted mean. Section 2.2 explains how these means are computed recursively. The new locally linear architecture figure in Appendix E illustrates these dependencies. The model directly generates these matrices; no explicit linearization is performed.
During training, matrices are regenerated at every rollout step from the predicted state, and prediction losses train the model end-to-end. During control, matrices are evaluated along a nominal trajectory and held fixed within each QP solve, then recomputed using the updated input plan. Sections 2.3 and 2.4 now explain these distinct procedures explicitly, with direct references to the corresponding equations and Algorithm 3.