acceptodds
Under review as a conference paper at ICLR 2027

Learning nonlinear embeddings for control of neural dynamics

Abstract

Control of neural activity could provide a way to study causal interactions within neural populations and develop adaptive interventions for neurological disorders. However, doing so is hindered by the complexity of neural population dynamics and their response to external inputs. Prior work on neural control has largely relied on linear or generalized linear models, whose simplicity enables tractable estimation and control design, but which may not capture sufficient complexity in neural dynamics for accurate control. Here, we develop a model that learns nonlinear embeddings of neural population activity with linear or locally linear latent dynamics using control-focused objectives. Combined with recursive state estimation, this model captures nonlinearity while enabling tractable model-based control. We validate our model and controller in simulation, across benchmark systems relevant to neuroscience, spanning both linear and nonlinear dynamics. Across these systems, our models outperform the baselines in stabilizing activity at desired target setpoints and in restoring movement trajectories following simulated lesions. Together, these simulation results show that learning control-relevant embeddings and latent dynamics provides a practical approach toward future systems for model-based control of nonlinear neural population activity.

Primary area:
applications to neuroscience & cognitive science
Keywords:
optimal feedback control, model-predictive control, latent state-space model, neural stimulation, system identification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_uwqoOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

The paper proposes a method for learning control-oriented locally linear method neural dynamics. For this, an encoder is employed to compress the neural observations into a (jointly) learned latent space. The (locally) linear latent dynamics are nonlinear manifold meaning that the state space matrices are functions of the (approximate/coarse) state but held fixed (and with that linear) over the prediction horizon. After training the encoder and the latent dynamics, a receding-horizon (linear) MPC can be used for closed-loop control The proposed method, coined ConFINE, is evaluated on the (ground-truth) dynamics of synthetic nonlinear manifolds, task-trained RNNs (not sure what the point is of learning the dynamics of another NN) and simulations of stroke recovery.

Critical strengths
  1. The claimed contribution of learning control-oriented but nonlinear embeddings for neural dynamics is worthwile and interesting.
  2. The results demonstrate a clear advantage of employing linear instead of nonlinear latent dynamics.
Critical weaknesses
  1. The presentation quality and clarity is lacking in parts. Examples include (i) the introduction does not sufficiently introduce the (non-expert) reader to and motivate the problem setting of learning neural dynamics, (ii) the lack of a methodology overview figures, (iii) core parts of the methodology that are crucial for understanding the paper are hidden in the appendix (particularly appendices E and F), including the generative model and what is used for, the locally linear terms, Algorithm 3, etc, (iv) how the local linearization is chosen (i.e., on which state the state space matrices are evaluated/conditioned on) and how this interacts with the training rollouts and the receding-horizon control remains unclear and confusing (see questions below).
  2. The validation is (largely) / entirely done on either toy reference dynamics (the nonlinear manifolds from Section 3.1) or on already trained, other neural networks functioning as plants (specifically learning the RNN dynamics in Sections 3.2 and 3.3). This seems curious and it is not clear what the point is of learning from already trained neural networks; wouldn't you prefer to train directly on the source dataset? The authors should validate their method on original datasets representative for neural dynamics (the focus of the paper) instead of (only) such synthetic data.
Questions for authors
  1. How is the state computed on which the state-space matrices are evaluated on? I.e., how does the model choose its locally linear dynamics? How is this taken into account during training: For each horizon contained in the dataset, does the training supervise a single linearization? How is this aspect handled during closed-loop control? Does the receding horizon MPC update the linearization (i.e., the state-space matrices) during receding-horizon control? Sometimes, the authors refer in this context to a nominal trajectory. Where do you get this from?
  2. Could you include an _Oracle MPC in Figure? So employing a (nonlinear) dynamics on the ground-truth dynamics? This would provide a good upper-bound for the performance we could expect.¨
  3. Are there any other nonlinear dynamical baselines that should/could be considered for learning the latent dynamics?
5 comments

Thank you for the helpful review. We have worked to address each point carefully through additional experiments and clarifications, as detailed below.

W1. Presentation quality and clarity

Thank you for these suggestions. We have substantially improved the clarity of the presentation by expanding the introductory motivation (Section 1), adding overview figures (Fig 1A-B, Fig A22), and clarifying the model, inference, training, and control procedures (Sections 2.1–2.4). Changes are highlighted in red. We address each point below.

  1. The introduction does not sufficiently introduce the (non-expert) reader to and motivate the problem setting of learning neural dynamics.

Neural population dynamics describe how activity across neurons evolves through circuit interactions and responses to external inputs. Learning these dynamics allows us to predict how candidate stimulation inputs affect future activity and select inputs toward neural or behavioral targets. Because these dynamics and input responses are unknown, they must be identified from recordings paired with stimulation. We now explain this motivation at the beginning of Section 1, with references to the identification and control diagrams in Fig. 1A–B.

  1. The lack of a methodology overview figure.

We added an overview of open-loop identification and closed-loop control (Fig. 1A–B), alongside the graphical model (Fig. 1C). Fig. A22 additionally illustrates the locally linear architecture, including how state estimation supplies the states used for matrix generation.

  1. Core parts of the methodology that are crucial for understanding the paper are hidden in the appendix, including the generative model and what it is used for, the locally linear terms, Algorithm 3, etc.

For clarity, we have renamed Section 2.1 “Generative Model” and clarified these components and the generation of locally linear matrices. Sections 2.2–2.4 explain their use in inference, training, and control, supported by Figs. 1 and A22. In particular, Section 2.4 now explains the procedure underlying Algorithm 3. Only the detailed classical inference/control equations and algorithms – Kalman filtering and model predictive control – remain in the appendices, included for reference, with precise annotations added wherever they are invoked in the main text.

  1. How the local linearization is chosen (i.e., on which state the state-space matrices are evaluated/conditioned on).

Section 2.1 now explicitly specifies the conditioning states: transition matrices are generated from the filtered mean, while observation matrices are generated from the predicted mean. Section 2.2 explains how these means are computed recursively. The new locally linear architecture figure in Appendix E illustrates these dependencies. The model directly generates these matrices; no explicit linearization is performed.

  1. How this interacts with training rollouts and receding-horizon control remains unclear.

During training, matrices are regenerated at every rollout step from the predicted state, and prediction losses train the model end-to-end. During control, matrices are evaluated along a nominal trajectory and held fixed within each QP solve, then recomputed using the updated input plan. Sections 2.3 and 2.4 now explain these distinct procedures explicitly, with direct references to the corresponding equations and Algorithm 3.

W2. Simulations vs real data

It is not clear what the point is of learning from already trained neural networks; wouldn't you prefer to train directly on the source dataset? The authors should validate their method on original datasets representative for neural dynamics..

We have now validated our model on real neural recordings for modeling and prediction as a complementary validation. Specifically, we fitted ConFINE to real stimulation-driven neural recordings, consisting of cortical surface recordings (μECoG) acquired during optogenetic (light-based) stimulation in nonhuman primates [2]. We now provide these evaluations in Section 3.4, “Modeling in-vivo input-driven neural dynamics” of the manuscript. We find that our model is able to predict future neural activity in response to stimulation better than baselines. Since model-predictive control (MPC) relies on high-fidelity future prediction of neural responses to stimulation, the strong prediction of ConFINE serves as a prerequisite for future in-vivo control experiments in the brain.

We note that previously recorded neural data allow us to validate modeling and prediction, as in our new Section 3.4, but do not directly evaluate closed-loop control: each input selected by the controller affects the activity observed next, whereas recordings contain responses to a fixed stimulation sequence. Simulated neural circuits therefore complement this validation by allowing us to apply controller-selected inputs and measure their effects on activity and behavior. In particular, the grasping circuit fitted to recorded primate kinematics provides a biologically motivated setting for system identification and closed-loop testing, with access to ground-truth dynamics.

We agree that demonstrating closed-loop control in vivo would be the strongest validation. Such experiments, however, involve substantial experimental and safety considerations, making simulation a useful and necessary first step. As we note in the Introduction, closed-loop continuous control using a dynamical model has remained elusive at the level of neural populations in real brains: existing demonstrations are limited to the firing rate of single neurons using linear models [1]. Our Discussion identifies achieving such population-level control in real brains as a critical next step for the field of neuroscience. Our simulations establish feasibility toward this goal.

[1] Bolus, M. F., et al. (2021). State-space optimal feedback control of optogenetically driven neural activity. Journal of Neural Engineering, 18, 036006.

[2] Yazdan-Shahmorad, A., et al. (2018). Targeted cortical reorganization using optogenetics in non-human primates. eLife, 7, e31034.

Q1. Method clarifications

Thank you for the opportunity to clarify these points, we answer each one-by-one in detail below, with further references to the corresponding additions to the manuscript.

How is the state computed on which the state-space matrices are evaluated? How does the model choose its locally linear dynamics?

A Kalman filter recursively computes the state estimate from encoded neural observations and known inputs. A learned neural network maps the filtered mean to transition matrices and the predicted mean to observation matrices. These computations are now explicit in Sections 2.1–2.2 and illustrated in the new Appendix E architecture figure.

During training, does each prediction horizon supervise a single linearization?

No. Starting from the filtered state, each rollout step regenerates the transition matrices from the preceding predicted mean; observation matrices are evaluated at the resulting predicted mean. Prediction losses train the model end-to-end across the specified horizons. Section 2.3 now states this explicitly.

How is this handled during closed-loop control?

At each control step, we roll out candidate inputs from the filtered state, evaluate matrices along that nominal trajectory, and hold them fixed while solving a QP for updated inputs. This procedure is now described in Section 2.4, with references to the nominal-rollout equation, control objective, and Algorithm 3.

Does receding-horizon MPC update the state-space matrices?

Yes. Matrices are recomputed along the nominal trajectory between QP iterations. After applying the first optimized input, the controller replans from the updated filtered state at the next control step. The learned network parameters remain fixed. Section 2.4 now clarifies these updates.

Where does the nominal trajectory come from?

The nominal trajectory is a model rollout from the current filtered state under a candidate input sequence, initialized from the preceding control solution (zeros initially). Each QP’s updated inputs generate the next nominal trajectory. Section 2.4 now defines this trajectory and points directly to its equation; initialization and sequence shifting are detailed in Algorithm 3.

Q2. Oracle MPC comparison

Could you include an Oracle MPC in Figure?

Tables 1–2 already report oracle MPC performance using the true plant dynamics and state, and we agree that showing this reference in the figures makes the comparison clearer. We have added it to the manifold sweeps (Figs. A1-A3), reach-RNN evaluation plots (Figs. 4, A7–A9) and the new sample-efficiency and ablation experiments (Figs. A10–A11). We also added an “Oracle (true plant)” row to the stroke-recovery table (Table 3) and reference lines in Figs. A12–A13, using MPC through the true nonlinear lesioned circuit.

Q3. Nonlinear baselines

Are there any other nonlinear dynamical baselines that should/could be considered for learning the latent dynamics?

We have added nonlinear gated recurrent unit (GRU) RNN baselines with varying hidden-state sizes and training budgets as a representative nonlinear recurrent baseline to test whether ConFINE’s advantages persist against more expressive dynamics. We evaluate them on the reaching RNN plants (Section 3.2), where even with 8x as many training updates, GRUs have higher mean control costs than LL-ConFINE on all three plants at both input-penalty settings (Table 2). LL-ConFINE also matches or exceeds the best GRU configuration trained on 4x sequences, even with 8x updates (Fig. A10). GRU controllers require 190–352 ms per control step, versus <1 ms for ConFINE and <7 ms for LL-ConFINE on the same CPU, with GRU runtime dominated by nonlinear planning (Table 4). Additional comparisons appear in Figs. 4 and A7–A9. On the simulated stroke-recovery task (Section 3.3), LL-ConFINE achieves higher recovery than GRUs across all four lesions under behavioral supervision, including when GRUs receive 8x as many training updates (Table 3). We include the GRUs in the recovery and prediction analyses in Figs. A12–A21.

We also tested GRUs on the torus-rotation plant from the main text. With ConFINE’s training protocol and CEM planning, the GRU achieved a control cost of 3.43 versus 0.57 for ConFINE and 0.24 for the oracle. Larger hidden states (32 and 128 dimensions) did not improve performance. As such, these results are not included in the paper. We investigated two potential sources of poor generalization: training inputs are bounded by ±0.5 and change every step, whereas holding targets requires larger, sustained inputs (median steady-state magnitude: 2.6). On a slower-rotation plant, where most required inputs fall within the training range, GRU cost decreased to 1.80. Adding sustained-input training sequences reduced it further to 1.31, still above ConFINE’s 0.55 (table below).

GRU training dataTorus-rotation plantGRU, 1× updatesGRU, 8× updatesConFINEOracle
Random inputsOriginal3.463.430.570.24
Random inputsSlower rotation2.081.800.550.24
Random + sustained inputsSlower rotation1.931.310.550.24

Cost: distance to target over the last 10 steps (lower is better). 1× / 8×: ConFINE's training budget and 8 times it.

We additionally implemented a stochastic nonlinear state-space model [1][2] with a GRU filter, an MLP transition and readout, a 32-dimensional latent state, and CEM planning. On reaching RNN plants B and C, its control costs were 2.1–4.6x those of ConFINE, exceeding LDM’s costs at both input penalties (table below). Additional data and optimization did not close the gap: on plant C, even with 16x the training data and 8x the training updates, its cost remained 6.9 versus 3.6 for ConFINE. Likewise, these results are not included in the paper.

ModelB, R = 10⁻²B, R = 10⁻³C, R = 10⁻²C, R = 10⁻³
ConFINE2.09 ± 0.021.75 ± 0.023.57 ± 0.033.41 ± 0.04
LDM2.53 ± 0.092.46 ± 0.094.22 ± 0.114.07 ± 0.12
Stochastic SSM7.57 ± 0.177.99 ± 0.177.61 ± 0.127.48 ± 0.11

Closed-loop cost as in Table 2 (mean ± s.e., 4096 training sequences, lower is better).

We also implemented a Mamba baseline, trained with the same multi-step objective and evaluated with the same CEM planner as the GRUs. Under matched CPU benchmarking conditions, each control step took several seconds, compared with 190–352 ms for the GRUs and approximately 3 ms for ConFINE. We therefore did not pursue a broader Mamba evaluation: this implementation’s planning latency substantially exceeded the timing requirements of the neural feedback applications discussed in the paper.

[1] Krishnan, R. G., Shalit, U., and Sontag, D. Deep Kalman Filters. arXiv:1511.05121, 2015.

[2] Krishnan, R. G., Shalit, U., and Sontag, D. Structured Inference Networks for Nonlinear State Space Models. AAAI, 2017.

We appreciate the reviewer’s constructive feedback and hope the additional results and clarifications resolve the concerns raised. If so, we would be grateful if you would consider raising your score.

Reviewer_wH9FOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ConFINE, a controlled state-space model for learning and controlling neural dynamics. It uses a nonlinear encoder to represent high-dimensional neural observations and learns input-driven linear or locally linear dynamics in the latent space. A Kalman filter estimates the hidden states, while model predictive controller (MPC) plans stimulation inputs to drive the system toward desired neural or behavioral targets. The method is evaluated in simulations involving nonlinear manifolds, a task-trained RNN, and a simulated grasping circuit. Across these settings, ConFINE learns the underlying neural dynamics and controls neural populations to reach desired targets with improved performance and accuracy.

Critical strengths
  1. I generally appreciate the authors’ overall control-oriented design. ConFINE learns dynamics from a limited amount of open-loop stimulation data and then uses state estimation within a state-space model for closed-loop control, combining state estimation and closed-loop manipulation into a complete pipeline. For neural stimulation, avoiding repeated closed-loop exploration of the real system during learning is a practical design choice. Besides, I also appreciate the authors’ treatment of nonlinear structure in neural dynamics. ConFINE uses a nonlinear encoder to represent neural observations and learns a latent representation that is qualitatively consistent with the ground-truth latent space.
  2. The combination of multi-step prediction, Kalman filtering, and MPC is one of the valuable aspects of this work. Kalman filtering estimates the latent state from the current observation, multi-step prediction allows the model to predict the future responses that are relevant for control, and MPC optimizes the stimulation inputs based on these predictions. The simulation experiments are quite thorough. I particularly appreciate the four requirements proposed by the authors for neural control: nonlinear representation, state estimation under noise and partial observability, learning from limited open-loop data, and support for constrained inputs and target switching.
Critical weaknesses
  1. The comparison set is relatively narrow. LDM and AE+LDM demonstrate the effects of nonlinear representation and joint training, but they are relatively insufficient to show the advantages of ConFINE over other linear/nonlinear models. Could the authors include other state-space models, or models such as RNNs and Mamba, and compare their performance under the same conditions? The current set of baselines is still limited. There is also an issue for the task-driven RNN experiment setting. The task-driven RNN was originally trained with the target position u_t as its input, while the same input is later treated as an optimizable stimulation signal. If u_t is continuously set equal to the target position, would that already accomplish the task?
  2. Validation on real data is insufficient. All control experiments are conducted in simulation. Although the grasping model from Michaels et al. (PNAS 2020) was trained using real (kinematic) data, it may not substitute for neural validation of ConFINE itself. Could the authors show applicability on the method to real (spiking) neural signals? I understand that conducting closed-loop stimulation experiments in a real system may be difficult, but the authors could at least provide modeling and prediction results on real neural data.
Questions for authors
  1. How much time does state estimation and MPC optimization take per step for ConFINE and LL-ConFINE, respectively? What hardware was used, and can the method satisfy the control frequency required by the possible experiment condition? The paper repeatedly emphasizes real-time control, but currently provides no runtime or computational-cost analysis.
  2. Appendix D.3 evaluates prediction error under random stimulation. Does the rollout remain accurate under the stimulation sequences selected by MPC? How does the prediction error change as the prediction horizon increases?
  3. What is the rationale for choosing an eight-step rollout horizon?
  4. Do the authors provide anonymous code or supplementary materials? Do the authors plan to release the code, model, and configuration files required to reproduce the experiments?
  5. In Section 3.2, the behavioral output z_t is obtained through the linear readout, whereas u_t represents the target position during the original RNN training. Why is this task input subsequently optimized as a stimulation signal?
5 comments

We appreciate the reviewer’s careful reading and helpful feedback. Below, we respond to each point in detail and describe the revisions and additional analyses to address their concerns.

W1. Additional baselines

Could the authors include other state-space models, or models such as RNNs and Mamba, and compare their performance under the same conditions? The current set of baselines is still limited.

Primary experiments: GRU

We thank the reviewer for this suggestion. We have added nonlinear gated recurrent unit (GRU) RNN baselines with varying hidden-state sizes and training budgets as a representative nonlinear recurrent baseline to test whether ConFINE’s advantages persist against more expressive dynamics.

We evaluate them on the reaching RNN plants (Section 3.2), where even with 8x as many training updates, GRUs have higher mean control costs than LL-ConFINE on all three plants at both input-penalty settings (Table 2). LL-ConFINE also matches or exceeds the best GRU configuration trained on 4x sequences, even with 8x updates (Fig. A10). GRU controllers require 190–352 ms per control step, versus <1 ms for ConFINE and <7 ms for LL-ConFINE on the same CPU, with GRU runtime dominated by nonlinear planning (Table 4). Additional comparisons appear in Figs. 4 and A7–A9. On the simulated stroke-recovery task (Section 3.3), LL-ConFINE achieves higher recovery than GRUs across all four lesions under behavioral supervision, including when GRUs receive 8x as many training updates (Table 3). We include the GRUs in the recovery and prediction analyses in Figs. A12–A21.

We also tested GRUs on the torus-rotation plant from the main text. With ConFINE’s training protocol and CEM planning, the GRU achieved a control cost of 3.43 versus 0.57 for ConFINE and 0.24 for the oracle. Larger hidden states (32 and 128 dimensions) did not improve performance. As such, these results are not included in the paper. We investigated two potential sources of poor generalization: training inputs are bounded by ±0.5 and change every step, whereas holding targets requires larger, sustained inputs (median steady-state magnitude: 2.6). On a slower-rotation plant, where most required inputs fall within the training range, GRU cost decreased to 1.80. Adding sustained-input training sequences reduced it further to 1.31, still above ConFINE’s 0.55 (table below).

GRU training dataTorus-rotation plantGRU, 1× updatesGRU, 8× updatesConFINEOracle
Random inputsOriginal3.463.430.570.24
Random inputsSlower rotation2.081.800.550.24
Random + sustained inputsSlower rotation1.931.310.550.24

Cost: distance to target over the last 10 steps (lower is better). 1× / 8×: ConFINE's training budget and 8 times it.

Additional exploration: Nonlinear stochastic SSM and Mamba

We additionally implemented a stochastic nonlinear state-space model [1][2] with a GRU filter, an MLP transition and readout, a 32-dimensional latent state, and CEM planning. On reaching RNN plants B and C, its control costs were 2.1–4.6x those of ConFINE, exceeding LDM’s costs at both input penalties (table below). Additional data and optimization did not close the gap: on plant C, even with 16x the training data and 8x the training updates, its cost remained 6.9 versus 3.6 for ConFINE. Likewise, these results are not included in the paper.

ModelB, R = 10⁻²B, R = 10⁻³C, R = 10⁻²C, R = 10⁻³
ConFINE2.09 ± 0.021.75 ± 0.023.57 ± 0.033.41 ± 0.04
LDM2.53 ± 0.092.46 ± 0.094.22 ± 0.114.07 ± 0.12
Stochastic SSM7.57 ± 0.177.99 ± 0.177.61 ± 0.127.48 ± 0.11

Closed-loop cost as in Table 2 (mean ± s.e., 4096 training sequences, lower is better).

We also implemented a Mamba baseline, trained with the same multi-step objective and evaluated with the same CEM planner as the GRUs. Under matched CPU benchmarking conditions, each control step took several seconds, compared with 190–352 ms for the GRUs and approximately 3 ms for ConFINE. We therefore did not pursue a broader Mamba evaluation: this implementation’s planning latency substantially exceeded the timing requirements of the neural feedback applications discussed in the paper.

[1] Krishnan, R. G., Shalit, U., and Sontag, D. Deep Kalman Filters. arXiv:1511.05121, 2015.

[2] Krishnan, R. G., Shalit, U., and Sontag, D. Structured Inference Networks for Nonlinear State Space Models. AAAI, 2017.

W2. Task-driven RNN experiment setting

The task-driven RNN was originally trained with the target position u_t as its input, while the same input is later treated as an optimizable stimulation signal. If u_t is continuously set equal to the target position, would that already accomplish the task?

Yes, setting u_t = z* would accomplish the task, and this is by design: this built-in solution merely guarantees that the control problem is well-posed (i.e., that a solution exists), not that it is easy to solve. In particular, the controller is never told that a constant input suffices, let alone which one. It must solve for the stimulation sequence at every step from its learned model, which is fit to input-output responses to random stimulation with no access to the RNN’s weights, training objective, or built-in solution. Thus, it remains a challenge to control the RNN using such a learned model. Please also see the detailed response under the related Q5, below.

W3. Real data modeling

Could the authors show applicability on the method to real (spiking) neural signals? I understand that conducting closed-loop stimulation experiments in a real system may be difficult, but the authors could at least provide modeling and prediction results on real neural data.

We have now validated our model on real neural recordings for modeling and prediction. Specifically, we fitted ConFINE to real stimulation-driven neural recordings, consisting of cortical surface recordings (μECoG) acquired during optogenetic stimulation in nonhuman primates [1]. We have added these evaluations in Section 3.4, “Modeling in-vivo input-driven neural dynamics” of the manuscript.

ConFINE achieves higher prediction accuracy than LDM, AE+LDM, and GRU at multi-step prediction, with its advantage widening at longer prediction horizons. Since MPC relies on high-fidelity future prediction of neural responses to stimulation, the strong prediction of ConFINE serves as a prerequisite for future in-vivo control experiments in the brain. This is particularly relevant when the stimulation cost is strongly penalized. New simulation results (Fig A11, App C.6) show that longer rollouts become more beneficial under larger input penalties. This is especially important for neural stimulation, as minimizing total delivered input is often critical for both safety and power consumption considerations.

[1] Yazdan-Shahmorad, A., Silversmith, D. B., Kharazia, V., & Sabes, P. N. (2018). Targeted cortical reorganization using optogenetics in non-human primates. eLife, 7, e31034.

Q1. Time per control step

How much time does state estimation and MPC optimization take per step for ConFINE and LL-ConFINE, respectively? What hardware was used, and can the method satisfy the control frequency required by the possible experiment condition?

We have added wall-clock as well as FLOP measurements for filtering, control, and total time per step during closed-loop control on an AMD EPYC 9354 CPU, using the models fit to the RNN. The results are described in detail in the newly added Section C.4. In summary, after optimizing the solver, we achieve a computation time of 0.67 ms per step for the globally-linear variant, and 6.4 ms for the locally-linear variant of ConFINE, resulting in potential real-time control at rates of up to ~1.5 kHz or ~150 Hz, respectively, comparable to rates previously demonstrated in state-space control (1 kHz) [1], or adaptive DBS (10 Hz) [2]. We also emphasize an important advantage of our method over nonlinear approaches: it does not require explicit re-linearization by design, since the dynamics are locally linear, and we can use the state-dependent transition matrices generated by a nominal trajectory in the QP directly.

[1] Bolus et al. (2021): State-space optimal feedback control of optogenetically driven neural activity, Journal of Neural Engineering

[2] He et al. (2021): Closed-Loop Deep Brain Stimulation for Essential Tremor Based on Thalamic Local Field Potentials, Movement Disorders

Q2. Prediction under MPC and increasing horizon

Appendix D.3 evaluates prediction error under random stimulation. Does the rollout remain accurate under the stimulation sequences selected by MPC?

We added this evaluation in Fig. A17 and direct comparisons with random stimulation in Figs. A18–A21 (top: random; bottom: MPC-selected; identical normalization). Errors are generally higher under MPC-selected stimulation, but the models’ relative performance is largely preserved. In particular, supervised LL-ConFINE retains the lowest behavioral prediction error across all four lesions throughout the planning horizon. Fig. A17 further shows that behavioral prediction error under MPC-selected stimulation closely tracks recovery. Neural prediction errors likewise generally increase under MPC-selected stimulation (Figs. A18–A19). Notably, GRU baselines often predict neural observations better, but this does not translate into better behavioral prediction or recovery. Without behavioral supervision, behavioral errors can be large under either stimulation condition (Fig. A20).

How does the prediction error change as the prediction horizon increases?

Figs. A18–A21 show prediction error across the full eight-step MPC horizon under both stimulation conditions. Errors generally increase gradually. For supervised LL-ConFINE, behavioral NMSE under MPC-selected stimulation increases approximately from 15% to 20% for M1 lesions, 6% to 9% for AIP lesions, 7% to 11% for connection lesions, and 1% to 2% for combined lesions. Supervised LL-ConFINE retains the lowest behavioral prediction error across all four lesions throughout the horizon.

Q3. Rollout horizon rationale

What is the rationale for choosing an eight-step rollout horizon?

We chose eight steps empirically: training performance largely saturates by this horizon, and extending the MPC planning horizon beyond eight steps gives little additional benefit at the input penalty weights (i.e. R in the MPC cost) used in our main RNN experiments. To make this explicit, we added separate sweeps of both horizons in Appendix C.6, Figure A11. For training, prediction and closed-loop control performance generally improve with longer horizons, with diminishing gains beyond approximately 4-8 steps. For planning, at low input penalties, performance is largely insensitive to the horizon, or saturates by 4-8 steps. At larger penalties, longer planning horizons continue to improve performance. A likely explanation for this dependence on R is that stronger inputs can move the system toward the target faster, but incur a larger input cost. With a short horizon, the planner sees only a few steps of the resulting improvement in tracking, which may not outweigh that input cost when R is large. A longer horizon also counts the subsequent steps spent closer to the target, making stronger corrective inputs worthwhile. When R is small, their input cost is already low enough that extending the horizon has little effect on this tradeoff.

Q4. Code availability

Do the authors provide anonymous code or supplementary materials? Do the authors plan to release the code, model, and configuration files required to reproduce the experiments?

Anonymized code to reproduce all experiments, including the models, plants, and their configuration files, is available in the attached zip file, and will be fully available online after publication.

Reviewer_aXqbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ConFINE, a method that maps high dimensional neural activity into non-linear embeddings while maintaining linear or locally linear latent dynamics to enable tractable, real-time control. This specific architecture is designed to meet four requirements- retaining non-linear expressivity, enabling real-time online state estimation, learning efficiently from open-loop data, and supporting real-time control for new targets. Results on three simulated settings demonstrate that this approach successfully drives neural activity to target set-points on the manifold plants, drives behavioural readouts to targets in the RNN plants, and can effectively restore movement trajectories following simulated lesions in the grasping circuit.

Critical strengths
  1. The method is online. The latent state is estimated recursively and causally with a Kalman filter, so missing samples are handled
  2. Identification needs no closed-loop exploration, which fits the safety limits of neural stimulation.
Critical weaknesses
  1. Real-time operation is stated as a requirement but is not quantified. Reporting wall-clock time per control step with comparisons for filtering and MPC would support this claim. This seems most relevant for the locally linear planner, which at every control step re-linearizes the dynamics and re-solves the quadratic program five times (App. G.4). Sample efficiency is also stated as a requirement but is not directly evaluated. All experiments use a fixed training budget. A sweep over the amount of identification data, compared with the baselines, would show how much perturbation data the method needs in practice.
  2. The paper ablates the KL term and compares globally versus locally linear dynamics, but not the prediction horizon, the latent dimension or the encoder architecture. These ablations would clarify which components drive performance.
  3. The authors acknowledge that in vivo closed-loop control is a next step, and that is understandable. Still, even offline identification and prediction on a real stimulation dataset would strengthen the paper. TBFM (Bryan et al., 2025) has done this with real optogenetic stimulation data from non-human primates, and DFINE and BRAID show that similar model classes can be fit to real neural recordings.

References :

[1] DFINE: H. Abbaspourazad, E. Erturk, B. Pesaran, M. M. Shanechi. Dynamical flexible inference of nonlinear latent factors and structures in neural population activity. Nature Biomedical Engineering, 8(1):85–108, 2024.

[2] BRAID: P. Vahidi, O. G. Sani, M. M. Shanechi. BRAID: Input-Driven Nonlinear Dynamical Modeling of Neural-Behavioral Data. ICLR 2025, arXiv:2509.18627, 2025.

[3] TBFM: M. J. Bryan, F. Schwock, A. Yazdan-Shahmorad, R. P. N. Rao. Temporal basis function models for closed-loop neural stimulation. Journal of Neural Engineering, 22(5):056012, 2025.

Questions for authors
  1. A sizable gap to the oracle remains on the RNN plants. ConFINE+KL improves consistently over the baselines, closing roughly 30–60% of the gap between the best baseline and the oracle across plants A–C. Can you explain what limits the remaining gap (model error, estimation error, or planning)?
  2. Which ConFINE variant is used for the reach RNN plants? If it is the globally linear one, then the locally linear variant, which the Discussion describes as capturing nonlinear dynamics, is evaluated only on the grasping circuit. How does it perform on the reach RNN plants, whose dynamics are nonlinear?
4 comments

We appreciate the reviewer’s careful reading and helpful feedback, particularly their suggestions for evaluating runtime, sample efficiency, architectural choices, and performance on real neural data. Their feedback has guided several additions and clarifications, and we address each point in detail below.

W1.1. Time per control step

Real-time operation is stated as a requirement but is not quantified. Reporting wall-clock time per control step with comparisons for filtering and MPC would support this claim.

We have added wall-clock as well as FLOP measurements for filtering, control, and total time per step during closed-loop control, using the models fit to the RNN. The results are described in detail in the newly added Section C.4. In summary, after optimizing the solver, we achieve a computation time of 0.67 ms per step for the globally-linear variant, and 6.4 ms for the locally-linear variant of ConFINE, resulting in potential real-time control at rates of up to ~1.5 kHz or ~150 Hz, respectively, comparable to rates previously demonstrated in state-space control (1 kHz) [1], or adaptive DBS (10 Hz) [2]. We also emphasize an important advantage of our method over nonlinear approaches: it does not require explicit re-linearization by design, since the dynamics are locally linear, and we can use the state-dependent transition matrices generated by a nominal trajectory in the QP directly.

[1] Bolus et al. (2021): State-space optimal feedback control of optogenetically driven neural activity, Journal of Neural Engineering

[2] He et al. (2021): Closed-Loop Deep Brain Stimulation for Essential Tremor Based on Thalamic Local Field Potentials, Movement Disorders

W1.2. Sample efficiency

Sample efficiency is also stated as a requirement but is not directly evaluated. A sweep over the amount of identification data, compared with the baselines, would show how much perturbation data the method needs in practice.

We have added evaluations of control performance on the reach-RNN plants as a function of training set size (App C.5, Figure A10). On plants A and B, globally linear ConFINE achieves lower mean control costs than all evaluated baselines with only 256 training sequences, including GRUs trained with 8x the standard number of updates. Across all three plants, LL-ConFINE trained on 4096 sequences achieves control costs lower or comparable to those of the best-performing GRU configuration trained on 16384 sequences. LL-ConFINE’s performance largely plateaus by 4096 sequences, whereas the GRU approaches this performance level when trained with 4x the dataset size and 8x the standard number of training updates (while remaining too slow for real-time control, as demonstrated above).

W2. Ablations

The paper ablates the KL term and compares globally versus locally linear dynamics, but not the prediction horizon, the latent dimension or the encoder architecture. These ablations would clarify which components drive performance.

We have added ablations of these features using the three reach-RNN plants as testbeds (App C.6, Figure A11). We vary the maximum training prediction horizon from 1 to 16 steps, latent dimension from 4 to 64, encoder/decoder width from 32 to 512 units per hidden layer, and encoder/decoder activation function across ELU, ReLU, and tanh. Both prediction and control performance generally improve with increasing training horizon and latent dimension, with diminishing gains beyond approximately 4–8 steps and 16–32 latent dimensions. Activation choice has relatively little effect on control. Interestingly, increasing encoder/decoder width consistently reduces prediction error but produces no corresponding improvement in control. One possible explanation is that smaller networks already capture the neural subspaces relevant to control, while additional capacity primarily improves prediction in subspaces irrelevant to the control objective.

W3. Real stimulation dataset

Still, even offline identification and prediction on a real stimulation dataset would strengthen the paper. TBFM (Bryan et al., 2025) has done this with real optogenetic stimulation data from non-human primates.

We have validated our model on real neural recordings for modeling and prediction, as suggested by the Reviewer, using one sample session from the recommended dataset. We have added these evaluations in Section 3.4, “Modeling in-vivo input-driven neural dynamics” of the manuscript. ConFINE achieves higher prediction accuracy than LDM and AE+LDM at multi-step prediction, with its advantage widening at longer prediction horizons. Since MPC relies on high-fidelity future prediction of neural responses to stimulation, the strong prediction of ConFINE serves as a prerequisite for future in-vivo control experiments in the brain. This is particularly relevant when the stimulation cost is strongly penalized. New simulation results (Fig A11, App C.6) show that longer rollouts become more beneficial under larger input penalties. This is especially important for neural stimulation, as minimizing total delivered input is often critical for both safety and power consumption considerations.

Q1. Gap to the oracle

A sizable gap to the oracle remains on the RNN plants. Can you explain what limits the remaining gap (model error, estimation error, or planning)?

Precisely separating these contributions is difficult because they interact: model error affects both estimation and planning, and estimation error perturbs the states from which planning is done. Our additional experiments nevertheless help identify what contributes to the oracle gap.

First, LL-ConFINE further reduces the globally linear model’s remaining oracle gap on plant C, indicating that limitations of the dynamics representation partly contribute to the gap. On the other hand, extending the MPC planning horizon from 8 to 16 steps yields little further improvement at either input penalty used in the main experiments, suggesting that the horizon is not a major limitation under these conditions.

Finally, even with LL-ConFINE, the gap is larger on plant C than on plant B, which has the same initial-state distribution but lower process noise. This suggests that greater dynamics flexibility does not fully resolve the difficulties introduced by noise. Higher process noise makes the model’s state predictions less reliable, placing greater demands on state estimation from noisy observations. Any remaining estimation error then affects planning. We therefore suspect that noise-sensitive state estimation, together with residual model error, contributes substantially to the remaining gap.

Q2. Locally linear variant on reach RNN plants

How does [the locally linear variant] perform on the reach RNN plants, whose dynamics are nonlinear?

The original reach-RNN experiments demonstrate the globally linear version of ConFINE. We have added the locally linear variant to Table 2 and Figures 4, A8, and included it in the new real-time operation (App. C.4, Table 4) and sample efficiency (App C.5, Figure A10) metrics. As described in Q1, the locally linear variant outperforms the globally linear one, and even remains comparable to fully nonlinear models (GRU), even when they are trained with 8x more updates and 4x more data (Figure A10).

We thank the reviewer again for the suggestions that motivated these additional experiments and clarifications. If you find that the new evidence addresses your concerns, we would be grateful if you would consider revising your score upward.

Reviewer_aXqbOpenReview ↗

Response to the authors

I would like to thank the authors for their in-depth rebuttal which addressed all of my concerns. I would be happy to raise my scores. I look forward to seeing the method extended to continuous closed-loop control in vivo.

We thank the reviewer for their encouraging follow-up and are glad that our response addressed all of their concerns. We share their enthusiasm for extending the method to closed-loop control in vivo.

We also appreciate their willingness to raise their scores. The original scores still appear on our end -- we would appreciate it if you could update them when convenient.

Comments on the paper

2 comments

We thank the reviewers for their thoughtful comments and constructive suggestions. We appreciate their recognition of ConFINE’s control-oriented design, nonlinear representations, recursive state estimation, and identification from open-loop data without closed-loop exploration. We have addressed their comments individually and substantially expanded the experiments and clarified the manuscript, as summarized below. We have uploaded a new version of the manuscript, with changes highlighted in red.

  1. Validation on biological neural recordings. We added modeling and prediction results on cortical surface recordings (μECoG) acquired during optogenetic stimulation in nonhuman primates, where ConFINE outperforms LDM, AE+LDM, and GRU baselines at multi-step prediction, with larger advantages at longer horizons (Section 3.4). These results complement our simulated control experiments. As clarified in the Introduction and Discussion, in-vivo closed-loop control of population dynamics remains elusive for the entire field, and establishing feasibility in simulation is a necessary step toward it, which is the step this paper takes.

  2. Expanded nonlinear comparisons and ablations.

    1. Nonlinear baselines: We added GRU baselines with varying hidden-state sizes and training budgets to the reaching and stroke-recovery experiments, where LL-ConFINE achieves better control even against GRUs trained with eight times as many updates. We also investigated a nonlinear stochastic SSM and Mamba as alternatives; preliminary experiments showed substantially higher control costs for the stochastic SSM and prohibitive planning latency for Mamba, supporting ConFINE’s balance of control performance and computational efficiency.
    2. Ablations: We added sweeps of training and planning horizons, latent dimension, encoder/decoder width, and activation function, evaluating both prediction and control performance (Appendix C.6). These analyses support the eight-step horizon and show that improved neural prediction alone does not necessarily yield better control.
  3. Additional control, prediction, and efficiency evaluations.

    1. Locally linear dynamics and oracle comparisons: We added LL-ConFINE to the reaching-RNN experiments, demonstrating further improvements over globally linear ConFINE, and expanded oracle references across evaluation figures and the stroke-recovery results. We also discuss how model limitations and noise-sensitive estimation contribute to the remaining oracle gap.
    2. Prediction during control: We now evaluate prediction across the planning horizon under both random and MPC-selected stimulation. Although errors generally increase under MPC-selected inputs, supervised LL-ConFINE retains the lowest behavioral prediction error across all four lesions throughout the horizon.
    3. Runtime: We added hardware details, wall-clock timings, and FLOP measurements for filtering and control (Appendix C.4). On the benchmark CPU, total computation takes 0.67 ms per step for ConFINE and 6.4 ms for LL-ConFINE, versus 190–352 ms for GRU controllers.
    4. Sample efficiency: We added control-performance sweeps over identification dataset size (Appendix C.5). Across all three reaching plants, LL-ConFINE trained on 4,096 sequences matches or exceeds the best GRU configuration trained on four times as much data and eight times as many updates.
  4. Presentation, methodological clarity, and reproducibility.

    1. Overview figures and explanations: We expanded the introductory motivation and added diagrams of open-loop identification, closed-loop control, and the locally linear architecture. The main text now explicitly explains matrix generation, training rollouts, nominal trajectories, and iterative MPC updates.
    2. Reaching-RNN task design: We clarified the distinction between designing the simulated plant and identifying and controlling it. Reusing its task-input channel guarantees reachable targets, while the learned controllers receive no knowledge of the plant’s weights, training objective, or built-in solution.
    3. Code availability: Anonymized code reproducing all experiments, including models, plants, and configuration files, is now provided in the supplementary materials and will be publicly available online after publication.

We hope these additions and clarifications address the reviewers’ concerns and strengthen the evidence for ConFINE’s contribution. If the reviewers find their concerns resolved, we would be grateful if they would consider raising their scores.

We again thank all of the Reviewers for their review of our work. We believe we have addressed the concerns that were raised, and substantially strengthened both our results and paper presentation/clarity. As the discussion period ends tomorrow, we kindly ask the Reviewers to let us know if there is anything else we can clarify. If they are satisfied with our comments and updated manuscript we kindly ask them to consider raising their score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.