- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The neuroscience community is increasingly interested in building latent dynamics models of neural population activity to analyze how the brain transforms sensory information and state into behavior and decisions. Such models usually assume the structure of an experiment, with trials that have known start and end times. However, the field is moving towards less structured paradigms, such as naturalistic behavior without concrete task boundaries and sleep, which violate these assumptions. Most existing techniques also fail to account for unobserved neural populations and external inputs, unless explicitly provided to the model, which is not always possible or may hurt interpretability. This paper develops Neubridge, extending the Schrödinger Bridge formulation to learn latent stochastic dynamics and inferred inputs from unobserved populations, with a focus on recordings that lack explicit task boundaries. The authors show promising model fits to experimental data recorded during sleep, with optogenetic stimulation that lead to switching the phase of sleep. The authors compare to several existing baseline methods and show that Neubridge outperforms the other methods in terms of model fits and flow field inference. It can also predict the correct timing of photostimulation perturbations without being explicitly trained to do so. Altogether, this model appears to combine the concept of inferred inputs from unobserved populations and external stimulation with flow field inference. While both components exist in the literature, this paper is the first to combine them and demonstrate advantages on real experimental data.
Critical strengths
Their model, Neubridge, is validated on real experimental data and compared to a number of popular alternatives. Neubridge demonstrates better model fits and more interpretable flow fields than these competing models. The inferred inputs also show the ability to infer experimentally-relevant information, such as optogenetic simulation perturbations upstream of the recorded populations.
Critical weaknesses
-
Inferred inputs from unobserved populations is a known and important features of LFADS. While the authors compare Neubridge to LFADS on reconstruction, the inferred input aspect of LFADS is ignored. Since this is considered a core motivation for the paper, this feature of both models should be compared. Moreover, the paper would be further strengthed by a synthetic dataset that included unobserved inputs beyond the noise introduced in the Lorenz attractor example. I would be very interested to see how well the inferred inputs correspond to the true external stimuli when known.
-
I found it unclear what was specific about the design decisions in Neubridge to the stated problem of training latent dynamics models without explicit trial structure. I can understand how the initial state prior, which was formulated in a way that conditions only on a window of history, and inferred inputs can help with this. But these are both standard practice in LFADS as well, so not clearly new.
Questions for authors
- How do the inferred inputs of Neubridge and LFADS compare?
- What about the design decisions in Neubridge help with the stated problem of training on data without explicit experimental trial structure?
We thank the reviewer for your constructive comments!
W1 Response: (Inferred inputs from unobserved populations is a known and important features of LFADS..., ...the paper would be further strengthed by a synthetic dataset that included unobserved inputs...)
We thank the reviewer for the suggestions. We have now included unobserved inputs to the synthetic dataset and show that NeuBridge can recover these inputs. We added two examples visualizing the inferred corrections of the autonomous process to account for the unobserved inputs (Figure 2e).
In addition, we also applied LFADS to the optogenetic dataset. Similar to NeuBridge, LFADS also shows an increase in inferred input for the optogenetic dataset (Supplementary Figure S2). But the key additional advantage of NeuBridge is that the correction is defined directly relative to the autonomous reference dynamics within the same latent space. Thus, the effect of an inferred input can be visualized as an observation-informed deviation (Q) from the autonomous trajectory R, allowing us to localize where in the latent landscape additional control is required and how that control reshapes the autonomous dynamics (see Figure 4f as an example). In LFADS, by contrast, the inferred input is fed into the recurrent generator and therefore is not directly represented as a correction vector within the latent factor space. We have clarified this distinction in the revised manuscript.
W2 & Q2 Response: (What about the design decisions in Neubridge help with the stated problem of training on data without explicit experimental trial structure?)
We agree that conditioning the initial state on preceding history and inferring observation-dependent inputs are not individually unique to NeuBridge and are also used in LFADS. The design choice that is specific to the absence of trial structure is instead the segment-based Schrödinger bridge formulation. NeuBridge can sample arbitrary segments from a continuous recording, infer their start and end latent distributions, and learn the dynamics connecting these boundary distributions. By repeatedly sampling segments that span different time intervals and different locations in latent space, the same reference process R is trained across the broader latent landscape, without requiring trial-aligned initial conditions or repeated trajectories.
In addition, NeuBridge models the diffusion term as a function of latent state, allowing the level of stochasticity to vary across the latent landscape. This is particularly relevant for spontaneous sleep recordings, where the variability of neural population activity is likely to differ across brain states and transitions.
Q1 Response: (How do the inferred inputs of Neubridge and LFADS compare?)
In LFADS, the inferred input is fed into the generator at each step, thereby modifying the generator state and, subsequently, the low-dimensional latent factors. The inferred input for LFADS is thus not represented in the same shared latent space as the internal dynamics. In NeuBridge, the correction is defined directly relative to the autonomous reference dynamics (R) within the same latent space. Thus, the effect of an inferred input can be visualized as an observation-informed deviation (Q) from the autonomous trajectory R, allowing us to localize where in the latent landscape additional control is required and how that control reshapes the autonomous dynamics.