- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a stimulus-conditioned generative model of population spiking activity at 1 ms resolution. It treats neural encoding as conditional generation in the latent space of a frozen spike tokenizer, a per-unit VAE developed in a concurrent submission. That tokenizer maps each unit binarized spike train to one continuous token per 100 ms period. A latent flow-matching transformer, conditioned on stimulus features through per-unit cross-attention queries, learns the joint distribution of tokens across units and time. Samples are then decoded deterministically back to spike trains, so all trial-to-trial variability comes from the initial noise of the flow and happens in the latent. The result is a stochastic emulator of the stimulus-conditioned response distribution p(Y | x) [Y being the spike trains and x the stimulus], rather than of the mean response alone or than a assumed-variability structure. I emphasize on the fact that the model is non-autoregressive, it generates the full window jointly (i.e. acausal generation within a window).
The model is evaluated on two public datasets that differ in species, modality, and recording technology. The first is macaque V4 responses to natural images, using a jointly fine-tuned DINOv3 encoder. The second is ferret auditory cortex responses to natural sounds, using a convolutional encoder trained from scratch on cochleagrams.
Evaluation covers four axes: conditional-mean prediction, single-unit and shared variability (Fano factors and noise correlations), spike timing at 1 ms (ACG and CCG), and completion of partially observed responses. The baselines are an independent Poisson encoder, a temporal extension of FlowFA, and a reimplementation of another latent-state model of Schmidt et al. (2025). All baselines operate on 20 ms count bins.
A (small in my opinion) ablation study is presented. An auxiliary projected CRPS loss is introduced specifically for the proposed model (called the SpikeDiT) in order to correct the over-dispersion of spike counts that the authors observe when training with flow matching alone, and to calibrate both single-unit variability and pairwise covariance on the decoded spikes rather than only in the latent space.
Critical strengths
-
Monte Carlo convergence is analysed explicitly and the metrics are careful and well defined.
- Because the model is defined only through samples, every reported statistic is a Monte Carlo estimate. The authors study convergence with the number of draws on both datasets. They also use the analytic limit of the Poisson baseline as a sanity check, which shows that the reported numbers are stable.
-
Evaluation goes well beyond the mean.
-
Besides conditional-mean prediction (which is a sanity check more than a result, in my opinion), the model is assessed on single-neuron dispersion, on both the pattern and the magnitude of shared variability, and on spike timing.
-
The completion experiment includes a well-designed control. It shows that the gain comes from trial-specific shared variability rather than from additional stimulus information.
-
(Weak point: see weaknesses on pedagogy.)
-
-
Generality across two very different datasets.
- The two datasets differ in species (macaque vs ferret), sensory modality, brain area (V4 vs auditory cortex), recording technology, population size and response duration.
-
Figure quality.
- The figures are clear, well designed, and carry the paper message effectively. Figure 1 gives an immediate overview of both the method and the main results, and Figure 2 makes the pipeline easy to follow (even if I would swap left and right panels). The results figures are informative.
-
The latent-regularization ablation is a valuable experiment.
- It is consistent with the broader observation of the field that a well-regularized latent space, not just accurate reconstruction, is important for latent generative modeling.
-
The limitations section is honest and well thought out.
- The authors name the specific costs of their design: no tractable likelihood, no interpretable latent state, approximate conditioning, dependence on the tokenizer, acausal stimulus context. Each is framed against baselines that do not share it, which tells the reader when a latent-variable model would still be preferable.
Critical weaknesses
I will separate the weaknesses into two categories reflecting my personal assessment: (1) non-decisive points, which I would welcome being addressed but which will not affect my score, and (2) decisive points, which, if addressed, would lead me to increase my score.
(1) NON-DECISIVE POINTS
-
The paper could explain things more pedagogically e.g. why going beyond the mean matters.
-
The introduction motivates the full distribution mostly by citation, without a concrete question that only a distributional model can answer (e.g., the impact of noise correlations on coding, or planning trial and neuron counts). The most compelling use cases, in silico experiments relying on variability, appear only at the end of the discussion and would be stronger in the introduction.
-
Space for this could come from the implementation and training paragraphs (lines 276 to 299). The optimizer settings, betas, learning-rate schedule, warm-up steps, gradient clipping and flow-time distribution are already listed in Table S1. A one or two sentence summary pointing to Appendix C/F would be enough in the main text.
-
-
Sensitivity of the CRPS term is not explored.
- The dose hyperparam. differs between datasets (0.05 vs 0.01). A sensitivity analysis would show how robust the calibration is.
-
Minor presentation issues.
- Figure 2 might read more naturally with the left and right panels swapped.
-
~Line 80, it is said that no binning is done, i disagree with this formulation, and I think that "no coarse" binning would be more honest.
(2) DECISIVE POINTS
-
No baseline for spike timing or for completion.
-
All baselines work with 20 ms counts, so the ACG/CCG comparison is won partly by construction. A baseline that models timing explicitly, such as a GLM with spike-history filters (Pillow et al., 2008), would make the timing result more informative. The independent Poisson model is a useful sanity check, but a correlated Poisson model or similar would be a stronger baseline.
-
LDNS is discussed at length as the closest work but is not compared empirically.
-
I would like the authors either to explain why this comparison is not possible or not relevant (I may be missing an obvious point here, and I am genuinely curious), or to add it.
-
Since other baselines were adapted to fit the framework and ensure a fair comparison, the same could be done for LDNS if needed. Notably, the companion paper already implements (I think) LDNS and LDNS-Perceiver as population tokenizers, so this comparison should be feasible with limited effort.
-
-
-
Strong dependence on an unreleased tokenizer from a concurrent submission.
-
I partially read the concurrent paper in the appendix to better understand the submission. My understanding is that the present paper contribution is the latent generative pipeline built on top of that tokenizer. The tokenizer choice should therefore be discussed, and I think an ablation is important: either generating directly in spike (or count) space, or using another published encoder/decoder.
-
The sentence "For neural data, spike tokenizers now provide such a latent space by encoding each neuron's spike train into continuous tokens (Anonymous et al., 2026)" should be tempered and the citations broadened (e.g., LDNS also learns a continuous latent space for spikes, at the population level).
-
Questions for authors
See the (2) DECISIVE POINTS of the weaknesses.
Summary of the discussion and final revision
We thank the reviewers for a discussion that strengthened both the experiments and their presentation. We have uploaded our final discussion-period revision, now titled “Stimulus-conditioned flow matching captures shared variability in population spike trains”. The paper asks the same scientific question throughout: can a single encoding model predict the mean response to a stimulus accurately while also learning how neurons vary together across repeats, at 1 ms resolution? This final reorganization retains the main figures and all results from the preceding revision. We changed the order of the results so that the controls added during rebuttal are now better integrated in the paper.
Here is a summary of the changes:
This re-organization was directly shaped by the rebuttal’s experiments, for which we are grateful to the reviewers for suggesting:
In the final discussion, Reviewer vwTv judged the scientific content to meet the ICLR standard and suggested giving the joint generator greater prominence. Reviewer uWdH also recognized the improvement and raised their recommendation, while encouraging a similar shift in emphasis. We took that feedback seriously, and have spent the last day of rebuttal to go in that direction. The paper still retains its scientific question and main figures, but now places the controls added during rebuttal directly after the result they help explain, leading, in our opinion, to an overall better and more coherent presentation.