acceptodds
Withdrawn from ICLR 2027

A discrete generative model of neuronal spiking activity on microelectrode arrays

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative models of neural activity could help characterize tissue dynamics, compare experimental conditions, and simulate population activity for applications ranging from disease and drug-response studies to closed-loop experimentation. Existing approaches, however, typically assume a fixed set of sorted neurons, whereas high-density microelectrode arrays produce extremely sparse, array-wide binary spike volumes in which the observed subset of electrodes varies across assays. We introduce a discrete generative model that represents this activity using a shared vocabulary of spatiotemporal motifs. A residual vector-quantized autoencoder learns the motif vocabulary, while a factorized masked transformer predicts where activity occurs and which motif appears at each active location. We evaluate the model on 31 assays spanning human brain organoids and acute *ex vivo* human hippocampal tissue. The learned motifs are broadly reused: assay identity explains only of the entropy in motif use, and motif overlap across tissue types is comparable to overlap within them. When representation quality is evaluated independently of the generative prior, our approach achieves the voxel-level reconstruction average precision of a matched flat tokenizer. For masked completion and free generation, the full model achieves – the site-level average precision of the matched generative baseline and outperforms it across all four families of generation metrics. These results establish a compact, reusable representation for array-wide spiking activity without learned assay-specific parameters, providing a scalable foundation for generative modeling across diverse neural preparations.

Primary area:
applications to neuroscience & cognitive science
Keywords:
generative models, vector quantization, discrete representation learning, masked generative transformers, neural spiking activity, high-density microelectrode arrays, brain organoids

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HNctOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a discrete generative model of spontaneous spiking activity recorded on high-density microelectrode arrays, where each recording observes only a small and varying subset of the array, with a potentially different subset in each recording. Rather than modeling a fixed set of channels or sorted units, the authors place every recording on the full array footprint, so that recordings with different observed subsets share one common spatial canvas.

The model is "discrete" in the sense that a VQ-VAE maps patches of activity to a finite set of symbols, and a masked transformer prior (MaskGIT) generates sequences of these symbols. The idea is that the symbols correspond to spatiotemporal activity motifs, which the authors claim are shared, at least in part, across recordings and datasets.

The structure of the paper and the lack of a problem formalization make the contributions hard to identify (even if some contributions are stated explicitely by the authors, I'm focusing on the contributions that I am really able to identify in the manuscript), in my opinion, but I would describe them as twofold: (1) a discrete generative model adapted to extremely sparse array-wide spike volumes, combining a residual VQ-VAE with a blank token for empty patches and a prior factorized into where activity occurs and which motif appears there, applied to organoid and ex vivo hippocampal recordings; (2) evidence that the learned motifs are reused across recordings and across the two tissue types, obtained with a single shared model that has no learned per-recording parameters. To support (1), the authors report better reconstruction and completion accuracy than a flat-codebook MaskGIT baseline, while acknowledging that a static per-recording site map and supervised inpainters perform better on completion. To support (2), they report that recording identity explains only 9% of motif usage entropy and that motif overlap across tissue types is comparable to overlap within them, evaluated on held-out segments of the training recordings.

Critical strengths
  • The problem setting is relevant and under-explored: modeling array-wide activity across recordings whose observed electrodes differ.

  • The idea of encoding population activity as a shared motif vocabulary is appealing and I think could lead to new kind of latent structure/interpretability/understanding of neuronal computation.

  • The reporting is transparent. The paper discloses negative results (simple baselines beating the method), and provides the associated code.

Critical weaknesses

I found the paper difficult to follow: the presentation is dense, key definitions are scattered across the main text and appendices, and the flow of ideas is often interrupted by forward references and defensive statements. Some of my comments below may therefore reflect misunderstandings rather than actual flaws, and I welcome corrections from the authors. More broadly, I believe the current presentation substantially hinders a thorough technical assessment, and a clearer structure would allow readers to evaluate the contributions on their merits.

  • The problem is never formalized.

    • There is no clear probabilistic statement of the task in a standard generative modeling framework. Equation 3 is a good start, but there is no derivation of this generative model and assumptions are not properly discussed.
  • Generalization and scalability are not tested.

    • The claim of no per-assay parameters is a design property. The random assay code makes generalization to unseen recordings impossible (but I may be wrong on this, and if this is the case, I’m curious on how the author would do it?)
  • The pipeline is complex without comprehensive ablation.

    • Six stages and roughly twenty loss terms and they are not individually ablated; yet they are not classical in the VQ-VAE or the neural generative field.
  • The presentation is hard to follow and defensive.

    • The introduction concedes limitations before stating the problem, uses undefined terms, discusses evaluation before the idea, and points mainly to Sections 5 and 6, making Sections 2 to 4 feel secondary. Section 2 mixes baseline justification, method claims, and provenance. It omits the motif-detection literature (seqNMF, PPSeq, SPADE, Cell Assembly Detection).

    • Figure 1 is a cluttered block diagram whose caption does not explain its encoding (e.g., the blue boxes). Figure 2 panel A message is not clear and the panel B right part is under-discussed.

  • Limited accessibility for a general machine learning audience.

  • The paper assumes substantial familiarity with HD-MEAs and neural recording practice. Key concepts such as how the array works, why only about 1,000 of 26,400 electrodes can be read at once, what electrode routing means, what spike sorting produces, and why recordings differ in their observed electrodes are mentioned only briefly or scattered across Sections 1, 3, and the appendices. Yet these points motivate the whole approach. For an ICLR audience, a short self-contained primer would make the problem, and the relevance of the proposed design, much easier to grasp. It could cover the recording hardware, the path from voltage to binary spike volumes, and the two tissue preparations, ideally with a simple illustrative figure. In its current form, the paper fit with a general machine learning venue is weakened by this missing background.

  • The experiments do not establish why the method should be preferred over alternatives. On the paper own completion tasks, a static per-assay site map, which carries no clip-specific information, outperforms all learned models in every setting. The directly supervised convolutional baselines also outperform the proposed method on most completion settings. The only learned baseline the method consistently beats is MaskGIT-flat, which the authors constructed themselves and which differs from the proposed model in several components at once (tokenizer architecture, codebook structure, blank-token routing, and auxiliary losses). As a result, none of the proposed design choices is isolated by a controlled comparison, and it remains unclear which of them, if any, drives the observed gains. The authors argue that their contribution is a reusable discrete representation rather than imputation accuracy, but the usefulness of this representation is never demonstrated.

Questions for authors

I find the core idea genuinely interesting: representing array-wide activity from recordings with different observed electrodes as a shared vocabulary of motifs could open new ways to compare preparations and study population structure. However, I believe the paper in its current form would require modifications too extensive to be addressed within a rebuttal for me to consider it publishable in the main track. In particular, the formalization needs to be deeply reworked: the generative task, what distribution is being modeled, and how the conditioning relates to the observed data should be stated explicitly before the method.

Formal task. Can you state the target distribution at the voxel level, for example p(X_hidden | X_visible, g_r), and explain how the token-level factorization of eq. 3 and the decoder together define it ? Is the decoder deterministic or sampled?

Usefulness of the representation. A downstream demonstration would substantially strengthen the claim that the motif vocabulary is useful. Examples include predicting tissue type or recording condition from motif usage, or comparing the learned motifs with those found by established methods such as seqNMF, PPSeq or SPADE.

Conditioning on the hidden region. The local descriptor l(X) is computed from the full clip, hidden region included, and its first feature is log mean firing density. How do completion accuracy and the generation families A to D change when l is computed from the visible region only, or omitted? Without this, it is unclear how much of the reported performance comes from information about the answer.

Per-recording code. With 31 fixed random codes and a learned mapper, how does your model differ in practice from one with a learned embedding table? What concrete benefit does the absence of per-recording parameters provide, given that the model cannot be applied to an unseen recording?

Reviewer_QGLKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the generative modeling of population-level neural spiking activity recorded using high-density microelectrode arrays (HD-MEAs). To address challenges such as the extreme sparsity of spike data and variations in electrode configurations across recording assays, the authors propose a discrete generative framework. Specifically, a Residual VQ-VAE is first employed to encode high-dimensional spike volumes into a shared vocabulary of discrete spatiotemporal motifs. Two MaskGIT Transformers are then used to predict where neural activity occurs (“Where”) and which motif is associated with each location (“What”), respectively. Experiments are conducted across 31 assays and four generative tasks. The results demonstrate that the proposed method outperforms the MaskGIT-flat baseline in both oracle reconstruction and site-level generation.

Critical strengths
  1. Developing a unified generative framework for high-dimensional and sparse neural population spiking activity is a research direction of meaningful value.
  2. The method is thoughtfully designed to account for the inherent sparsity of spike data. Specifically, the dedicated blank token prevents inactive regions from consuming codebook capacity, while the activity/motif factorization separates the modeling of activity sparsity from the generation of specific spatiotemporal patterns.
  3. The experimental analysis is relatively comprehensive. The authors provide oracle-code reconstruction results, motif reuse analysis, and several ablation studies, which collectively offer empirical support for the main motivations underlying the proposed method.
Critical weaknesses
  1. The methodological novelty of this work is somewhat limited. The main components, including Residual VQ-VAE and MaskGIT, are adopted from existing work.
  2. Although the proposed method achieves substantial improvements in oracle reconstruction, the full model still underperforms the 3D U-Net in voxel-level generation, and the learned prior achieves only approximately 4–6% of the oracle voxel-level performance. This raises the question of whether the high-quality discrete representations can be effectively translated into accurate spike generation, potentially weakening the paper's central claim regarding the effectiveness of the proposed generative framework.
  3. All train/test splits are performed temporally within each assay, without evaluating generalization to unseen assays or biological preparations. Given that the dataset is derived from only a small number of biological preparations, the current motif reuse analysis is insufficient to establish the model's ability to generalize across recordings.
  4. The primary baseline, MaskGIT-flat, differs from the proposed method in several aspects simultaneously, including the encoder architecture, codebook structure, and blank-token design. Consequently, it is difficult to determine which components are responsible for the observed performance improvements, leaving the individual contributions of the proposed design choices unclear. Additional controlled ablation studies would help isolate the effects of these components and quantify their respective contributions to the overall performance gains.
Questions for authors

see weakness.

Reviewer_Pf2GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a discrete generative model for sparse neuronal spike volumes recorded using high-density microelectrode arrays. A residual vector-quantized autoencoder learns a shared vocabulary of spatiotemporal motifs, while separate masked transformers model activity locations and motif identities. Evaluation covers 31 assays from human brain organoids and ex vivo hippocampal tissue, using temporal splits within assays.

Critical strengths

The paper addresses an important problem in neural data modeling: representing and generating highly sparse population activity across multiple recordings. The problem has clear relevance to computational neuroscience, and progress in this direction could support neural data analysis, simulation, and experimental investigation.

Critical weaknesses
  1. The target-informed evaluation limits the predictive claims, and the count baseline is insufficient. Section 4.3 explicitly computes the local descriptor from the entire clip, including the hidden region. This defines a legitimate descriptor-controlled completion task, but it does not establish forecasting from observed activity alone. More specifically, the supplied whole-clip firing density determines the total spike count; subtracting the observed spikes then determines the hidden spike count, up to numerical precision. Appendix N instead compares against a weaker control that distributes the total density proportionally over the hidden volume. Consequently, outperforming that control does not establish meaningful learned count prediction. This materially weakens the interpretation of the reported count-discrimination advantage.

  2. The evidence for faithful generation of neuronal dynamics remains insufficient. Although the tokenizer reconstructs substantially better than the selected peer, the prior retains only a small fraction of its oracle voxel-level AP, and the within-token temporal profiles are close to uniform. The directly supervised convolutional models outperform the proposed method on voxel AP across all four settings, while Appendix N reports approximately sixfold excess expected spike counts in the decoded probability field. These findings concern central properties of spike generation rather than peripheral implementation details.

Questions for authors
  1. The principal architectural comparison does not isolate the proposed mechanisms. MaskGIT-flat changes the tokenizer backbone, quantization structure, blank handling, and auxiliary objectives simultaneously. Therefore, the reconstruction advantage cannot be attributed specifically to residual quantization or sparse routing. Similarly, the benefit of the activity–motif factorization is not established against an otherwise identical single-prior model.

  2. The statistical analysis needs to account for dependence between observations. Evaluation uses multiple clips per assay, sampling burst windows with replacement, and some assays are separate conversions of the same acquisition. Appendix A provides reassuring evidence against training-to-test crossover between the duplicate acquisitions, but this does not make their test observations independent. Clip-level Wilcoxon tests and multiple-comparison correction do not address this clustering.

  3. Vocabulary reuse should be distinguished more carefully from biological motif discovery. Rarefaction and exclusion of the blank token are appropriate controls, but shared code usage can also arise from common firing rates, sparse local events, and acquisition geometry. The label-shuffle analysis tests assay dependence in code allocation; it does not establish that the vocabulary captures meaningful neuronal sequences. Controls that disrupt temporal or spatial organization while preserving relevant marginal statistics would make this interpretation more convincing. The current random assay identifiers and within-assay splits support sharing among known assays, rather than transfer to new preparations.

  4. The observation model and sampling scope require clearer interpretation. Fewer than 4% of electrodes are routed in each assay, yet the model operates over the full array canvas. The manuscript should clarify how an unrouted location is distinguished from a recorded location with no spike, and how this distinction enters training and evaluation. Moreover, all clips come from detected burst windows. The demonstrated distribution is therefore burst-conditioned activity, and broader claims about spontaneous-activity simulation should acknowledge that inter-burst silence and continuous recording dynamics are not validated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.