- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
NLDisco is an open-source pipeline for finding interpretable latents in neural population activity by training shallow, overcomplete, sparse encoder-decoder (SED) models, i.e., sparse dictionary / sparse autoencoder variants, with batch top-k sparsity so that individual latents act as candidate feature detectors. It introduces window SEDs (a flat-window encoder and a transformer-window encoder over timebins, with an optional shift-equivariant mode) and Matryoshka nested levels to handle multi-scale and absorbed features, plus an evaluation suite adapted from SAEBench. It is validated on simulated hippocampal place-cell spike data (ground-truth recovery), macaque motor-cortex data (comparison against CEBRA and LangevinFlow on two chosen features), and an 18-hour mouse RSC/SC foraging recording (one latent tracks wheel activity).
Critical strengths
- The pipeline is complete, usable and well motivated. From spike-sorting ingestion through training, evaluation and latent inspection, NLDisco lowers the barrier to sparse-latent analysis of neural data and supports multiple modalities and paradigms, which is consequential for adoption by experimental neuroscientists.
- The synthetic demonstration is a clean, falsifiable sanity check. Showing that matched latents recover simulated place fields, and that Matryoshka levels recover nested broad/narrow fields with source-activity traceback, convincingly demonstrates that the sparse dictionary behaves as intended.
- The architectural adaptations are thoughtful. Window SEDs and Matryoshka levels are sensible, low-overhead responses to the temporal and multi-scale structure of neural data, and the shift-equivariant mode is a natural extension.
Critical weaknesses
- The evaluation does not support the central interpretability claim. Latent interpretation is manual and subjective, feature selection is post hoc, and the motor-cortex comparison is explicitly a targeted search for pre-specified features. Cov/Spec/Sel/AUROC are computed on latents chosen to match known features. Without a quantitative, selection-bias-controlled measure of interpretability or discovery yield, the paper cannot substantiate that NLDisco yields more interpretable latents than CEBRA or LangevinFlow, i.e., this might not be guaranteed.
- Methodological novelty is limited and benchmarking is narrow. The pipeline largely re-packages established sparse-dictionary techniques; comparisons use only two baselines and do not report standard reconstruction/decoding benchmarks such as NLB-style co-smoothing, so the contribution reads as primarily software rather than a demonstrated advance in neural representation learning.
- Stability of the discovered features is not quantified. It is unclear how robust the reported latents are across random seeds, dictionary widths, sparsity levels, and sessions or animals, which matters because the scientific claims rest on individual hand-inspected latents.
Questions for authors
- Can you provide a quantitative, selection-bias-controlled measure of interpretability or discovery, for example, automated feature-labeling agreement, inter-rater reliability from a human study, or discovery recall over a held-out feature set?
- How stable are the reported features across random seeds, dictionary widths, sparsity levels, and across sessions or animals for the real datasets? Reporting stability statistics seems necessary.
- Since the only data requirement is a predefined spatiotemporal binning, how does NLDisco perform on standardized reconstruction (is the reconstruction as good as baselines) and decoding benchmarks?
2 comments
- “Methodological novelty is limited and benchmarking is narrow. The pipeline largely re-packages established sparse-dictionary techniques; comparisons use only two baselines and do not report standard reconstruction/decoding benchmarks such as NLB-style co-smoothing, so the contribution reads as primarily software rather than a demonstrated advance in neural representation learning.”
We agree that much of NLDisco brings together established sparse-dictionary techniques. However, we see this as a strength: practicing researchers benefit from a coherent pipeline with sensible defaults that can also be customized to their needs. Combining and adapting existing ideas is a common and productive approach in modern ML; for example, the Transformer architecture (Vaswani et al. 2017) brought together existing deep learning mechanisms, e.g. self-attention and skip connections.
In addition, the paper introduces the Window SED, which extends sparse encoder-decoders to capture neural representations over temporal windows, and adapts Matryoshka nesting to address representations at different scales.
Regarding baselines, in addition to CEBRA-Time and LangevinFlow, we compare NLDisco to PCA in the synthetic experiment (Section 4.1).
Regarding reconstruction and decoding benchmarks: our primary goal is to discover interpretable neural representations, and we include decoding as a secondary evaluation to show that NLDisco’s representations retain useful behavioral information alongside their interpretability. NLB-style co-smoothing directly measures the predictability of held-out neural activity, but we did not include it because our primary focus is interpretability.
Given these goals, does the rationale for our evaluation choices make sense? Are there particular baseline methods or analyses you would still like to see, and what question would they help address?
- “Stability of the discovered features is not quantified. How stable are the reported features across random seeds, dictionary widths, sparsity levels, and across sessions or animals for the real datasets? Reporting stability statistics seems necessary.”
We agree that a systematic analysis of feature stability across training configurations and recordings would strengthen the paper.
We do show some evidence of recurrence in the current experiments. In Section 4.1, the synthetic place-field features are recovered across five independently trained models with random seeds, and we report the mean and variability of their activation curves. In Section 4.2, behavioral descriptions developed from a reference model with dictionary width 256 can also be matched to latents in a separately trained model with width 192.
However, we agree that these examples do not provide a systematic characterization of real-data feature stability across seeds, dictionary widths, or sparsity levels. To address this, we will investigate reporting recovery rates and variation in feature-association metrics across multiple seeds and training configurations. For the proposed statistics, a straightforward choice is the fraction of runs in which each feature is recovered. We also plan to analyze a second, new Aeon dataset (4.3) to assess whether similar features can be found in another session and animal. We would welcome your feedback on whether these analyses would address your concern.
- Since the only data requirement is a predefined spatiotemporal binning, how does NLDisco perform on standardized reconstruction (is the reconstruction as good as baselines) and decoding benchmarks?
As mentioned in our response to point 2, our primary focus is interpretable feature discovery. We evaluate reconstruction quality within NLDisco and report behavioral decoding comparisons with CEBRA-Time and LangevinFlow in the macaque experiment, but we do not currently report a standardized reconstruction or decoding benchmark. We agree, however, that scoring on a reconstruction benchmark would be be useful complementary evidence about the information retained by NLDisco’s representations, so we will aim to include this before the rebuttal deadline.
We hope these clarifications and suggested edits are clear, and we look forward to your response, and to any additional specific suggestions you may have. Thanks again for the thoughtful review!
Thanks for the constructive review! We’ll attempt to consolidate your critiques and questions into individual points we’ll respond to one-by-one, and answer each in-line in our reply. We believe we cover all critiques and questions brought up in your review, but if we left something meaningful out please let us know!
Our full response covers more than 5,000 characters, so we’ll leave this over multiple replies.
We will also ask you for clarifications in bold.
Can you confirm what you mean exactly by “the evaluation does not support the central interpretability claim”?
We agree that our current experiments do not fully establish that NLDisco definitively yields more interpretable latents or more reliable end-to-end discovery than CEBRA-Time or LangevinFlow. To address this limitation, we are considering an additional end-to-end discovery comparison on a second, new Aeon recording (4.3), that would pursue two of your suggested analyses. First, we will define a set of behavioral features independently of all models, search for them using the same procedure across methods, and evaluate the selected latents on held-out data. This would let us quantify how often each method recovers the independently specified features. Second, we will ask multiple raters to independently assess interpretations of a sample of latents from each method. We would independently inspect latents from each method on a “training” / “discovery” split, record candidate feature descriptions, and evaluate the selected coordinates and descriptions on held-out data using shared criteria. We would report the number and proportion of inspected coordinates yielding validated interpretations, as well as any distinct features surfaced by each method. This analysis would require substantial effort, but we will investigate whether we can complete it before the rebuttal deadline.
We recognize that neither analysis alone provides a universal measure of interpretability. Together, however, they would help assess the consistency of interpretations, and we believe they would satisfy your ask for a “selection-bias-controlled measure of interpretability” – we would be interested to hear if you agree, and why if not?
We would also like to address these points: 1a. “Latent interpretation is manual and subjective” 1b. “the motor-cortex comparison is explicitly a targeted search for pre-specified features” 1c. “feature selection is post hoc” 1d. “Cov/Spec/Sel/AUROC are computed on latents chosen to match known features.” On 1a: We agree that interpreting real-data latents involves manual inspection and judgment. In our experiments, we used visualization dashboards to examine latent activations alongside neural activity and behavioral variables, with AI agents assisting in the analysis of candidate features. We will expand the appendix to describe this workflow and how we selected latents for inspection.
On 1b: To clarify our results in 4.2, the behavioral descriptions were developed by inspecting a reference NLDisco model, then used to search for corresponding features in a separately trained NLDisco model and the baseline representations: we did not search for pre-specified features.
On 1c and 1d: we’re a bit confused because it seems 1c contradicts 1b and 1d: claims of the feature being “pre-specified” vs. “post hoc”. To be clear, we believe the reported metrics in 1d provide a useful comparison of how individual latents align with these descriptions. Features aren't known a priori: we see when a latent fires, then try to come up with a description of that occurrence (i.e. a feature), and then see how well the latent activity matches the features, scored by these metrics.