- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduced JEDI, a hierarchical hypernetwork model that generates low-rank autonomous RNNs to fit neural data, and the model can also learn a context embedding for each trial. The model is evaluated on both simulated data and real data and the model shows more interpretability.
Critical strengths
- The approach is clear. The hypernetwork allows trial-specific dynamics while sharing parameters across trials. Generating low-rank connectivity provides a practical structural constraint and makes the learned networks good for dynamical analysis.
- The authors did evaluation beyond reconstruction accuracy. The paper investigates embedding structure, perturbation robustness, eigenvalue spectra, fixed points, and Lyapunov exponents.
Critical weaknesses
- The experiments do not establish that varying recurrent weights is necessary to explain contextual differences. In the simulated data, the teacher’s connectivity is fixed across tasks. However, JEDI absorbs these differences into context-dependent connectivity. I appreciate the authors for acknowledging this limitation, but it directly affects their central interpretation. A shared-weight RNN with context-dependent inputs would be an important baseline.
- The fairness of the model comparisons requires further clarification. Although the methods use the same embedding dimensionality, their parameter counts may differ substantially. Please report the total number of trainable parameters. Also, section E.4 says baselines get the same first bin as JEDI, could the authors discuss how you give this information to VAE and LFADS?
- In Fig. 5D, exact values and tested pairs are not reported, and CEBRA and LFADS appear to classify reach direction better than JEDI. Could the authors discuss more?
- The claim that JEDI learns a “clear ring-like structure” appears to rely on 2D PCA visualization. However, their circular geometry is less evident. Could the authors quantify it?
Minor:
- A.2: line 795 has W_ext and line 800 has another W_ext, are they different?
Overall, I appreciate this paper’s idea of using a hypernetwork over learned context embeddings to generate condition-specific, low-rank RNN. However, several weaknesses should be addressed to strengthen the paper.
Questions for authors
Please see section Critical Weaknesses.
3 comments
Response to Reviewer K3vT (2/2)
W3. In Fig. 5D, exact values and tested ...
We recomputed Fig. 5D on all 208 trials with 10× repeated 5-fold cross-validation. The table gives the exact values (reach-direction accuracy, mean ± s.d. over the cross-validation splits) and the test of each method against JEDI (corrected resampled t-test [1], Holm-corrected).
| Method | Accuracy | vs JEDI |
|---|---|---|
| JEDI (ours) | 0.914 ± 0.029 | – |
| LFADS, initial condition | 0.918 ± 0.035 | p = 1 |
| LFADS, initial condition + controller | 0.946 ± 0.023 | p = 0.39 |
| CEBRA (trained with the direction labels) | 1.000 ± 0.000 | p = 7×10⁻⁶ |
LFADS is not significantly better than JEDI. CEBRA is better because it is trained with the direction labels to separate them [2], whereas JEDI's context is learned without labels, only to reconstruct each trial's activity.
Action. App. F.7 of the revised paper reports the exact Fig. 5D values and the tested pairs, and Section 3.4 states that direction is near ceiling for every representation.
W4. The claim that JEDI learns a “clear ...
We agree, and we are happy to have quantified it. On held-out contexts, we measured whether the direction centroids lie in cyclic order, RSA with angular distance [3], and whether the centroids form a loop. The loop is measured with persistent homology, as the lifetime of the centroids' largest one-dimensional hole (H1), tested against a label-shuffle null (500 shuffles; minimum p = 0.002), as for head-direction rings [4].
| Model | Cyclic order↑ | RSA↑ | Loop (H1) p↓ |
|---|---|---|---|
| JEDI | 8/8 | 0.68–0.81 | 0.002 |
The direction centroids lie in a ring in every seed. All eight are in cyclic order, distances follow angle, and the loop is significant. The loop also stays significant on native time bins and in a second monkey (all 11 fits, p ≤ 0.018; RSA 0.60–0.83).
Action. The ring table is in App. F.8 of the revised paper, and Section 3.4 cites these metrics for the ring claim.
Minor 1. A.2: line 795 has W_ext and line ...
Minor. W_ext. Thank you for kindly spotting this. Both denote the same input matrix; we will unify the notation and dimensions.
Action. App. A.2 of the revised paper uses the single symbol W_ext with one stated dimension.
Thank you again for your precise questions. If our responses address your concerns, we would be grateful if you would consider raising your score.
References
[1] Nadeau, Claude, and Yoshua Bengio. “Inference for the Generalization Error.” Machine Learning, vol. 52, 2003, pp. 239–281.
[2] Schneider, Steffen, et al. “Learnable Latent Embeddings for Joint Behavioural and Neural Analysis.” Nature, vol. 617, 2023, pp. 360–368.
[3] Kriegeskorte, Nikolaus, et al. “Representational Similarity Analysis: Connecting the Branches of Systems Neuroscience.” Frontiers in Systems Neuroscience, vol. 2, 2008, p. 4.
[4] Chaudhuri, Rishidev, et al. “The Intrinsic Attractor Manifold and Population Dynamics of a Canonical Cognitive Circuit across Waking and Sleep.” Nature Neuroscience, vol. 22, 2019, pp. 1512–1520.
I thank the author for answering my questions and addressing my concerns. The quantification results, especially for parameter matched comparison is highly helpful, although it may show JEDI is only performing a little better. I now have no further questions to the author and will consider raising my score to positive after discussion with other reviewers and the AC in the next phase.
Response to Reviewer K3vT (1/2)
Thank you for your careful review. We appreciate your note that the approach is clear and that the evaluation goes beyond reconstruction accuracy. We address each of your points below. Unless noted, R² is on held-out trials, with ranges over ≥3 seeds.
We agree that this is an important baseline, and we would kindly refer you to CQ1, which reports it at matched capacity, together with teachers whose ground truth is known. Where connectivity differs across conditions only JEDI generalizes (0.95 vs −0.52), and where only inputs differ, JEDI given the input keeps its connectivity shared.
The number of trainable parameters are listed in the table below, where synthetic data have 200 neurons and monkey data 117. The context-input RNN is matched to JEDI, and JEDI has 14–42× fewer parameters than the VAE. We also trained parameter-matched versions of the VAE and LFADS (below). We matched parameters by changing only layer widths, keeping each architecture otherwise unchanged. The context-input RNN shares one low-rank connectivity across conditions, and the width of the hidden layer that maps each context to its constant input is set so that its total parameter count equals JEDI's. For the VAE, the hidden layers of the encoder and decoder shrink from 128 to 9 units on the monkey data and to 3 on the synthetic data. For LFADS, the initial-condition and controller-input encoders, the controller, the generator, the initial-condition dimension and the factors grow from 16 to 60 units on the monkey data and to 57 on the synthetic data.
On the first bin, thank you for kindly checking this. App. E.4 was inaccurate. In the submission, only JEDI started each held-out trial from its recorded first bin. The VAE decoder received only its latent, and LFADS inferred its initial condition from the whole trial during fitting and used the condition-mean initial condition on held-out trials. We will correct E.4. For the rebuttal, we retrained both baselines so that they also receive each trial's first time bin. The VAE decoder takes [z; x(0)], and the LFADS initial-condition encoder sees only the first bin instead of the whole trial, with its controller set to the direction mean. All models are then matched on the information available at t = 0, and JEDI leads (0.39–0.40 vs 0.37–0.38 for the VAE and 0.35 for LFADS). When every model instead sees the whole test trial, LFADS benefits most, since its controller adds trial-specific input at every time step, yet JEDI still leads (0.53 vs 0.49). The table below reports held-out R² on monkey execution in both settings. Its first column uses baselines retrained to receive the first bin, and the second column uses the original baselines. The parameter-matched rows use a VAE hidden layer of 9 units and LFADS sizes of 60.
With parameters matched to JEDI, LFADS improves and reaches JEDI on the whole test trial (0.52–0.53 vs 0.53), while JEDI still leads with the first bin (0.39–0.40 vs 0.37); the smaller VAE becomes worse and unstable.
Action. App. B.7 of the revised paper lists the parameter counts, and App. E.4 is corrected to state how each baseline receives the first bin, with this table.