- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper attempts to highlight a particular issue which faces cognitive (computational) neuroscientists when fitting small models to data. As an example they fit tiny (1–2 unit) gated RNNs to the choices of mice performing a probabilistic reversal-learning task. It shows that, across different data folds, standard GRUs can produce near-identical choice logits while their weights and hidden-state trajectories differ (solution degeneracy, in the words of the authors). This effectively shows that these models cannot be used out of the box for research in which one attempts to treat hidden variables as useful readouts for cognitive mechanisms. The authors then propose the introduction of biological constraints and model restrictions, ultimately arriving at “monoGRU”: a GRU unit with ReLU activations, an energy penalty, a weight sparsity inducing mechanism, no reset gate, a single scalar update gate and no decoder bias. They report that its weights and hidden states are much more similar across fits with no loss of predictive performance, allowing more clear interpretation of it’s hidden states in terms of cognitive mechanisms.
Critical strengths
First, the setup of the problem in the Introduction and the first half of Section 2 (up to Table 1) gives a clear and good reasoning for how one cannot treat trainable algorithms with a high degree of expressivity and necessarily expect that they will find a consistent solution across folds. It also helps to illuminate this very important problem in cognitive computational neuroscience: the danger of fitting overly expressive models to data and then interpreting them in a mechanistic fashion.
Second, the manner in which comparisons are made, via nested cross-validation and within vs between subject statistical testing appears appropriate and well constructed and well presented.
Critical weaknesses
First, when shifting from posing the problem to posing a solution (in the final form of monoGRU) the work passes through an undesirably handtuned, non-generalizable, and non-rigorous process. The authors introduce a range of interventions on the model (biological constraints (BC), decoder bias (DB) removal, and light vs mono GRU) without much of an explanation as to why these are the set to choose over or how another practitioner could hope to make similar choices. Furthermore, the choices of biological restrictions and simplifications are not organised, ordered, or rigorously searched. The appendix shows that a specific subset of combinations were checked but even these do not cover all possible options (e.g. lightGRU+BC−DB is missing, as is monoGRU−DB-BC among others). Therefore it is not possible to isolate which interventions are responsible for the effects seen. In effect this leaves the results of this model search as seeming to be a random set of uninterpretable choices which cannot be guaranteed to produce the same effect if applied to an alternate task’s data.
Second, it is unclear whether the statistics and approaches used might favour the models selected in the end. Models trained with Tanh non-linearities are allowed permutations and sign-flipping when finding equivalent parameterizations. This is equivalent to allowing their input weights to be permuted or to be sign reversed only. ReLUs are instead allowed permutations and scaling. However, scaling of ReLUs allows for a much greater degree of tuning of the alignment between models than tanh can possible achieve. This leaves me, as a reader, unsure if the alignment statistics might be skewed toward ReLU non-linearities from the outset. Furthermore, some alignment statistics are being computed for weights and update gates, however are these not of different shapes depending upon the model’s construction (GRU vs monoGRU)? If so, how can alignment be compared across these methods?
This leads to a third and final consideration which is missing: the models are never compared in parameter matched cases. When simplifying from a GRU to a monoGRU or light GRU the number of parameters in the model change. This is already a reduction in the level of overparameterization of these models. Parameter matched model comparison is a norm in machine learning and neuroscience when comparing methods, therefore it seems that some of these comparisons might be unfair from the outset.
Questions for authors
Did you also test a range of regularization methods in isolation? Regularization is also a method which is commonly used to restrict the set of solutions found by trained models (linear or otherwise).
Do you think the alignment of your models might have also (independently) gotten better with more data? One common effect in model training is the effect of an underdetermined solution if there is insufficient data.
Was your search over models principled in anyway? How did you make choices about what to ablate or remove from models and what to add to the objective function when searching for a model without such degeneracy?
Note, there are a few textual issues with your work including:
- Citations missing brackets throughout the paper (see L134/L135/L177/L280-281 etc)
- “Hidden units accumulate integrate reward history” (L265) is not a parsable sentence
- L288-289 use of a colon is weird
We would like to thank the reviewers for their detailed and constructive feedback. We have decided to withdraw the submission at this stage, but your comments will be valuable in updating the work for a future submission.
We note that the previous upload was not reflective of the finished manuscript; this was an editorial mistake made during upload. We have since replaced this with the revised manuscript that both addresses the comments made relating to this, as well as the vast majority of all three of the reviewer’s comments.