- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether neural population representations can help an artificial agent learn to use a hidden context in decision making. In a simulated IBL task, a recurrent reinforcement-learning agent receives noisy visual evidence, its previous action and reward, but no explicit block identity. An auxiliary loss aligns an eight-dimensional policy embedding with a 27-condition table derived from CEBRA embeddings of mouse recordings. The paper compares brain, Bayesian and shuffled teachers, includes a no-transfer baseline, and examines a time-conditioned CEBRA variant. It reports improved accuracy on zero-contrast trials in biased blocks, with brain-trained and Bayes-trained students showing similar performance over part of the transfer-weight range.
Critical strengths
- Transferring information about an uncued context extends neural supervision beyond current-stimulus representations. Zero contrast removes current sensory information about stimulus side while retaining the block-dependent side distribution, making this subset a useful probe of history-dependent behavior.
- The no-transfer baseline at alpha = 0, transfer-weight sweep, Bayesian teacher and per-seed shuffled table provide useful comparisons. The time-conditioned CEBRA variant further tests whether behavior-conditioned encoder fitting is essential to the observed benefit. The eight training seeds also make variability more visible.
- Building the condition table from held-out trial chunks is a useful precaution against reusing encoder-fitting samples. The authors explicitly acknowledge both the deterioration on nonzero-contrast trials and the possible contribution of choice-related activity to the neural separation analysis.
Critical weaknesses
- The curves labelled Bayes in Figures 2 and 3 report students trained with a Bayesian auxiliary target. They do not show how accurately the observer itself decides from its belief. Consequently, similarity between the two trained students does not establish proximity to an independent Bayesian policy. A direct observer baseline on the same episodes would separate the capability of the inference rule from the student's ability to learn it. Moreover, the observer's constant switch hazard of 1/50 does not exactly match geometric block durations clipped to 20-80 trials. Its status as an optimal reference therefore needs qualification. For the full task, the relevant comparison should include an observer combining history with noisy visual evidence, without access to true contrast at decision time.
- Zero-contrast accuracy is useful for probing history-dependent behavior, but the overall benefit of neural alignment remains unclear. The reported gains concern zero-contrast trials in biased blocks, while the authors acknowledge declining accuracy on nonzero-contrast trials as alignment strengthens. The paper does not quantify this decline or report aggregate accuracy or reward, so it is unclear whether the gains outweigh the losses. Zero contrast accounts for only one fifth of simulated trials in expectation, and the main analysis further restricts this subset to biased blocks. Results for all nine signed contrasts, weighted aggregate performance, and separate unbiased/biased phases are needed to assess this tradeoff.
- The paper already includes a non-neural Bayes teacher, but it encodes only a continuous belief on one axis, without contrast. In comparison, the brain teacher supplies an eight-dimensional target indexed by three belief bins and nine signed contrasts. Shuffling this table breaks the correspondence between conditions and targets; outperforming shuffle therefore does not show whether neural recordings add value beyond the condition labels themselves. A more direct control would construct eight-dimensional targets from the same belief bins and signed contrasts, without using mouse neural recordings, and match target scale and training budget. This would test the additional benefit of the recorded neural geometry. The statement at line 185 that no belief label is used anywhere in construction should also be narrowed to time-conditioned CEBRA fitting: constructing and querying the condition table still require belief-bin labels, a dependency acknowledged at lines 371-372.
- Figures 2(c) and 3(c) describe learning over training epochs. They do not show how a trained policy adapts within an episode when the hidden block changes. The pooled score cannot distinguish better decisions within stable blocks from faster recovery after switches, or reduced influence of the initial unbiased phase. Accuracy aligned to block switches, with the first biased block separated from subsequent blocks, would clarify what the neural teacher changes.
Questions for authors
- The Bayes curves in Figures 2 and 3 show choices made by a GRU trained with Bayesian supervision. Could you add a baseline in which the Bayesian model itself predicts the left/right probabilities and chooses the more likely side, without training a GRU? Separately, does the target for the student at trial t use the belief updated with that trial's true side, or the belief before that update (lines 442-449)?
- For each teacher and transfer weight, what are the accuracies at all nine signed contrasts and the overall accuracy or reward? Please separate biased from unbiased trials and quantify whether the zero-contrast gain offsets the nonzero-contrast loss. Confidence intervals over independent test episodes would make this trade-off easier to assess.
- Does the neural teacher outperform a non-neural eight-dimensional target derived from the same belief bins and signed contrasts, with matched target scale and training budget? Separately, what motivates reinforcement learning for this task? How would accuracy and learning efficiency change if the recurrent policy were trained with supervised cross-entropy on the correct side in place of the policy-gradient objective, without the value loss?
- Could you report accuracy by time since the most recent block switch, separating the first biased block from later blocks? Does neural alignment improve performance within stable blocks, speed recovery after switches, or both? Please distinguish these within-episode effects from convergence across training epochs.
- The authors acknowledge that choice-related activity may contribute to the zero-contrast separation in Figure 4. Does it persist when left/right choice proportions are matched across belief bins, or when embeddings use only pre-stimulus activity? These analyses would help assess the belief interpretation. Figure 4(b,c) also compares embeddings trained on correct trials only with those trained on all trials, including errors. Do they use the same recording sessions and comparable trial counts per condition?