acceptodds
Under review as a conference paper at ICLR 2027

NEURAL POPULATION EMBEDDINGS AS A TEACHER FOR BELIEF-GUIDED DECISION MAKING

Abstract

Low-dimensional structure in neural population activity often resembles the representational geometry that emerges in artificial networks trained on the same task, which suggests that recorded neural activity could serve as a teacher for artificial agents. Prior brain-tuning work, including its single application to decision-making, has transferred representations tied to the stimulus presented on each trial. To our knowledge, no study has asked whether neural activity can transmit a latent context that is absent from the input and must be inferred across trials from the history of choices and outcomes. Here we address this question using the International Brain Laboratory (IBL) task, in which mice choose the side of a visual stimulus of varying contrast while the prior probability of that side switches between uncued biased blocks, such that on zero-contrast trials the animal must rely on its inferred belief alone. Specifically, we train a recurrent agent on a simulated version of the same protocol and, building on the Brain2Model paradigm, add a transfer loss that aligns a low-dimensional belief embedding within the agent to a teacher constructed from CEBRA embeddings of brain-wide IBL population activity averaged by trial condition. We compare this brain teacher against a noise-less Bayesian ideal-observer teacher and a shuffled control that retains the same embedding vectors under a permuted condition assignment. On zero-contrast trials, the brain teacher matches the ideal-observer teacher across the mid-range of transfer weights despite receiving only a coarse belief label rather than the observer’s posterior, and it outperforms the shuffled control by a margin that grows with transfer weight while also accelerating learning. Importantly, this advantage cannot be attributed to regularization, since all three teachers produce the same small gain at low transfer weight, and it replicates with a self-supervised embedding constructed without any belief labels. Moreover, within the neural embedding itself, the separation between low- and high-prior states is largest at zero contrast, where the prior is the only information available to the animal. Together, these results provide the first demonstration that neural population data can substitute for a hand-designed teacher when the task-relevant variable is latent and must be inferred across trials.

Primary area:
reinforcement learning
Keywords:
brain2model, transfer learning, neural embeddings, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZVC9OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether neural population representations can help an artificial agent learn to use a hidden context in decision making. In a simulated IBL task, a recurrent reinforcement-learning agent receives noisy visual evidence, its previous action and reward, but no explicit block identity. An auxiliary loss aligns an eight-dimensional policy embedding with a 27-condition table derived from CEBRA embeddings of mouse recordings. The paper compares brain, Bayesian and shuffled teachers, includes a no-transfer baseline, and examines a time-conditioned CEBRA variant. It reports improved accuracy on zero-contrast trials in biased blocks, with brain-trained and Bayes-trained students showing similar performance over part of the transfer-weight range.

Critical strengths
  1. Transferring information about an uncued context extends neural supervision beyond current-stimulus representations. Zero contrast removes current sensory information about stimulus side while retaining the block-dependent side distribution, making this subset a useful probe of history-dependent behavior.
  2. The no-transfer baseline at alpha = 0, transfer-weight sweep, Bayesian teacher and per-seed shuffled table provide useful comparisons. The time-conditioned CEBRA variant further tests whether behavior-conditioned encoder fitting is essential to the observed benefit. The eight training seeds also make variability more visible.
  3. Building the condition table from held-out trial chunks is a useful precaution against reusing encoder-fitting samples. The authors explicitly acknowledge both the deterioration on nonzero-contrast trials and the possible contribution of choice-related activity to the neural separation analysis.
Critical weaknesses
  1. The curves labelled Bayes in Figures 2 and 3 report students trained with a Bayesian auxiliary target. They do not show how accurately the observer itself decides from its belief. Consequently, similarity between the two trained students does not establish proximity to an independent Bayesian policy. A direct observer baseline on the same episodes would separate the capability of the inference rule from the student's ability to learn it. Moreover, the observer's constant switch hazard of 1/50 does not exactly match geometric block durations clipped to 20-80 trials. Its status as an optimal reference therefore needs qualification. For the full task, the relevant comparison should include an observer combining history with noisy visual evidence, without access to true contrast at decision time.
  2. Zero-contrast accuracy is useful for probing history-dependent behavior, but the overall benefit of neural alignment remains unclear. The reported gains concern zero-contrast trials in biased blocks, while the authors acknowledge declining accuracy on nonzero-contrast trials as alignment strengthens. The paper does not quantify this decline or report aggregate accuracy or reward, so it is unclear whether the gains outweigh the losses. Zero contrast accounts for only one fifth of simulated trials in expectation, and the main analysis further restricts this subset to biased blocks. Results for all nine signed contrasts, weighted aggregate performance, and separate unbiased/biased phases are needed to assess this tradeoff.
  3. The paper already includes a non-neural Bayes teacher, but it encodes only a continuous belief on one axis, without contrast. In comparison, the brain teacher supplies an eight-dimensional target indexed by three belief bins and nine signed contrasts. Shuffling this table breaks the correspondence between conditions and targets; outperforming shuffle therefore does not show whether neural recordings add value beyond the condition labels themselves. A more direct control would construct eight-dimensional targets from the same belief bins and signed contrasts, without using mouse neural recordings, and match target scale and training budget. This would test the additional benefit of the recorded neural geometry. The statement at line 185 that no belief label is used anywhere in construction should also be narrowed to time-conditioned CEBRA fitting: constructing and querying the condition table still require belief-bin labels, a dependency acknowledged at lines 371-372.
  4. Figures 2(c) and 3(c) describe learning over training epochs. They do not show how a trained policy adapts within an episode when the hidden block changes. The pooled score cannot distinguish better decisions within stable blocks from faster recovery after switches, or reduced influence of the initial unbiased phase. Accuracy aligned to block switches, with the first biased block separated from subsequent blocks, would clarify what the neural teacher changes.
Questions for authors
  1. The Bayes curves in Figures 2 and 3 show choices made by a GRU trained with Bayesian supervision. Could you add a baseline in which the Bayesian model itself predicts the left/right probabilities and chooses the more likely side, without training a GRU? Separately, does the target for the student at trial t use the belief updated with that trial's true side, or the belief before that update (lines 442-449)?
  2. For each teacher and transfer weight, what are the accuracies at all nine signed contrasts and the overall accuracy or reward? Please separate biased from unbiased trials and quantify whether the zero-contrast gain offsets the nonzero-contrast loss. Confidence intervals over independent test episodes would make this trade-off easier to assess.
  3. Does the neural teacher outperform a non-neural eight-dimensional target derived from the same belief bins and signed contrasts, with matched target scale and training budget? Separately, what motivates reinforcement learning for this task? How would accuracy and learning efficiency change if the recurrent policy were trained with supervised cross-entropy on the correct side in place of the policy-gradient objective, without the value loss?
  4. Could you report accuracy by time since the most recent block switch, separating the first biased block from later blocks? Does neural alignment improve performance within stable blocks, speed recovery after switches, or both? Please distinguish these within-episode effects from convergence across training epochs.
  5. The authors acknowledge that choice-related activity may contribute to the zero-contrast separation in Figure 4. Does it persist when left/right choice proportions are matched across belief bins, or when embeddings use only pre-stimulus activity? These analyses would help assess the belief interpretation. Figure 4(b,c) also compares embeddings trained on correct trials only with those trained on all trials, including errors. Do they use the same recording sessions and comparable trial counts per condition?
Reviewer_5RZ7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends the Brain2Model framework to prove that a teacher model (CEBRA) can provide useful latent information from neural data to a recurrent network student trying to decode a hidden state. The experiments use an IBL dataset with the block prior used as the hidden state to decode. A key benefit of this dataset, compared to previous analysis on human intracranial data, is the wide distribution of the brain region recorded. The paper compares its brain teacher model to two baselines: a bayes teacher (knowing the hidden state generative model) and a shuffle teacher (permutes CEBRA embedding assignment from their trial condition).

Critical strengths
  • The paper's results include relevant baseline methods and provide valuable analysis of its results and acknowledge several limitations.
  • The paper explores new directions in brain-tuning studies with transmitting from neural activity a hidden state and not stimulus encoding.
Critical weaknesses
  • The model architecture or main component are not ablated and the teacher model study is limited to CEBRA.
  • Some key clarification are needed to understand the method and interpret some results.

Note: the code is not shared at submisson time even if some details are provided for reproducibility and a justification is provided (pending intellectual-property).

Questions for authors

Even if the paper provides an original framework, I would lean for a weak rejection of the paper because the generalization of the method is limited at the moment. The results provided use a single student architecture with no ablation on its components and the teacher model is limited to one method (CEBRA). Additionally, some key points would benefit from clarification.

Questions that might require additional experiments (ordered by importance):

  • Q1: Could you reproduce your experiment with belief embeddings from different models? This would probe the teacher framework generalization and also what type of teachers and inductive biases are needed to make the framework function. Methods like a traditional linear dimensionality reduction (e.g., PCA, FA), a variational autoencoder (e.g., pi-VAE) or a latent dynamical model (e.g., LFADS) or mask auto encoder transformer (e.g., NDT) would be great additions.
  • Q2: Is there a rationale behind using 8 as the belief embedding dimension? Could the embedding dimension be ablated by increasing and decreasing its dimension?
  • Q3: Did you try additional architecture for the student, and what are the key changes to the Brain2Model framework?
  • Q4: Could you comment on why this decision:

"All teachers reported here use tables built from trials on which the mouse chose correctly."

When the stimulus contrast is null, would this make the neural activity correlated to the mouse wheel movement be correlated to the block prior? In particular, the trial window is [-0.3, 0.2] from the stimulus onset. Is this window always excluding the movement onset of the mouse, and why not base the window on movement onset instead?

  • Q5: The embedding from CEBRA seems to be population activity averaged by trial condition. Why is this averaged and not each trial embedding used? Without averaging, the embeddings might capture the potential evolution of the block prior encoded in the population activity across different trials.

Clarification questions:

  • Q6: Could you provide some insights on how this framework can be used in another context than this experiment and analysis?
  • Q7: Could you confirm that you used the IBL Brain Wide Map dataset [1]? If so, did you use neural recording from the repeated site probe or for all probes? This impacts the data assumption between embeddings collected across different brain regions and spanning the mouse brain or embeddings from the same regions.
  • Q8: Could you explain:
    • how the embedding separation is computed? What metric is used?
    • what is a "belief bin"?
    • what does the value head predict?

[1] Angelaki, Dora, et al. "A brain-wide map of neural activity during complex behaviour." Nature 645.8079 (2025): 177-191.

Additional feedback not affecting the decision assessment:

  • Some additional background in the supplementary material on the original Brain2Model framework and on the collection of CEBRA's embeddings would help readers not familiar with those.
  • Information about computational cost would be beneficial to report in the additional material.
Reviewer_DiqzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether recorded neural population activity can teach RNN a latent variable that must be inferred across trials. The authors simulate the IBL two-alternative task, in which the prior over stimulus side switches between uncued blocks. A GRU agent trained with actor–critic receives noisy signed evidence, its previous action and the previous reward. The policy reads only from a linear 8-dimensional embedding, which during training is pulled toward a teacher target by an additive MSE term weighted by α.

Three teachers are compared: a Bayes teacher (the ideal-observer posterior, placed on one dimension), a brain teacher (a 27-entry lookup table, 3 belief bins × 9 signed contrasts, of mean CEBRA embeddings of IBL activity, either label-supervised or time-conditioned), and a shuffle teacher (the same vectors with permuted assignment).

The brain teacher is statistically indistinguishable from Bayes at mid-range α, falls behind it at α = 30, and exceeds shuffle by a margin that grows with α. All teachers give a ~0.02 gain at small α, which the authors attribute to regularisation. The brain–shuffle result replicates with the self-supervised embedding, which stays ~0.01 below Bayes. In the brain table, low/high-prior separation is largest at zero contrast.

Critical strengths
  1. The question is well posed. Brain-tuning has so far transferred stimulus-locked representations and asking whether neural data can teach a latent, across-trial variable is a natural and useful next step. The IBL task is suitable here because it is a standardised brain-wide dataset whose conditions can be matched one-to-one with a simulated agent.
  2. Several design choices are careful: the shuffle control keeps the identical multiset of vectors, scale and covariance; all teachers are rescaled to a common target RMS so that α is comparable; and the condition tables are built out of sample with contiguous 50-trial chunks, which respects the autocorrelation of block belief.
  3. The paper is candid about some of its limitations, including the label supervision of CEBRA, the choice confound at zero contrast in Fig. 4, and the loss of accuracy on non-zero-contrast trials.
Critical weaknesses
  1. The Bayes and brain teachers receive different information, so the main comparison is not controlled. The brain target is indexed by the noiseless signed contrast, so it carries the current trial's stimulus, which the Bayes teacher never sees. Because the policy is a linear readout of the same 8-dimensional embedding, the Bayes teacher pulls the embedding away from the evidence that the task loss needs on non-zero-contrast trials; the brain teacher does not. The comparison therefore confounds “neural vs. hand-designed” with “belief only vs. belief + stimulus”. A matched control is straightforward: a Bayes teacher in the same format (a 27-entry table of posterior × signed contrast, or a 2-d target [belief, signed contrast]).
  2. The Bayes target is rank-1. It occupies dimension 0 only, and the remaining 7 dimensions have target 0. Under the MSE transfer loss, large α (up to 30) forces those 7 dimensions toward zero and collapses the embedding onto one axis, whereas the brain and shuffle targets span up to 8 dimensions. This forces the supervised Bayesian teacher to discard information, making the comparison unfair.
  3. Even with this advantage, the brain teacher does not outperform Bayes. At mid-range α the largest difference is 0.0034 (p = 0.095); at α = 30 Bayes is ahead (p = 0.0005); with the self-supervised embedding the brain teacher stays ~0.01 below Bayes at every α ≥ 1 (Fig. 3b), and its AULC is below Bayes throughout (Fig. 3d).
  4. The brain teacher is not independent of the ideal observer. The three belief bins are thresholds (0.35/0.65) of the Bayes posterior, computed with the same generative model, and the supervised CEBRA model is conditioned on these labels. The statement that the brain teacher “gets no explicit generative model” and can “substitute for a hand-designed teacher” is therefore overstated: its lookup key is the ideal observer. Moreover, belief explains only 2.21% (supervised) and 0.21% (self-supervised) of table variance, so it is unclear how much of the benefit comes from neural belief geometry at all. It is also unclear how much of the between-condition geometry comes from the recordings rather than from CEBRA itself: with discrete labels, the InfoNCE objective already pushes classes apart and tends toward a near-uniform arrangement, so label-supervised CEBRA can produce a structured 27-entry table from almost any input.
Questions for authors
  1. Could you add a Bayes teacher in the same format as the brain teacher (a 27-entry table of posterior × signed contrast, or a 2-d target [belief, signed contrast]), and a Bayes teacher with the unused dimensions masked out of the loss?
  2. What is the ideal observer's zero-contrast accuracy under your environment, and what are overall accuracy and psychometric curves for each teacher at each α?
  3. What teacher results do you obtain from a table built by label-supervised CEBRA fit on shuffled neural data or Gaussian noise with the same 27 labels?
  4. How do the results change with a smaller GRU (e.g., 4–32 units) and without dropout? Does the teacher benefit survive a well-tuned baseline, including a supervised one (cross-entropy on the rewarded side)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.