acceptodds
Under review as a conference paper at ICLR 2027

Learning Predictive Representations of Neural Population Dynamics

Abstract

Neural interfaces and population analyses increasingly require representations that can support several uses without learning a new neural encoder for each task. A central challenge is to learn such states from neural activity itself while retaining information about how the population evolves. We introduce Neural Predictive Coding (NPC), a self-supervised framework that uses future population activity as the learning signal. A causal encoder summarizes past and present activity into a compact state and contrastively predicts learned future targets across multiple timescales. Across motor control, spatial navigation, visual perception, and cognitive timing in three species, frozen NPC states improve forecasts of neurons excluded from pretraining when added to recent-activity summaries. These states also permit task decoding across all four functions, including mean of 0.50 for motor velocity and 0.58 for spatial position. Task-associated trajectories admit fitted updates that improve neural forecasts over static states in every function; in vision, evolved NPC states yield a 60% higher mean likelihood-gain score than CEBRA-Time. In a separate source-disjoint mouse cohort, pretraining improves likelihood-gain scores by 7% over scratch training at matched target updates and full query coverage, although longer scratch training performs better. Future-neural prediction thus provides a common training task for reusable readouts and quantitative analysis of population-state evolution.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Predictive representation learning, neural population dynamics, self-supervised learning, neural decoding, brain-computer interface

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nKdYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Neural Predictive Coding (NPC), a self-supervised objective for binned spiking activity that follows the contrastive predictive coding template. A per bin encoder and a causal GRU over a short history window (300 ms, or 500 ms for navigation) produce a 128 dimensional state, and horizon specific affine heads contrast the true future population embedding at 100, 200, and 400 ms against negatives drawn from temporally separated bins of the same sequence and from other sequences matched on total population count. Frozen states are evaluated across 26 recordings covering motor control (macaque), navigation (rat CA1), vision (mouse, Allen Visual Behavior Neuropixels), and interval timing (macaque). The evaluation has three endpoints: Poisson forecasts of neurons withheld from pretraining, task probes under label budgets, and fitted state evolution, where per horizon affine transitions in a 16 dimensional projection are read out through a fixed emission map. Additional studies cover adaptation to source disjoint mice, common trunk objective controls on five recordings, and trained versus random encoder controls on seven recordings. Comparators are a causal adaptation of CEBRA-Time and NDT2 trained from scratch with a local causal adapter. NPC generally exceeds CEBRA-Time, results against NDT2 are mixed (better neural forecasting in vision and timing, worse in motor and navigation, and worse task decoding in three of four functions), fitted transitions improve over static readouts, and pretraining beats a 250 update scratch control but loses to a longer scratch control from 10 percent query coverage onward. The objective controls suggest that the true past future pairing and multiple horizons carry most of the benefit, while the structured negative sampler contributes little. The authors frame all within recording comparisons as exploratory, and no main benchmark comparison reaches statistical significance.

Critical strengths

1.Careful multi endpoint evaluation with unusually candid reporting. The paper separately tests three properties of a frozen representation (forecasting of withheld neurons, task accessibility under label budgets, and readout fixed state evolution) across four brain functions, three species, and 26 recordings, and it adds mechanism oriented controls (shuffled futures, direct future count prediction, temporal contrast, trained versus random encoders). Post hoc decisions, a failed pre specified transfer criterion, and unfavorable comparisons are all disclosed rather than hidden. This matters because it makes the mixed and negative findings credible, and the evaluation protocol itself is a reusable contribution for assessing self supervised objectives on neural population data.

2.Two findings with real value for the field. First, in the common trunk diagnostic, contrasting true futures beats shuffled future training and direct Poisson prediction of future counts on both endpoints in all five recordings (Tables 2 and 7), which identifies the past future association itself, rather than architecture or added features, as the source of utility. Second, predictive training can reduce task accessibility: trained NPC decodes image identity worse than its own random initialization (Table 5) while forecasting neural activity better (Table 6), and random NDT2 encoders already decode image identity at 0.94 and above. This is a concrete and non obvious caution for objective design in neural foundation models.

Critical weaknesses

1.Limited novelty and no consistent advantage over the strongest baseline. NPC maps almost one to one onto CPC (van den Oord et al., 2018): per step encoder, GRU context, step specific linear predictors, and a contrastive loss. Its main neural specific element, the count matched and within sequence negative sampler, adds essentially nothing to forecasting relative to global negatives (plus 0.003 bits per spike, 2 of 5 recordings; Table 2). Against NDT2, the only strong comparator, NPC is lower on 400 ms forecasting in motor and navigation and lower on task decoding in three of four functions, with a large gap in vision (0.62 versus roughly 0.96 balanced accuracy; Table 4). The CEBRA-Time comparator is a local causal adaptation whose tuning is not described, so wins against it carry limited weight. Expected baselines are absent: DCA (explicitly omitted) or nonlinear predictive information methods, a Poisson future count forecasting model in the main benchmark (it appears only in the five recording diagnostic), raw history decoders for the task probes, and Neural Latents Benchmark standard metrics even though two of its datasets already appear in Appendix E.

2.The evidence is exploratory by construction, and several key analyses have design problems, so method level conclusions are not supported. (a) There is no independent evaluation: within recording comparisons use validation splits that informed result driven tuning applied only to NPC (the motor encoder variant and the navigation checkpoint and learning rate schedule; Appendix B), and the transfer cohort had been inspected before (Appendix G). (b) The significance tests cannot detect effects by design. With n equals 4 animals, an exact two sided paired sign flip test has a minimum attainable p of 0.125, so no four animal contrast can reach 0.05 even before Holm correction, and with n equals 6 the minimum of 0.031 cannot survive Holm over four or eight tests. The null results in Figure 2b, Table 4, and Section 5.3 are therefore guaranteed and carry no information. The single significant result (Section 5.4, p equals 0.0469) is exactly the floor attainable when all eight mice agree, in a post hoc emphasized condition. (c) The incremental readout test behind the first claim of the abstract appears mis specified: in Table 6 the history plus state readout scores below the state only readout in 11 of 14 cells, and seven increments are negative, which suggests the shared regularization is dominated by the high dimensional history block rather than measuring value beyond history. (d) The static reference in the evolution analysis assumes the future equals the present; NPC's static vision forecast is negative (minus 0.175 bits per spike), so the 0.434 gain and the 60 percent advantage over CEBRA-Time largely reflect a degenerate reference and the differing intrinsic timescales of the two representations, and the per horizon transitions are fitted independently, so dynamical consistency is untested. (e) The transfer control is weak: a 5,250 update scratch model wins from 10 percent query coverage onward, the pre specified 10 and 25 percent criterion failed, and the arm that would show whether pretraining helps at convergence (longer adaptation or full fine tuning) is missing.

Questions for authors

1.In the incremental readout, are history and state features penalized by a single shared regularization strength? Please report state only, history only, and combined readouts for the main benchmark with block specific penalties. If NPC alone clearly beats history alone and the combined model at least matches both, the forecasting claim becomes considerably stronger.

  1. Can you freeze all configurations and evaluate on recordings not used in development, with a comparable tuning budget for the baselines? Additional Allen Visual Behavior mice would also allow the sign flip tests to reach significance in principle.

3.Please add: DCA or a nonlinear predictive information baseline; the Poisson future count GRU in the main benchmark, matched in wall clock time or trained to convergence given the 20 to 29 fold training time difference reported in Appendix E; raw history decoders for the task probes; and NDT2 with a 128 dimensional projection to control state dimensionality.

4.Under a common constant rate reference, how does the evolved forecast compare with a direct horizon specific readout from the same 16 dimensional projection, which upper bounds any linear transition? Do fitted transitions compose, that is, does the 200 ms transition approximate the 100 ms transition applied twice? Without these, it is unclear what evolved versus static establishes beyond differences in representation timescale.

5.Please add transfer arms with 5,250 adaptation updates and with full fine tuning, and a variant where negatives and rate strata are restricted to the same fraction, so that the study can speak to data efficiency rather than only restricted query sampling.

The paper is a careful, honest, and labor intensive empirical study, and its evaluation protocol and controls are genuinely useful. However, the method is an established idea transplanted to spiking data with its main novel component shown to contribute little, there is no consistent or statistically supported advantage over the strongest baseline, all evidence is exploratory with result driven tuning applied only to the proposed method, and the transfer result rests on a post hoc emphasis after the pre specified criterion failed. For the applications to neuroscience and cognitive science area, the paper also delivers no new neuroscientific finding. It is above clear rejection because of its rigor and transparency, and I would raise my score if a confirmatory evaluation on independent cohorts with frozen configurations showed NPC matching or beating strong baselines on neural forecasting with the readout issues fixed.

5 comments
AuthorsOpenReview ↗

Response to Reviewer nKdY (1/4)

Thank you for highlighting the careful evaluation and the findings on true future pairing and task accessibility. We address i) contribution and scientific value, ii) strength of evidence, iii) readout and baseline controls, and iv) dynamics and transfer.

W1: What does NPC contribute, and where does it offer an advantage?

A: We contribute a compact predictive model for neural forecasting and task decoding, a reusable evaluation framework across four brain functions, and controlled evidence showing what predictive training adds. Building on CPC [1], these contributions are:

i) NPC uses one compact representation for both neural forecasting and task decoding. Within each recording, NPC learns to predict future population representations. We then freeze the encoder and use its state to forecast activity in neurons excluded from pretraining and decode task variables. The encoder stays fixed as the readout changes.

ii) Our evaluation compares forecasting, task decoding and fitted state evolution under matched input conditions across four brain functions. It covers 26 recordings and 18 animals spanning motor, navigation, vision, and timing. NPC achieves higher task-decoding means than CEBRA-Time [2] in all four functions and higher forecasting means in motor and vision. It exceeds NDT2 [3] in vision and timing forecasting. Visual NPC uses 0.11–0.12 million parameters versus approximately 3.32 million for the evaluated NDT2 model. After correcting readout regularization, NPC also adds forecasting value beyond recent-count summaries in all four function means (Q1).

iii) Matched controls show that true future pairing and multiple prediction horizons improve neural forecasting. Trained NPC forecasts better than its random initialization in all seven screened recordings. True future pairing improves forecasting and decoding in all five diagnostic recordings, and multiple prediction horizons improve forecasting in all five. The visual controls also show that forecasting gains and linear task accessibility respond differently to training, giving a concrete guide for choosing neural learning objectives.

Structured negatives improve forecasting in two and decoding in four of the five diagnostic recordings. The baseline comparisons use causal CEBRA-Time and locally adapted, scratch-trained NDT2; broader pretrained-model and standardized NLB [4] comparisons remain open.

W2: What do the development results and statistical tests establish?

A: We report animal-level effects and distinguish development evidence from independent confirmation. Seeds and sessions are averaged within animals. With four/six animals, the minimum two-sided exact p-values are 0.1250/0.0313, versus 0.0063 at the first step of each eight-contrast Holm family. Adjusted p-values for the primary forecasting and task comparisons range from 0.25 to 1.00. NPC received additional motor/navigation development tuning; the frozen-cohort follow-up is reported in Q2. The historical transfer p = 0.0469 is six-fraction Holm-adjusted, with a raw minimum of 0.0078, and retains its post-hoc interpretation.

AuthorsOpenReview ↗

Response to Reviewer nKdY (2/4)

Q1: Does NPC add predictive value after correcting the shared readout penalty?

A: Combining NPC’s learned state with recent spike-count summaries improves mean forecasting scores over either alone in all four brain functions. We replaced the shared penalty with separately calibrated penalties for history and state. We refitted all 234 model–recording–seed combinations with frozen encoders, matched observations and targets, and inner-only selection that also permits either block to be excluded.

The additional 400-ms scores are in bits/spike. H denotes the specified recent-count summary, Z the learned state, and “Selected” the inner-selected standalone or combined readout.

FunctionModelHZH+ZSelected
MotorNPC0.05920.07830.08150.0815
MotorCEBRA-Time0.05920.05560.06950.0695
MotorNDT20.05920.08050.08180.0818
NavigationNPC0.16990.23120.23320.2330
NavigationCEBRA-Time0.16990.21870.23440.2344
NavigationNDT20.16990.25530.25590.2549
VisionNPC0.23950.30340.32180.3218
VisionCEBRA-Time0.23950.15320.26000.2600
VisionNDT20.23950.24200.28480.2848
TimingNPC0.22510.29600.29920.2988
TimingCEBRA-Time0.22510.29140.30350.3021
TimingNDT20.22510.26080.26300.2621

The gains hold across all four function means. NPC’s increments over H / Z are +0.0223 / +0.0033 (motor), +0.0633 / +0.0021 (navigation), +0.0823 / +0.0184 (vision), and +0.0742 / +0.0032 (timing), calculated before rounding. Combined-below-state cases decrease from 52/78 to 15/78; four combined-below-history cases remain. These complete results distinguish function-level gains from individual outcomes.

Selected fits passed convergence checks; some unselected candidates did not, and penalty-boundary selections are retained in the audit. All methods use uniform fresh fits because archived scores were not uniformly reproduced. The historical fourteen-cell random-feature screen remains separate and unrepaired.

Q2: Is there an independent evaluation with frozen configurations?

A: We have established an independent-cohort evaluation with configurations fixed for NPC and both baselines. The protocol specifies 12 previously unused mice, matched training-update budgets, and selection using inner calibration only. Recorded coverage at the cutoff is 12 of 36 model–recording evaluations. The matching applies to update counts.

AuthorsOpenReview ↗

Response to Reviewer nKdY (3/4)

Q3a: How does NPC compare with a predictive-information baseline?

A: NPC exceeds DCA’s state-only and combined forecasting means on the six visual development mice. The additional comparison with DCA [5] uses common targets, anchors and independently regularized readouts, in bits/spike:

ModelHZH+Z / selected
Original frozen NPC0.23950.30340.3218
DCA0.23950.28830.2970

NPC uses a 128-dimensional state; DCA uses 240 projection-history features. NPC scores average three training seeds, and training budgets differ.

Q3b: How does NPC compare with count prediction at matched training time?

A: NPC’s combined readout exceeds both standalone readouts in all six mice at matched training time. Both methods start from the same encoder weights and share observations and measured training-step time on the same GPU; calibration time is excluded. This additional visual comparison uses one training seed.

ModelHZH+Z / selected
Fresh NPC0.23950.28320.3061
Future-count GRU0.23950.30130.3120

NPC exceeds count prediction in three mice; the count GRU has the higher cohort mean, 0.3120 versus 0.3061. These are fixed-time results, with convergence untested. Fresh NPC’s 0.3061 and the original frozen model’s 0.3218 come from separate training runs; the submitted five-recording comparison retains its fixed-update scope.

Q3c: What do raw-history task decoders reveal?

A: The raw-history control supports separate evaluation of neural forecasting and task decoding. All six visual mice are complete. Linear probes use the same ordered 300-ms observations and labeled anchors; NPC/random scores average three seeds within mouse. Balanced accuracy is:

Label budgetRaw historyNPCRandom NPC
1%0.83580.31820.2766
5%0.98770.44350.4927
10%0.99260.49140.5498
25%0.99390.52720.5912
100%0.99500.54960.6290

Raw history has the higher mean at every budget, and trained NPC is below random NPC in five mice at full labels. Together with the submitted trained/random forecasting controls, this shows how predictive training changes linear task accessibility. These probes differ from the submission’s inner-selected probe families; small label budgets can omit classes. The result concerns linear accessibility, with other brain functions still untested by this control.

Q3d: Does NPC’s advantage persist when NDT2’s readout dimension is matched?

A: NPC retains higher visual and timing forecasting means when NDT2 is projected to 128 dimensions. The additional state-only comparison is:

FunctionNDT2 PCA256NDT2 PCA128NPC native128
Motor0.0804710.0803190.078288
Navigation0.2552600.2548260.231167
Vision0.2419880.2400150.303389
Timing0.2608060.2603880.296013

The function-level ordering is preserved: NPC leads in vision and timing; NDT2 leads in motor and navigation. This matches readout dimension, while encoder capacities differ. The numerical audit records tolerance exceedances and two flagged fits per arm. The largest final-score discrepancy is 0.0000184 bits/spike.

Reviewer_Ev4jOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript explores whether representations learned from neural datasets with contrastive predictive coding facilitate prediction of future neural activity; decoding of task activity (e.g., hand velocity); and adaptation to new recording sessions. The answer in all cases is a qualified "yes": the CPC-trained model does support these uses, although typically not better than other, existing models.

Critical strengths

The experiments are logically organized and rigorous, and their limitations and those of the results are very honestly stated. The problem of learning useful neural representations from unlabelled data is important and unsolved.

Critical weaknesses

The primary weakness of this paper is that the network's performance is compared against only two other models (NDT2 and CEBRA-Time), and the comparison seldom favors the CPC network! Most comparisons (Fig. 2b, Fig. 3b, Fig. 4b) are not statistically significant.

For future prediction of neural activity, a useful point of comparison is (e.g.) the leaderboard of the Neural Latents Benchmark, which scores dozens of models on exactly this measure. And for any of the measures, it would be useful to compare against the many proposed pre-training techniques for spiking data that have been proposed in the last 5 years (there are now perhaps a dozen, but to name a few: MYOW [Azabou 2021], Swap-VAE [Liu 2021, NeurIPS], POYO [Azabou 2023]).

CPC applied to neural data is also not novel. As the authors must know (since they compared against it), CEBRA-Time is trained with a very similar objective. There is also at least one pre-print on applying wav2vec (very close to CPC) to ECoG data (Yuan 2025, https://arxiv.org/pdf/2405.18639).

In sum, it is not necessary to show that this CPC-trained network outperforms all other approaches for the investigation to be useful (this reader is in fact relieved to review a paper without the tables of bolded entries and "SoTA" claims). But since the proposed framework is not particularly novel (even in the context of neural data), nor obviously better than other approaches along any particular dimension, it is not clear what we learn from the results.

Finally, the Methods and Results (and some parts of other sections) were clearly written by Claude. I believe this is acceptable by the ICLR guidelines. But this reviewer, at least, finds Claude's prose hard to read. The manuscript could be made much more accessible with a re-write by an English-speaking human.

Questions for authors
  • Can the authors specify clear advantages of their approach to (e.g.) CEBRA-Time, and explain those differences in terms of differences in the architectures or training?
  • Can the authors provide comparisons against other techniques for pre-training on spiking (or other neural) data?
  • How does future prediction compare to the Neural Latents Benchmark?
  • What specifically do we learn from this study about applying CPC to neural data that the field didn't know before?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Ev4j (1/4)

Thank you for recognizing the rigor of the experiments and the importance of learning useful representations from unlabeled neural data. We address three weaknesses—i) comparison scope and statistical evidence, ii) contribution beyond CPC, and iii) presentation—followed by your four questions on CEBRA-Time, additional baselines, NLB, and the scientific findings.

W1: Do the baseline comparisons establish a useful advantage?

A: NPC achieves higher task means than CEBRA-Time across all four functions and higher forecasting means than NDT2 in vision and timing. We chose CEBRA-Time [1] and NDT2 [2] to represent temporal contrast and masked spike reconstruction. Their comparison with explicit future prediction tests three learning objectives under common causal observations. The new readout analysis independently calibrates history and state penalties for all 234 model–recording–seed combinations. NPC adds forecasting value beyond the specified history summaries in all four function means. Q1 gives the complete comparisons; Q2 adds DCA and a time-matched count GRU.

Animal-level effect sizes and directions quantify these gains. Seeds and sessions are averaged within animals. With four/six animals, the smallest two-sided exact p-values are 0.125/0.03125; the first threshold in each eight-contrast Holm family is 0.00625. Adjusted p-values range from 0.25 to 1.00 for both forecasting and task comparisons.

W2: What does NPC contribute beyond existing CPC applications?

A: NPC contributes a compact predictive neural model and controlled evidence of what predictive training adds to forecasting and task decoding. NPC builds on CPC [3], and we acknowledge earlier neural applications, including CEBRA-Time and Yuan and Makin’s ECoG wav2vec work [4]. Our contribution lies in connecting the learned state, its downstream uses, and interventions on its training.

The advance has three parts: i) a 128-dimensional causal state trained to predict future population embeddings and reused for multiple readouts within each recording; ii) matched evaluation across 26 recordings and 18 animals, separating forecasting of neurons excluded from pretraining, task decoding and fitted state evolution; and iii) controls identifying the benefits of true future pairing and multi-horizon prediction, and their distinct effects on task accessibility. The submitted controls in Section 5.5 and Appendices D–E provide the learning evidence; Q4 explains the resulting scientific lessons.

W3: How can the Methods and Results be easier to follow?

A: We would like to first thank you for your suggestions on the wording. We have prepared detailed revisions for these two sections to make them fluent. Here, we also want to give an explanation of the two sections.

NPC follows three steps: learn from neural history, freeze the encoder, and evaluate forecasting and task decoding. The encoder receives past and present neural activity; future activity supplies training targets. Behavioral labels enter the task readout, and reserved neurons enter the forecasting readout, after encoder training. Within a recording, these readouts share the same frozen encoder. We have reorganized and improved the explanation around what is learned, what is measured, and what the result establishes.

AuthorsOpenReview ↗

Response to Reviewer Ev4j (2/4)

Q1: What advantages does NPC have over CEBRA-Time, and which design choices differ?

A: In the work, NPC achieved competitive performance against CEBRA-Time and NDT2, two representative methods based on temporal contrastive learning and masked spike reconstruction, respectively. These results demonstrate the value of future prediction as a novel training objective for neural representations that support both forecasting and task decoding across four brain functions.

NPC achieves higher forecasting means in motor and vision and higher task-decoding means in all four functions than the evaluated CEBRA-Time model. The additional combined-readout comparison at 400 ms uses identical observations, targets and grouped partitions, with separate history/state penalties. Scores are in bits/spike:

FunctionNPCCEBRA-TimeNDT2
Motor0.08150.06950.0818
Navigation0.23320.23440.2559
Vision0.32180.26000.2848
Timing0.29920.30350.2630

NPC also exceeds NDT2’s forecasting means in vision and timing with a compact model. Visual NPC uses 0.11–0.12 million parameters versus approximately 3.32 million for the evaluated NDT2 implementation. The visual NPC–CEBRA-Time gain is +0.0618 bits/spike, positive in five of six mice in each seed.

The same frozen NPC states support task decoding across all four functions. Submitted full-label, inner-selected task readouts give:

Function / task metricNPCCEBRA-TimeNDT2
Motor / R²0.49560.30980.5418
Navigation / R²0.57780.51420.5921
Vision / balanced accuracy0.62390.49380.9579
Timing / R²0.22420.14670.2205

NPC exceeds CEBRA-Time in every task mean and NDT2 in timing; NDT2 has higher means in motor, navigation and vision. These submitted probes select the readout family on inner data and differ from the additional linear controls in Q4.

The design difference includes recurrent context with prediction heads for several future horizons. Our CEBRA-Time uses the official convolutional encoder with a locally aligned causal window and +100-ms temporal positives. NPC uses a GRU and separate predictors at 100, 200 and 400 ms. Both states use past and present input. The main ranking includes architecture and training differences; the shared-encoder controls in Q4 test individual training choices. We use official baseline components with local causal adaptations and scratch-trained NDT2. The original learning-rate grid was {0.0003, 0.001}; motor/navigation development included additional NPC tuning.

AuthorsOpenReview ↗

Response to Reviewer Ev4j (3/4)

Q2: What additional neural-learning baselines have been evaluated?

A: We completed DCA and a training-time-matched count GRU on all six visual development mice. They add linear predictive-information learning and direct future-count prediction to the comparison.

NPC exceeds DCA’s state-only and combined forecasting means. The DCA comparison [5] uses matched targets, anchors and independently regularized readouts, in bits/spike:

ModelHistoryStateCombined
Original frozen NPC0.23950.30340.3218
DCA0.23950.28830.2970

DCA selects among 100/200-ms windows and three starts using inner data. Its projection histories provide 240 features; NPC uses a 128-dimensional state and averages three training seeds. Feature dimensions and training budgets differ.

Fresh NPC adds forecasting value beyond history in all six mice under the matched training-time budget. The separate count-control results are:

ModelHistoryStateCombined / inner-selected
Fresh NPC0.23950.28320.3061
Future-count GRU0.23950.30130.3120

Both methods share initial trunk weights, observations, seed zero and learning rate. Count prediction receives the same measured training-step time on the same GPU, excluding calibration: approximately 20,000 updates versus NPC’s 4,000. NPC’s combined readout exceeds both standalone readouts in all six mice; NPC exceeds count prediction in three, while count prediction has the higher cohort mean. This fixed-time experiment leaves convergence untested. Fresh NPC’s 0.3061 and the original frozen NPC’s 0.3218 come from separate training runs.

The raw-history control clarifies linear task accessibility. All six visual development mice use the same ordered 300-ms observations and labeled anchors across arms:

Label budgetRaw historyNPCRandom NPC
1%0.83580.31820.2766
5%0.98770.44350.4927
10%0.99260.49140.5498
25%0.99390.52720.5912
100%0.99500.54960.6290

Scores are balanced accuracies, with three NPC/random seeds averaged within mouse. Raw history has the higher mean at every budget; NPC is below random NPC in five mice at full labels. These linear probes measure accessibility, rather than total information content, and differ from the submitted probe-family-selected results. Small label budgets can omit classes.

Q3: How does the forecasting evaluation relate to the Neural Latents Benchmark?

A: Our evaluation tests what a frozen neural representation predicts under a fixed causal input window. NLB [6] provides standardized co-smoothing and forward-prediction tasks. Our analysis fits matched readouts to neurons excluded from encoder pretraining and evaluates multiple representation uses. Neuron masks, splits, horizons, reference rates and readout protocols differ from NLB. A direct leaderboard comparison requires evaluation using NLB’s prescribed protocol; that evaluation remains open. The contribution alongside NLB is the separation of representation learning, neural forecasting and task accessibility under controlled observations.

Reviewer_FGGUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an approach to encode population neural activities into "generic-purpose" latent states, termed Neural Predictive Coding (NPC). NPC is trained to a GRU to encode feature-mapped binned population activities, then the encoded latent states are projected to contrastively discriminate multiple sampled candidates at different horizons, using a horizon modified designed InfoNCE loss function. Once trained, the latent state are tested to (1) forecast held-out neurons during the same recording session; (2) decode behavioural variables; (3) forecast evolution of low-dimensional neural state, visualised with PCA.

The paper compared NPC against CEBRA (Schneider et al., Nature 2023) and NDT2 (Ye et al., NeurIPS 2023) for the above-mentioned tests on 4 tasks, 26 recordings, and 18 animals, and reported some improvements, albeit the authors were honest that some improvements were not significant.

Critical strengths

Machine learning approaches to causal forecasting of neural state and the evaluation of these states is an under-addressed realm at ICLR.

Perhaps the strongest part of the paper, in my opinion, is the novelty of evaluation methodologies. For example, the fixed-emission dynamics test is a new idea that tests whether a (linearly) transformed latent state is predictable of future held-out neural activity. The scale of experimental recordings used is also broad. This can be advertised as a standard evaluation framework for neural latent state prediction models. In fact, the paper may be better presented as a report on evaluation methodologies, rather than focusing on NPC.

Some results from validating NPC on the task-dependent tests are interesting. For example, the authors found that the forecasting and decoding ability of neural state may not agree in the mouse visual perception task. The paper could benefit from a reframing and adding a handful of explanation of the observations.

Critical weaknesses

The improvement in performance of NPC over baseline comparison models, as reported, were not substantive. NPC was on par with CEBRA-Time and NDT2 in 3 tasks, only overperforming CEBRA-Time in vision tasks (by inspection of Figure 2a and Figure 6, no p-values were reported); it also underperformed NDT2 in several decoding tasks.

The novelty of the proposed NPC remains limited. Its components are chained together from previous works: it inherited the architecture and loss function from Contrastive Predictive Coding (van den Oord et al., 2018) and CEBRA. There was also no reported advantages in parameter or training efficiencies.

The architecture of NPC (MLP encoder + GRU + projection / horizon heads), as described in the paper, seem to be rather small in parameter size. The results could benefit from a quick parameter search, applied independently to each experimental session.

The paper seem to have missed some more current ML for spike models, for example the NDT3 (Ye et al., NeurIPS 2025) and several successors of the POYO line.

Questions for authors
  • I believe for each animal and each of the tasks the authors fitted NPC on, the number of input neurons is different. Could the authors report the complexities of the tasks, and the significance of the effective dimensionality of zt in each task/recording?
  • The abstract states to learn a “representations that can support several uses without learning a new neural encoder for each task”, but to my understanding, the authors trained a separate encoder for each recording (line 186). Could the authors clarify this - perhaps a more important question is, is there evidence that the same encoding model in the same animal is (somewhat) transferrable across sessions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer FGGU (1/4)

Thank you for highlighting the evaluation methodology and recording breadth of the work. We greatly appreciate your suggestion to make the scientific contribution clearer, and address your concerns step-by-step below in four parts: i) novelty and significance, ii) model scale and comparisons, iii) task complexity and effective dimensionality, and iv) encoder reuse and cross-session transfer.

W1: What does NPC contribute to neural representation modeling, and why is it significant?

A: NPC learns a compact causal state by predicting future population representations and demonstrates its value for neural forecasting and task decoding across four brain functions. The contribution connects a concrete neural learning objective to measurable benefits and controlled evidence of what training adds.

NPC proves the effectiveness of predictve learning targets. LFADS [1] and NDT2 [2] use generative reconstruction or masked spike reconstruction; CEBRA [3] organizes embeddings through temporal or behavioral relationships; MYOW [4] and Swap-VAE [5] use mined or augmented views. DCA [6] and CPC [7] establish the predictive-learning tradition on which NPC builds.

NPC explicitly predicts learned neural futures at several horizons. Its causal encoder summarizes past and present activity, and horizon-specific predictors map that state to future population embeddings without behavioral labels during pretraining. Building on CPC, this neural-population implementation provides a compact state for multiple downstream readouts.

The three contributions address the model, its evaluation and the scientific findings:

i) A compact predictive-state model. NPC learns a causal, recording-specific state by predicting future population embeddings at multiple horizons; the frozen state supports competitive neural forecasting and task readouts across four functions.

ii) A reusable framework for evaluating neural representations. Matched observations and frozen encoders separate withheld-neuron forecasting, task accessibility and fitted state evolution, while trained/random and common-trunk controls test what learning adds.

iii) Controlled findings that guide objective choice. Training improves NPC forecasting in all seven screened recordings, and true future pairing improves both endpoints in all five diagnostic recordings; the visual controls show that forecasting gains can accompany reduced linear task accessibility.

We chose two representative methods, CEBRA-Time and NDT2, to provide informative tests of the learning objective. They compare temporal contrast and masked reconstruction with explicit future prediction under a common observed-history constraint.

AuthorsOpenReview ↗

Response to Reviewer FGGU (2/4)

NPC adds forecasting value across all four brain functions. After independently calibrating history/state penalties for every method, adding NPC states improves all four function means over the specified recent-count history summaries. The new combined 400-ms scores are in bits/spike:

FunctionHistory onlyNPC + historyCEBRA-Time + historyNDT2 + history
Motor0.05920.08150.06950.0818
Navigation0.16990.23320.23440.2559
Vision0.23950.32180.26000.2848
Timing0.22510.29920.30350.2630

NPC achieves forecasting gains against both comparator objectives. It exceeds CEBRA-Time in motor and vision and NDT2 in vision and timing. Combined with the positive increments over history in all four function means, these results demonstrate predictive utility across motor, navigation, vision and timing.

The same frozen states also support task decoding across all four functions. The submitted full-label, inner-selected task readouts give:

Function / task metricNPCCEBRA-TimeNDT2
Motor / R²0.49560.30980.5418
Navigation / R²0.57780.51420.5921
Vision / balanced accuracy0.62390.49380.9579
Timing / R²0.22420.14670.2205

NPC exceeds CEBRA-Time in every task mean. Against NDT2, NPC reaches 0.2242 versus 0.2205 R² in timing; NDT2 has higher means in motor, navigation and vision. The same frozen NPC states therefore support both neural forecasting and task readouts across all four functions. These submitted, inner-selected probes are distinct from the new linear raw-history controls.

The scientific significance is the link between predictive training and downstream usefulness. The matched visual controls show that training improves neural forecasting while reducing linear image accessibility. Evaluating both endpoints on the same encoders identifies how objective choice changes what downstream readouts recover. The study provides a compact predictive model and experimental evidence for choosing neural learning objectives according to the intended use.

AuthorsOpenReview ↗

Response to Reviewer FGGU (3/4)

W2: Is the model appropriately scaled, and are the comparisons sufficiently strong and current?

A: NPC uses one compact encoder per recording for both neural forecasting and task decoding. A per-bin MLP with a 64-unit hidden layer produces 16-dimensional population features; a 128-unit GRU and normalized projection form the 128-dimensional state. Separate prediction heads target future population embeddings at 100, 200 and 400 ms. The encoder receives 300 ms of causal history, extended to 500 ms for navigation. This design separates population encoding, temporal integration and future prediction, while allowing multiple readouts to share one frozen encoder.

The core dimensions are fixed across recordings, with learning rates and checkpoints selected by inner neural loss. The main benchmark uses AdamW, batch size 128, weight decay 0.01, gradient clipping at 1 and up to 12,000 updates. The initial learning-rate grid is {0.0003, 0.001}; motor/navigation development adds 0.0001 and evaluates every 100 rather than 500 updates. Seed zero selects the learning rate, and each of three seeds retains its inner-best checkpoint. Motor also uses a population-rate skip. These motor/navigation refinements followed development-result inspection, so the comparison includes additional NPC tuning.

NPC achieves these results with 82,240–123,264 parameters across the 26 recordings. Its 128-dimensional state supports multiple downstream readouts from the same frozen encoder within each recording. It also achieves higher mean task scores than CEBRA-Time across all four functions and higher mean forecasting scores than NDT2 in vision and timing. We selected CEBRA-Time [3] and NDT2 [2] to represent temporal contrastive learning and masked spike reconstruction, respectively; NPC explicitly trains its state to predict future population embeddings at multiple horizons. NPC also achieves higher forecasting means than CEBRA-Time in motor and vision. Against NDT2, its forecasting means are 0.3218 versus 0.2848 bits/spike in vision and 0.2992 versus 0.2630 in timing. Visual NPC uses 0.11–0.12 million parameters versus approximately 3.32 million for the evaluated NDT2 implementation. These comparisons hold observed history, targets, and evaluation partitions common, with independently calibrated readouts for every method. We use official encoder/backbone components with local causal adaptations and train NDT2 from scratch.

We added predictive-information and direct-count controls on all six visual development mice. NPC’s combined forecasting mean is 0.3218 versus DCA’s 0.2970 bits/spike; DCA uses 240 projection-history features versus NPC’s 128-dimensional state, with different training budgets. In a separate fresh-training comparison, NPC reaches 0.3061 and the count GRU 0.3120 under matched training-step time, excluding calibration. NPC’s combined readout exceeds both history-only and state-only readouts in all six mice, and exceeds the count GRU in three. This separates predictive-state effectiveness from the choice between contrastive and count-prediction objectives.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Many thanks to the Area Chair and all three reviewers for their thoughtful comments and suggestions! We appreciate the reviewers’ recognition of our comprehensive evaluation methodology and broad recording coverage (FGGU), rigorous experimental design (Ev4j), and informative training controls and valuable insights into forecasting and task accessibility (nKdY).

Our work advances future prediction as a general strategy for learning neural representations that support multiple downstream uses. We contribute i) a compact predictive framework that learns without task labels and supports forecasting and decoding with one frozen encoder per recording, ii) a reusable evaluation protocol across four brain functions that assesses both predictive power and task accessibility, and iii) scientific findings that guide predictive neural modeling, identifying how true future pairing and multiple horizons strengthen forecasting, and how training shapes the information accessible to task decoding. These contributions provide a modeling strategy and evaluation criteria for developing neural models that combine predictive power with broad downstream usefulness.

Our study covers 26 recordings from 18 animals (macaques, rats and mice) across motor control, navigation, vision and interval timing, evaluating forecasting, task decoding and fitted state evolution. In rebuttal, we address the reviewers’ concerns as follows:

i) Clarifying NPC’s contribution. NPC connects compact predictive states, reusable evaluation across brain functions, and controlled findings that guide objective design.

ii) Demonstrating forecasting value beyond recent activity. Refitting all combinations with separate history/state penalties shows that combining NPC states with recent spike-count summaries improves mean forecasting over either alone in all four functions.

iii) Identifying NPC’s strengths through additional comparisons. On six visual development mice, NPC exceeds DCA’s forecasting mean, while time-matched count prediction and raw-history decoding achieve higher means than NPC in forecasting and linear decoding, respectively.

iv) Strengthening the evaluation of learned dynamics. Direct forecasts exceed fitted state evolution in all four function means, while transition-composition tests quantify consistency across prediction horizons.

The detailed responses provide the supporting results and address each concern point by point. Thank you again for helping us improve this work!

Dear Reviewer,

Thank you very much for your time and thoughtful review. As the discussion deadline approaches, could you please take a look at the authors' rebuttal and share your thoughts in the discussion?

Many thanks for your continued help!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.