acceptodds
Under review as a conference paper at ICLR 2027

Behavioral Geometry Guides Generalizable Neural Representation Learning

Abstract

Neural recordings can change across sessions even when behavior remains similar. This makes behavior a natural reference for learning shared representations, but pointwise prediction alone does not specify how these representations should be organized. We introduce Behavioral Relational Guidance (BeRG), which uses behavioral geometry as a structural prior for neural pretraining. A frozen behavioral encoder maps behavioral histories and task context to state representations. For each state, a soft neighborhood assigns probability weights to states from other sessions of the same task, giving higher weights to more similar behavioral representations. BeRG matches these neighborhood distributions in a neural decoding subspace, jointly optimizing relational and decoding objectives. The framework supports fixed and learned encoders, while inference requires only neural inputs. We evaluate hand-velocity decoding from macaque motor-cortical recordings during center-out and random-target reaching. Across six neural architectures, BeRG reduces normalized mean squared decoding error by approximately 15–23% on validation data. Compared with fixed-feature guidance, learned guidance increases mean for 250-ms-ahead velocity readout from frozen representations by approximately 20%. It also improves neighborhood-based velocity readout and behavioral similarity among cross-session neighbors, with benefits in both the decoding subspace and final hidden representations. In longitudinal recordings during a finger-movement task, BeRG further improves decoding on previously unseen recording dates after few-shot calibration. These findings point to a broader role for behavioral data as structural supervision, offering a route toward neural models that remain useful across readouts and recordings.

Primary area:
applications to neuroscience & cognitive science
Keywords:
neural population activity, motor decoding, behavioral geometry, representation learning, multisession pretraining

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_LpMqOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper leverages fixed behavior encoding for neural decoder training, and proposed BeRG to match the distributional space between neural and behavior states of similar tasks. BeRG can leverage behavior information explicitly, with trained encoder from scratch and with pretrained model, and is compatible with various neural decoder backbone showing improved performance.

Critical strengths
  • The use of frozen behavior encoder and distribution matching is novel
  • Method demonstrate improved performance over many decoder backbones
  • Conducted future probing and representation analysis
Critical weaknesses
  • The writing of the paper makes it difficult to gauge the performance of the proposed methods, especially in the results section. Experiment details are not clear.
  • In some of the experiments, the improvement of using BeRG over base is relatively small or slightly worse, more comprehensive investigations are likely needed.
  • Code is not provided
Questions for authors
  1. Since both decoding and relational guidance are in the loss function (both leverage behavior targets), a key question is whether the relational guidance mainly acts as a regularizer. Can the proposed method operates without decoding loss?
  2. Can the author elaborate how the source-training bank was selected? What is the definition of the same task for random-target sessions?
  3. How bank size affects the performance of the proposed method?
  4. Please clarify split details on both datasets. For example for Perich datasets, what are development endpoints and how do the 19 sessions represent it.
  5. Can the author provide citation for all the baseline, for example I am not clear if POYO-light is just a variant of POYO methods proposed in Azabou 2023
  6. There seems to be only POYO as a recent neural decoder baseline, can BeRG be applied to other popular decoder methods like NDT? In addition, the choice of biGRU is non-common among neural decoder, have the authors tried normal GRU as well?
  7. Please explain the training details for POYO-light. What do the authors mean by “these runs do not use an official pretrained checkpoint” (line 259) and “BeRG arm resumes from weights at epochs 35 and 50 without restoring optimizer or RNG states” (appendix D3)
  8. What are the potential reasons for a worse performance in MC maze? I understand this is from different subject and conducting maze tasks. I am curious if this is a dataset / lab difference or task difference. More finetuning datasets could likely provide some insights

Format related:

  1. Submission pdf is likely not generated properly, all cross-references to figures/tables/citations are disabled.
  2. Figure 3 is being cut-off
  3. Citation on Perich dataset is likely wrong in date, I believe it should be 2025
8 comments
AuthorsOpenReview ↗

Response to Reviewer LpMq (1/6): Relational supervision and training without the decoding loss

Thank you for the careful review. We address the methodological, experimental, and presentation concerns below.

Q1, concern summary. Does relational guidance mainly act as a regularizer when both objectives use behavior, and can the method learn representations without the decoding loss?

What the two objectives constrain.

Relational guidance supplies behavior-dependent structural supervision and can also regularize decoder training. The decoding loss constrains current behavioral predictions at each state, whereas the relational objective constrains how neural states from different sessions preserve neighborhoods defined by the behavioral reference. The two objectives therefore constrain prediction accuracy and cross-session behavioral organization along the same decoding path.

Training without decoding supervision.

To test representation learning without the decoding loss, we removed this loss during neural representation training, froze the resulting models, and fitted linear ridge readouts from their final hidden states using a common protocol. Table R1 reports the resulting readouts alongside random initialization, decoding-only Base, and joint LIGHT training.

Table R1. Frozen behavioral readouts with and without decoding supervision.

Training conditionCurrent velocity R2R^2100-ms future velocity R2R^2250-ms future velocity R2R^2
Random initialization−0.0420±0.0013-0.0420 \pm 0.0013−0.0458±0.0009-0.0458 \pm 0.0009−0.0393±0.0010-0.0393 \pm 0.0010
Decoding only (Base)0.8165±0.00150.8165 \pm 0.00150.6978±0.00510.6978 \pm 0.00510.3652±0.01660.3652 \pm 0.0166
Relation only0.6584±0.03520.6584 \pm 0.03520.6490±0.03090.6490 \pm 0.03090.5228±0.01830.5228 \pm 0.0183
Joint decoding and relations (LIGHT)0.8630±0.00130.8630 \pm 0.00130.7834±0.00140.7834 \pm 0.00140.5860±0.00290.5860 \pm 0.0029

Window Transformer final-hidden ridge readouts on Perich source validation. Scores are session-macro R2R^2, reported as mean ±\pm sample SD across neural-training seeds 11, 22, and 33, conditional on the original frozen LIGHT teacher. Each query session is excluded from readout fitting; its training data remain included in neural pretraining. These scores differ from trained-decoder pooled R2R^2.

The relational objective can learn behaviorally informative representations without direct decoding supervision during representation training, while the joint objective gives stronger current-behavior readout under the tested recipe. We summarize this finding in Section 4.1 and report the ablation protocol and unified readout results in Appendix D.1.

AuthorsOpenReview ↗

Response to Reviewer LpMq (2/6): Source-training bank, task definition, and bank size

Q2–Q3, concern summary. How is the source-training bank constructed, what counts as the same task for random-target sessions, and how does bank size affect performance?

Cache, bank, and per-anchor candidates.

The full source-training cache, dynamic bank, and per-anchor candidate set serve different roles. The cache contains valid paired neural and behavioral states from source-training intervals. Each active task uses a session-balanced, cyclically refreshed bank of 256 states. For each anchor, its own session is excluded before selecting 64 distinct candidates from near-to-distant behavioral-similarity ranks, with at most four states per session. In Perich, “same task” denotes the CO or RT paradigm; target changes within RT do not create new task labels. Behavioral history and available target and phase context distinguish movement states within each paradigm. We have clarified the task definition in Section 3.3 and reorganized Appendix G.3 into a six-step procedure from cache construction and anchor selection to relational-loss computation.

Bank size and candidate-quality evidence.

Bank size determines the comparison pool, whereas the 64 candidates support an individual anchor’s relational distribution. All 256 bank states undergo neural forward passes and contribute to session centering, so changing bank size also affects computation and the centering estimates. Table R2 summarizes the earlier candidate-quality diagnostic.

Table R2. Candidate quality in an early source-training diagnostic.

Candidate constructionMean best-candidate teacher cosine similarityMedian best-candidate rank in the full eligible pool
Session-fixed sets of 64 candidates0.89914,073.5
Refreshed 256-state bank with 64 candidates selected per anchor0.94841,101

Source-training diagnostic using 3,168 anchors from 99 sessions; 32 banks were evaluated for the dynamic construction. Higher cosine similarity and lower rank indicate closer behavioral candidates.

The dynamic construction provides closer behavioral neighbors in this diagnostic. The comparison jointly changes bank refresh and candidate selection; the independent effect of bank size on decoding performance under the final recipe has not been systematically evaluated. We report these diagnostics and their scope in Appendix G.3.

AuthorsOpenReview ↗

Response to Reviewer LpMq (3/6): Data splits and development endpoints

Q4, concern summary. How are the two datasets split? In particular, what are the Perich development endpoints and the role of the 19 sessions?

Perich teacher fitting, selection, and development.

The 19 Perich sessions form a teacher-selection set stratified by animal and task, covering the five animal–task combinations present in the source data. Learned behavioral teachers are fitted on training intervals from the other 80 sessions, with checkpoints selected using validation intervals from these 19 sessions. Neural pretraining and the relation bank use training intervals from all 99 source sessions. The 19 sessions are held out from teacher fitting while their training intervals still participate in neural pretraining. “Development endpoints” refers to source-validation evaluations used during development, because the 99-session aggregate includes validation data used for teacher selection. Evaluations on held-out TEST intervals within source sessions are reported separately in Appendix C.1.

Table R3. Perich session split for behavioral-teacher fitting and selection.

AnimalTaskSource sessionsTeacher fittingTeacher selection
CCO534310
CRT15123
JCO321
MCO22184
MRT651
Total998019

LIGHT and MOMENT use the same session split. Teacher fitting uses TRAIN intervals from 80 sessions; selection uses VALID intervals from the other 19. Neural pretraining and bank construction use TRAIN intervals from all 99 sessions.

LINK source, development, and final dates.

LINK is divided chronologically into 94 source, 17 development, and 45 final dates. Within source dates, retained trials 1–270 are used for training and 271–300 for source validation. The separate 17 development dates determine the training and calibration protocol. Final dates are excluded from source-model fitting and protocol selection, but recent50 evaluation uses labeled trials 251–300 from each final date for calibration and scores trials 301–375. Source validation, development dates, and final-date calibration and scoring therefore serve distinct roles. We have clarified these roles in the experimental setup in Section 4 and Appendix A.1, including the stratified session counts; target calibration and source-session TEST protocols are detailed in Appendices B.1 and C.1, respectively.

Reviewer_JTPWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an auxiliary training loss called Behavioral Relational Guidance. It is for neural decoders that predict movement velocity from motor cortex recordings. A frozen behavioral encoder is used to decide which time points from other sessions are behaviorally similar to each other, and the neural model is trained so that its own representations have the same similarity structure, alongside the normal decoding loss. The behavioral encoder is only used during training.

The authors show that BeRG improves velocity decoding across six neural architectures, makes future velocity easier to read out from the learned representations, and improves decoding on new recording dates and an unseen monkey after a small amount of calibration.

Critical strengths

The method is simple and seems to work across a wide range of architectures. BeRG improves decoding for all six backbones tested, including Transformers, a TCN, a BiGRU, an MLP and a version of POYO, and since the behavioral encoder is removed at inference there is no added cost at test time. This matters because it makes the method easy to drop into existing decoding pipelines.

Additionally, the observation that decoding accuracy on its own does not tell you much about how the representation is organized. The fixed and learned behavioral encoders give almost the same current decoding accuracy, but the learned one makes future velocity much easier to read out from the frozen representations. Similarly, the Base model already supports good linear decoding at the decoding subspace, but its nearest neighbors from other sessions are not behaviorally similar, and BeRG changes this a lot. I think this is a useful point for the field, since most neural decoding work is compared only on decoding R², and it suggests that two models with the same R² can be very different in how reusable their representations are.

Critical weaknesses

The main weakness is that the evidence for the core claim is thin. Almost all of the Perich results, including the main table, the encoder comparisons, the readout analyses and the cross-animal results, are from a single training seed, and the gains are small (around 0.03 R²). The authors say that additional repetitions exist in their experiment archive but are not reported, which makes it hard to know how much of the gain is real. The controls also leave a lot unexplained. When the relations between the behavioral and neural candidates are shuffled, about two thirds of the gain over Base remains, so the part that is actually due to the correct behavioral relations is quite small. BeRG also takes around four times the training time of Base because of the extra candidate forwards, and Base is not given matched compute or data exposure. Finally, the main results are on validation data from sessions used in pretraining, some of which were also used to select the behavioral encoder, so they are a development result and not a held-out test.

The second weakness is the positioning against prior work and the lack of baselines. BeRG is only compared against Base and variants of itself, not against other published methods like CEBRA, even though the paper cites them. There is also closely related work that is not cited, such as CANDY (Jiang et al., NeurIPS 2025) and PNBA (Zhu et al., ICML 2025), which both align neural representations to behavior across sessions on monkey reaching data, as well as earlier work on stabilizing decoders across days (e.g. Gallego et al., 2020; Degenhart et al., 2020). Since the loss builds directly on PKT, the novelty is mainly in the application. The generalization claims in the title also seem stronger than the results support, since without calibration BeRG does worse than Base on the held-out LINK dates and on MC-Maze.

Questions for authors

Questions that could change my assessment:

The paper says additional seed repetitions for the Perich experiments exist in the experiment archive. Could the authors report the main table and the shuffle and extra decoding controls over these seeds, with the variation across seeds? Given the size of the gains, this is the most important thing for me.

How does BeRG compare to other methods that align neural representations to behavior across sessions, such as CEBRA, CANDY or PNBA, using the same backbones and data? CANDY in particular uses a very similar idea of ordering latent distances by behavioral distance. And for the cross-session claims, how does it compare to an alignment-based approach such as Gallego et al. (2020) or Degenhart et al. (2020)?

Does Base improve if it is given matched compute, for example more updates or access to the same candidate windows? This would help separate the effect of the relational loss from the extra computation and data exposure.

5 comments
AuthorsOpenReview ↗

Response to Reviewer JTPW (1/4): Training replication and held-out test evaluation

CW1 / Q1, concern summary. Can the archived training repetitions be reported with their variability, and what held-out test evidence supports the core encoder and readout findings beyond source-validation development results?

Neural-training repetitions.

We now report three neural-training seeds for the Window Transformer and its main controls. The paired current-decoding gain of LIGHT over Base is 0.0306±0.00240.0306 \pm 0.0024 R2R^2 (mean ±\pm sample SD across training seeds). The current-decoding gain is positive in all three tested training repetitions. The six-backbone comparison in manuscript Table 1 still uses one pretraining seed per condition; repetitions focus on the Window Transformer used for the core encoder comparisons and control analyses. Table R3 below summarizes these results, with individual-seed statistics in Appendix D.1.

Held-out temporal test evaluation.

The original source-validation comparisons are development evaluations and include sessions used for behavioral-teacher selection. Appendix A.2 reports positive current-decoding test gains for three backbones in the historical single-seed evaluation. We additionally fixed the trained Base, Explicit, and LIGHT models and evaluated future readout using a common protocol on held-out time intervals within the source sessions. TEST is not used for model or readout-setting selection in this evaluation.

Table R1. Future-velocity readout on held-out temporal TEST intervals. Scores are session-macro joint-xy R2R^2 over 25,344 common queries from 99 source sessions. Values are mean ±\pm sample SD across three neural-training seeds. Each LIGHT row uses one independently fitted, fixed behavioral teacher; paired gains subtract Explicit within neural seed.

ConditionTeacher seed250-ms future R2R^2Paired gain over Explicit
Base—0.3660±0.01150.3660 \pm 0.0115—
Explicit—0.4919±0.01080.4919 \pm 0.0108—
LIGHT110.5905±0.00250.5905 \pm 0.0025+0.09859±0.01063+0.09859 \pm 0.01063
LIGHT220.5925±0.00110.5925 \pm 0.0011+0.10065±0.01038+0.10065 \pm 0.01038

The approximately 0.10 R2R^2 future-readout advantage over Explicit recurs with both teachers, with positive differences in all three neural-training seeds for each teacher. This reproduces the core difference across the tested teacher-fitting and neural-training repetitions. These test intervals belong to the source sessions used for pretraining. Appendix C.1 reports the protocol and complete statistics, with teacher-fitting details in Appendix D.2.

Repeated neighborhood analyses.

We also report three-seed neighborhood analyses. Table R2 shows the layer-dependent pattern under centered-cosine distance: both behavioral references improve neighborhood readout over Base, while the relative performance of Explicit and LIGHT depends on the layer. Appendix C.2 additionally reports standardized-Euclidean results. Three-seed Monkey T adaptation results appear in Table R5 below and Appendix B.2.

Table R2. Repeated neighborhood readout. Current velocity is predicted from 20 other-session neighbors using centered-cosine distance. Scores are session-macro R2R^2 on the original 25,344 source-validation queries, reported as mean ±\pm sample SD across three neural-training seeds.

RepresentationBaseExplicitLIGHT
Block 30.3782±0.01060.3782 \pm 0.01060.6176±0.01270.6176 \pm 0.01270.6032±0.01180.6032 \pm 0.0118
Final hidden0.7656±0.01460.7656 \pm 0.01460.8395±0.00180.8395 \pm 0.00180.8505±0.00260.8505 \pm 0.0026
Common-Port0.2185±0.03380.2185 \pm 0.03380.8320±0.00090.8320 \pm 0.00090.8469±0.00260.8469 \pm 0.0026
AuthorsOpenReview ↗

Response to Reviewer JTPW (2/4): Additional supervision, training budgets, and relational guidance

CW1 / Q3, concern summary. How much benefit remains after accounting for the gains retained by shuffled guidance, additional candidate exposure, and longer Base training?

Stronger supervision and randomized controls.

Base improves with additional supervision or longer training, but the stronger controls below still fall below LIGHT. Because the original Shuffle retains behavior-dependent candidate selection, we also added a control with randomized candidate ranking and shuffled correspondence.

Table R3. Candidate-supervision and training-budget controls. Scores are trained-decoder pooled normalized R2R^2 on Perich source validation, reported as mean ±\pm sample SD across three neural-training seeds. All conditions use 20,000 updates except the extended Base. Differences are paired within seed.

ConditionCurrent-decoding R2R^2LIGHT minus conditionPositive differences
Base0.85713±0.001410.85713 \pm 0.00141+0.03060±0.00237+0.03060 \pm 0.002373/3
Extra candidate decoding0.87442±0.000870.87442 \pm 0.00087+0.01331±0.00185+0.01331 \pm 0.001853/3
Original shuffled guidance0.87844±0.000910.87844 \pm 0.00091+0.00928±0.00188+0.00928 \pm 0.001883/3
Random ranking + shuffled correspondence0.87690±0.001810.87690 \pm 0.00181+0.01083±0.00089+0.01083 \pm 0.000893/3
Extended Base (172,600 updates)0.87652±0.000140.87652 \pm 0.00014+0.01121±0.00096+0.01121 \pm 0.000963/3
LIGHT0.88773±0.000980.88773 \pm 0.00098——

LIGHT retains an incremental benefit over the tested candidate-supervision and randomized controls in every training seed. These comparisons do not provide an additive causal decomposition of individual design choices. Appendix D.1 reports the control definitions and individual-seed results.

Longer Base training.

The extended Base uses approximately 7%–10% more actual student-training time than LIGHT's timing reference, yet LIGHT retains a mean gain of 0.0112 R2R^2. Thus, within the tested budget range, longer ordinary decoding training does not recover the same performance. Appendix E.2 reports the timing definitions and budget–performance curves.

Candidate-exposure-matched LINK comparison.

In a separate single-seed LINK source-training comparison, LIGHT and extra decoding share initialization, main windows, and identical candidate windows at every update.

Table R4. LINK candidate-exposure-matched comparison at 40,000 updates. Scores are source-validation date-macro R2R^2 over 94 dates, using neural-training seed 11. Exposure counts are repeated window accesses, not unique samples.

ConditionSource-validation R2R^2Main-window exposuresCandidate-window exposures
Extra decoding0.53931.92 million10.24 million
LIGHT0.55431.92 million10.24 million

LIGHT retains a 0.0150 R2R^2 gain at this fixed endpoint. Extra decoding performs better at earlier update budgets, so this result supports a candidate-exposure-matched benefit at the tested endpoint. This source-domain comparison is separate from held-out-date calibration. Complete results are in Appendix E.3.

AuthorsOpenReview ↗

Response to Reviewer JTPW (3/4): Relation to prior methods and the specific contribution

CW2 / Q2, concern summary. How does BeRG differ from PKT, CEBRA, CANDY, PNBA, and earlier cross-day alignment methods? What do the available comparisons establish about its specific contribution?

Motivation.

Neural recordings are costly to acquire, whereas behavior can be measured independently, with robotics and movement simulation offering potential additional data sources. This motivates learning representations of behavioral states and temporal structure from behavior alone and using them to guide neural decoding. Our longer-term aim is to improve data efficiency and transfer across recordings when paired neural–behavioral data are limited. BeRG investigates behavioral relational supervision as a step toward this goal.

Integration with decoder training.

Building on PKT's probabilistic neighborhood distillation, we construct candidates across sessions within the same task, use session-balanced sampling and neural centering confined to the relation branch, and apply guidance to a subspace on the decoding path. Our contribution is this integration of behavioral reference construction with decoder training and its empirical evaluation.

Behavior-based alignment methods.

CEBRA organizes contrastive sampling using behavioral or temporal conditions, including in multi-session training. CANDY aligns neural embeddings across sessions through continuous behavioral-distance rankings and jointly learns shared linear dynamics. BeRG constructs fixed or learned behavioral references from history and task context, then matches the resulting candidate-neighborhood probability distributions in the neural decoding subspace. The distinction concerns how behavioral relationships are defined and transferred.

PNBA jointly learns probabilistic neural and behavioral latents with bidirectional generative constraints. BeRG keeps its behavioral reference frozen during neural training and transfers relationships between states without directly aligning the two modalities' latent coordinates. BLEND uses a teacher receiving both neural activity and behavior to supervise a neural-only student through neural reconstruction and distillation. BeRG's behavioral reference receives no neural observations and supplies cross-session state-neighborhood targets alongside current-behavior decoding.

Cross-day alignment and comparison scope.

Gallego et al. and Degenhart et al. support stable decoding through cross-day neural-manifold alignment; BeRG intervenes during shared-decoder pretraining. Our matched-backbone controls evaluate the incremental benefit of this supervision; direct comparisons with these methods under the same data and protocols have not been conducted. We have added CANDY, PNBA, and earlier alignment studies to Section 2 with brief methodological distinctions. The neighborhood construction and decoding-path constraint are specified in Sections 3.2 and 3.3.

Reviewer_b4ESOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors train a neural decoder on 99 sessions of the Perich monkey reachin datasets. Each session gets its own learned input projection, all sessions share a common transformer backbone, and the main novelty is an extra loss. For each timepoint, 64 timepoints from other sessions under the same task are drawn. A frozen behavior encoder maps each timepoint and task context to a vector. Then, a softmax over cosine similarities results in a sofrt distribution over behavioral neighbors. The softmax is computed in a smaller 32dim subspace with a KL term that is designed to pull the neural distribution toward the behavioral subspace. The authors present three different behavior encoders: (i) explicit is not leared and used behavioral history and context only. (ii) light is a small transformer trained to forecast and reconstruct beahvior. (iii) moment is a frozen time-series foudnation model plus learned adapter. All encoders receive the context, which is target location and phase of the trial. The contribution of this work is a training time loss that pulls neural states with similar behavior together across recording sessions with similar tasks, which improves inference time performance. In particular, the authors claim that their method leads to (1) behavioral decoding gain on every backbone that they tried, (2) frozen representations that are able to forecast behavior 250 ms into the future better, and (3) better few shot calibration in new sessions and new subjects.

Critical strengths

The problem matters for brain-computer interfaces (BCIs), where decoders often need quick recalibration as sessions and subjects have substantial drift. The method proposed here is straightfoward and well presented. It adds a loss, used only during training, that uses behavioral similarity to organize neural representations across sessions, without needing to match neurons or timestamps between recordings. Inference is unchanged, since the teacher is dropped and only a new readout per recoding needs to be learned, as is the standard for novel recordings. The authors show a gain in decoding on six backbones, with gaines for each one. The paper also includes useful ablations, uses public datasets and has very clear figures.

Critical weaknesses
  1. The paper never compares BeRG with an existing method. Every neural result compares a backbone with and without the new loss. The authors do not compare their method against very similar prior state of the art:
  • multi-session CEBRA [5], which already pairs states across sessions by behavioral similarity
  • BLEND [6], which also uses behavior only during training

POYO/POYO+ and NDT checkpoints are readily available not tested either, and no comparison with other alignment methods [1-4] was performed on the LINK datasets. POYO also holds out Monkey T and reports the performance when only a single adapter is trained, resulting in R² 0.90 (CO) and 0.78 (RT). This paper reports 0.73 to 0.76, though it is unclear if these scores are comparable. The contribution is an extra loss term, so its value critically depends on whether this contribution actually beats existing methods. The gain also shrinks on the strongest backbone (+0.018 R² for POYO-Light, +0.012 under the older POYO-style evaluation), so it may not hold up against a stronger baseline.

  1. The results do not show that behavioral geometry actually causes the gain. The decoding gains are fairly small (0.018 to 0.048 R²), and the paper’s own controls account for most of them. On the Window Transformer, plain decoding on the extra candidate windows recovers half of the +0.0326 gain, which leaves roughly only 0.01 R² that depends on correct behavioral matching. Unfortunately, the behavioral geometry analyses were only computed on a single seed and can not be easily interpreted. Overall, it is unclear if the behavioral geometry is aiding neuronal representations, which is the claimed central contribution of this work.

Other weaknesses

  • Novelty is limited. The loss is inspired by PKT [7] applied across sessions, which is very similar to multi-session CEBRA [5], BLEND [6] and contrastive regression [8].
  • The writing is hard to follow. Teacher inputs and candidate sampling appear only in the appendix, the light condition is not clearly defined, and many paragraphs end by listing what results “do not establish.”

References: [1] Gallego et al., (CCA alignment of latent dynamics across days), 2020, https://www.nature.com/articles/s41593-019-0555-4

[2] Degenhart et al., (manifold alignment for BCI stabilization), 2020, https://www.nature.com/articles/s41551-020-0542-9

[3] Farshchian et al., (ADAN), 2019, https://arxiv.org/abs/1810.00045

[4] Karpowicz et al., (NoMAD), 2025, https://www.nature.com/articles/s41467-025-59652-y

[5] Schneider et al., (CEBRA), 2023, https://www.nature.com/articles/s41586-023-06031-6

[6] Guo et al., (BLEND), 2025, https://arxiv.org/abs/2410.13872

[7] Passalis & Tefas, (PKT), 2018, https://arxiv.org/abs/1803.10837

[8] Zha et al., (Rank-N-Contrast), 2023, https://arxiv.org/abs/2210.01189

Questions for authors
  1. How does BeRG compare with multi-session CEBRA plus a linear decoder, and with BLEND, on the same splits?

  2. Does BeRG still help when added to an official POYO+ or NDT3 checkpoint?

  3. What are the Monkey T results under POYO’s evaluation protocol?

  4. Does a Base trained to predict future velocity directly match LIGHT on the future readout?

  5. What is your explanation that BeRG hurts zero-shot decoding on the LINK dataset?

7 comments
AuthorsOpenReview ↗

Response to Reviewer b4ES (1/6): Method design and relation to prior work

Thank you for your detailed comments on method positioning, baseline strength, representation analysis, transfer, and presentation. We organize our response into six comments below.

CW1 / Q1 and novelty concern, summary. How does BeRG compare with multi-session CEBRA and BLEND on the same splits, and what distinguishes its design from PKT and contrastive regression?

BeRG investigates how relationships between behavioral states can provide additional supervision during the pretraining of neural decoders across recordings. Our specific contribution combines a separate behavioral reference, neighborhood construction across sessions within the same task, and relational guidance on the decoding path. This allows behavioral relationships and current-behavior prediction to jointly train the shared neural backbone.

Relation to PKT. BeRG builds on PKT's probabilistic neighborhood distillation and adapts the construction and application of relational targets to heterogeneous neural recordings. Candidate states come from other sessions of the same task, with session balancing and sampling across near-to-distant behavioral similarity ranks. Within-session centering is confined to the neural relation branch, preserving the uncentered states used for decoding. Relation matching acts on the Common-Port subspace on the behavioral decoding path and is optimized jointly with current-behavior prediction. These designs adapt relational distillation to behavioral supervision across heterogeneous recordings.

Distinction from CEBRA and Rank-N-Contrast. CEBRA uses behavioral or temporal auxiliary variables to organize contrastive sampling and learn decodable neural representations, including through multi-session training. Rank-N-Contrast organizes its contrastive objective according to relative distances between continuous labels. BeRG uses a separate behavioral encoder to map behavioral history and task context into soft neighborhood distributions over candidate states, then matches these distributions in the actual decoding subspace. Behavioral state relationships therefore shape the pretrained model jointly with decoding error, providing supervision beyond the current output labels.

Distinction from BLEND. BLEND's teacher receives both neural activity and behavior, targets neural reconstruction, and transfers knowledge to a student that receives neural inputs alone. BeRG's behavioral encoder receives no neural observations, remains frozen during neural training, and supplies relational targets between behavioral states across sessions. The neural student jointly optimizes relation matching and current-behavior decoding. Thus, both methods use behavioral information during training, but differ in the teacher's inputs, the supervision transferred, and the student's training objective.

AuthorsOpenReview ↗

Response to Reviewer b4ES (2/6): Matched POYO-Light retraining, Monkey T scoring, and scale

CW1 / Q2–Q3, concern summary. Does BeRG retain a benefit on a stronger backbone, has it been evaluated with official POYO+ or NDT3 checkpoints, and are the Monkey T scores comparable with POYO’s evaluation protocol?

Matched POYO-Light retraining.

We added paired POYO-Light retraining with matched initialization, main-window sampling, optimization settings, and full-state recovery rules.

POYO-Light results on the shared Perich source-validation set, using pooled R2R^2 over 99 source sessions and one paired neural-training seed:

Table R1. Matched POYO-Light retraining

ComparisonBaseBeRG-LIGHTLIGHT − Base
Matched retraining, epoch 600.89280.9027+0.0099

This supports an incremental benefit from behavioral relational guidance on this stronger backbone within our study. The new comparison is listed separately in Table 1, with training protocols and subgroup results in Appendix D.3.

Monkey T scoring protocol.

For Monkey T, the approximately 0.73–0.76 scores in our paper compare Window Transformer Base and LIGHT after pooling CO and RT, whereas POYO's approximately 0.90/0.78 scores refer to CO and RT separately, with CO evaluated during successful reaching periods. We therefore added task-specific results and re-scored predictions from fixed models.

The following scores use the same two pretraining seeds (22 and 33), 100% training-support budget, and output-averaged session-macro R2R^2. The two CO rows change only the scoring interval for the same fixed models.

Table R2. Monkey T scoring with fixed models

Task and scoring intervalBaseLIGHTLIGHT − Base
CO, all archived test queries0.74580.7803+0.0345
CO, successful reaching periods0.81260.8446+0.0320
RT, continuous test queries0.70760.7326+0.0250

Re-scoring the same models shows that the scoring interval substantially affects absolute performance, while the benefit of BeRG is retained in both tasks. This analysis aligns selected scoring choices; model, data-split, and adaptation differences remain. The protocol comparison and complete task-specific results are reported in Appendix B.2.

Official checkpoints and scale.

Our POYO-Light experiment uses the lightweight variant in this study, trained from a shared initialization. Within our available compute budget, we have not completed systematic paired evaluations using official pretrained POYO, POYO+, or NDT3 checkpoints, and benefits at larger model and data scales remain untested. Our current relational construction connects sessions within the same task. Future work will expand task coverage in both behavioral and neural data, develop cross-task relations that preserve task context, and evaluate this supervision across model and pretraining-data scales.

AuthorsOpenReview ↗

Response to Reviewer b4ES (3/6): Direct future supervision and interpretation of future readout

Q4, concern summary. Can a Base model trained to predict future velocity directly match LIGHT on the future-readout evaluation?

DirectFuture control.

Direct future supervision can match and exceed LIGHT's 250-ms future readout. We added a DirectFuture control that retains Base's current-decoding objective and adds an auxiliary objective predicting velocity 250 ms ahead. After training, we freeze the neural models and refit the same linear readout.

The table reports mean ±\pm sample SD over three neural-training seeds on source validation. Current decoding uses native pooled R2R^2; future readout uses session-macro R2R^2 from a frozen-model ridge probe. These are distinct evaluation endpoints.

Table R3. Direct future supervision and frozen future readout

Training conditionCurrent-decoding R2R^2250-ms future-readout R2R^2
Base0.8571±0.00140.8571 \pm 0.00140.3652±0.01660.3652 \pm 0.0166
LIGHT0.8877±0.00100.8877 \pm 0.00100.5860±0.00290.5860 \pm 0.0029
DirectFuture0.8589±0.00020.8589 \pm 0.00020.6049±0.00240.6049 \pm 0.0024

Strong future readout can also be obtained through direct future supervision, while LIGHT provides higher current-decoding performance in this comparison. We therefore use future readout as a complementary representation probe and assess supervision through both outcomes. DirectFuture's auxiliary multiplier was selected using seed 11 and fixed for seeds 22 and 33. The complete comparison and training settings are reported in Appendices C.1 and D.1.

Predictive supervision and teacher inputs.

Our behavioral forecasting objective is intended to learn the temporal structure of movement states and transfer that structure into neural decoding through relational supervision. Motor-related neural activity can contain information about upcoming behavior, providing a physiological motivation for predictive behavioral representations. In BeRG-LIGHT, observed future trajectories supervise behavioral-teacher fitting, whereas relational targets are generated from behavioral history and task context without directly supplying each sample's future trajectory to the state encoder. Some Perich task context is retrospective training information, as detailed in Appendix D.2. At inference, the behavioral teacher is removed and only neural inputs are required. This design uses temporal structure learned from behavior to guide neural representations, with future readout probing their linearly accessible predictive information.

Reviewer_c9YjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Behavioral Relational Guidance (BeRG), which uses a frozen behavioral encoder to define soft neighborhoods across recording sessions and matches these relationships within a neural decoding subspace. Training combines relational supervision with current-behavior decoding, while inference requires neural inputs alone. The experiments evaluate decoding across several backbones, frozen-representation readouts, neighborhood organization, and supervised adaptation to held-out recording dates and one held-out animal.

Critical strengths

The problem is well motivated: behavioral relationships offer a plausible reference for learning representations across heterogeneous neural recordings. Current decoding improves across the evaluated backbones. The fixed-feature, shuffled-correspondence, and extra-decoding controls provide useful evidence about components of the method.

Critical weaknesses
  1. The advantage over strong, comparably trained baselines remains insufficiently established. In Table 2, Base achieves R²=0.8561, extra decoding(Line 2) reaches 0.8735, shuffled guidance with BeRG-LIGHT(Line 3) reaches 0.8775, and LIGHT reaches 0.8887. The gap to LIGHT therefore decreases from 0.0326 against Base to 0.0152 against extra decoding and 0.0112 against shuffled guidance. These results make baseline strength and training exposure central to assessing the contribution.And why line 2 and 3 have omitted index?I don’t find the reason that these four result can’t be achieved by formula or something else in caption. Table S13 lists nominal LINK training counts of 10.24 million candidate-window exposures alongside 1.92 million main windows, making the additional candidate count approximately 5.33 times the main-window count. Equal optimizer updates therefore leave a substantial exposure difference. The Perich extra-decoding control is not a full computation match, and shuffling retains behavior-dependent candidate selection. A candidate-exposure-matched supervised baseline and budget–performance curves would clarify the remaining advantage. Appendix D.3 further reports that POYO-Light’s BeRG arm restarts from weights at epochs 35 and 50 without restoring optimizer or RNG states, but Base trains continuously.I think it is not properly aligned and may introduce of bias. The contribution also needs clearer experimental positioning against related supervision. PKT transfers neighborhood distributions, CEBRA uses behavioral information to organize neural embeddings, and BLEND distills behavioral guidance into neural-only models. I think you can make a representative controlled comparison, or an ablation isolating BeRG’s cross-session construction to prove your particular design useful.

  2. The learned encoder’s added value is readout-dependent and remains untested in the key calibration experiment. Explicit nearly matches LIGHT in current decoding, with R²=0.8880 versus 0.8887. LIGHT’s clearest advantage is 250-ms future-velocity readout, at 0.5891 versus Explicit’s 0.4898, but its teacher is trained with future-behavior targets. The history-conditioned analysis controls velocity history while leaving other teacher inputs and task context uncontrolled.Another ablation experiment may solve this problem. Table S8 provides a useful counterpoint: for final-hidden future-count prediction, Explicit scores 0.0884 versus LIGHT’s 0.0808 bits/spike; for masked-unit reconstruction, the scores are 0.1451 versus 0.1369. Thus, the learned interface’s advantage in future behavior does not carry over consistently to the additional readouts. Appendix D.1 also reports a LIGHT-FUTURE minus LIGHT difference of −0.00951 at 250 ms, with a paired-session 95% interval of [−0.02236, 0.00366].But the difference may a little tiny to support the advantage.Its poorer current decoding indicates an objective tradeoff under the tested recipe, whose sensitivity to auxiliary weighting remains unresolved. These results make Explicit particularly important for interpreting LINK, where only Base and LIGHT are compared. Consistent LIGHT–Explicit gains on held-out dates would connect the learned encoder to the strongest practical result. Comparable calibration performance would shift the contribution toward fixed behavioral structural supervision. The missing comparison leaves that distinction unresolved.

  3. The central learned-versus-fixed representation finding lacks independent validation and training replication. Table S2B prove the Base–LIGHT current-decoding improvement: the Window Transformer test gain is +0.0330, close to the source-validation gain of +0.0326. The validation gap concerns the principal encoder and future-readout comparisons. Table S6 reports the 0.4898-versus-0.5891 Explicit–LIGHT difference using only seed 11 on source validation, including sessions used for teacher selection. Excluding a query session from ridge fitting still leaves it included in backbone pretraining. Appendix D.2 additionally states that each learned teacher configuration uses one selected checkpoint from one teacher-training seed. The reported comparisons consequently leave both teacher-fitting and neural-training variability insufficiently characterized. Existing archived repetitions and an independent test evaluation of the core Base/Explicit/LIGHT future readouts would establish how stable this finding is beyond the selected teacher and trained models.

  4. The adaptation advantage depends on calibration, and animal-level robustness remains insufficiently characterized. The Perich source animals contribute 68, 3, and 28 sessions. Under session-macro averaging, two animals therefore carry approximately 97% of the weight. Per-animal results and an animal-balanced aggregate are needed to determine how broadly the improvements are distributed. Cross-animal adaptation is then evaluated on one target animal with one pretraining seed. Table S3 shows that the LINK advantage changes sign with calibration: without calibration, LIGHT scores 0.0979 versus Base’s 0.1252, a paired difference of −0.0273±0.0058; after 50 labeled trials, the difference becomes +0.0375±0.0045. These three-seed results establish a benefit conditional on recent labels within one animal. Table S4 further reports MC-Maze differences of −0.0089 and −0.0071 at 5% and 100% training-label budgets, respectively, under a joint task, recording, and interface shift.Again, the difference may caused by randomness.

The strongest practical evidence therefore concerns supervised calibration in familiar-task settings. Evaluating another target animal where task coverage permits would strengthen the broader reuse claim. For Monkey T, the reported training-support fractions also exclude target-validation labels used for checkpoint selection; the total target-side labeling requirement should be stated.

Questions for authors
  1. Does LIGHT retain its advantage under comparable candidate exposure and training budgets, and against a representative related behavioral-supervision method?
  2. How does Explicit perform under the existing LINK calibration protocol with paired pretraining seeds?
  3. What independent test results and additional training repetitions support the main encoder and future-readout comparisons?
  4. How do animal-balanced gains compare with session-macro gains, and can another held-out target animal be evaluated using existing data?
7 comments
AuthorsOpenReview ↗

Response to Reviewer c9Yj (1/6): Stronger baselines and training fairness

Thank you for the detailed review. We added controls, repetitions, and analyses and revised the manuscript. Our response has six comments.

CW1 / Q1, summary. Does LIGHT benefit over stronger, comparably trained controls? Why were Table 2 entries missing, and was POYO-Light retraining symmetric?

Stronger Perich controls.

Matching optimizer updates alone does not match candidate exposure or training expenditure. We added Perich controls with candidate-window decoding supervision, randomized guidance, and longer Base training. These narrow the original gap, but LIGHT retains a mean gain of approximately 0.009–0.013 R2R^2 over these controls, positive in all three neural-training seeds for each comparison.

Table R1. Stronger Perich controls

ConditionUpdatesCurrent R2R^2, mean ±\pm SDPaired LIGHT minus control
Base (20,000 updates)20,0000.85713±0.001410.85713 \pm 0.00141+0.03060±0.00237+0.03060 \pm 0.00237
Extra candidate decoding20,0000.87442±0.000870.87442 \pm 0.00087+0.01331±0.00185+0.01331 \pm 0.00185
Original shuffled guidance20,0000.87844±0.000910.87844 \pm 0.00091+0.00928±0.00188+0.00928 \pm 0.00188
Random-candidate shuffled guidance20,0000.87690±0.001810.87690 \pm 0.00181+0.01083±0.00089+0.01083 \pm 0.00089
Longer-trained Base (172,600 updates)172,6000.87652±0.000140.87652 \pm 0.00014+0.01121±0.00096+0.01121 \pm 0.00096
LIGHT (20,000 updates)20,0000.88773±0.000980.88773 \pm 0.00098—

Window Transformer source validation. Current R2R^2 is native-decoder pooled R2R^2; entries and paired differences are mean ±\pm sample SD over three neural-training seeds (11, 22, 33). Differences are computed within seed before averaging.

Extra decoding uses current-velocity labels from the additional candidate windows. LIGHT's gain over it supports value beyond this additional supervision. Because correspondence shuffling retains behavior-dependent selection, we also combined random ranking with shuffled correspondence, retaining the original anchor sampling and task/session constraints. LIGHT's gain supports its relational construction and guidance jointly. Base trained slightly longer than LIGHT's student-time reference plateaus in the latter budget range and remains below LIGHT. Thus, under the tested recipe and budgets, extra ordinary decoding supervision or training time does not recover the same benefit.

Completing the missing Table 2 entries.

Table 2 rows 2–3 lacked frozen-readout evaluations, which the caption did not explain sufficiently. Future and history-conditioned scores require separate readout fits and cannot be inferred from current decoding. We completed them with the original models and protocols (Table R2). Both controls remain below LIGHT in linear accessibility of future information despite improved current decoding. The four missing values and details are now in Table 2 and Appendix D.1.

Table R2. Completed frozen readouts for the original Table 2 checkpoints

ConditionCurrent pooled R2R^2250-ms future session-macro R2R^2History-conditioned ΔR2\Delta R^2
Base0.85610.3783+0.1168
Extra candidate decoding0.87350.4710+0.1834
Original shuffled LIGHT0.87750.4184+0.1443
LIGHT0.88870.5891+0.2939

Original seed-11 source-validation models. Current decoding is pooled; future readout is session-macro at 250 ms. Conditional ΔR2\Delta R^2 is the increment over the original history-only reader. The Extra/Shuffle future and conditional cells are newly completed; Base/LIGHT are references. This original shuffle differs from the random-candidate control in Table R1.

Symmetric POYO-Light retraining.

We retrained POYO-Light pairs from the same initialization, matching main-window sampling, augmentation, optimizer, learning-rate schedule, and full-state recovery rules. The fixed-endpoint results retain a decoding benefit under this symmetric procedure (Table R3). The paired results are listed separately from historical runs in manuscript Table 1, with protocols and animal-level results in Appendix D.3.

Table R3. POYO-Light with a symmetric training procedure

ConditionSource-validation pooled R2R^2Difference from Base
Base0.8928—
BeRG-LIGHT0.9027+0.0099

Fixed epoch-60 endpoints, one paired neural-training seed (42), source-validation pooled R2R^2. The matched quantity is the training procedure; BeRG still uses candidate computation.

Perich controls and budget curves are reported in Section 4.1 and Appendices D.1 and E.2.

AuthorsOpenReview ↗

Response to Reviewer c9Yj (2/6): LINK controls and related methods

CW1 / Q1, summary. Does the benefit survive matched exposure? What do cross-date candidates add, and how does BeRG differ from prior methods?

At 40K updates, LIGHT exceeds extra current-velocity supervision by 0.0150 in source-validation R2R^2 and by 0.0296 on held-out dates after calibration with 50 recent labeled trials. Relational guidance thus adds value on the same candidate windows, including after target-date calibration. Extra decoding is stronger at early budgets; an equal-student-time extended Base control on LINK was not run.

Table R4. LINK source-validation performance across training budgets

Condition10K R2R^220K R2R^230K R2R^240K R2R^2Student minutes at 40K
Base0.35640.42800.47030.494932.49
LIGHT0.42490.48810.53760.554392.30
Extra decoding0.47990.51450.53620.539355.99
WITHIN0.41450.48810.53310.5502173.49

One neural-training seed (11); date-macro R2R^2 over 94 source dates. All arms share saved initialization and main-window stream. At 40K, each has 1.92 million main-window exposures; LIGHT/Extra share candidate banks and 10.24 million candidate-window forwards. Student time excludes teacher fitting and precomputation. Matched candidate exposure does not match computation or time. Table R5 gives held-out-date calibration.

WITHIN draws candidates from different trials within the same date. It retains most of the benefit, differing from LIGHT by only 0.0004 R2R^2 after calibration. This supports relational supervision but does not establish a unique advantage of cross-date construction.

Table R5. Held-out-date calibration of the new LINK exposure controls

ConditionZero-label R2R^2Recent50 R2R^2Recent50 Δ\Delta vs BasePositive dates vs Base
Base0.14200.4012——
LIGHT0.09540.4385+0.037334/45
Extra decoding0.09500.4089+0.007725/45
WITHIN0.09040.4381+0.036936/45

New four-arm cohort, one pretraining seed (11), fixed 40K checkpoints, 45 held-out dates. Recent50 uses the same 50 recent labeled trials and 300 read-in/readout updates, freezing the backbone and resetting the pretrained state for each date. Zero-label uses no adaptation. Dates are not training replicates. After calibration, LIGHT−Extra is +0.0296 and LIGHT−WITHIN is +0.0004.

PKT. BeRG builds on probabilistic neighborhood distillation, adapting it to heterogeneous neural recordings. Candidates come from other sessions of the same task, with session balancing, per-session quotas, and sampling across near-to-distant behavioral similarity ranks. Neural states are centered within sessions only in the relational branch; decoding uses uncentered states. Relation matching acts on Common-Port on the actual decoding path and jointly trains the backbone with current-behavior prediction.

BLEND. Its teacher receives neural activity and behavior, targets neural reconstruction, and distills outputs, features, or neural correlation structure into a neural-only student. BeRG's behavioral encoder receives no neural data and supplies soft behavioral neighborhoods across sessions. Its student jointly optimizes relation matching and current-behavior decoding. The teacher inputs, transferred supervision, and student objectives thus differ.

CEBRA. It uses behavioral or temporal auxiliary variables to organize contrastive sampling and learn decodable neural representations, including multi-session training. BeRG's separate behavioral encoder supplies soft relational targets that shape the shared multi-recording decoder backbone through the decoding subspace, jointly with current-behavior loss. At inference, the relational branch is removed and the projection and linear readout can be folded together, preserving the neural-to-behavior interface. Our matched-backbone experiments measure the incremental benefit of relational guidance during pretraining of the same decoder.

LINK controls and budget curves appear in Appendix E.3, calibration in Appendix B.1, and related-method positioning in Section 2.

AuthorsOpenReview ↗

Response to Reviewer c9Yj (3/6): Learned behavioral encoders and future-target supervision

CW2, concern summary. What does a learned encoder add beyond Explicit? Could its future-readout advantage reflect future-target supervision or teacher inputs omitted from the original conditional analysis?

Motivation and behavioral evidence.

Behavioral data are more accessible and scalable than neural recordings. We aim to learn movement states and temporal structure from behavior, then use their relationships to supervise neural decoding. Explicit constructs behavioral features directly, LIGHT learns from behavioral data, and MOMENT adapts features from a general pretrained time-series model. This study uses recorded behavior; robotics simulation is a possible future data source.

In the original forecasting evaluation, LIGHT and the MOMENT interface achieved mean nMSEs of 0.0637 and 0.0873 across three horizons, respectively (Appendix D.2). Probes of frozen LIGHT also recover current velocity and recent velocity changes. These findings support predictive training for representing movement state and temporal evolution; downstream neural readouts assess its value for relational guidance.

Future targets and teacher information access.

We train behavioral encoders on future targets to turn predictive temporal structure into relational supervision. Motor-related neural activity can precede observable behavior and carry upcoming-movement information, motivating predictive behavioral representations (Paninski et al., 2004). In BeRG-LIGHT and BeRG-MOMENT, relational targets use state representations computed from behavioral history and task context, without directly supplying each sample's realized future trajectory to the state encoder. Future trajectories supervise behavioral model fitting and can shape predictive states. Some Perich task context is defined retrospectively, with its training-time information access specified in the appendix. At neural inference, the teacher is removed and only neural inputs are required.

Readout-dependent value and complete conditioning.

Explicit and LIGHT achieve similar current decoding; Explicit performs better on final-hidden future-count prediction and masked-unit reconstruction, whereas LIGHT provides stronger future-velocity readout. We added a conditional linear readout with the full teacher-visible kinematic history and task context as its baseline. Adding neural features yields approximately 0.083 more incremental R2R^2 for LIGHT than Explicit, consistently across three neural-training seeds. Thus, under this conditional linear evaluation, LIGHT provides more linearly accessible future-predictive information even when these behavioral inputs are supplied directly to the reader.

DirectFuture directly supervises the student to predict future velocity. Its 250-ms readout is approximately 0.605 versus LIGHT's 0.586, while current decoding is 0.859 versus 0.888. Direct future supervision can produce stronger future readout, while LIGHT combines higher current decoding with strong future readout. We therefore assess both endpoints when interpreting each supervision approach.

Table R6. Repeated current, future, and full-input-conditioned readouts

ConditionCurrent R2R^2250-ms future R2R^2Full-input conditional ΔR2\Delta R^2
Base0.8571±0.00140.8571 \pm 0.00140.3652±0.01660.3652 \pm 0.0166+0.0455±0.0083+0.0455 \pm 0.0083
Explicit0.8884±0.00060.8884 \pm 0.00060.4885±0.00810.4885 \pm 0.0081+0.0978±0.0069+0.0978 \pm 0.0069
LIGHT0.8877±0.00100.8877 \pm 0.00100.5860±0.00290.5860 \pm 0.0029+0.1804±0.0024+0.1804 \pm 0.0024
DirectFuture0.8589±0.00020.8589 \pm 0.00020.6049±0.00240.6049 \pm 0.0024+0.1763±0.0030+0.1763 \pm 0.0030

Source validation, mean ±\pm sample SD over three neural-training seeds. Current R2R^2 is pooled; future and conditional readouts are session-macro. The conditional increment is

ΔR2=R2(B2+H)−R2(B2).\Delta R^2 = R^2(B_2 + H) - R^2(B_2).

Here, B2B_2 includes the full teacher-visible kinematic history and task context, and HH is the neural representation. Ordinary and conditional readers use 25,344 and 24,447 common eligible queries, respectively. DirectFuture uses the weight selected on seed-11 source validation and then fixed across seeds. The paired LIGHT−Explicit conditional increment is +0.0826±0.0058+0.0826 \pm 0.0058 (3/3 positive).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the chairs and reviewers for their time and thoughtful suggestions.

We appreciate the reviewers’ recognition of the following strengths:

  1. The practical importance of cross-session neural decoding and calibration (c9Yj, b4ES).
  2. A simple training-time method that improves multiple backbones while preserving neural-only inference (b4ES, JTPW).
  3. Future-readout and representation analyses that provide information beyond current-decoding accuracy (JTPW, LpMq).

Core contributions and significance

  1. Behavior as a shared reference across recordings. BeRG turns comparatively accessible behavioral data into structured supervision for neural pretraining, without requiring neuron or timestamp correspondence. This provides a way to learn jointly from costly, heterogeneous neural recordings.

  2. Decoder reuse as recordings change. Training-only guidance improves multiple tested backbones and supports supervised calibration to new dates and adaptation to a held-out animal within familiar tasks. This addresses recording variability, a practical obstacle to sustained BCI use, while preserving neural-only inference.

  3. Revealing what decoding accuracy misses. Fixed and learned guidance can produce similar current decoding yet different future-behavior accessibility. These comparisons show how behavioral supervision changes the information available in neural representations, beyond improvements in current-task accuracy.

Main concerns addressed

The main questions concerned training fairness, repeatability, and the interpretation and scope of the gains. We addressed them as follows:

  1. Training fairness. Stronger supervision and budget controls, together with matched POYO-Light retraining, retain gains under the tested protocols. This supports the incremental value of relational guidance (Section 4.1; Appendices D.1, D.3, and E.2–E.3).

  2. Repeatability. Additional neural and teacher training runs and held-out test intervals within source sessions reproduce the core future-readout finding (Section 4.2; Appendices C.1 and D.2).

  3. Interpretation. Relation-only, direct-future, conditional-readout, and geometry analyses clarify the complementary role of relational guidance and the readout-dependent benefits of learned encoders (Appendices C.1–C.2 and D.1).

  4. Transfer scope. Animal-balanced source analyses retain decoding gains, and repeated adaptation evaluations support supervised reuse within familiar tasks. We clarify label requirements and retain zero-shot decreases and mixed cross-task results (Section 4.4; Appendices A.2 and B.1–B.2).

  5. Presentation. We clarified methods, backbone definitions, sampling, splits, and metrics; updated tables and captions; and corrected references, figure layout, and PDF navigation (Sections 3–4; Appendices A and G).

We will include selected key code and the materials needed for reproduction in the conference supplementary files.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.