acceptodds
Under review as a conference paper at ICLR 2027

Representing Short-Term Movement to Reveal Animal Identity and Learning Progress

Abstract

How learning progresses over time and varies across individuals is a longstanding question in cognitive and behavioral science. Behavioral changes are observable outcomes of learning and have been a long-established bedrock for understanding learning progress. However, a major challenge is identifying observable behavioral measures that faithfully capture learning progression. Existing approaches commonly rely on predefined behavioral features or task-level performance, while fine-scale posture and movement in freely behaving animals remain less explored. In this paper, we ask: *can short-term movement trajectories reveal learning progress and individual identity?* We introduce a self-supervised representation learning framework built on a simple hypothesis: behavioral characteristics associated with identity and learning remain relatively stable over nearby time intervals. Using temporal proximity as supervision, our framework learns compact representations directly from movement trajectories without manually defining behavioral states or kinematic features. Using behavior data from freely behaving rats, we show that the learned representations not only improve prediction of identity and learning-related readouts over existing methods, but also provide a tool for investigating broader scientific questions about when and how these signals appear in behavior. Specifically, trajectories as short as one second contain informative learning and identity signals. Egocentric pose dynamics are more predictive of identity, whereas allocentric motion is more predictive of learning-related variation, and both signals are strongest at decision junctions. We further use a representation-conditioned diffusion model to inspect movement patterns associated with identity and learning progress. Our findings establish short-term movement as an informative behavioral “fingerprint” for studying learning and individual differences.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Keywords: animal behavior, self-supervised representation learning, conditional diffusion, trajectory reconstruction, latent interpolation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nQbwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRAIL, a self-supervised framework for learning compact representations of short animal movement trajectories. The input is the 2D trajectory of three keypoints (nose, neck, body) obtained by DeepLabCut from top-view video at 11 Hz, together with 25 maze waypoints as context. The encoder follows Wayformer and Q-Former: coordinates receive multi-frequency encodings, eight learned queries summarise all tokens, and the first query is projected to a 16-dimensional representation. Training combines a temporal-proximity contrastive loss, in which segments of the same recording partition within 10% of its duration are positives and all others are negatives, with a Gaussian negative log-likelihood reconstruction loss. The data come from the Y-maze spatial learning experiments of Shridhar et al.: six rats learning an alternation rule and eight rats learning a probabilistic rule, five days each, 275 sessions, about 45 hours. Evaluation freezes the encoder and fits linear probes for identity, recording day, trial-level proficiency, and success rate and reward count within ±30 seconds. TRAIL is reported to outperform classical state-space models, deep behavioural representation methods, time-series foundation models and a multi-task supervised variant, and to remain competitive on the MABe22 mouse triplet benchmark.

Critical strengths
  1. The learning-progress half of the problem is well posed. Longitudinal recordings of the same individual across days of learning, under a quantifiable reward rule with trial-level outcomes, are hard to obtain for humans, and rodent spatial-learning experiments are an appropriate setting for asking whether movement changes with learning.
  2. The breadth of the evaluation and the reporting of uncertainty. Baselines span classical state-space models, deep behavioural representation methods, time-series foundation models and a supervised variant; the appendix reports animal-bootstrap confidence intervals and paired comparisons, and adds held-out-session, held-out-day and new-animal experiments.
  3. Translating representation-space directions back into movement is an interesting attempt. Using a diffusion model conditioned on the frozen representation with DDIM inversion to inspect what the representation encodes is a reasonable direction for hypothesis generation.
Critical weaknesses

W1. The input consists of only three keypoints, which does not constitute pose or motion in any meaningful sense; what the model learns from it is most likely trajectory and projection-induced variation rather than movement, which bears directly on whether the paper's claim can be substantiated. (a) Sec. 4 states that only the nose, neck and body keypoints are used because they are "the most reliably tracked using DeepLabCut". For a mammal, movement is defined by limb articulation, and the three points contain no limb information at all. Nevertheless, the abstract and introduction repeatedly refer to "fine-scale posture", "limb posture" and "egocentric pose dynamics", none of which is present in the data. (b) More fundamentally, nose, neck and body correspond in three dimensions to a single joint, the neck, whose range of motion is itself limited. The true body configuration should therefore be very similar across the great majority of frames, and the variation in what the paper calls "local pose" arises mainly from projection under a single fixed camera: the animal's position in the field of view and the pitch and roll of its head relative to the camera change the projected lengths and angle of the two segments, and none of this is articulation. In other words, most of the learnable variation in the Local Pose view is projection geometry and tracking noise rather than movement. Given that the Global Trajectory view, which contains only position and heading, already reaches 72.1% identity accuracy (Table 3), the more plausible reading is that the model learns identity and day from where the animal is, where it is going and how its head is tilted relative to the camera, not from how it moves. The paper's central claim is that short-term movement reveals identity and learning progress; on the present data this cannot be distinguished from the claim that trajectory and projection-induced variation reveal them. (c) The model itself is not restricted to three keypoints, so this is a choice of data rather than a limitation of the method. Data of this kind can serve as an auxiliary evaluation but cannot be the sole basis for the central claim. The authors should redo the main experiments on recordings with limb tracking and quantify how much of the local-pose variation is explained by position in the field of view and head pitch; otherwise the claims and terminology should be restricted to short-term trajectories of the head and trunk.

W2. The identity question neither requires nor is well served by a study on six rats. Learning progress calls for longitudinal animal data (see S1), but whether short movement reveals individual identity does not: human motion data offer several established identity benchmarks with many individuals and full-body pose, for example the BOXRR-23 corpus of VR head and hand motion from over 100,000 users, the public gait-recognition datasets, and the recent BALLER120 benchmark built around individual differences in a single repeated action. The identity conclusion here rests on a single cohort of six rats, a six-way classification, three keypoints and one maze. A "movement fingerprint" established under these conditions does not generalise, yet the title and abstract state it in general terms. The authors should validate the identity half on at least one human motion dataset, or explicitly scope the claim to rats in a Y-maze.

W3. The main evaluation task is defined by the authors, and the method is a combination of existing components without a new mechanism. On the evaluation side, the five readouts, the protocol and the split are the authors' own design on a single laboratory's dataset; the only external benchmark is MABe22, where TRAIL is merely competitive and weaker than hBehaveMAE on frame-level tasks. On the method side, the components of TRAIL all come from existing work: Wayformer-style alignment and multi-frequency encoding, the Q-Former of BLIP-2, an InfoNCE contrastive loss, Gaussian reconstruction, and DiT with DDIM inversion. Overall the method reads as an engineering combination of these parts, without a particular insight or novelty, and without a design targeted at the identity or learning-progress question. The idea the paper presents as its core hypothesis, that temporally nearby segments should be positives, also has close counterparts in self-supervised time-series learning.

Questions for authors
  1. Is there an animal pose dataset with more than three keypoints, including the limbs, on which the main experiments of this paper could be repeated? If no such data currently exist, could the authors add manual limb annotations to the existing recordings? Three keypoints are not sufficient to support the paper's conclusions about movement.
  2. How much of the variation in the Local Pose view is explained by the animal's position in the field of view and the pitch of its head relative to the camera, for example by regressing the projected segment lengths and angle on position? How much identity accuracy does Local Pose retain once this component is removed?
  3. One of the paper's conclusions is that short-term movement trajectories reveal individual identity. Can this conclusion be verified on a human motion dataset? Specifically, could the authors report identity recognition accuracy on a human dataset with subject identity labels, full-body keypoints and far more than six individuals? To my knowledge such datasets are readily available, for example motion-capture datasets with subject identifiers, gait-recognition datasets and large-scale VR motion corpora.
3 comments
AuthorsOpenReview ↗

Response to Reviewer nQbw (1/2)

We thank Reviewer nQbw for their feedback. Our study uses short movement trajectories to investigate individual differences and changes during spatial learning. We address your concerns below.

W1. Movement information captured by three keypoints

We agree that three keypoints provide limited coverage of limb articulation, and we respectfully disagree that this makes them insufficient for studying movement. The nose, neck, and body keypoints capture how each rat moves through the maze and how its head and trunk move relative to each other. Both are relevant to our question of how short movements relate to identity and learning across days.

(a) Relevance to spatial navigation. The tracked keypoints capture navigation and brief head and neck movements during behaviors such as scanning and stalling. We would also want to clarify some misunderstanding. While “limb posture” is mentioned in the introduction in the context of discussing prior motor-learning studies, “fine-scale posture” and “egocentric pose dynamics” refer to subtle changes in the tracked head–neck–body dynamics, which we analyze in Section 5 and Table 3.

(b) Camera setup and movement decomposition. The spatial-learning dataset uses a fixed overhead camera that covers the entire maze across rats and recording days, with calibration to minimize distortion [1]. From each Full Movement trajectory, we derive Local Pose by normalizing position and heading to capture changes in relative body configuration. Global Trajectory retains changes in position and heading and omits relative body configuration. The 72.1% identity accuracy for Global Trajectory shows that navigation patterns carry identity information. Local Pose reaches 77.1%, and Full Movement reaches 84.0% (Table 3), indicating that navigation and relative head–trunk movement carry complementary information about identity.

(c) Dataset suitability and additional annotations. The three reliably tracked keypoints are sufficient to detect identity and learning-related information in this task. Limb tracking would add detail about gait and articulation, while the available keypoints already capture the navigation measures central to our question, including where rats go, how they turn, and how they scan. We will clarify that our conclusions concern the movements captured by these three keypoints.

W2. Cohort size and scope of the identity findings

We appreciate the reviewer’s concern about the scope of the six-rat identity experiment. Our question is whether short movement sequences distinguish individual rats as their behavior varies across days of controlled spatial learning. Repeated recordings let us examine this variation within the same animals, and the three keypoints capture the relevant navigation and head–trunk movements (W1). A large human motion benchmark would test identity recognition in a different population. The longitudinal rodent recordings directly address our question about individuality during spatial learning.

Our evidence extends beyond the original six-rat cohort. Appendix B.1 reports similar performance in eight additional rats learning a different reward rule in the Y-maze. As detailed in our general response, we also evaluated six additional rats recorded over five days in a four-arm Plus-maze. These results broaden the evidence across cohorts, reward rules, and maze geometries. We will clarify that our identity findings apply to the rodent settings evaluated in this study.

AuthorsOpenReview ↗

Response to Reviewer nQbw (2/2)

W3. Scientific contribution, external evaluation, and methodological design

Scientific objective and evaluation. Through the main spatial-learning dataset, our central question is whether short movement trajectories carry information about individual identity and learning progress, and how that information appears in behavior. The five downstream tasks provide measurable proxies for these concepts. We use them to evaluate the representation in Section 4, after TRAIL learns from trajectories without predefined behavioral features or downstream labels. Section 5 then uses the representation to examine when and where these signals are strongest and how different movement views contribute.

Methodological design and novelty. We agree that TRAIL builds on established components. Our contribution lies in adapting and evaluating these components to answer specific questions about individual identity and learning-related behavior, rather than just combining established components for beating baselines. We adapt trajectory modeling from autonomous driving to learn behavioral representations and use multi-frequency encoding to capture subtle changes in keypoint coordinates over time. The temporal contrastive objective encourages nearby trajectories to have similar representations because they may share behavioral characteristics, while reconstruction encourages the representation to retain movement detail in the representation. Together, these choices let TRAIL capture information about identity and learning without task-specific supervision. Appendix C (p. 23) examines the design choices through ablations, and Section 5 shows how different movement views carry complementary information and how these signals vary across temporal and spatial contexts.

Performance on MABe22 dataset. Alongside the spatial-learning dataset, we evaluate TRAIL on the established MABe22 Mouse Triplets benchmark, which records three mice interacting in an open arena. This complementary setting lets us compare TRAIL with existing methods beyond the spatial-learning experiments. TRAIL performs well on within-clip identity and sequence-level tasks, while hBehaveMAE performs better on frame-level tasks. Frame-level tasks require precise localization of brief interactions in time, whereas TRAIL summarizes short movement windows to capture individual characteristics and movement variation associated with longer-term behavioral change. These results clarify the kinds of behavioral information TRAIL captures and the tasks for which its representation is best suited.

Question 1. Richer pose data and limb annotation.

Please refer to our response to W1(a) and (c).

Question 2. Position and projection effects on Local Pose.

Please refer to our response to W1(b).

Question 3. Identity validation on human motion data.

Please refer to our response to W2.

Ethics concern — Animal-use approval

Thank you for highlighting this omission. Our study reuses the published data from Shridhar et al. [1] and involves no new animal experiments. We will add an ethics statement citing the original study’s animal-use approval.

[1] Shridhar et al., “Goal Location Beta Oscillations and Sharp Wave–Ripples Are Complementary Markers of Learning-Related Changes in Hippocampal–Neocortical Networks,” eNeuro, 2026

Thank you again for your valuable feedback, which has helped us improve the manuscript. We welcome any remaining questions and would appreciate your reconsideration of the score if our responses have addressed your concerns.

Reviewer_nQbwOpenReview ↗

Remaining concern: the choice of dataset and the reliability of the tracked keypoints

Thank you for the response and for the additional Plus-maze results.

My main concern remains the choice of dataset in the experiments, for two reasons.

First, I am not convinced that a dataset with three keypoints (nose, neck and body) and a primary cohort of six rats is sufficient to support the claim that short-term movement reveals individual identity and learning progress.

Second, the response states that "the three reliably tracked keypoints are sufficient to detect identity and learning-related information in this task". However, the keypoints are obtained with DeepLabCut, which is a markerless pose estimation tool. Since DeepLabCut trains a network on a small set of manually labelled frames, almost all keypoint coordinates used in the paper are network predictions and not manual labels. On what basis are the three keypoints considered "reliably tracked"? The paper gives no criterion for this, such as a likelihood threshold, no tracking error with respect to manual labels, and no fraction of removed frames. Predicted coordinates carry prediction errors. How do these errors affect the results? In particular, the nose-neck and neck-body segments are short, so the tracking error may account for a considerable part of the variation in the Local Pose view. In addition, a tracking error that differs systematically between animals could itself contribute to the Identity accuracy.

Reviewer_4t31OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper raises an question about how learning progress can be measured from behavioral data, highlighting the limitations of existing analyses that rely on manually defined features and task-level performance. The authors proposed a self-supervised pretraining method for movement-trajectory data collected from rats and mice in a Y-shaped maze. The pretrained model is then fine-tuned on multiple downstream tasks, including individual recognition, elapsed-day prediction, and earned-reward prediction. The proposed method outperforms a range of baseline methods.

Critical strengths

The authors proposed a pretraining method whose downstream performance on individual recognition, elapsed-day prediction, proficiency prediction, and earned-reward prediction surpasses that of existing methods.

Critical weaknesses

Mismatch between research motivation and what is actually done

The authors state that "a major challenge is identifying observable behavioral measures that faithfully capture learning progression" (a point with which I agree), and go on to note that "Existing approaches commonly rely on predefined behavioral features or task-level performance." Given this framing, I expected the paper to propose a new metric of learning maturity that could replace or complement existing methods. In reality, however, what the paper explores is a high-performing mapping from behavioral recordings to existing metrics such as success rate and rewards earned (i.e., task-level performance). I therefore do not see how the proposed method enables experimentalists to recognize learning maturity beyond the existing criteria.

The visualization of learning-related behavioral changes (Section 6) is also unclear. Figure 6 appears to show some "day-related differences in movement execution" (though I am unsure what specific values are plotted, especially in the bottom-left panel), but I do not see how these could serve as a metric for learning maturity.

Similarly, I find it misleading to conclude that "fine-scale movement patterns in freely behaving animals differ across individuals and change systematically over days of learning." The proposed pretraining method encourages distinct representations for data "from temporally distant segments or different sessions," so the individual and temporal differences in movement patterns are likely amplified relative to those in the raw recordings. A more appropriate statement would be something like "fine-scale movement data exhibit detectable individual differences." I also do not understand what "systematic changes over days of learning" the study claims to observe.

I believe the study could be improved by including a baseline dataset consisting solely of behaviors from "matured" animals, whose task performance has already reached a ceiling. Such a dataset would disentangle temporal change from the learning progress embedded in animal behavior. If the model can still distinguish the recording dates of behaviors exhibited by matured animals, the features it detects are more likely to be time-dependent rather than learning-related.

Research Scope Overgeneralized

In the abstract, introduction, and conclusion, the proposed method is advertised as a general solution to animal behavioral studies, while only rodent data were actually used. The model architecture is likewise specialized for the rodent-specific, maze-based experimental design. It therefore remains unclear whether the proposed method would be as effective for other "freely moving" animals (e.g., monkeys in their cages).

I believe the scope of the study should be narrowed to rodents, unless additional experiments are performed on animals with different body shapes and environments.

Missing Baselines for Visualizing Characteristic Behaviors

While the classification and regression performance of the proposed method is evaluated against existing models, its visualization of learning-related behaviors is not compared with general-purpose methods. For example, one could build an individual/reward predictor and visualize where it attends, using SHAP, Grad-CAM (for CNNs), or attention weights (for Transformers).

Questions for authors
  • What would your model produce when given a dataset collected from matured individuals, whose task performance has already reached a ceiling? If the model still captures day-related features in this setting, would you still argue that these features encode learning progress?
  • I do not understand what the bottom-left panel of Figure 6 represents. What do the two axes characterizing body velocity represent? Is this a PCA of the latent representations?
  • The authors should explain what "NeRF-style multi-frequency encoding" is for readers unfamiliar with 3D scene recognition and generation.
  • Given the original research question concerning the measurement of learning progress, the evaluation of individual recognition appears supplementary to me. Is individual identification truly inseparable from learning progress in an essential way, or is the improved individual identifiability more of a byproduct of the pretraining design (which contrasts different sessions, necessarily performed by different individuals)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4t31 (1/2)

We would like to thank reviewer 4t31 for their valuable review and would like to address their concerns, point by point, as follows:

Weakness 1: Research motivation and claim mismatch

We acknowledge that our framing could lead readers to expect a new measure of learning maturity, and apologize that our writing can lead to such misleading expectations. We would humbly want to provide a better narrative as follows. Our scientific goal is to learn a representation that helps us understand how movement differs across individuals and how it changes during learning. To this end, we develop TRAIL to learn directly from short trajectories through temporal contrastive learning and reconstruction, without any predefined behavioral features or downstream labels.

To assert weather the learned representation already capture meaningful behavior concept, we consider five downstream tasks—identity, recording day, proficiency, success rate, and reward—as proxies toward a broader concepts of individuality and learning progress. The results in Tables 1 and 2 show that TRAIL captures useful behavioral information. Building on these results, Section 5 examines when and where these signals are strongest and how different movement views contribute, and Section 6 translates shifts in the learned representation into observable changes in turning speed and head–trunk dynamics, helping us inspect which aspects of movement TRAIL captures.

We agree that temporal contrast can amplify individual and temporal differences. Our contrastive objective is based on the hypothesis that trajectories close in time tend to share behavioral characteristics that may vary over longer intervals. The representation’s ability to predict proficiency, success rate, and reward suggests that it captures movement variation associated with learning-related outcomes. We use recording day as an indirect proxy for learning progress, and it can also reflect other changes that occur over time. We will clarify in the introduction how the downstream tasks evaluate the representation and how the subsequent analyses use it to investigate behavior. We will also revise the conclusion to describe movement variation associated with learning-related outcomes more precisely.

Weakness 2: Overclaiming scope

We agree that our findings should reflect the scope of the rodent settings we evaluated. We deliberately chose the spatial-learning dataset because it follows the same rats as they learn over several days under controlled conditions, a design suited to studying how movement changes during learning. Comparable longitudinal recordings are difficult to collect, especially across species. As detailed in our general response, we repeated the analyses with six additional rats recorded over five days in a Plus-maze. The results support the maze-center finding in a second cohort and maze geometry.

TRAIL can also operate without maze information. The Plus-maze experiments demonstrate its use with a different layout, and MABe22 evaluates its representations in an open arena without maze inputs. We will clarify the scope of these findings in the abstract, introduction, and conclusion. Generalization to other species remains an open question.

Weakness 3: Other visualization baseline

Thank you for suggesting these comparisons. As discussed in our response to Weakness 1, our goal is to use TRAIL’s learned representation to investigate individual differences and behavioral changes across days. In Section 6, we use a diffusion model to translate shifts in the representation into observable kinematic changes, such as differences in turning speed across days and how head–trunk dynamics change across animals. These examples help us examine what the representation captures beyond the five downstream readouts.

While SHAP and Grad-CAM would help identify which inputs contribute to a particular prediction, our diffusion interpretation actually serves a different purpose: it makes the movement patterns associated with shifts in the representation visible. We will clarify in Section 6 that these visualizations serve as a tool for behavioral investigation, rather than a quantitative benchmark to compare visualization methods.

AuthorsOpenReview ↗

Response to Reviewer 4t31 (2/2)

Question 1: Dataset collected from expert individuals

Thank you for the great suggestion. Different individual learn at different rats and reach different levels of task performance. As an initial look at later-stage behavior, we examined individual performance across two final days 4–5 in the spatial-learning data, however it may represent different maturity stages across animals. In both the Y-maze and Plus-maze cohorts, identity remained decodable during these final days, while proficiency prediction error varied across rats. Because learning unfolds over time, examining identity and proficiency together can help us investigate how each rat’s movement evolves and when it stabilizes. If recording day remained decodable after an independently verified performance plateau, we would interpret it as evidence of temporal variation beyond measured task improvement. We believe TRAIL can help us investigate that variation by directing attention to specific movement patterns in each rat’s behavioral record.

Y-maze — Per-rat results on Days 4–5

RatIdentity Acc ↑Proficiency MSE ↓
A85.38%0.004
B98.66%0.007
C82.17%0.012
D77.98%0.031
E97.37%0.007
F89.04%0.013

Plus-maze — Per-rat results on Days 4–5

RatIdentity Acc ↑Proficiency MSE ↓
A280.65%0.023
B269.64%0.014
C278.79%0.027
D287.18%0.023
E289.71%0.014
F283.33%0.024

Question 2: Clarification about Figure 6

Thank you for pointing out the ambiguity. In Section 6, we use a diffusion model to convert behavioral representation encoded by TRAIL (t-SNE plot in Figure 5) back to observable trajectory movement. Figure 6 conducts an interpretation analysis over the same type of turning movement across the first and last day (left) and between two rats (right). Following the movement-view analysis in Table 3, the bottom-left panel traces the body keypoint’s path through the maze, and the bottom-right panel shows the tracked skeleton. In the bottom-left panel, each dot marks the rat’s current position, the fading trail shows its recent path, and successive tiles advance through time. The plot above shows body speed during each movement (left), and individual body ratio for each individual (right) over the same interval.

We start with an Observed turn from Day 1 (left) or Rat A (right) and decode its TRAIL representation to obtain the Reconstructed turn. We then shift that representation toward the representation of a Reference turn from Day 5 (left) or Rat B (right) and decode the result. The Transformed turn shows how that shift changes the movement, helping us inspect differences in turning across days and in body configuration across rats. We will revise the figure and caption to clarify what each panel shows and how we generate the transformed turns.

Question 3: Explanation about “NeRF-style multi-frequency encoder

Thank you for asking us to clarify this term. Small shifts in keypoint positions can reveal subtle changes in how a rat moves. To help TRAIL capture these shifts, we adapt NeRF’s coordinate encoding [1] and represent each coordinate using sine and cosine functions at 10 frequency levels. Lower frequencies vary slowly with position, while higher frequencies respond more strongly to small shifts. This gives TRAIL features for learning both overall movement and subtle postural changes. We will clarify this in the manuscript.

[1] Mildenhall et al., “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis,” ECCV 2020.

Question 4: Role of identity recognition and the pretraining objective

Thank you for raising this question. As discussed in our response to Weakness 1, we study both how movement differs across animals and how it changes during learning. Table 3 illustrates why we evaluate both: Local Pose better predicts identity, while Global Trajectory better predicts learning-related outcomes. We agree that negative pairs from different animals can implicitly encourage identity separation. Our results show that the same representation retains information relevant to both questions without explicit identity or learning-related labels during pretraining.

Again, we thank you for your valuable feedback, which has helped us improve our study substantially. We welcome any remaining questions and would appreciate your reconsideration of the score if our responses have addressed your concerns.

Reviewer_4t31OpenReview ↗

Comment by 4t31

I appreciate the authors' rebuttal. I will retain my original score because some of my concerns persist.

Regarding Q1, I interpreted the reported "variation" as insufficiency of the current dataset to answer to my question. I think a stable baseline representing the maturity state is crucial as a baseline (or a control condition) for evaluating the proposed metthod.

And Q3 is not addressed appropriately in the revised manuscript. The authors should report the exact math (in Appendix) and make their manuscript self-explanatory. I don't think NeRF is a universal jargon outside the 3D vision community.

I was not convinced by the authors' response to Weakness 3, either. Standard visualization techniques would at least be applicable (though not necessarily effective) to the animal data if combined with appropriate models. The authors should at least acknowledge and discuss them in the manuscript.

Reviewer_rQn4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces TRAIL, an encoder transformer-based model for rat movement trajectory windows adapted from Wayformer (Nayakanti et al., 2022), trained with a contrastive loss and an auxiliary reconstruction objective. The model is trained and applied on a spatial learning task (Shridhar et al., 2026) to predict animal identity, day of recording, proficiency, success rate, and reward on the task; beating all or most alternative models. TRAIL is also tested on the behavioural benchmark MABe22 (Sun et al., 2023), on which it reaches performances on-par with other models both in short- and long-term contexts. The model's learnt representations are analysed to investigate broader learning-related features of movement, showing that (among other things) comparing segments further than 60 seconds apart slightly reduces performance on some downstream target metrics and short 1-second segments give results close to the best performance. Finally, movement is generated from the model's learnt representations with a diffusion model and interpretations are given for turning bouts, showing that trained rats turn faster and identity is best described by body elongation.

Critical strengths

Performance on the downstream decoded features (identity, day of recording, proficiency, reward) are better (around 13% improvement w.r.t. second best on identity and day) or slightly better (around 3% improvement w.r.t. second best on proficiency and reward) than multiple alternative models, representing a valid alternative for neuroscientists or other experimentalists when in need to decode such features from movement trajectories.

Critical weaknesses

Performance on decoded feature is the only critical contribution I identified, but it should be validated from multiple other aspects to gauge whether the introduced method is to be preferred over alternatives. The first main expansion would be the introduction of an additional dataset, because the model is currently compared exclusively on a single dataset collected during a Y-maze task. While MABe22 is introduced as a second dataset for evaluating representation quality, the learning-related analysis only covers the Y-maze task. Moreover, it would be important to know the compute resources and time needed for training, dependance on the set of parameters and difficulty of successfully fitting them (i.e., is a parameter sweep highly recommended?), and amount of training data needed.

I am particularly concerned about Sections 5 and 6 (titled "Scientific analysis and insight" and "Interpretation", respectively). In Section 5, the authors state using TRAIL to investigate broader scientific questions: at what timescale behaviour varies in relation to learning, how long must a trajectory window be to capture signals useful to decode identity and learning-related features, and which movement patterns reveal individual identity and learning progress. First of all, investigation is done on a single dataset where rats performed a Y-maze rewarded task (Shridhar et al., 2026), and it must be clearly stated results must be interpreted relative to such circumstances. Furthermore, regarding the authors' claimed findings:

  • The statement that "treating segments more than 60 seconds apart as behaviourally uniform substantially reduces downstream performance" - relative to results shown in Figure 3 (left) - is inflated considering that results just slightly worsen only on two of three metrics when considering longer gaps.
  • Local pose being more informative than global trajectory when decoding animal identity could very easily be confounded by single animal dimension (i.e., joint organisation in space), or annotator bias. The same confound could be driving the results when plotting with t-SNE. A control experiment could help address these worries.

While I appreciate the introduction of the diffusion model to investigate visible changes in the behaviours learned by the model, from my understanding the main take-away from Figure 6 (left) is that animals tend to change their speed profile when tunring with increased exposure to the learning task, and from Figure 6 (right) is that animals have different body-ratio compositions. These results do not provide quantitative population-level validation, giving limited insights into their generality or scientific significance.

The authors do not discuss the limitations of their work, nor do they commit to releasing their code or provide a reproducibility statement.

Questions for authors

See weaknesses above for major concerns. Questions that could help clarify some parts of the paper:

  • Could you please clarify how the Proficiency, Success Rate, Reward metrics are defined and how MSE is derived?
  • In Section 4, a "multi-task supervised variant" of TRAIL is introduced. What results are derived from this version of the model?
5 comments
AuthorsOpenReview ↗

Response to Reviewer rQn4 (1/2)

We thank the reviewer for recognizing TRAIL’s downstream performance and for the suggestions on generalization, interpretation, and reproducibility. Below, we address each concern.

Answer to Weakness 1: Learning-related analysis in additional dataset.

Thank you for pointing this out. We agree that our scientific analyses in the manuscript covered only the spatial-learning Y-maze dataset (Table 4, p. 7), limiting the scope of our findings. We extended the analyses to a dataset with a different maze geometry, comprising six additional rats recorded over five days in a four-arm Plus-maze. The analyses on this new dataset further support our initial claim in Section 5 that signals of individuality and behavior are most apparent at the maze center, where two movement motifs, Turning and Going Straight, occur in the Plus-shaped maze, consistent with the Turning-only in the Y-shape analyses (Table 4c, p.7). We provide the full Plus-maze analysis below and will include it in the revised manuscript.

(a) Maze Regions

RegionIdentity ↑Day ↓Proficiency ↓
Straight arm70.801.5750.021
Arm endpoint81.411.3040.017
Maze center85.511.2740.014

(b) Recording Days

Recording dayIdentity ↑Proficiency ↓
Day 179.290.022
Day 283.120.010
Day 377.780.012
Day 483.600.021
Day 577.500.019

(c) Movement Motifs

Movement motifIdentity ↑Day ↓Proficiency ↓
Remaining still84.391.2420.017
Scanning67.621.7160.021
Going straight93.581.0170.010
Turning82.811.2850.015
Turning back80.771.1910.013

Answer to Weakness 2: Interpretation of the 60-second timescale window

Thank you for pointing this out. We agree that “substantially reduces” overstates the effect. We interpret 60 seconds as an empirical choice for this spatial-learning task, where rats learn movement patterns tied to the task’s demands. Across the windows tested in Figure 3 (left), identity accuracy is near its plateau at 60 seconds, while Day and Proficiency errors are near their minima. Shorter windows may exclude informative trajectories from the same behavioral state, while wider windows may pair trajectories from different states. We will revise the text to explain this trade-off and remove the stronger claim of a one-minute behavioral timescale.

Answer to Weakness 3: Potential body-proportion bias in local-pose analysis

Thank you for suggesting this control experiment. We considered whether body proportions provide a shortcut for identity decoding. We initially expected the measured segment lengths to remain stable, but observed substantial variation during turning and scanning, where both identity and learning signals are strongest (Table 4c, p. 7). To quantify how much identity information segment lengths alone carry, we used each trajectory’s mean nose–neck and neck–body lengths as the sole inputs to predictors for all five downstream tasks. The best predictor achieved 42.69% identity accuracy, compared with 77.1% for Local Pose, showing that Local Pose carries substantial identity information beyond these two features.

Prediction from mean segment lengths compared with Local Pose as reference

MethodIdentity ↑Day ↓Proficiency ↓
Logistic/Ridge Regression29.25%1.9730.021
KNN41.31%1.8350.020
SVM41.37%1.8710.020
TabPFN 3.042.69%1.8320.020
Local Pose (reference)77.1%1.610.019
AuthorsOpenReview ↗

Response to Reviewer rQn4 (2/2)

Answer to Weakness 4: Quantitative population-level validation for the interpretation

Thank you for raising this point. Figure 6 illustrates the findings from Section 5 by translating shifts in the learned representation into visible movement changes. The left panel shows how a rat’s speed profile during a turn shifts toward that of a later-day example. The right panel shows how the relative positions of the head, neck, and trunk during a turn shift toward those of a different individual. These examples make individual differences and behavioral changes across days observable through kinematic features such as turning speed and relative head–trunk configuration. To clarify what you mean by “population-level validation,” would quantifying these patterns across rats in the original recordings address your concern, or would you also like us to assess how consistently the diffusion transformations reproduce them?

Answer to Weakness 5: Spatial-learning targets, TRAIL implementation, and computational cost

Thank you for asking us to clarify these details. In the Y-maze and Plus-maze datasets, rats move freely through the maze and visit its arms for food or water rewards. For each trial, we record whether the rat received a reward and estimate a performance score following [1]. We derive three learning-related targets for each input trajectory. Proficiency is the score for the trial in progress at the trajectory’s final frame. Success Rate and Reward are the fraction of rewarded trials and the total number of rewards, respectively, within 30 seconds on either side of that frame. We predict these targets from the learned trajectory representation using ridge regression and report mean squared error (MSE).

Training TRAIL takes approximately 22 hours per spatial-learning dataset and 20 hours for MABe22 on a single NVIDIA A100 GPU with 32 GB of memory (Appendix A.1, Tables 5–6). For a fair comparison, we sweep hyperparameters for all methods and select configurations based on validation performance on Identity and Day (Appendix A.2, Tables 7–8). We use these two targets because they are readily available in comparable recordings, and we reserve Proficiency, Success Rate, and Reward for the learning-related evaluation.

[1] Smith et al., “Dynamic Analysis of Learning in Behavioral Experiments,” Journal of Neuroscience, 2004.

Answer to Question 1: clarify how downstream benchmarks are derived

Please see our response to Weakness 5, which explains how Proficiency, Success Rate, and Reward are derived and how MSE is calculated.

Answer to Question 2: Multi-task supervised variant

We apologize for the unclear clarification in the manuscript. Table 1 reports that multi-task supervised variant as the baseline. We call it a TRAIL variant because it shares TRAIL’s trajectory embedding, encoder architecture, and hyperparameters. The variant trains directly on all five downstream targets, while TRAIL learns through temporal contrast and trajectory reconstruction. TRAIL’s stronger downstream performance shows that these self-supervised objectives can capture more useful behavioral information without explicit identity or learning-related labels during representing learning.

Again, we appreciate your valuable opinions on improving our study. Please also kindly give us a chance to clarify any remaining unclear points, or consider raising the score if you find our response satisfactory.

I thank the authors for taking the time to respond to my concerns.

I appreciate the introduction of a new four-arm plus maze configuration. However, results on the plus maze are limited to corroborating Section 5's claim that signals of individuality and behaviour are most apparent at the maze center. Importantly, the authors do not provide a performance comparison with other models, as in Table 1.

With respect to weakness 2, it is unclear how you would materially address this in the main text.

With respect to weakness 4, I think both directions would strengthen results, possibly the first route would be more informative because as of now, Section 6 remains particularly limited in providing new scientific insights.

Overall, I think the additions to the study are headed in the right direction. On the other hand, the authors do not precisely specify how they intend to revise the manuscript, and an updated version of the article is not provided as of now. I will maintain my score.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (1/2)

Dear reviewers, We thank all three reviewers for their thoughtful feedback. We appreciate the recognition of TRAIL’s strong downstream performance (rQn4, 4t31), the value of the longitudinal learning setting and the breadth of our evaluation and uncertainty reporting (nQbw), and the potential of diffusion-based visualization for inspecting movement differences and generating hypotheses (nQbw, rQn4). Below, we clarify our scientific objective, present the additional Plus-maze results, and address shared concerns about the interpretation of our findings.

Scientific objective and contribution

Individual variation in behavior and cognitive function is an important scientific question that remains challenging to tackle. Our goal is to understand which short-term movement patterns distinguish individuals, how they vary with learning, and when and where these differences are most apparent. The spatial-learning dataset [1] lets us follow the same rats as they learn across days under controlled conditions. TRAIL learns directly from their trajectories through temporal contrastive learning and reconstruction, without predefined behavioral descriptors or downstream labels.

We use five downstream tasks—identity, recording day, proficiency, success rate, and reward—to assess whether the representation retains information relevant to individuality and learning progress. We then use the representation to investigate behavior. Section 5 examines how these signals vary across time, maze regions, and movement views. Section 6 translates shifts in the representation into observable movement changes. Our contribution includes adapting established components to this scientific question, testing those choices through ablations, and using the resulting representation to investigate behavior. See our responses to 4t31’s W1 and nQbw’s W3.

[1] Shridhar et al., “Goal Location Beta Oscillations and Sharp Wave–Ripples Are Complementary Markers of Learning-Related Changes in Hippocampal–Neocortical Networks,” eNeuro, 2026

Additional longitudinal evaluation in a different maze geometry

To examine our findings in another maze environment, we evaluated six additional rats recorded over five days in a four-arm Plus-maze. We repeated the Section 5 analyses across animals, maze regions, recording days, and movement motifs. The most informative motif differed across settings: turning in the Y-maze and going straight in the Plus-maze, where rats can also traverse the center in a straight line. The shared result supports the importance of decision regions, while the difference in motifs suggests a question for further study about how maze geometry shapes the information carried by movement. Identity is reported as percentage accuracy. Day and Proficiency are reported as mean squared error.

(a) Maze Regions

RegionIdentity ↑Day ↓Proficiency ↓
Straight arm70.801.5750.021
Arm endpoint81.411.3040.017
Maze center85.511.2740.014

(b) Recording Days

Recording dayIdentity ↑Proficiency ↓
Day 179.290.022
Day 283.120.010
Day 377.780.012
Day 483.600.021
Day 577.500.019

(c) Movement Motifs

Movement motifIdentity ↑Day ↓Proficiency ↓
Remaining still84.391.2420.017
Scanning67.621.7160.021
Going straight93.581.0170.010
Turning82.811.2850.015
Turning back80.771.1910.013
AuthorsOpenReview ↗

General Response (2/2)

Trajectory representation and learning-related signal

While the spatial-learning dataset tracks rat movement through only three keypoints, these prominent body locations capture both navigation and subtle head–trunk changes relevant to our scientific questions. Separating Local Pose and Global Trajectory lets us examine the information each view carries. In our additional control, predictors using only mean nose–neck and neck–body lengths achieved at most 42.69% identity accuracy, compared with 77.1% for Local Pose. This shows that Local Pose carries identity information beyond those two features. We discuss the control (rQn4’s W3) and the measurement scope and possible projection effects (nQbw’s W1).

Temporal contrast can amplify individual and temporal differences. The downstream results show that the learned variation is associated with measured task outcomes, while recording day can also capture other changes over time. The late-day Y-maze and Plus-maze results motivate closer inspection of each rat’s learning trajectory. Testing-day decoding after a performance plateau would require establishing that plateau from the behavioral records (4t31’s Q1). We will also clarify that 60 seconds is an empirical choice for the positive-pair window in this task (rQn4’s W2).

Interpretation visualization in Figure 6

While our central goal is to learn a meaningful representation that encodes individual identity and learning progress, in Section 6 we use a diffusion model to see how this information appears in the trajectories themselves. Figure 6 illustrates this with three-second turns. On the left, shifting the representation toward a later-day reference changes the turn’s speed profile. On the right, shifting it toward a reference from another rat changes the head–trunk configuration. These examples show which kinematic features change with each shift and suggest patterns to test across animals. We will revise the figure and caption to clarify what each panel shows and how we generate the transformed turns. We further discuss the meaning of Figure 6 in our response to rQn4’s W4 and clarify the Figure 6 visualization in our responses to 4t31’s Q2.

Once again, we thank the reviewers for their feedback, which helped us clarify TRAIL’s contribution as a tool for studying individual differences and learning-related changes in movement. In the revised manuscript, we will include the Plus-maze results and the body-length control, scope our conclusions to the rodent settings studied, and clarify what the quantitative analyses and visualizations each support. We provide further details in our individual responses and welcome any remaining questions.

AuthorsOpenReview ↗

Follow-up on our Author Responses

Dear Reviewers,

Thank you again for your reviews. If you have a chance to read our response before the discussion closes on November 18, we'd appreciate your feedback and would be happy to clarify any remaining questions. If our responses address your concerns, we'd be grateful if you would consider revisiting your scores.

Thank you for your time.

Dear Reviewers and Area Chair,

We have uploaded the revised manuscript, with changes highlighted in blue. Thank you for your patience and feedback, and we sincerely apologize for the long delay. The main revisions are:

  1. Abstract and introduction: Clarified the progression from our central scientific question to TRAIL’s design, quantitative validation of its learned representations, behavioral investigation, and inspection of the representations through observable movement changes.
  2. Section 4 (Experiments): Expanded the dataset description, clarified the downstream targets and evaluation procedure, and discussed the MABe22 results.
  3. Section 5 (Scientific Findings): Added the body-proportion control (Table 4) and Plus-maze analysis (Table 6), and revised the 60-second timescale claim.
  4. Section 6: Rewrote the section as “From Representations to Observable Dynamics” to clarify its purpose, and revised Figure 6’s caption to explain how representation shifts translate into movement changes.
  5. Section 7 (Discussion and Conclusion): Added an explicit discussion of limitations, scope, and future directions.

We hope these revisions clarify the contribution and address the concerns raised. We welcome further feedback and questions.

Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.