- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
FreeCAB is a benchmark pairing freely moving dorsal CA1 miniscope calcium imaging with synchronized 8-camera behavioral video across 58 curated sessions. It defines matched encoding and decoding tasks on 2D position and reconstructed 3D pose, and evaluates behavioral-state dependence, cross-session adaptation, and joint encoding–decoding training across general-purpose and neural-domain baselines.
Critical strengths
The paired multi-view video and calcium data is a valuable resource for neural–behavior modeling under free behavior.
The quality of the 3D tracking appears to be high, with low cross-session variance.
Critical weaknesses
Table 3 says nine of the ten held-out sessions come from animals that also contribute held-in sessions, so the cross-session evaluation is largely within-animal. An included leave-animal-out split would be a stronger test of transfer.
The main text spends most of its space on the simple general-purpose baselines. I would prefer to see that material condensed and the foundation-model benchmarks the work is primarily concerned with moved into the main text.
Section 3.3 describes exclusion of sessions for insufficient behavioral coverage and alignment. It would be helpful to retain or co-release a less-filtered version of the session list, as this would enable additional analyses, particularly for self-supervised methods that do not depend on behavioral labels. This is a very mild weakness.
Questions for authors
Do you split over mouse identity, or only over sessions?
Do you have any setup for zero-shot or forecasting settings that are becoming more popular? (e.g. SPINT, Le et al. 2025, arXiv:2507.08402, for zero-shot; POCO, Duan et al. 2025, arXiv:2506.14957, for forecasting)
Will you release the unfiltered/complete data? The multiview camera data may be useful for methods like BEAST3D, Wang et al. 2026, arXiv:2606.02937.
3 comments
Response to Reviewer aKN2 (Part 2/3)
W2: The main text spends most of its space on the simple general-purpose baselines. I would prefer to see that material condensed and the foundation-model benchmarks the work is primarily concerned with moved into the main text.
Q2: Do you have any setup for zero-shot or forecasting settings that are becoming more popular? (e.g. SPINT, Le et al. 2025, arXiv:2507.08402, for zero-shot; POCO, Duan et al. 2025, arXiv:2506.14957, for forecasting)
Neural-domain baselines (W2). We have moved all six neural-domain baselines—LFADS, NeuroFormer, NDT2, CEBRA, POYO+, and NEDS—into the main results (Section 5.1, Table 2), covering 2D-position and 3D-pose decoding alongside available NEDS encoding results. These evaluations use ten reporting sessions and three seeds.
Additional prediction tasks (Q2). Following your suggestion, we have added supplementary evaluations with SPINT [1] and POCO [2] to explore label-free transfer and neural forecasting within FreeCAB.
Label-free adaptation with SPINT. We adapted SPINT to FreeCAB's 30-frame neural-to-position decoding task, using 41 source sessions from 11 mice and three animal-disjoint target sessions. Each target session provides 300 unlabeled neural frames for calibration, without behavioral labels or parameter updates. SPINT achieves a mean of −0.1071, compared with −0.0354 for a source-global position-mean baseline. Since calibration uses unlabeled target neural activity, this is a label-free adaptation setting rather than strict zero-shot transfer. Our evaluation also uses whole-window decoding instead of SPINT's original last-timestep protocol.
Neural forecasting with POCO. We evaluated POCO on a 48-to-16 neural forecasting task, predicting 16 future neural frames from 48 context frames. Across ten sessions and three seeds, POCO achieves and Pearson , outperforming the persistence baseline (). The mean improvement is +0.1568 (paired 95% CI: [0.0737, 0.2165]).
These experiments extend FreeCAB to additional neural prediction settings while keeping encoding and decoding as its primary benchmark tasks. Full protocols and results are provided in Appendix E.8 (Table 21 for POCO).
References
[1] Le et al. SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding. NeurIPS 2025.
[2] Duan et al. POCO: Scalable Neural Forecasting through Population Conditioning. NeurIPS 2025.
Response to Reviewer aKN2 (Part 3/3)
W3: Section 3.3 describes exclusion of sessions for insufficient behavioral coverage and alignment. It would be helpful to retain or co-release a less-filtered version of the session list, as this would enable additional analyses, particularly for self-supervised methods that do not depend on behavioral labels. This is a very mild weakness.
Q3: Will you release the unfiltered/complete data? The multiview camera data may be useful for methods like BEAST3D, Wang et al. 2026, arXiv:2606.02937.
Thank you for your interest in the broader FreeCAB dataset and its potential applications to methods such as BEAST3D.
Yes. Upon acceptance, we plan to publicly release the additional unfiltered recordings and corresponding multi-view behavioral videos, alongside the curated 58-session benchmark. This will enable researchers to explore alternative preprocessing and tracking pipelines, develop new behavioral representations, and evaluate methods beyond the benchmark's predefined tasks.
The curated benchmark will remain available with its standardized preprocessing, quality-control criteria, and evaluation splits to support reproducible comparisons.
Response to Reviewer aKN2 (Part 1/3)
Thank you for raising this question. Our original split is session-disjoint, so held-in and held-out sessions may come from the same mouse. This evaluates adaptation to new recording sessions, whereas animal-disjoint evaluation tests transfer to previously unseen animals.
We have therefore added an animal-disjoint NEDS 2D-position decoding experiment. All recordings from three target mice (M01, M04, and M10) are excluded from source training and validation, leaving 41 source sessions from 11 mice. Scratch and Pretrain+FT are compared on identical target-session splits using three downstream seeds.
Pretraining improves decoding on two target mice (M01: ΔR2=+0.0205; M04: +0.0059), while performance decreases on the third (M10: −0.0136), yielding a mean gain of +0.0043. These results extend our evaluation to unseen animals and show that transfer benefits vary across target recordings.
We have clarified the original split in Section 4.2 and added the animal-disjoint results in Appendix E.5 (Table 18).