acceptodds
Under review as a conference paper at ICLR 2027

CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

Abstract

Calcium imaging has enabled large-scale recordings of neural population activity across diverse animals, laboratories, and experimental paradigms. However, these datasets remain highly fragmented: Existing approaches are often designed for individual datasets or tasks, making it unclear whether their learned representations are reusable for new calcium imaging trace datasets. To tackle this gap, we present CAPT, a Continuous Autoregressive Population Transformer for calcium population dynamics. CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively with a mean squared error objective, enabling end-to-end training and adaptation to diverse downstream tasks. We first pretrain CAPT on a large-scale mouse calcium imaging dataset and evaluate its transferability across independent mouse, larval zebrafish, and C. elegans datasets collected by different laboratories. In these transfer settings, the pretrained backbone is frozen and only adaptation modules, i.e., neuron and session embeddings or task-specific heads, are updated. Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines, demonstrating that a single continuous autoregressive backbone can learn representations that transfer beyond the original pretraining distribution. Beyond predictive performance, multimodal analyses using NeuroPAL annotations in C. elegans datasets show that CAPT embeddings form a shared functional space across datasets and capture cell-identity-related structure. Together, these results suggest that continuous autoregressive modeling provides an effective framework for learning reusable calcium population representations that transfer across datasets, experimental paradigms, and species. Code is available at https://anonymous.4open.science/r/project_code_2026-0AC8/

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neural decoding, neural population dynamics forcasting, multitasking, calcium traces

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_aV76OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces CAPT, a continuous autoregressive Transformer for modeling calcium imaging population activity across datasets and species. The proposed framework combines continuous patch tokenization, a dual-axis Transformer backbone, dynamic residual gating, and neuron-aware QKV modulation. The model is pretrained on a large-scale mouse calcium imaging dataset using an autoregressive forecasting objective and subsequently evaluated on independent mouse, larval zebrafish, and C. elegans datasets. During transfer, the pretrained backbone remains frozen, while dataset-specific neuron and session embeddings are optimized for forecasting, followed by task-specific readout training for behavior decoding.

The authors report improved forecasting and decoding performance over several existing baselines, including POCO, POYO+, and CalM. They further investigate the biological relevance of the learned representations using NeuroPAL annotations in C. elegans, showing that the adapted neuron embeddings support cross-dataset cell-identity classification.

The central contribution of this work is an empirical demonstration that a mouse-pretrained autoregressive backbone can be reused across heterogeneous calcium imaging datasets with limited parameter adaptation. However, the extent to which these results establish the transfer of neural dynamics, rather than generic calcium signal statistics or architectural inductive biases, remains unclear. Similarly, the NeuroPAL analysis provides evidence of identity-related information in the adapted embeddings, but does not directly establish that conserved neural dynamics are represented across species.

Critical strengths
  1. Biologically motivated evaluation beyond predictive performance

One notable strength of this work is the incorporation of NeuroPAL annotations to investigate whether learned neuron embeddings capture biologically meaningful information. Rather than relying exclusively on forecasting and decoding performance, the authors examine whether the representations support cell-identity classification across independent C. elegans datasets. I particularly appreciate this attempt to connect neural representation learning with biological interpretability, as such evaluations remain relatively uncommon in neural foundation model research. Although the results do not directly establish the transfer of conserved neural dynamics, they provide an interesting empirical observation and a potentially useful direction for future investigations.

  1. Broad empirical evaluation across heterogeneous calcium imaging datasets

The evaluation spans multiple independently collected datasets across mice, larval zebrafish, and C. elegans, covering different recording conditions, sampling rates, and experimental paradigms. This breadth is valuable because cross-dataset generalization remains an important challenge in neural population modeling. The frozen-backbone adaptation protocol also provides a useful empirical setting for investigating model reusability. While the results do not fully establish the underlying mechanism of transfer, the extensive evaluation offers a potentially valuable benchmark for future work.

Critical weaknesses
  1. The central claim of cross-species neural dynamics transfer is not sufficiently substantiated.

My primary concern is the substantial gap between the paper’s central scientific claim and the evidence provided. The authors demonstrate that a mouse-pretrained backbone can be adapted to calcium imaging datasets from other species by optimizing target-specific neuron and session embeddings. However, successful adaptation does not necessarily imply that representations of neural population dynamics have been transferred across species.

In particular, the observed performance could be explained by generic temporal properties of calcium fluorescence signals, such as autocorrelation, smoothness, or indicator-dependent kinetics, rather than conserved neural dynamics. This distinction is especially important given the substantial differences in sampling rates, calcium indicators, and biological timescales across the evaluated datasets. The paper does not provide sufficient controlled experiments or mechanistic analyses to distinguish these competing explanations.

The NeuroPAL experiments are an interesting attempt to establish biological relevance, but they remain indirect evidence. Cross-dataset cell-identity classification demonstrates that the adapted embeddings contain identity-related information; it does not establish that this information originates from mouse pretraining or reflects conserved neural dynamics. Furthermore, the failure of independently trained models to produce directly comparable embeddings is not sufficient evidence that the shared coordinate system learned through the frozen backbone captures biologically conserved structure.

Consequently, the experiments establish the practical reusability of a pretrained backbone, but do not sufficiently substantiate the stronger scientific interpretation of cross-species neural dynamics transfer. This distinction is central to my assessment because cross-species transferability is presented as one of the paper’s primary contributions.

  1. The methodological contributions and the role of pretraining are insufficiently isolated.

My second concern is the limited evidence establishing which components of CAPT are responsible for its reported improvements, particularly under cross-species transfer. The proposed framework largely combines existing modeling techniques, including continuous patch tokenization, dual-axis attention, dynamic residual gating, neuron-aware QKV modulation, and residual autoregressive prediction. While their combination yields competitive empirical performance, the paper does not convincingly establish a new methodological principle underlying transferability.

More importantly, the current experimental design does not adequately distinguish the contribution of pretrained representations from that of the architecture itself. For example, a randomly initialized frozen backbone with trainable target-specific embeddings would provide an important control for determining whether the observed transfer performance genuinely depends on knowledge acquired during mouse pretraining. Similarly, target-only training and controlled comparisons of adaptation strategies would help clarify the contribution of source-domain pretraining.

Although the authors provide architectural ablations, these primarily examine forecasting performance rather than the cross-species transfer settings that motivate the paper. Therefore, the reported improvements cannot be confidently attributed to the proposed components or to the autoregressive pretraining strategy.

Without these controls, it remains unclear whether CAPT introduces a genuinely transferable representation learning approach or primarily offers an effective architecture for adapting to heterogeneous calcium imaging datasets. This substantially limits the methodological novelty and the strength of the paper’s conclusions.

Questions for authors

Q1. What evidence supports the transfer of neural dynamics rather than generic calcium fluorescence statistics?

The paper’s central claim concerns transferable representations of neural population dynamics across species. However, calcium fluorescence signals contain substantial temporal structure arising from indicator kinetics, autocorrelation, and recording-specific characteristics. How do the authors distinguish the transfer of biologically meaningful neural dynamics from the reuse of generic temporal statistics shared across calcium imaging datasets?

In particular, have the authors considered controls that preserve basic signal statistics while disrupting population-level neural interactions? Such comparisons would help determine whether CAPT transfers neural population structure or primarily models generic fluorescence dynamics.

Q2. How much of the reported transfer performance actually depends on source-domain pretraining?

Could the authors clarify whether they evaluated a randomly initialized, frozen CAPT backbone with the same target-specific embedding adaptation procedure?

This comparison is particularly important because the current results do not isolate the contribution of mouse pretraining from the architectural inductive biases of CAPT. A comparison against target-only training would further clarify whether source-domain pretraining provides a meaningful advantage over learning directly from the target dataset.

Q3. What is the evidence that the proposed architectural components specifically improve cross-species transferability?

The paper introduces several architectural modifications, including dynamic residual gating, neuron-aware QKV modulation, and residual-last prediction. However, the reported architectural ablations primarily focus on forecasting performance within a limited number of sessions.

Do these components also improve transfer performance on zebrafish and C. elegans datasets? In particular, is the complete architecture consistently superior to a simpler continuous autoregressive dual-axis Transformer under the same frozen-backbone adaptation protocol?

This distinction would materially affect my assessment of the methodological contribution, as the central motivation of the paper is transferability rather than source-domain forecasting accuracy.

Q4. How do the authors account for differences in physical timescales across species and recording conditions?

The datasets differ substantially in sampling frequency and calcium indicators. For example, the mouse pretraining dataset is sampled at 6 Hz, whereas the C. elegans datasets are sampled at approximately 1.67 Hz. Consequently, a fixed-length temporal patch corresponds to substantially different physical durations across datasets.

How does CAPT account for these differences when transferring temporal representations? Were any experiments conducted to determine whether the reported transfer performance is robust to changes in sampling frequency or temporal resampling?

Without such controls, it is difficult to determine whether the backbone learns transferable temporal dynamics or simply adapts to dataset-specific temporal statistics through newly optimized embeddings.

Q5. What evidence establishes that the NeuroPAL results reflect biologically meaningful shared representations?

I appreciate the NeuroPAL-based analysis, as it attempts to evaluate learned representations beyond predictive performance. However, cross-dataset cell-identity classification does not necessarily establish that the embeddings encode conserved neural dynamics.

Could the authors clarify what specific evidence supports this interpretation? In particular, would comparable identity-classification performance emerge from a randomly initialized frozen backbone or a backbone pretrained on temporally matched surrogate calcium signals?

Additionally, the independently trained CAPT models used in Table 15 do not necessarily share a common embedding coordinate system. Therefore, their poor cross-dataset classification performance may reflect representational misalignment rather than the absence of biologically meaningful information.

How do the authors distinguish these possibilities?

Q6. Are the improvements over existing baselines attributable to the pretrained backbone or to differences in adaptation and decoding protocols?

CAPT employs a two-stage adaptation procedure, first optimizing target-specific neuron and session embeddings and subsequently training a nonlinear behavior decoder. The compared baselines differ in their architectures, objectives, and adaptation strategies.

Could the authors clarify whether the baselines were evaluated under comparable adaptation budgets and decoder capacities? In particular, would CAPT retain its advantage if the same decoding architecture and comparable trainable parameter budgets were used across methods?

This clarification is important for attributing the observed performance improvements to the proposed representation learning framework rather than differences in downstream adaptation capacity.

9 comments
AuthorsOpenReview ↗

Rebuttal part I

We thank the reviewer for the detailed comments and questions with sufficient explanation.

[W1 & Q1]:

We thank the reviewer for this important question. To investigate whether CAPT benefits from population-level information beyond individual-neuron temporal statistics, we conduct three additional experiments: (1) removing neuron-axis attention to prevent cross-neuron communication; (2) pretraining CAPT on surrogate source data with disrupted cross-neuron relationships; and (3) introducing an AR(2) baseline to evaluate forecasting based on simple temporal dependencies.

We construct two surrogate-data controls: (1) IAAFT (iterative amplitude-adjusted Fourier transform), independently generating surrogate traces that preserve the amplitude distribution and approximately preserve the power spectrum; (2) trial shuffle, independently permuting trial assignments for each neuron. These controls disrupt different aspects of cross-neuron temporal structure while preserving selected single-neuron signal statistics. All three pretrained models use the same three Tseng source sessions and identical training budgets.

For IAAFT, we first independently shuffle each neuron’s trace within each trial. At each iteration, we take its FFT, replace the magnitudes with the original trace’s magnitudes while keeping the current phases, and apply the inverse FFT. We then replace the smallest resulting value with the original minimum, the second smallest with the original second-smallest value, and so on. We repeat this up to 200 times, retaining the surrogate with the smallest Fourier-magnitude error.

For the AR(2) baseline, we independently fit each neuron's autoregressive coefficients with xt=b+α1xt−1+α2xt−2+ϵtx_{t} =b+\alpha_{1}x_{t-1}+\alpha_{2}x_{t-2}+\epsilon_{t} using least squares on the training trials only. At test time, the model is initialized with the observed context and recursively predicts the forecasting horizon.

We evaluate all models on three Tseng sessions and three Sun sessions under source-domain and cross-dataset transfer settings, respectively. For the surrogate-pretraining controls, only the source pretraining data are modified, while the target adaptation and evaluation data remain unchanged. This allows us to examine the contribution of source-domain population structure to subsequent transfer.

As shown below, CAPT consistently outperforms all ablation baselines on both Tseng and Sun. In particular, removing neuron-axis attention or disrupting cross-neuron temporal structure during pretraining leads to substantial performance degradation. These results suggest that population-level temporal information contributes to CAPT's forecasting and transfer performance beyond simple individual-neuron autoregressive statistics.

MethodTseng 3-session Test correlationSun 3-session transfer Test correlation
CAPT0.5166 ± 0.06180.3522 ± 0.1105
CAPT (no neuron attention)0.4045 ± 0.05110.2467 ± 0.0369
CAPT (IAAFT)0.3126 ± 0.03750.2882 ± 0.1037
CAPT (trial-shuffled)0.3560 ± 0.03430.2785 ± 0.1097
AR(2)0.2521 ± 0.02620.1822 ± 0.0100
AuthorsOpenReview ↗

Rebuttal part II

[W1 & Q2]:

We further evaluate a randomly initialized, frozen CAPT backbone using the same target-specific embedding adaptation protocol. As shown below, the pretrained backbone consistently outperforms the random backbone across transfer settings, demonstrating that source-domain pretraining provides substantial benefits beyond the architectural inductive biases alone.

Datasetrandomly initialized, frozen CAPT backbonePretrained backboneΔ Corr
AHB0.1900 ± 0.08300.4003 ± 0.0197+0.2102
Brysch0.1616 ± 0.02780.2129 ± 0.0249+0.0513
CaLiAli0.2074 ± 0.01020.2970 ± 0.0082+0.0895
Park0.1804 ± 0.01020.2576 ± 0.0173+0.0771
Atanas0.1554 ± 0.08250.2575 ± 0.1062+0.1021
Copper0.1937 ± 0.08350.2823 ± 0.1298+0.0886

To directly assess the contribution of source-domain pretraining, we compare mouse-pretrained CAPT with the same architecture trained from scratch on each target dataset. The pretrained model is evaluated using frozen-backbone adaptation, whereas the target-only model is trained with all backbone parameters learnable. As shown below, frozen-backbone CAPT generally achieves comparable or better performance than target-only training, despite keeping the pretrained backbone fixed. Especially, the largest forecasting improvements are observed on the two C. elegans datasets.

DatasetTrain from scratchEmbedding adaptation
AHB0.4088 ± 0.07570.4003 ± 0.0197
Brysch0.2094 ± 0.00470.2129 ± 0.0249
CaLiAli0.2956 ± 0.00770.2970 ± 0.0082
Park0.2467 ± 0.01320.2576 ± 0.0173
Atanas0.2291 ± 0.09130.2575 ± 0.1062
Copper0.2257 ± 0.11370.2823 ± 0.1298
Dataset / behaviorTrain from scratchEmbedding adaptationΔ R2
Atanas / head angle0.5668 ± 0.16150.5918 ± 0.1522+0.0250
Copper / velocity0.7429 ± 0.09290.7503 ± 0.0948+0.0074
Brysch / eye position0.5029 ± 0.16730.5250 ± 0.1503+0.0221

[W2 & Q3]:

We conduct architectural ablations under the same frozen-backbone transfer protocol as required. The full CAPT consistently outperforms the simplified continuous autoregressive Transformer without dynamic gating, QKV modulation, or residual-last prediction across transfer settings. These results demonstrate the robust benefit of the proposed architectural components for transfer performance.

DatasetSessionSimplified CAPTCAPTΔ Corr
AHB30.3818 ± 0.01770.4003 ± 0.0197+0.0185
Brysch290.1895 ± 0.02460.2129 ± 0.0249+0.0234
CaLiAli30.2833 ± 0.00790.2970 ± 0.0082+0.0136
Park30.2473 ± 0.01740.2576 ± 0.0173+0.0103
Atanas800.2372 ± 0.10410.2575 ± 0.1062+0.0203
Copper270.2634 ± 0.11850.2823 ± 0.1298+0.0188

[W1 & Q4]:

To examine the effect of different sampling rates, we resample the same three sessions of the Tseng and Sun dataset at 0.5× and 2× the original sampling rate. We transfer the same pretrained CAPT backbone to each version following the embedding-only fine-tuning protocol without any sampling-rate-specific modifications to the model architecture or patch length. During test evaluation, we adjust the context and forecasting horizon proportionally to cover the same physical durations across different sampling rates. We observe model stays robust across different sampling rates.

Together with the comparison against a simple autoregressive baseline in [W1 & Q1], these results suggest that CAPT benefits from population-level structure beyond simple individual-neuron statistics, while remaining robust to changes in sampling rate.

DatasetDownsample ×0.5Original ×1Upsample ×2
Context → Horizon24 → 1648 → 3296 → 64
Tseng 3 sessions0.3418 ± 0.07260.3502 ± 0.06490.3483 ± 0.0764
Sun 3 sessions0.3850 ± 0.11440.3966 ± 0.10670.4085 ± 0.1029
AuthorsOpenReview ↗

Rebuttal part III

[W2 & Q5]:

To address this concern, we repeat the NeuroPAL classification experiments using a shared, randomly initialized backbone and backbones pretrained on the 3-sessions surrogate Tseng data described in Q1. For a controlled comparison, we also pretrain a backbone on the same three original Tseng sessions. The backbone pretrained on real calcium recordings substantially outperforms both the randomly initialized backbone and those pretrained on surrogate data that preserve single-neuron temporal statistics. These results demonstrate that our advantage cannot be explained solely by the constraint of an arbitrary shared backbone or by single-neuron calcium signal statistics, but also depends on population-level temporal structure learned during pretraining.

We agree that the result in Table 15 does not necessarily indicate an absence of biologically meaningful information. However, in practical cross-dataset applications, where no prior cell correspondences or external alignment reference is available, such misalignment is itself a substantial limitation, as the learned representations cannot be directly compared or transferred. CAPT addresses this issue by adapting different datasets into a shared pretrained backbone, providing a common representation space while retaining cross-dataset cell-identity-related structure.

Balanced Acc/ Macro F1189-session pretrainedRandom backbone
Pooled52.47 / 53.4027.04 / 26.19
Atanas → Copper36.45 / 38.1424.42 / 22.12
Copper → Atanas35.35 / 37.2320.56 / 20.83
Balanced Acc/ Macro F13-session pretrained3-session IAAFT3-session trial shuffle
Pooled44.71 / 45.1634.66 / 35.2834.17 / 33.45
Atanas → Copper35.25 / 35.3920.16 / 20.4319.94 / 20.48
Copper → Atanas31.57 / 33.7720.66 / 21.3927.45 / 26.44

[W2 & Q6]:

We thank the reviewer for this important question. POYO+ is primarily designed for supervised neural decoding, while CAPT employs a self-supervised forecasting backbone followed by a task-specific decoder.

We first compare the two methods with matched model sizes (27M parameters) and comparable training budgets. As shown below, CAPT consistently outperforms POYO+ on both the pretraining and cross-species transfer datasets, suggesting that the performance advantage cannot be explained by model size alone.

Setting27M POYO+27M CAPT
Tseng held-in decoding macro R²0.71270.7941
Atanas head-angle R²−0.00080.5918
Copper velocity R²0.05280.7503

To further examine whether these improvements arise primarily from differences in decoder capacity or adaptation procedures, we conducted an additional common-decoder experiment. Following each method's original pretraining and target-adaptation pipeline, we extracted the features from the final backbone outputs and froze the adapted feature extractors. We used PCA to reduce the CAPT feature dimensionality to match that of POYO+, and trained newly initialized decoders with the same POYO+-style cross-attention decoder. Despite this dimensionality reduction and the replacement of its native decoder, CAPT retains a substantial decoding advantage under this shared readout architecture.

These results show that CAPT's decoding advantage persists even when both methods use an identical, newly initialized decoder architecture, indicating that the improvement cannot be explained solely by differences in their native decoder designs or capacities.

Dataset27M CAPT + common decoder27M POYO+ + common decoder
Atanas head-angle R2R^20.3235 ± 0.2319−0.0317 ± 0.0700
Copper velocity0.7244 ± 0.09520.0588 ± 0.2649

We sincerely thank the reviewer again for the careful feedback. We hope our responses have addressed your concerns, and would be happy to provide further clarification if needed.

Reviewer_Ku4QOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAPT predicts continuous calcium patches using a dual-axis autoregressive transformer with residual-last prediction, dynamic residual gating, and neuron-aware attention modulation. After mouse pretraining, it freezes the backbone and adapts target neuron embeddings, followed by behavior readouts. Forecasting and decoding are evaluated across three species.

Critical strengths
  1. Broad transfer evaluation: Eight target datasets span three species under heterogeneous recording conditions.

  2. Substantial reported gains: Held-out Tseng forecasting correlation reaches 0.4677, versus 0.3383 for CalM and 0.3322 for POCO.

  3. Useful identity representations: Cross-dataset NeuroPAL balanced accuracy reaches 36.45%/35.35%, versus 4.35% chance, supporting transferable cell-identity information.

Critical weaknesses
  1. The distinction from CalM is not clear. CalM [1] already supplies the dual-axis backbone, both predictive tasks, and frozen held-out adaptation. CAPT should explicitly distinguish pretraining sources and adaptation protocols, rather than leaving the added capability implicit.
  2. Continuous-versus-discrete attribution is unresolved. Sections 3.1–3.3 change tokenization and loss alongside gating, QKV modulation, and residual prediction. The component ablations help, but do not compare continuous and discrete variants under otherwise matched settings. Consequently, the complete-system gains in Figures 2–3 cannot be attributed specifically to continuous modeling.
  3. Pretraining benefit and adaptation cost remain unquantified. Figure 3 lacks target-only CAPT controls for forecasting/decoding. Beating other transferred models does not establish positive transfer over training from scratch.

References: [1] CalM: A Self-Supervised Foundation Model for Population Dynamics in Calcium Imaging Data. In ICML 2026.

Questions for authors
  1. How do CAPT's pretraining sources and transfer protocols differ from CalM's published Sun/Atanas experiments, and what capability is newly established?
  2. Can a controlled comparison cross continuous/discrete modeling with the auxiliary modules, holding patch length, data, and training budgets fixed?
  3. Does mouse-pretrained CAPT outperform target-only CAPT with identical target data and comparable optimization budgets?
  4. How many parameters are adapted, and how much target training time is required, including embeddings and behavior heads?
4 comments
AuthorsOpenReview ↗

Rebuttal part I

We thank the reviewer for the thoughtful comments.

[W1 & Q1]:

We thank the reviewer for this important question.

CalM did not systematically investigate cross-dataset and cross-species transfer in its original paper. Its experiments on Sun. and Atanas involved training on the target dataset's held-in recordings rather than transferring a mouse-pretrained backbone. In contrast, our work explicitly investigates cross-dataset and cross-species transfer.

In our experiments, both CalM and CAPT are pretrained on the same Tseng mouse dataset. During CalM adaptation, the VQ module and DAT backbone are frozen, with only target-specific neuron and session embeddings updated for forecasting, followed by training task-specific decoding heads for behavior decoding. CAPT follows the same transfer protocol but directly models continuous calcium patches without the VQ module.

Our experiments show that CAPT, combining continuous autoregressive modeling with dynamic gating and QKV modulation, substantially outperforms CalM under the same pretraining data and adaptation protocol. These results demonstrate the effectiveness of our framework for cross-dataset and cross-species transfer.

We will clarify these distinctions in the revised paper.

[W2 & Q2]:

We further conduct a controlled comparison between CalM and CAPT, equipping both models with dynamic gating and QKV modulation. We use the same temporal patch length (8), pretraining data (30 Tseng held-in sessions), transfer datasets, and backbone training budget. CalM additionally requires a separate VQ training stage, which is not included in the matched backbone training budget. The results are shown below.

Under these matched settings, CAPT consistently outperforms CalM across all evaluation datasets.

DatasetSessionsCalM (corr)CAPT (corr)
Tseng held-in300.35940.4121
Atanas800.20080.2463
Copper270.22180.2924
AHB30.14780.3729
Park30.18760.2431
AuthorsOpenReview ↗

Rebuttal part II

[W3 & Q3]:

To directly assess the contribution of source-domain pretraining, we compare mouse-pretrained CAPT with the same architecture trained from scratch on each target dataset. The pretrained model is evaluated using frozen-backbone adaptation, whereas the target-only model is trained with all backbone parameters learnable.

As shown below, frozen-backbone CAPT generally achieves comparable or better performance than target-only training, despite keeping the pretrained backbone fixed. In particular, the largest forecasting improvements are observed on the two C. elegans datasets. Since embedding-only adaptation freezes the backbone and thus has less trainable parameters and compute than training from scratch, we additionally compare full fine-tuning with scratch training under exactly matched target-data and optimization budgets. Full fine-tuning consistently outperforms training from scratch across all evaluated datasets, further demonstrating the benefit of source-domain pretraining.

DatasetTrain from scratchEmbedding adaptationFull fine-tuning
AHB0.4088 ± 0.07570.4003 ± 0.01970.4677 ± 0.0781
Brysch0.2094 ± 0.00470.2129 ± 0.02490.2217 ± 0.0254
CaLiAli0.2956 ± 0.00770.2970 ± 0.00820.3109 ± 0.0077
Park0.2467 ± 0.01320.2576 ± 0.01730.2589 ± 0.0157
Atanas0.2291 ± 0.09130.2575 ± 0.10620.2625 ± 0.0938
Copper0.2257 ± 0.11370.2823 ± 0.12980.2848 ± 0.1169
Dataset / behaviorTrain from scratchEmbedding adaptationΔ R2
Atanas / head angle0.5668 ± 0.16150.5918 ± 0.1522+0.0250
Copper / velocity0.7429 ± 0.09290.7503 ± 0.0948+0.0074
Brysch / eye position0.5029 ± 0.16730.5250 ± 0.1503+0.0221

[Q4]:

For a new session with N neurons, forecasting adaptation updates (N + 1)D parameters for the neuron and session embeddings, where D is the model dimension. For decoding, the neuron-specific readout introduces N × B × L × R parameters, where B is the number of behavioral channels, L is the patch length, and R is the decoder rank, together with additional shared decoder parameters.

For example, with D = 512, L = 8, R = 16, B = 1 in C. elegans dataset, the complete forecasting-to-decoding adaptation pipeline optimizes 640N+131,602640N + 131,602 parameters across its two stages, without updating the pretrained backbone. The forecasting and decoding adaptation times on a single-GPU for the Copper C. elegans dataset are reported in the table below.

StageTraining time / epochTime for 200 epochs
Embedding adaptation9.03 ± 0.37 s30.11 min
Behavior head0.73 ± 0.11 s2.44 min

We further report the computational cost of embedding-only adaptation on the same Copper dataset. The results are shown below.

MethodTime / epochPeak memoryTraining TFLOPs / epoch
Embedding-only fine-tuning9.03 ± 0.37 s12.49 GiB430.02 ± 19.28

We sincerely thank the reviewer again for the constructive feedback. We hope our responses have addressed your concerns, and would be happy to provide further clarification if needed.

Thank you for the clarification and additional experiments. My remaining concern is whether residual-last prediction was also controlled in the matched CalM/CAPT comparison. If not, please qualify the attribution to reflect this remaining difference, rather than attributing the entire gain to continuous modeling alone. Overall, I am inclined to raise my score.

AuthorsOpenReview ↗

Further clarification

We sincerely thank the reviewer for acknowledging our rebuttal and additional experimental evidence.

Regarding the matched CalM/CAPT comparison in [W2 & Q2], residual-last prediction is designed for continuous trace prediction and cannot be directly applied to CalM's discrete token prediction. To avoid this confounding factor, neither CAPT nor CalM uses residual-last prediction in this ablation experiment. Both models incorporate dynamic gating and QKV modulation under the same pretraining data, patch length, and backbone training budget. Therefore, the observed performance gains cannot be attributed to residual-last prediction, further supporting the effectiveness of CAPT's continuous autoregressive formulation compared with CalM's discrete modeling framework under these matched settings.

We have incorporated new experimental results and clarify the comparison protocol in the revised manuscript. If the reviewer identifies any important results or clarifications that should be further incorporated into the revised manuscript, we would be glad to make the corresponding revisions.

We sincerely thank the reviewer again for the careful and constructive feedback.

Reviewer_CSiXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper makes use of continuous tokenization, autoregressive end-to-end training, neuron-aware QKV modulation and dynamic residual gating to model calcium imaging data. The authors pretrained their model on an imaging dataset of 8 mice across 286 sessions and evaluated it on held-out data of the same dataset and, importantly, 8 other multi-session datasets from different labs and containing also different animals (other mice, but also larval zebrafish and C. elegans). They report a substantial improvement over previous models in neural population forecasting, behavior decoding (using an extra head) and classification of neuron identities by training an SVM on their learned neuron embeddings, which generalizes to an independently collected, held-out C. elegans dataset.

Critical strengths

The evaluation is broad, covering nine datasets from three species and different labs, two tasks, and several baselines. Throughout experiments, datasets and tasks, this paper’s model shows mostly substantial improvements in performance compared to the baselines. Particularly, the model is effective at transferring learned representations to other species with fundamentally different nervous system organization.

Critical weaknesses
  • No reference for how well frozen transfer performs compared to training on the target data. The results show that frozen CAPT representations transfer to the held-out datasets better than the baselines’ representations do, but it is unclear how they compare to a model trained directly on those datasets. CAPT may substantially outperform the baselines and still fall well short of a CAPT model trained from scratch or fully fine-tuned on the target data. The only related reference in the paper is the Tseng dataset, where held-in sessions (seen during pretraining) can be compared with held-out sessions (frozen backbone, new embeddings). The small gap suggests that frozen transfer loses little within the same dataset, but this compares different animals from the same lab and setup and says little about the heterogeneous transfer datasets. Appendix E (Table 12) compares fine-tuned and in-domain trained POYO+ on the two C. elegans datasets, but this concerns a baseline rather than CAPT, and only decoding. From-scratch or fine-tuned references would clarify the practical value of frozen transfer.
  • Source of the performance gain is unexplained. CAPT deviates from CalM in several entangled ways: simplified autoregressive loss, continuous tokenization, dynamic gating, QKV modulation, residual-last prediction. The latter three components are ablated in Appendix D (Table 10), but only on a small subset of three sessions whose differences are small relative to the reported standard deviations, and the ablation does not include a CalM-style variant. It therefore remains open which of these changes is responsible for the large gains over CalM on the pretraining and transfer datasets. The gain over POCO, which is also continuous, shows the tokenization alone cannot be the full explanation. A more controlled ablation would clarify which design choices matter for representation transfer.
  • NeuroPAL analysis does not necessarily show biological meaning of the learned representations. The paper interprets the cross-dataset classification of neuron identities as evidence that CAPT embeddings capture biologically meaningful structure. However, a shared frozen backbone provides a common coordinate system by construction, and C. elegans neurons are stereotyped across animals, so the cross-dataset alignment may not depend on what was learned in mouse pretraining. Table 15 shows that a shared backbone is necessary, but not that its learned content matters. A control with a randomly initialized frozen backbone would show whether the learned representations matter.

The paper could be a valuable contribution to the field, but the central claims are not yet fully supported by the presented controls. I would be happy to raise my score if the authors address the first three weaknesses.

Things to improve the paper that did not impact the score:

  • This paper would benefit from more motivation for main architectural decisions, particularly the choice of autoregressive objective and the switch from CalM’s discrete tokenization to continuous patches, even though CalM gives explicit arguments for the discrete design.
  • The introduction describes transfer as ‘poorly understood’, ‘less studied’ and ‘underexplored’, but POCO reports negative results for cross-dataset and multi-species pretraining, and POYO+ and CalM reuse a pretrained backbone across sessions and animals for several tasks. The contribution relative to these should be positioned more precisely.
  • It could be mentioned that QKV modulation doesn’t seem to have a substantial impact on forecasting, but on the quality of the embeddings (according to Appendix D (Tables 10 and 11)).
  • It would be helpful to make the differences between CAPT and baselines more clear in related work section or Appendix B
Questions for authors
  • The NeuroPAL analysis uses POCO as the only baseline. CalM has neuron embeddings and is run on both C. elegans datasets (Figure 3E, F). How do CalM embeddings perform in the same cross-dataset classification, and why was CalM not included?
  • Which sessions are used for the ablations in Tables 9 to 11, and does “all components” in Table 10 correspond to the main model?
  • Is the depthwise temporal convolution used in all reported decoding results, including for the baselines?
  • How are the neuron embeddings pooled for the NeuroPAL classification?
5 comments
AuthorsOpenReview ↗

Rebuttal part I

We thank the reviewer for the detailed comments and questions with sufficient explanation.

[W1]:

We additionally conduct from-scratch training and full fine-tuning on the target datasets, using the same CAPT architecture. The results are shown below. Full fine-tuning consistently outperforms training from scratch, supporting the benefit of source-domain pretraining. Meanwhile, our original frozen-backbone adaptation achieves competitive performance without updating the pretrained backbone.

DatasetRandom backboneTrain from scratchEmbedding adaptationFull fine-tuning
AHB0.1900 ± 0.08300.4088 ± 0.07570.4003 ± 0.01970.4677 ± 0.0781
Brysch0.1616 ± 0.02780.2094 ± 0.00470.2129 ± 0.02490.2217 ± 0.0254
CaLiAli0.2074 ± 0.01020.2956 ± 0.00770.2970 ± 0.00820.3109 ± 0.0077
Park0.1804 ± 0.01020.2467 ± 0.01320.2576 ± 0.01730.2589 ± 0.0157
Atanas0.1554 ± 0.08250.2291 ± 0.09130.2575 ± 0.10620.2625 ± 0.0938
Copper0.1937 ± 0.08350.2257 ± 0.11370.2823 ± 0.12980.2848 ± 0.1169
Dataset / behaviorTrain from scratchEmbedding adaptationΔ R2
Atanas / head angle0.5668 ± 0.16150.5918 ± 0.1522+0.0250
Copper / velocity0.7429 ± 0.09290.7503 ± 0.0948+0.0074
Brysch / eye position0.5029 ± 0.16730.5250 ± 0.1503+0.0221

[W2]:

We further perform architectural ablations using models pretrained on the full source dataset and evaluate their performance under the same frozen-backbone transfer protocol.

The simplified continuous CAPT already substantially outperforms CalM across transfer datasets, while the additional architectural components consistently provide further improvements. These results support the effectiveness of our continuous autoregressive formulation and the proposed auxiliary components.

DatasetSessionsCalMCAPT w/o gating, QKV modulation and residual-lastCAPTΔ Corr
AHB30.0952 ± 0.03240.3818 ± 0.01770.4003 ± 0.0197+0.0185
Brysch290.1342 ± 0.02480.1895 ± 0.02460.2129 ± 0.0249+0.0234
CaLiAli30.1119 ± 0.02820.2833 ± 0.00790.2970 ± 0.0082+0.0137
Park30.1333 ± 0.02280.2473 ± 0.01740.2576 ± 0.0173+0.0103
Atanas800.1780 ± 0.08460.2372 ± 0.10410.2575 ± 0.1062+0.0203
Copper270.2075 ± 0.11420.2634 ± 0.11850.2823 ± 0.1298+0.0189

We further conduct matched architectural ablations on CAPT and CalM using the same 12-session subset. CAPT consistently achieves higher forecasting performance across all configurations, further supporting the effectiveness of our continuous autoregressive framework.

ConfigurationCAPTCalM
Baseline0.47430.3567
+ Dynamic gating0.48380.3317
+ QKV modulation0.48710.3688
+ All mechanisms0.49170.3452

For POCO, although it also models continuous calcium signals, the architecture differs substantially from CAPT. POCO combines a POYO-based population encoder with a FiLM-conditioned MLP forecaster, while CAPT uses a dual-axis autoregressive Transformer for continuous patch prediction. The consistent performance gap suggests that continuous modeling alone is insufficient to explain the improvements, and the overall autoregressive architecture also plays an important role.

[W3]:

We further evaluate a shared, randomly initialized frozen backbone on the two C. elegans datasets. As shown below, the pretrained backbone substantially outperforms the random backbone in both within-dataset and cross-dataset NeuroPAL classification, demonstrating that the learned representations provide benefits beyond a shared embedding coordinate system.

pretrained / randomAccuracyBalanced AccMacro F1
Pooled52.44 / 26.5652.47 / 27.0453.40 / 26.19
Atanas69.89 / 41.9770.08 / 41.9870.10 / 41.75
Copper75.70 / 63.8075.66 / 63.7575.75 / 63.68
Atanas → Copper37.03 / 23.8136.45 / 24.4238.14 / 22.12
Copper → Atanas38.76 / 22.5535.35 / 20.5637.23 / 20.83
AuthorsOpenReview ↗

Rebuttal part II

We try to concisely answer weaknesses from 4 to 7.

[W4]:

Our motivation for continuous tokenization is to model calcium traces directly in their original continuous space, without requiring a separately trained discrete tokenizer. In our experiments, we found that continuous tokenization works well and the importance of patch length is demonstrated by the ablation results in Table 9, and CAPT performs much better on transfer tasks empirically.

For the autoregressive objective, we are motivated by its success in language processing and its natural applicability to temporal modeling. We adopt the autoregressive objective to directly model temporal dependencies in calcium traces while supporting flexible forecasting horizons. Our experiments support this design choice, while identifying the optimal architecture for neural data remains an interesting direction for future work.

[W5]:

We thank the reviewer for the suggestion. POCO explored multi-species pretraining and cross-dataset fine-tuning, but did not systematically study frozen-backbone cross-species transfer. POYO+ mainly focused on supervised decoding, while CalM trained on target-domain held-in recordings for its C. elegans experiments, rather than transferring a mouse-pretrained backbone. In contrast, we systematically evaluate whether a single mouse-pretrained, multitask backbone can transfer across independent datasets and species. We will clarify these distinctions in the revised paper.

[W6]:

We thank the reviewer and will clarify this distinction in the revised paper.

[W7]:

We briefly summarize the key methodological differences below. POCO combines a POYO-based population encoder with a FiLM-conditioned MLP for neural forecasting. POYO+ uses a Perceiver IO-based architecture with neuron-specific embeddings and latent cross-attention for supervised behavior decoding. CalM combines discrete neural tokenization with a dual-axis autoregressive Transformer for forecasting and decoding. In contrast, CAPT directly predicts continuous calcium patches autoregressively, with dynamic residual gating and neuron-aware QKV modulation, and evaluates multitask, frozen-backbone transfer across datasets and species.

For weaknesses from 4 to 7, we thank the reviewer for these helpful suggestions and will revise the corresponding discussions to improve clarity and precision.

AuthorsOpenReview ↗

Rebuttal part III

[Q1]:

We originally selected POCO as the self-supervised baseline based on its forecasting performance. However, we agree that CalM is also an important comparator for neuron embedding analysis. We therefore evaluate CalM under the same NeuroPAL classification protocol, as shown below.

CAPT consistently outperforms CalM across all three settings, with a particularly notable improvement in the Copper → Atanas direction. We will include these results in the revised paper.

Setup (CAPT / CalM)Balanced accuracyMacro F1
Pooled0.5247 ± 0.0209 / 0.5099 ± 0.01230.5340 ± 0.0219 / 0.5279 ± 0.0117
Atanas → Copper0.3645 ± 0.0091 / 0.3584 ± 0.01380.3814 ± 0.0084 / 0.3730 ± 0.0149
Copper → Atanas0.3535 ± 0.0079 / 0.2488 ± 0.02030.3723 ± 0.0084 / 0.2630 ± 0.0277

[Q2]:

In Table 10 and 11, we randomly selected three sessions from the Tseng held-out set for pretraining, covering two subjects and three cortical areas, with AM and PM from subject 10, and VISa from subject 6. Table 9 uses 12 sessions from the Tseng held-in set, with one L2/3 session and one L5 session randomly selected from each of six subjects (3, 4, 5, 7, 8, and 9).

The "All components" configuration in Table 10 corresponds to the full CAPT architecture used in the main experiments.

[Q3]:

CAPT and CalM both use depthwise temporal convolution in their decoding heads. For POYO+, we follow its original implementation without adding an extra convolution layer, ensuring that the baseline retains its native decoding architecture.

[Q4]:

We use the learned static neuron embeddings rather than temporally pooled Transformer outputs. For each C. elegans dataset, we adapt neuron and session embeddings while keeping the pretrained backbone frozen.

For pooled classification, neuron embeddings from both datasets are combined with their NeuroPAL labels and split for SVM training and evaluation. For cross-dataset classification, the SVM is trained and selected on one dataset and directly evaluated on the other to prevent potential data leakage.

We sincerely thank the reviewer again for the thoughtful feedback. We hope our responses have addressed your concerns, and would be happy to provide further clarification if needed.

Reviewer_S3dKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CAPT is a transformer that learns from continuous Ca2+ imaging traces by predicting future activity. The authors pretrain it on mouse recordings, then keep the backbone fixed while fitting new embeddings or decoding heads for datasets from mice, zebrafish, and worms. The main contribution is demonstrating that a single pretrained model can support neural forecasting and behavior decoding across different recording conditions and species. Evaluation includes comparisons with neural-data and general-purpose models, ablations, and analyses relating the learned worm embeddings to known cell identities.

Critical strengths

Frozen-backbone transfer exps are strong. A model pretrained on mouse recordings is useful across 8 target datasets in mice, zebrafish, and worms, with adaptation restricted to embeddings and readouts.

NeuroPAL analysis provides a concrete test of what the embeddings capture. Cell-identity classification transfers between the 2 worm datasets and outperforms handcrafted trace statistics. Adapted embeddings may thus carry cell-related information shared across experiments.

Critical weaknesses

My main reservation is that the benefit of mouse pretraining is not isolated. The forecasting and decoding experiments lack a comparison with the same CAPT architecture trained only on target data. The NeuroPAL control in Table 15 addresses a related but different question: independently trained models can learn incompatible embedding coordinates, so their poor cross-dataset classification does not establish how much mouse pretraining contributes. A shared, randomly initialized frozen backbone would help separate the value of a common coordinate system from that of learned source representations. The transfer results are promising, but the contribution specifically attributable to pretraining remains unclear.

The statistical comparisons use sessions as the units of replication, although several datasets contain repeated recordings from the same animals. For example, the 97 held-out Tseng sessions come from only two mice, and the 29 Brysch sessions come from 3 fish. The tests in Table 14 do not account for this dependence. Summarizing improvements and uncertainty by animal would make the evidence for generalization across subjects more convincing.

Questions for authors
  1. What exactly does the “baseline” in Table 10 include, and which changes explain the cross-species gains? Does continuous prediction without the added gating and QKV modulation retain the transfer advantage?
  2. In Table 15, the comparison with independently trained models changes both pretraining and whether the datasets share an embedding coordinate system. What evidence shows that the mouse-trained features matter beyond providing a common fixed mapping into which the target embeddings are fitted?
  3. Appendix B specifies 40 context frames and 24 forecast frames for POCO. Please clarify whether CAPT and CalM receive the same context and are scored on exactly the same future time points and retained trials, particularly for datasets with variable-length trials.
3 comments
AuthorsOpenReview ↗

Rebuttal part I

We really appreciate the thoughtful comments from the reviewer.

[W1 & Q2]:

To directly assess the contribution of source-domain pretraining, we compare mouse-pretrained CAPT with the same architecture trained from scratch on each target dataset. The pretrained model is evaluated using frozen-backbone adaptation, whereas the target-only model is trained with all backbone parameters learnable.

As shown below, frozen-backbone CAPT generally achieves comparable or better performance than target-only training, despite keeping the pretrained backbone fixed. Especially, the largest forecasting improvements are observed on the two C. elegans datasets.

DatasetTrain from scratchEmbedding adaptation
AHB0.4088 ± 0.07570.4003 ± 0.0197
Brysch0.2094 ± 0.00470.2129 ± 0.0249
CaLiAli0.2956 ± 0.00770.2970 ± 0.0082
Park0.2467 ± 0.01320.2576 ± 0.0173
Atanas0.2291 ± 0.09130.2575 ± 0.1062
Copper0.2257 ± 0.11370.2823 ± 0.1298
Dataset / behaviorTrain from scratchEmbedding adaptationΔ R2
Atanas / head angle0.5668 ± 0.16150.5918 ± 0.1522+0.0250
Copper / velocity0.7429 ± 0.09290.7503 ± 0.0948+0.0074
Brysch / eye position0.5029 ± 0.16730.5250 ± 0.1503+0.0221

We also perform the experiment on shared, randomly initialized frozen backbone on the two C. elegans datasets. The pretrained backbone substantially outperforms the random backbone in both within-dataset and cross-dataset NeuroPAL classification, demonstrating that the learned pretrained representations provide benefits beyond a shared embedding coordinate system.

pretrained / randomBalanced AccMacro F1
Pooled52.47 / 27.0453.40 / 26.19
Atanas70.08 / 41.9870.10 / 41.75
Copper75.66 / 63.7575.75 / 63.68
Atanas → Copper36.45 / 24.4238.14 / 22.12
Copper → Atanas35.35 / 20.5637.23 / 20.83

[Q1]:

The baseline in Table 10 is CAPT without dynamic gating, QKV modulation, or residual-last prediction.

Since the original ablation was conducted only on the pretraining dataset, we further evaluate this simplified model under the same frozen-backbone transfer protocol. As shown below, continuous prediction without the added gating and QKV modulation already shows the transfer advantage, and the full CAPT consistently outperforms the simplified model across transfer settings, confirming the benefit of the added components for transfer performance.

DatasetPOCOCalMCAPT w/o auxiliary componentsFull CAPTΔ Corr (Full − w/o auxiliary components)
AHB0.3289 ± 0.01720.0952 ± 0.03240.3818 ± 0.01770.4003 ± 0.0197+0.0185
Brysch0.1872 ± 0.02390.1342 ± 0.02480.1895 ± 0.02460.2129 ± 0.0249+0.0234
CaLiAli0.2296 ± 0.00760.1119 ± 0.02820.2833 ± 0.00790.2970 ± 0.0082+0.0136
Park0.1908 ± 0.01410.1333 ± 0.02280.2473 ± 0.01740.2576 ± 0.0173+0.0103
Atanas0.2058 ± 0.10750.1780 ± 0.08460.2372 ± 0.10410.2575 ± 0.1062+0.0203
Copper0.2161 ± 0.10700.2075 ± 0.11420.2634 ± 0.11850.2823 ± 0.1298+0.0188

[Q3]:

We clarify that CAPT, CalM, and POCO use the same 40-frame context and 24-frame forecasting horizon in the standard transfer settings. All methods are evaluated on the same retained trials and future time points.

On the pretraining dataset, CAPT and CalM are originally evaluated using autoregressive rollout over the remaining trial, without a fixed forecasting horizon.

For a fair comparison, we additionally evaluate CAPT and CalM over the same 24-frame forecasting horizon as POCO. CAPT maintains a substantial advantage over both baselines on Tseng held-in and held-out sessions.

DatasetSessionPOCO Corr₍₂₄₎CalM Corr₍₂₄₎CAPT Corr₍₂₄₎
Tseng held-in1890.3668 ± 0.06050.3577 ± 0.08800.4957 ± 0.0647
Tseng held-out970.3322 ± 0.05110.3094 ± 0.07650.4311 ± 0.0605
AuthorsOpenReview ↗

Rebuttal part II

[W2]:

We agree that sessions from the same animal should not be treated as independent observations. We therefore aggregate session-level results within each animal and perform two-sided paired t-tests with Holm correction at the animal level. The updated results shown below (mean ± SD across animals), indicating that CAPT maintains strong performance across datasets, with statistically significant improvements in most comparisons. We will update the statistical analysis in the paper accordingly.

Result (animals)POCO / POYO+CalMCAPTp(vs. POCO / POYO+)p(vs. CalM)
Tseng held-in forecasting corr(n=6)0.3654 ± 0.03380.3783 ± 0.03210.5151 ± 0.02985.477 × 10⁻⁵ (***)7.768 × 10⁻⁶ (***)
Tseng held-out forecasting corr(n=2)0.3331 ± 0.02410.3377 ± 0.01590.4676 ± 0.0017——
Sun forecasting corr(n=13)0.2552 ± 0.03090.2603 ± 0.05410.3347 ± 0.04639.955 × 10⁻⁷ (***)2.335 × 10⁻¹⁰ (***)
Brysch forecasting corr(n=3)0.1882 ± 0.00630.1354 ± 0.00530.2149 ± 0.00640.0633 (n.s.)0.0139 (*)
Copper forecasting corr(n=27)0.2161 ± 0.10700.2075 ± 0.11420.2823 ± 0.12980.0048 (**)0.0043 (**)
CaLiAli forecasting corr(n=3)0.2296 ± 0.00760.1119 ± 0.02820.2970 ± 0.00820.0596 (n.s.)0.0370 (*)
Park forecasting corr(n=3)0.1908 ± 0.01410.1333 ± 0.02280.2576 ± 0.01730.0124 (*)0.0596 (n.s.)
Landmark forecasting corr(n=6)0.3968 ± 0.03030.1795 ± 0.05350.4645 ± 0.03582.345 × 10⁻⁴ (***)1.391 × 10⁻⁵ (***)
aHB forecasting corr(n=3)0.3289 ± 0.01720.0952 ± 0.03240.4003 ± 0.01970.1402 (n.s.)0.0533 (n.s.)
Atanas forecasting corr(n=80)0.2058 ± 0.10750.1780 ± 0.08460.2575 ± 0.10626.692 × 10⁻¹¹ (***)1.412 × 10⁻¹² (***)
Tseng held-in decoding pitch R²(n=6)0.7227 ± 0.05350.7835 ± 0.05080.8444 ± 0.03740.0011 (**)0.0036 (**)
Tseng held-in decoding roll R²(n=6)0.6723 ± 0.05840.7477 ± 0.04290.8013 ± 0.02840.0046 (**)0.0049 (**)
Tseng held-in decoding yaw R²(n=6)0.5868 ± 0.11360.7061 ± 0.06120.7307 ± 0.05370.0247 (*)0.0124 (*)
Tseng held-out decoding pitch R²(n=2)0.7086 ± 0.03450.7332 ± 0.06260.8136 ± 0.0597——
Tseng held-out decoding roll R²(n=2)0.6405 ± 0.11150.6937 ± 0.11800.7714 ± 0.0691——
Tseng held-out decoding yaw R²(n=2)0.5393 ± 0.10030.5993 ± 0.08050.6367 ± 0.0477——
Sun decoding R²(n=13)0.8584 ± 0.07370.9519 ± 0.03430.9572 ± 0.02496.129 × 10⁻⁴ (***)0.2411 (n.s.)
Brysch decoding R²(n=3)0.2520 ± 0.19650.3512 ± 0.05510.5066 ± 0.05870.2411 (n.s.)0.0247 (*)
Copper decoding velocity R²(n=27)0.2773 ± 0.21490.5882 ± 0.22130.7503 ± 0.09489.567 × 10⁻¹⁴ (***)0.0036 (**)
Atanas decoding head angle R²(n=80)−0.0124 ± 0.13490.0896 ± 0.27290.5918 ± 0.15226.380 × 10⁻⁴⁴ (***)1.994 × 10⁻³⁴ (***)

We sincerely thank the reviewer again for the constructive feedback. We hope our responses have addressed your concerns, and would be happy to provide further clarification if needed.

AuthorsOpenReview ↗

Further clarification

We sincerely thank the reviewer for the positive assessment of our additional experiments.

In our previous animal-level Tseng analysis, CAPT and CalM were evaluated using the original variable-length autoregressive rollout, whereas POCO was evaluated using a fixed 24-frame forecasting horizon. We apologize for not making this distinction clear.

To ensure a fair comparison across all three methods, we have recomputed the animal-level statistics for CAPT and CalM using the same 40-frame context and 24-frame forecasting horizon as POCO. For each animal, we average the session-level correlations computed over the matched 24-frame predictions, and then calculate the mean, standard deviation, and statistical tests across animals. The updated results are shown below.

Result (animals)POCOCalMCAPTp (vs. POCO)p (vs. CalM)
Tseng held-in Corr24 (n=6)0.3654 ± 0.03380.3572 ± 0.03450.4963 ± 0.03196.114 × 10⁻⁵ (***)1.110 × 10⁻⁵ (***)
Tseng held-out Corr24 (n=2)0.3331 ± 0.02410.3094 ± 0.00010.4317 ± 0.0163——

We have incorporated these matched-horizon results, together with the additional experimental evidence from our rebuttal, into the revised manuscript. If the reviewer identifies any important results or clarifications that should be further incorporated into the revised manuscript, we would be glad to make the corresponding revisions.

We sincerely thank the reviewer again for the careful and constructive feedback.

Comments on the paper

2 comments

Dear Reviewer,

Thank you for your time and constructive feedback. As the discussion deadline approaches, we would greatly appreciate it if you could review our rebuttal and additional experimental results, which directly address your main concerns.

We would welcome any further feedback or questions and sincerely appreciate your consideration.

Best regards, The Authors

AuthorsOpenReview ↗

Summarization of the paper and rebuttal

Dear AC and Reviewers,

We sincerely thank you for the time, careful reading, and constructive engagement throughout the discussion. We are grateful to the reviewers for their positive reassessment of our work and their willingness to raise their scores.

[Briefing]: CAPT is a continuous autoregressive Transformer for calcium population dynamics. After pretraining on a large-scale mouse dataset, its frozen backbone transfers to independent mouse, larval zebrafish, and C. elegans datasets, supports both neural forecasting and behavior decoding, and yields embeddings that carry cell-identity-related information.

[Strength]: Reviewers recognized the strong empirical performance of CAPT, breadth of our evaluation, the strength of frozen-backbone transfer, the value of the NeuroPAL analysis, and the clarity of the presentation.

[Rebuttal]: We addressed the reviewers' main concerns with additional controls. Comparisons with a random frozen backbone, training from scratch, and full fine-tuning confirm the contribution of source-domain pretraining. Surrogate-pretraining and neuron-attention ablations show that CAPT benefits from population-level structure beyond generic calcium signal statistics. Matched ablations clarify the role of the continuous formulation and the auxiliary components. Animal-level statistics and matched-horizon evaluations further support the robustness and fairness of our comparisons.

We have incorporated these results into the revised manuscript and refined the text accordingly. If the reviewers or AC have further suggestions on this article, we will be glad to incorporate them in the final version.

We sincerely thank the AC and all reviewers again, and we wish everyone all the best !

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.