- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BiXformer, a cross-attention transformer that separates inter-regional neural dynamics into leading and lagging streams using temporal attention masks and an orthogonal decoder. The model is evaluated on synthetic data for latent and delay recovery, and on real neural-behavioral and S1–M1 recordings to demonstrate its ability to identify interpretable temporal interactions.
Critical strengths
-
Interesting approach: Separating leading and lagging interactions using masked cross-attention and orthogonal decoding is a useful contribution. The synthetic results show promising performance in recovering latent dynamics and temporal delays.
-
Real-data applications: I appreciate the evaluation on both neural-behavioral recordings and simultaneous S1–M1 recordings. These demonstrate the potential of the method across different types of data and provide interpretable results. The presentation of these experiments is also clear.
Critical weaknesses
-
Limited synthetic validation: One of the main motivations for BiXformer is overcoming the linearity and stationarity assumptions of existing methods such as DLAG. However, the synthetic data are generated using stationary Gaussian processes and linear mixing. I would expect experiments with nonlinear and nonstationary dynamics to demonstrate the claimed advantages. It is also unclear why DLAG performs substantially worse on data that seem compatible with its assumptions.
-
Insufficient baseline comparisons: All baseline comparisons are limited to one synthetic dataset. Important related methods such as MR-SDS (ICLR 2024), MRDS-IR (ICLR 2025), and GLM-Transformer are not included in the experiments. Quantitative comparisons on the real datasets are also missing. These comparisons would help establish the advantages of BiXformer over existing methods.
-
Interpretation of directionality: Both streams reconstruct the target population using past or future activity from the source population. While this identifies temporal relationships, it does not necessarily establish physiological communication direction. This is particularly concerning for the S1–M1 experiment, where the attention profiles peak at zero lag (Appendix D.6), yet the learned components are interpreted as S1→M1 and M1→S1 communication.
-
Limited validation of the architectural constraints: The Stiefel decoder imposes orthogonality between the two streams, but it is unclear whether this assumption is appropriate for neural populations with potentially overlapping feedforward and feedback representations. The existing ablation focuses on reconstruction accuracy and removes both constraints together. It does not demonstrate how each constraint contributes to recovering the true latent streams and delays.
Questions for authors
-
Given the zero-lag attention profiles in the S1–M1 experiment, what evidence supports interpreting the two streams as separate communication directions?
-
Are the estimated sensory and motor delays in Section 4.2 consistent with previous physiological measurements? How sensitive are these estimates to temporal smoothing?
Update: Most of my concerns were addressed and authors performed valuable experiments during rebuttal hence I increased my score from 2 to 3 at the end of rebuttal session.
20 comments
Response to Reviewer ayzr (2/4)
W2
All baseline comparisons are limited to one synthetic dataset. Important related methods such as MR-SDS (ICLR 2024), MRDS-IR (ICLR 2025), and GLM-Transformer are not included in the experiments.
- We thank the reviewer for these references. We will revise the Related Work section to discuss MR-SDS (Karniol-Tambour et al., 2024) and MRDS-IR (Dowling & Savin, 2025). The methods in our original comparison were chosen to span the two properties BiXformer combines: DLAG, which estimates delays explicitly but assumes linear and stationary interactions, and MR-LFADS, which captures nonlinear dynamics but does not estimate delays.
- Following the reviewer's suggestion, we have added MRDS-IR (Dowling & Savin, 2025) to the model comparison (see table below). Although MRDS-IR does not explicitly model time delays, a delay estimate can be read post hoc from the peak of its learned cross-region impulse response filters. In both directions (X → Y and Y → X) and in all five cross-validation folds, these filters peaked at 5 ms. We report the resulting delay MAE in the table, noting that this readout is ours and that MRDS-IR was not designed to explicitly model delays. Like MR-LFADS, MRDS-IR is a recurrent state-space model, which likely explains its longer training time relative to BiXformer. The table also includes ADM (Li et al., 2025), which builds on DLAG's time-delay kernel by allowing delays to vary within a trial, and reduces training time by approximating each GP latent as a state-space model. The comparison now also spans a nonlinear and a time-varying dataset (see our response to W1). We will add this table to the appendix of the revised manuscript, together with a full description of each model's training procedure and hyperparameters. No model's hyperparameters, including BiXformer's, were tuned for these comparisons.
- We did not include MR-SDS or GLM-Transformer, for the following reasons. MR-SDS adds switching between discrete dynamical regimes to multi-region nonlinear latent dynamics. Our main synthetic datasets contain no contextual changes that would engage this switching, so we expect MR-SDS to perform similarly to the nonlinear dynamical models already in the comparison (MRDS-IR and MR-LFADS). GLM-Transformer models cross-area communication through a stationary GLM coupling term, a property shared with methods already in our comparison (DLAG, MRDS-IR). It also uses a Poisson likelihood on spike counts, so it cannot be applied to our continuous-valued synthetic data without modification. We are happy to run either method if the reviewer would find it useful, although given the limited time and the reasons above, we hope the current comparison is sufficient.
| Method | R² (mean ± std) | Delay MAE (ms) | Train time / fold (min) |
|---|---|---|---|
| RRR (r=4) | 0.341 ± 0.005 | N/A | N/A |
| DLAG | 0.590 ± 0.074 | 13.5 ± 8.9 | 93.9 ± 3.9 |
| ADM | 0.591 ± 0.017 | 19.3 ± 1.2 | 14.4 ± 0.2 |
| MRDS-IR | 0.842 ± 0.001 | 30.0 ± 0.0 | 50.72 ± 6.15 |
| MR-LFADS | 0.865 ± 0.001 | N/A | 78.82 ± 2.85 |
| BiXformer | 0.887 ± 0.018 | 1.0 ± 2.0 | 0.44 ± 0.01 |
Quantitative comparisons on the real datasets are also missing.
- We thank the reviewer for this suggestion. Since the real datasets have no ground truth for latents or delays, we compared predictive performance: we applied MR-LFADS to the neural–behavioral dataset and report its held-out R² and training time alongside BiXformer (table below). BiXformer achieved a higher held-out R² and trained far faster.
- We limited this to one method because MR-LFADS and the other methods take far longer to train than BiXformer (see the synthetic-data comparison table), which makes running all of them infeasible within the rebuttal period. These models are also sensitive to hyperparameters, and we did not run an extensive search, so the MR-LFADS R² is likely an underestimate. Please let us know whether this is sufficient.
| Method | R² (mean ± std) | Train time / fold (min) |
|---|---|---|
| MR-LFADS | 0.434 ± 0.041 | 224.8 ± 10.6 |
| BiXformer | 0.656 ± 0.013 | 1.40 ± 0.04 |
Response to Reviewer ayzr (3/4)
W3 and Q1
Both streams reconstruct the target population using past or future activity from the source population. While this identifies temporal relationships, it does not necessarily establish physiological communication direction. This is particularly concerning for the S1–M1 experiment, where the attention profiles peak at zero lag (Appendix D.6), yet the learned components are interpreted as S1→M1 and M1→S1 communication.
Given the zero-lag attention profiles in the S1–M1 experiment, what evidence supports interpreting the two streams as separate communication directions?
- We thank the reviewer for raising this point, and we agree: the two streams capture components of the target population explained by past versus future source activity, and this temporal ordering alone does not establish physiological communication direction. This holds for all methods that infer communication from observational recordings; establishing direct coupling would require perturbing specific pathways, which we view as important future work.
- To address the zero-lag concern, we retrained BiXformer with strictly directional masks (zero lag excluded from both streams) and compared the results with the original model (rebuttal_figure4). Held-out R² was essentially unchanged, and the PC1 time course of each stream was nearly identical between the two models. The decomposition therefore does not depend on the shared zero-lag bin.
- We also performed a condition-matched trial-shuffled control on the neural–behavioral dataset, pairing behavior from each trial with neural activity from a different trial of the same condition. Both reconstruction and directional structure collapse under the shuffle (rebuttal_figure1), indicating that the recovered streams depend on trial-specific correspondence between the two signals and not on task-locked responses alone. This provides further support for a communication channel.
- In light of this, the model's design and these controls support interpreting the two streams as separate communication channels. This does not unequivocally prove that communication is present, but it indicates a putative communication channel in each direction. We hope this addresses the reviewer's concern. If the reviewer would like to see additional analyses, we would be happy to include them.
Response to Reviewer ayzr (1/4)
We thank the reviewer for the encouraging and helpful feedback. We respond to each point below. We hope these responses address the reviewer's concerns, and that the reviewer will consider revisiting their score.
The rebuttal figures referenced below (rebuttal_figure1–5) are in the rebuttal figures folder of our anonymous repository: https://anonymous.4open.science/r/BiXformer-EB1B/
W1
We thank the reviewer for this feedback. To address it, we ran the model comparison on two additional synthetic datasets, which we will include in the appendix:
Nonlinear mixing (our synthetic dataset with ReLU mixing). This dataset uses the same GP-driven latents and ground-truth delays as our main synthetic dataset, but maps them to each population through a ReLU MLP instead of a linear projection. The latent dynamics remain linear; the relationship between the latents and the observed activity, and therefore between the two populations, is nonlinear, which violates the linear assumptions of methods such as DLAG. BiXformer achieves the highest held-out R² and the lowest delay error (see table below).
We view nonlinear mixing (as proposed by the reviewer) as the central test of inter-regional communication methods beyond the stationary-linear regime, because the complexity and nonlinearity of the signals passed between regions is what varies across brain areas and tasks, and because DLAG and related linear-Gaussian methods cannot accommodate it. The reviewer may additionally have intended to ask about nonlinear latent dynamics, which this benchmark does not test. Benchmarking on nonlinear latent dynamics probes a separate question, the form of within-region temporal structure, that is distinct from the inter-regional communication claim of the submitted model. We expect BiXformer's architecture to extend to nonlinear latent dynamics because its attention mechanism carries no linear-Gaussian prior on latent structure. If the nonlinear mixing benchmark does not fully address the reviewer's concern and the reviewer would like an additional benchmarking dataset, we would be glad to run it.
Please let us know if these comparisons are sufficient to alleviate the reviewer's concerns, or if additional benchmarking (time permitting) is necessary.
Regarding DLAG's poor performance, the delays and GP timescales in our synthetic dataset were sampled independently from a uniform distribution over a biologically plausible range; we will revise Appendix B.1 to state this explicitly. As a result, some of our cross-area latents had a delay/timescale ratio of less than or equal to 1. Under DLAG's squared exponential kernel, the cross-area covariance in this regime changes little between zero lag and the true delay, so the likelihood is nearly flat around the true delay and EM has little gradient to recover it. Although DLAG's log-likelihood and delay estimates converged in all folds (rebuttal_figure3), this flatness limits the precision of its delay estimates and, therefore, its prediction of the target population.