acceptodds
Under review as a conference paper at ICLR 2027

Inter-Regional Communication Benchmark '26: Simulated Datasets and Evaluation Metrics for Multi-Regional Models of Neural Population Activity

Abstract

Multi-region neural recordings have motivated a growing family of data-driven models that aim to infer the pathways and content of inter-regional communication. Evaluating these models is challenging because communication is typically unobserved, and accurate reconstruction of observed neural activity does not imply accurate recovery of unobserved communication. Model validation has therefore largely relied on synthetic datasets with ground-truth communication. However, due to the lack of standardized datasets and evaluation metrics, each existing model has been validated in a unique testbed, often designed alongside the model itself. This fragmented approach precludes systematic comparison across models and risks obscuring models' capabilities and limitations. To address this gap, we introduce a benchmark comprising 42 synthetic multi-region datasets and a suite of evaluation metrics for rigorous assessment of communication models. We demonstrate the benchmark on two communication models, revealing model limitations and conditions that impact model performance. Altogether, we believe this resource will stimulate and guide model development, and support scientific interpretation of large-scale multi-region neural recordings.

Primary area:
datasets and benchmarks
Keywords:
Computational Neuroscience, Multi-Region Neural Population Activity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_B1S2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces IRCB '26, a standardized benchmark for evaluating multi-region neural communication models. The gap they address is that modles published in this field all introduce thier own syntehtic dataset tailored to their specific model, preventing systematic comparison and potentially hiding failure modes. The benchmark contributes 42 synthetic datasets organized into three families of data-generating networks (DGNs): DELAYED-MEMORY, RELAY-DECISION, and a COGNITIVE TASK SUITE of 40 networks trained on 20 cognitive tasks adapted from Yang et al. 2019. Each DGN is a set of interconnected vanilla RNNs where each RNN represents a brain region. The benchmark also introduces a suite of evaluation metrics that separately assess neural activity reconstruction, effectome recovery, and communication fidelity and specificity. The authors demonstrate the benchmark on two communication models, reduced-rank regression (RRR) and MR-LFADS, revealing distinct limitations of each. Datasets, code, and evaluation scripts are publicly released.

Critical strengths
  1. Addresses a genuine and important gap. There is currently no standardized benchmark for multi-region communication models, and this fragmentation is a real problem for the field. The paper is well-placed in the literature and the motivation is compelling.

  2. The separation of evaluation metrics into activity reconstruction, effectome recovery, and communication fidelity/specificity is conceptually important and practically useful. The paper demonstrates convincingly that these can dissociate, which means existing single-metric evaluations may be misleading.

Critical weaknesses
  1. All DGNs are continuous-time vanilla RNNs. Models like mp-srSLDS assume the underlying dynamics switch between discrete linear regimes — a vanilla RNN DGN won't naturally expose that model's failure modes under regime changes. Gaussian process models like DLAG, MRM-GP1 and ADM2 make very different assumptions about temporal smoothness. The benchmark may be inadvertently tailored to the assumptions of continuous nonlinear dynamical models (like MR-LFADS) than to GP-based or switching models, which could bias which models look good on this benchmark.

  2. The claim is too broad. The paper claims to provide evaluation metrics that "standardize evaluations" but the benchmark specifically targets three failure modes. It does not cover:

  • Failure modes specific to Gaussian process or switching dynamical systems models
  • Non-stationarity and session-to-session drift
  • Robustness to model hyperparameter choices
  • Scaling to many regions
  • Closed-loop sensorimotor dynamics (acknowledged in limitations)
  • Heterogeneous region sizes
  • Models' ability to handle different timescales across regions The benchmark is a useful step toward standardization, but calling it a standardization of evaluations generally is an overreach. A more accurate framing would be "a benchmark targeting specific classes of failure modes in specific communication models."
  1. Evaluation metrics: A correlation coefficient or normalized RMSE might be more informative rather than cosine similarity for effectome. Also the measure for communication fidelity is linear and a non-linear measure could be more useful here. a model that encodes the communication content in a highly nonlinear representation could look artificially poor. The related work section mentions the main model families but does not systematically go through each model's synthetic dataset and say "this model used X type of dataset and evaluated with Y metric, and our benchmark covers/doesn't cover this." That kind of structured comparison would be expected in a benchmark paper and would also justify the claim that existing testbeds are insufficient. The limitations section touches on this but it's not done at the level of rigor I would expect. The paper covers most of what the community uses and adds the effectome framework. One thing notably absent from the benchmark is subspace alignment (principal angles between inferred and ground-truth communication subspaces), which is central to how RRR and DLAG-family models think about communication.

In my opinion a strong benchmark paper should:

  • Apply the benchmark to a representative set of existing models (not just 2)
  • Show which models fail on which dataset conditions and explain why based on their design choices
  • Compare the DGN structures to those used in each prior model's own paper and show the benchmark is a superset or a more challenging version

Minor weaknesses:

  • There are some missing related work from other family of models that authors did not mention: MRM-GP1, ADM2, CREIMBO3, CTAE4

  • The authors should mention that their data is solely spike rate and counts, real neural recordings often include LFP, multiunit activity, and calcium imaging signals, none of which are represented here.

  • Missing citations for restrictions of existing synthetic datasets, around line 57. Authors need to cite specific papers whose datasets exemplify these limitations.

References

1: MRM-GP (ICML 2024): Multi-Region Markovian Gaussian Process: An Efficient Method to Discover Directional Communications Across Multiple Brain Regions

2: ADM (ICML 2025): Learning Time-Varying Multi-Region Brain Communications via Scalable Markovian Gaussian Processes

3: CREIMBO (ICLR 2025): Cross-Regional Ensemble Interactions in Multi-view Brain Observations

4: CTAE (ICLR 2026): Coupled Transformer Autoencoder for Disentangling Multi-Region Neural Latent Dynamics

Questions for authors
  1. Why were DLAG, mDLAG, mp-srSLDS, and MR-SDS not benchmarked? What prevented their inclusion?

  2. How does the benchmark scale to larger numbers of regions, and was computational cost a factor in the design choices?

  3. How do the authors justify that vanilla RNN DGNs expose failure modes relevant to switching or GP-based models, given these models have fundamentally different inductive biases?

7 comments

We thank the Reviewer for the detailed and expert review. We were encouraged that the Reviewer found that our paper “addresses a genuine and important gap” and that our benchmark is “conceptually important and practically useful.” Below we respond sequentially to each of the Reviewer’s points. Following the Reviewer’s suggestions, we are benchmarking additional communication models, adding evaluation metrics, analyses and data-generating networks, and systematically comparing our benchmark with prior testbeds. Some of this work is still under way as we provide an initial response to Reviewers to encourage discussion. We will report conclusively on these ongoing threads at the end of the discussion period.

(A) Model-class bias

The Reviewer was concerned that, because every data-generating network (DGN) is a vanilla RNN, the benchmark may be inadvertently tailored to continuous nonlinear models such as MR-LFADS, and asked (Q3) how we might expose failure modes of other communication model classes built on switching or Gaussian-process (GP) dynamics. We agree that the benchmark should include DGNs that match the assumptions of switching and GP models, and we are therefore adding data-driven DGNs of both kinds. We are now fitting these models to real brain-wide electrophysiology recordings in mice performing a sensorimotor memory-guided decision task [1]. We will then use each fitted model as a DGN to create synthetic datasets that capture features of real neural data, with communication known by construction:

  • Switching DGNs, from mp-srSLDS [2] and MR-SDS [3] fit to these recordings, matching the state-dependent switching dynamics these models assume.
  • GP DGNs, from ADM [4] fit to these recordings, matching the temporal smoothness assumed by DLAG [5], mDLAG [6] and ADM.

Original rationale for RNN DGNs. RNNs make relatively few assumptions (e.g., compared to switching linear or GP models) and are known to be universal function approximators [7]. As such, RNNs can be trained to reproduce the dynamics of a switching linear network or a network with GP latents. The reverse is not true in general.

Empirically, we indeed observe switching behavior in our Cognitive Task Suite DGNs. These DGNs’ dynamics switch operating regimes at the go cue, when transitioning from the delay to response epochs. For example, in DGNs trained on multisensory decision-making, the leading principal axis of the DGN dynamics rotates by about 60° across the go cue, and linear dynamics fit within one phase (R2>0.83R^2 > 0.83) transfer poorly to the other (cross-phase R2R^2 as low as 0.28).

Timeline. Revised paper (at close of discussion period): the switching and GP DGNs fit to the Chen et al. recordings, with their fitting procedures and reconstruction quality, and the empirical switching analysis. Camera-ready version: synthetic datasets sampled from these DGNs, with code and benchmark results (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

(B) The claim is too broad

The Reviewer argued that our claim, that our evaluation metrics support “standardized evaluations,” is too broad. We now replace “standardized” with “consistent” in this usage. Elsewhere, we use “standardized” in its conventional sense for machine-learning benchmarks: a fixed evaluation protocol that makes results comparable across models, as in RobustBench [8], the Neural Latents Benchmark [9] and BrainWideBench [10], rather than coverage of every condition a model may face.

We appreciate the Reviewer’s point that our benchmark does not yet cover many challenging conditions, and we thank the Reviewer for identifying them. Acknowledging that no benchmark can possibly cover all conditions that may be encountered, we now expand our benchmark to cover many of the conditions that the Reviewer surfaced.

  • We are fitting the benchmarked communication models to brain-wide electrophysiology recordings in mice performing a sensorimotor memory-guided decision task [1] (also in response to Reviewer g3JZ), and we will release a preprocessing pipeline that transforms these publicly available data into the same format as our original released datasets, so that every model fit to them can be evaluated using our benchmark evaluation pipeline. These data bring:
    • Non-stationarity
    • Sensorimotor dynamics
    • Heterogeneous region sizes
    • Different timescales across regions
  • Because ground-truth communication is not available in real recordings, evaluations on these data focus on reconstruction quality. To bring ground-truth communication to these data, the data-driven DGNs described in (A) add:
    • GP and switching dynamical models
  • Following Reviewer vAUC’s suggestion, we have trained a new multitask DGN that performs all 20 Yang et al. [11] cognitive tasks, and we will add multitask DGN variants that vary in which regions receive inputs and produce readouts, the number of regions, the number of units in each region, the recurrent unit type and the noise level. These variants add:
    • Robustness to (DGN) model hyperparameter choices
    • Heterogeneous region sizes

We also appreciate the Reviewer’s suggestion that an appropriate framing would be “a benchmark targeting specific classes of failure modes in specific communication models.” We will adopt this description.

Timeline. Revised paper (at close of discussion period): the Reviewer’s framing, fits to the real recordings with reconstruction results, and the new multitask DGN. Camera-ready version: the data-driven DGNs and multitask DGN variants (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

(C) Evaluation metrics

The Reviewer suggested a correlation coefficient to complement cosine similarity for effectome evaluation, a nonlinear measure of communication fidelity, and subspace alignment. Following these suggestions, we are adding three metrics:

  • Effectome recovery: Pearson correlation, alongside cosine similarity.
  • Nonlinear communication fidelity: Nonlinear decoding of the true messages, fit and scored on separate trials, as a secondary metric, comparing kernel ridge regression and a small multilayer perceptron as the decoder. We agree that a model encoding the right content in a nonlinear form “could look artificially poor” under a linear measure. Linear decoding remains the primary score, because the form of a message matters as well as its content: the same information arriving as a pulse or as a step drives the receiving region differently, so a model that recovers the content but not the form may infer different dynamics in that region.
  • Subspace alignment: Principal angles between inferred and ground-truth communication subspaces, computed within the high-variance subspace of source-region activity, starting with the models whose messages are linear projections of that activity (observed neurons for RRR [12], inferred rates for MR-LFADS [13]). For switching latent-state models, whose messages are functions of source-region latents, we will map the message into source-neuron space through the model’s emissions: for mp-srSLDS [2] the message is linear, giving an exact subspace per discrete state; for MR-SDS [3] it is nonlinear, so we will use the dominant subspace of its Jacobian, averaged over inferred states. For GP-family models [4–6], no explicit messages are defined, but, as for RRR, the fitted parameters specify the source-neuron subspace that predicts target-region activity.

Timeline. Revised paper (at close of discussion period): all three metrics for the benchmarked models, with subspace alignment for RRR and MR-LFADS. Camera-ready version: subspace alignment for the remaining models.

(D) Prior testbeds

The Reviewer asked for a structured comparison of the synthetic datasets and metrics used to validate each prior model, and for the limitations of those existing approaches. We agree that this would be valuable, and to address this, we are adding appendix tables providing this information, showing that our benchmark is largely a superset of prior testbeds. In particular, prior testbeds typically lack two features central to ours: dynamics that perform behaviorally relevant computations, and unobserved influences, such as external inputs and unrecorded regions, that can confound recovery of communication structure. We will summarize these tables in the main text alongside the requested citations.

Timeline. Revised paper (at close of discussion period): the appendix tables, their summary in the main text, and the citations.

(E) Expand benchmarked models

The Reviewer asked why DLAG, mDLAG, mp-srSLDS and MR-SDS were not benchmarked (Q1). In response, we are extending the set of benchmarked communication models to include mp-srSLDS [2], MR-SDS [3], ADM [4], DLAG [5] and mDLAG [6]. We do not plan to include MRM-GP [14] because its released codebase supports only two regions. Our benchmark code will allow MRM-GP to be benchmarked in future work should its authors or others further develop MRM-GP to support >2 regions.

Timeline. Revised paper (at close of discussion period): results on four core datasets, namely Delayed-Memory, Relay-Decision, a new multitask network trained to perform all 20 Yang et al. [11] cognitive tasks (per Reviewer vAUC), and the public brain-wide Neuropixels electrophysiology recordings of Chen et al. [1] (per Reviewer g3JZ). Camera-ready version: results on all benchmark datasets (fitting and tuning now seven benchmarked models to more than 40 datasets requires more compute than the discussion period allows).

(F) Scaling and computational cost

The Reviewer asked how the benchmark scales to more regions and whether computational cost shaped the design (Q2). Computational cost did shape the design: we kept to four or fewer regions in our DGNs so that every communication model could be fit and appropriately tuned within a practical compute budget. The deep learning models are the main bottleneck, with MR-LFADS and MR-SDS taking 24–72 hours to fit a single model on a given benchmark dataset, and hyperparameter tuning often meant fitting tens or hundreds of variants of each communication model on each dataset.

The benchmark itself is not limited to four regions: the DGN training, communication model training, data format and evaluation pipeline accommodate any number of regions. However, the cost of fitting communication models does scale with the number of regions. The new multitask DGN variants described in (B) include DGNs with more regions (likely 6–8), which will show how each model’s accuracy and fitting cost scale.

Timeline. Revised paper (at close of discussion period): timing and tuning details for every benchmarked model. Camera-ready version: scaling information enabled by multitask DGN variants with >4 regions (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

Reviewer_vAUCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Inter-Regional Communication Benchmark '26, a benchmark designed to assess how well models of inter-regional communication in the brain can reconstruct activity, recover true causal structure, and stay robust to partial observability. To provide a testbed with known ground truth, they train multi-region RNNs with synthetically generated inter-region connectivity structure on a variety of different tasks, and propose a number of metrics for assessing model quality, spanning neural activity reconstruction, effectome reconstruction, and communication content recovery. They compare two commonly used methods against this benchmark, revealing substantial differences in their ability to recover the underlying causal connectivity structure.

Critical strengths

The paper is well written and easy to follow, and properly motivates the need for assessing model recovery of inter-region connectivity and communication in a setting where the ground truth is known. The tasks are designed in a thoughtful way to isolate interpretable failure modes that these methods may suffer from.

Critical weaknesses

A critical weakness of the current task suite is that all ground truth networks seem to be structured in a way that makes each region's function fully distinct/modularized. This is a strong assumption that, as the authors note in section 3.4, is not always justified. They go on to say that introducing the suite of networks trained on cognitive tasks addresses the counterfactual, but I don't see how it does, because in the cognitive tasks suite, the trained networks still have clear individualized roles assigned to each region (one region receives fixation cue, one receives stimulus, one is the readout, etc.)? Intuitively, this assumption should make this task of inferring interregion structure easier than it realistically should be assumed to be a priori in actual brain recordings.

In terms of the benchmarked methods in this paper (reduced-rank regression and MR-LFADS), its not clear in the exposition as to whether these models were fit on the same form of observations; I'm assuming the RRR models were fit to firing rates, and the MR-LFADS got Poisson observations? Are there any other differences? If so, is the comparison here necessarily fair?

Overall my assessment is borderline, but I would be open to raising my score if these concerns can be addressed.

Questions for authors
  • Why were networks trained individually on each cognitive suite in the Yang et al. multitask task suite, as opposed to using that task suite to train multiregion networks designed to do many tasks? I think the latter is more consistent with what one should expect in brain function, while using single-task networks might make the problem artificially easier than it deserves to be?
4 comments

We thank the Reviewer for the constructive comments and for being open to revisiting the assessment. We were encouraged that the Reviewer found our tasks “designed in a thoughtful way to isolate interpretable failure modes.” Below we respond sequentially to each of the Reviewer’s points. Following the Reviewer’s suggestions, we are training new multitask networks, analyzing how distributed the computation in the existing networks is, and clarifying how the benchmarked models were fit. Some of this work is still under way as we provide a brief initial response to Reviewers to encourage discussion. We will report conclusively on these ongoing threads at the end of the discussion period.

(A) Modularity

The Reviewer is right that in our initial suite of ground truth data-generating networks (DGNs), each external signal enters a single region, and each readout comes from a single region. We appreciate the Reviewer’s concern that this input/output structure may make inter-regional structure more modularized and easier to infer than in the brain. We first justify our rationale for adopting this structure, then present new analyses and DGNs that directly address the Reviewer’s concern.

DGN design rationale. First, this structure loosely mirrors sensory and motor organization in the brain, where each sensory modality enters through its own distinct pathway and motor output leaves through distinct pathways. Second, known entry points let us evaluate input recovery (“fidelity” and “specificity”) region by region, testing whether a communication model attributes each external input to the region that actually receives it in the ground truth DGN. That evaluation would be muddied if external signals entered several or all DGN regions. Third, the alternative of providing all inputs to all regions need not result in distributed computation. With a single readout region, all task computation could be confined to that region since it already receives all necessary inputs. If every region had its own readout, each region could perform the full computation independently.

How distributed are the original DGNs? To measure the extent to which task information is distributed across regions in the original DGNs, we decoded task variables from each region’s activity and from inter-regional communication. In the two DGNs trained to perform multisensory decision-making, the upcoming choice was decodable from all four regions during the delay period (cross-validated R² 0.38–0.85), including the region that receives only the fixation input and the region with no input or output role. Every communication pathway carried choice information as well. By contrast, the stimulus directions remained partly localized to the region that receives them (R² 0.70–0.73 there during stimulus presentation, versus 0.13–0.34 elsewhere). The original DGNs thus exhibited both distributed and localized representations.

New multitask DGNs designed to distribute computation. We trained a new multitask DGN designed to encourage distributed representations and computation (also addressing the Reviewer’s “Questions For Authors”; see (C) below). The DGN performs all 20 Yang et al. [1] Cognitive Suite tasks in a single network. The task cue enters only one region, and an auxiliary training objective requires the current task to be decodable from every region, so the network must communicate task information to all regions (held-out decoding accuracy ≥99.5% in every region). To test whether the computation itself is distributed, we silenced each inter-regional communication pathway in turn, replacing its messages with their across-trial average. The DGN performs 96% of trials correctly when intact (averaged over the 20 tasks), and silencing any one of 6 of the 8 pathways reduces success rates to 11%–70%, indicating distributed computation across regions. We are adding this DGN to the benchmark.

We will also add multitask DGN variants whose inputs and readouts target multiple or all regions, thereby directly avoiding “individualized roles assigned to each region” based on input/output architecture. With every variant, we will release both region-wise decoding of task variables, showing where task information is represented, and pathway-silencing results, showing how much task performance relies on each communication pathway. Users can then compare how communication models perform as a function of the degree of distributed representations and computations in the ground truth DGNs.

Timeline. Revised paper (at close of discussion period): the decoding analysis of the original DGNs; the new multitask DGN. Camera-ready version: the multitask DGN variants (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

(B) Same observation for both models

The Reviewer asked whether RRR and MR-LFADS were fit to the same form of observations, and therefore whether their comparison is fair. We agree that the manuscript was unclear on this point. RRR [2] and MR-LFADS [3] were fit to exactly the same spike-count observations, with the same training, validation and test splits. We will state this explicitly in the paper. All newly benchmarked models (MR-SDS [4], mp-srSLDS [5], ADM [6], mDLAG [7] and DLAG [8], per Reviewers g3JZ and B1S2) are also fit to the same spike counts. ADM, DLAG and mDLAG assume Gaussian observations, so they are fit to square-root-transformed spike counts, which stabilizes the variance, as we will detail in the appendix.

Timeline. Revised paper, posted at close of discussion period.

(C) Multi-task networks

The Reviewer asked why each Cognitive Task Suite network was trained on a single task rather than on many tasks at once. We agree that multitask DGNs would greatly strengthen the benchmark. As described in (A) above, we have successfully trained a new four-region DGN to perform all 20 Yang et al. [1] cognitive tasks, with a task cue that identifies the current task. Because a single network serves 20 tasks, its effectome is task-dependent, and we will report effectome recovery both overall and per task, for all benchmarked communication models including the newly added ones.

We will also add further multitask DGN variants. These DGNs will vary in which regions receive inputs, produce readouts, and communicate; their input and output representations; the number of regions; the number of units in each region; recurrent unit type (e.g., vanilla or gated); and noise level. Our benchmark will thereby test the robustness of communication models to a diversity of data-generating processes.

Timeline. Revised paper (at close of discussion period): the new multitask DGN along with per-task evaluations of all seven benchmarked communication models. Camera-ready version: the further multitask DGN variants.

We thank the Reviewer again for the thoughtful review, and we would welcome any further comments and feedback.

References

[1] Yang et al. Task representations in neural networks trained to perform many cognitive tasks. Nature Neuroscience, 2019.

[2] Semedo et al. Cortical areas interact through a communication subspace. Neuron, 2019.

[3] Liu et al. Accurate identification of communication between multiple interacting neural populations. ICML 2025.

[4] Karniol-Tambour et al. Modeling state-dependent communication between brain regions with switching nonlinear dynamical systems. ICLR 2024.

[5] Glaser et al. Recurrent switching dynamical systems models for multiple interacting neural populations. NeurIPS 2020.

[6] Li et al. Learning time-varying multi-region brain communications via scalable Markovian Gaussian processes. ICML 2025.

[7] Gokcen et al. Uncovering motifs of concurrent signaling across multiple neuronal populations. NeurIPS 2023.

[8] Gokcen et al. Disentangling the flow of signals between populations of neurons. Nature Computational Science, 2022.

Thanks to the authors for the clarifications and additional experiments. I still am not entirely convinced by the reasoning for why the DGNs should be designed with a perfectly modularized role structure a priori, as it still feels a bit circular to assume this structure even if it could be broadly accurate in some cases. This is not my field though, so I could be missing something about how these multiregion models are motivated in general. Based on these considerations, I will tentatively raise my score to a 3.

We thank the Reviewer again for the constructive comments and the openness to discussion. The revised paper and appendix are now uploaded. For each point below, we show where the revised paper delivers the work described in our initial response, and what we will complete for the camera-ready version.

(A) Modularity

The Reviewer was concerned that single input and readout regions may make the DGNs’ computation more modular than in the brain.

We have added two analyses to address this concern. We first decoded task variables from each region’s activity and from inter-regional communication in the original DGNs (Appendix H.1): beyond the two DGNs trained on multisensory decision-making reported in our initial response, the upcoming response was decodable from every region and communication pathway in all 26 networks trained on the other 13 tasks whose response is determined before the go cue. Since a broad representation does not by itself show that computation is distributed, we also silenced each communication pathway in every Cognitive Task Suite DGN and the multitask DGN, measuring how much task performance falls (Appendix H.2). We found that silencing a single pathway often brought performance close to chance: in the single-task DGNs nearly every pathway was needed for the task, and in the multitask DGN each task depended on three to six of the eight pathways, with different tasks relying on different subsets (Appendix H.2). Together with the decoding analysis, this shows that the DGNs’ behavior depends on communication among all four regions rather than on a single route from input to readout.

As this point was especially important to the Reviewer, we would welcome further discussion if any concerns remain.

Timeline. Revised paper (at close of discussion period): the decoding analysis and pathway silencing.

(B) Same observation for both models

The Reviewer asked whether the models were fit to the same observations.

We have revised the wording to make this explicit: all models are fit to the same spike-count data with the same data splits, and scored with the same evaluation (Section 3.2; Table 1 caption). DLAG, mDLAG and ADM assume Gaussian observations, so they are fit to variance-stabilized counts (square-root-transformed for DLAG and ADM, Anscombe-transformed for mDLAG), but their predictions are mapped back to expected spike counts and scored with the same metrics as every other model (Appendix C.2).

Timeline. Revised paper and camera-ready version: the clarified wording.

(C) Multi-task networks

The Reviewer asked why each network was trained on a single task rather than on many tasks at once. We agree that multitask DGNs strengthen the benchmark.

Following the Reviewer’s suggestion, we have added the multitask DGN trained on all 20 tasks to the benchmark (Section 3.2.3; Appendix A.2), the communication model fits to it, with effectome recovery overall (Table 1) and per task (Appendix G.2), and its pathway-silencing results (Appendix H.2).

Timeline. Revised paper: the multitask DGN, the communication model fits to it (ADM and DLAG over smaller latent grids) with overall and per-task effectome recovery, and its pathway-silencing results. Camera-ready version: ADM and DLAG on it over the full latent grids, and multitask DGN variants, including ones whose inputs and readouts target several or all regions, each with region-wise decoding and pathway-silencing results (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

We thank the Reviewer again for the thoughtful review, and we would welcome any further comments and feedback.

Reviewer_g3JZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors proposed a a benchmark named Inter-Regional Communication Benchmark, (IRCB ’26). The scientific aim of the task is to infer the multi-region pathways and content of inter-regional communications. The datasets overall is consisted of 42 synthetic inter-multi-region neural data subsets, which are generated and synthesized by task-trained RNNs. The benchmark also evaluateed the dataset on two competitive communication models i.e., RRR and MR-LFADS. Their results reveal their model's limitations and failure modes in this given task.

Critical strengths
  1. The focus of this work, for enhancing the scientific interpretation of large-scale multi-region neural recordings, plays a key role in neuroscience research.
  2. The paper is well-written and easy-to-follow for academic readers.
  3. The experimental results of IRCB'26, e.g., the comprehensive comparison Table 1 is abundant. The results also both measures the neural activity reconstruction of the models and also their brain-region communication specificity.
Critical weaknesses
  1. I agree that since communication is typically unobserved, thus synthetic datasets with ground-truth communication can be a choice. While that I guess that a maybe better for the benchmark to also contain some ground-truth neural data to complement the current full synthetic 42 datasets.
  2. The paper would benefit from a broader related-work section and more compared models in Table 1, e.g., state-of-the-art frameworks for modeling the communication across brain regions, including Multi-Region Markovian Gaussian Process [1] and ADM [2]. Currently, there only exists two models in Table 1: MR-LFADS, and RRR.

[1] Multi-Region Markovian Gaussian Process: An Efficient Method to Discover Directional Communications Across Multiple Brain Regions, ICML 2024.

[2] Learning Time-Varying Multi-Region Brain Communications via Scalable Markovian Gaussian Processes, ICML 2025, Oral.

Questions for authors

I have no further questions that would affect my assessment. Please refer to the Weaknesses section for my remaining concerns.

2 comments

We thank the Reviewer for their careful reading and constructive response. We were encouraged that the Reviewer sees our work as playing “a key role in neuroscience research” and found the paper “well-written and easy-to-follow.” Below we respond sequentially to each of the Reviewer’s points. Following the Reviewer’s suggestions, we are adding real neural recordings to the benchmark, benchmarking additional communication models, and broadening the related work. Some of this work is still under way as we provide a brief initial response to Reviewers to encourage discussion. We will report conclusively on these ongoing threads at the end of the discussion period.

(A) Real neural data

We appreciate that the Reviewer’s perspective matches ours that “communication is typically unobserved”, so a useful benchmark must focus on synthetic data where communication and related quantities are available as ground truth. This said, we agree that real neural data would provide significant value to the benchmark. We are now fitting communication models to the public brain-wide Neuropixels electrophysiology recordings of Chen et al. [1], and we will update the benchmark release with a preprocessing pipeline that transforms those publicly available data into the same format as our original released datasets. This enables every benchmarked model fit to these real data to be evaluated using our benchmark evaluation pipeline. Because ground-truth communication is not available in real recordings, evaluations on these data will focus on reconstruction quality.

To bring ground-truth communication to real data, we are fitting the switching models mp-srSLDS [2] and MR-SDS [3] and the GP-based ADM [4] to the Chen et al. recordings, and we will use each fitted model as a data-generating network (DGN). Datasets sampled from these DGNs capture features of real neural data, with communication known by construction.

Timeline. Revised paper (at close of discussion period): model fits to the Chen et al. recordings, with reconstruction results. Camera-ready version: the data-driven DGNs, with code, synthetic datasets and benchmark results (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

(B) Expand Benchmarked Models

We appreciate the Reviewer’s suggestion to benchmark additional existing communication models and to broaden the related work. In response, we are extending the set of benchmarked communication models to include mp-srSLDS [2], MR-SDS [3], ADM [4], mDLAG [5] and DLAG [6]. We do not plan to include MRM-GP [7] because its released codebase supports only two regions. Our benchmark code will allow MRM-GP to be benchmarked in future work should the authors or others further develop MRM-GP to support >2 regions.

We are applying these models to four core datasets: Delayed-Memory, Relay-Decision, a new multitask network trained to perform all 20 Cognitive Suite tasks (per Reviewer vAUC), and the Chen et al. electrophysiology recordings described above.

Correspondingly, we are also adding all of these communication models, along with CREIMBO [8] and CTAE [9] (per Reviewer B1S2), to the related work (Section 2).

Timeline. Revised paper (at close of discussion period): comparisons on the four core datasets and the expanded related work. Camera-ready version: results on all benchmark datasets.

We thank the Reviewer again for the thoughtful review, and we would welcome any further comments and feedback.

References

[1] Chen et al. Brain-wide neural activity underlying memory-guided movement. Cell, 2024.

[2] Glaser et al. Recurrent switching dynamical systems models for multiple interacting neural populations. NeurIPS 2020.

[3] Karniol-Tambour et al. Modeling state-dependent communication between brain regions with switching nonlinear dynamical systems. ICLR 2024.

[4] Li et al. Learning time-varying multi-region brain communications via scalable Markovian Gaussian processes. ICML 2025.

[5] Gokcen et al. Uncovering motifs of concurrent signaling across multiple neuronal populations. NeurIPS 2023.

[6] Gokcen et al. Disentangling the flow of signals between populations of neurons. Nature Computational Science, 2022.

[7] Li et al. Multi-region Markovian Gaussian process: an efficient method to discover directional communications across multiple brain regions. ICML 2024.

[8] Mudrik et al. CREIMBO: Cross-regional ensemble interactions in multi-view brain observations. ICLR 2025.

[9] Sristi et al. Coupled transformer autoencoder for disentangling multi-region neural latent dynamics. ICLR 2026.

We thank the Reviewer again for the constructive review. The revised paper and appendix are now uploaded. For each point below, we show where the revised paper delivers the work described in our initial response, and what we will complete for the camera-ready version.

(A) Real neural data

Following the Reviewer’s suggestion, we have added real neural recordings to the benchmark (Section 3.1): the public brain-wide Neuropixels recordings of Chen et al. (2024), together with a preprocessing pipeline that converts recordings from any mouse in the dataset into the benchmark’s format (Appendix B), so that every model is scored with the same evaluation code. We have fit every benchmarked communication model to them (Table 1): all models except RRR reconstruct held-out neurons comparably, which is why the synthetic DGNs, with known communication, are needed to tell them apart (Section 5).

We have also broadened the classes of dynamics in the benchmark by adding data-driven DGNs, built from the GP and switching models that best reconstruct the electrophysiology recordings, ADM and MR-SDS (Section 3.3). Our initial response named both mp-srSLDS and MR-SDS as switching DGNs; we keep only the better-fitting one, since the two capture much the same switching dynamics. Due to time constraints, we will provide benchmark results on these datasets in the camera-ready version.

Timeline. Revised paper (at close of discussion period): the recordings, the preprocessing pipeline, every model’s reconstruction of the recordings, and the two data-driven DGNs with their fits. Camera-ready version: benchmark results on the data-driven DGN datasets (fitting and tuning now seven benchmarked models to these new datasets requires more compute than the discussion period allows).

(B) Expand benchmarked models

As promised in our initial response, we have added five communication models to our benchmark (mp-srSLDS, MR-SDS, ADM, DLAG and mDLAG; Section 5; Appendices C.2 and C.3), compared them with RRR and MR-LFADS on four core datasets (Table 1), and expanded the related work to cover all seven benchmarked models together with MRM-GP, CREIMBO and CTAE (Section 2).

Timeline. Revised paper: the five added models on the core datasets (ADM and DLAG on Delayed-Memory over smaller latent grids), their fitting and tuning, and the related work. Camera-ready version: results on all benchmark datasets, and ADM and DLAG on Delayed-Memory over the full latent grids (fitting and tuning now seven benchmarked models to more than 40 datasets requires more compute than the discussion period allows).

We thank the Reviewer again for the thoughtful review, and we would welcome any further comments and feedback.

Comments on the paper

1 comment

We thank all three Reviewers for their careful and constructive reviews. We were encouraged that the Reviewers found that our benchmark “addresses a genuine and important gap” (Reviewer B1S2), with tasks “designed in a thoughtful way to isolate interpretable failure modes” (Reviewer vAUC), and that it can play “a key role in neuroscience research” (Reviewer g3JZ). The Reviewers also raised valuable concerns, chiefly the absence of real neural data, the small set of benchmarked models, and whether results on our task-trained networks generalize. In response, we are fitting the benchmarked models to public brain-wide Neuropixels recordings, expanding the benchmarked models from two to seven, and adding a multitask network that performs all 20 Cognitive Task Suite tasks, new evaluation metrics, and a structured comparison with prior testbeds. We will also release data-generating networks built by fitting switching and Gaussian-process models to real recordings, so that datasets with ground-truth communication also capture features of real neural data. We detail these changes, and when each will be reported, in our responses to each Reviewer below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.