acceptodds
Under review as a conference paper at ICLR 2027

Latent Brain State-Guided Mixture-of-Experts for EEG-to-fMRI Reconstruction

Abstract

Translating electroencephalography (EEG) into functional magnetic resonance imaging (fMRI) signals could enable region-resolved estimation of blood-oxygen-level-dependent (BOLD) dynamics and functional connectivity using only EEG at inference, without costly fMRI acquisition. However, this mapping is challenging because neurovascular coupling is temporally delayed and varies across brain regions, while existing approaches often fail to separate shared hemodynamic dynamics from region-specific decoding. To address this challenge, we propose Latent Brain State-Guided Mixture-of-Experts (**E2fMoE**), inspired by neurovascular coupling in which neural activity gives rise to delayed, region-dependent BOLD responses. Accordingly, E2fMoE performs shared-state inference before regional decoding. Specifically, a state-space mixer transforms EEG tokens into slowly evolving latent representations, while region-of-interest (ROI) queries extract region-specific information and route it through shared and specialized experts. Residual prediction and low-rank connectivity-aware calibration further refine reconstruction. Extensive experiments across four simultaneous EEG-fMRI datasets show that E2fMoE outperforms representative EEG representation models and EEG-to-fMRI baselines in temporal reconstruction and functional-connectivity preservation. Code is available in the supplementary material.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Latent Hemodynamic Modeling, ROI-Adaptive Expert Routing, Functional Connectivity-Preserving Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_y6VmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes E2fMoE for reconstructing regional BOLD signals from EEG. The model combines a shared latent-state module, an ROI-conditioned mixture-of-experts decoder, and FC-template calibration. It is evaluated on four simultaneous EEG-fMRI datasets for temporal reconstruction and FC preservation, with ablations and an expert routing analysis.

Critical strengths
  1. The model is tested on four datasets, including in a cross-subject setting (Table A2). On NeuroBOLT, it reaches a correlation of 0.575 and an MSE of 0.238 (Table 1).

  2. The ablations test each component separately, and the authors not only report overall scores but also analyze how ROIs are routed to experts.

Critical weaknesses
  1. The method description doesn’t match the released code. Section 3.2 describes recurrent state updates over EEG tokens, but the code pools Transformer-encoded temporal patches and applies HemoSSMBlock across scale vectors. The routing statistics also differ: the Heschl–E2 weight is 0.46 in Table A3 but about 0.282 in the plotting script. Without clarification, the claims about the architecture and expert specialization can’t be verified.

  2. The contribution of EEG to FC prediction is unclear. Removing the FC template lowers the FC score from 0.772 to 0.612 (Table A9), so the template plays a large role. A baseline that uses only the training-set template is needed to show how much scan-specific connectivity the model actually recovers from EEG.

Questions for authors
  1. Which code version and configuration produced the main results and ablations? How do they relate to Section 3.2 and to the routing statistics in Figure 4 and Table A3?

  2. Could you report a baseline that uses only the training-set FC template?

4 comments
AuthorsOpenReview ↗

[Part 1/2] Response to Reviewer gpRP

We sincerely thank the reviewer for the careful and constructive evaluation, and for acknowledging our extensive experiments and ablation studies. We address the reviewer's concerns point by point below:

Q1: Which code version and configurations produced the reported results, and how do they align with the temporal-state formulation and routing statistics?

A1: We apologize that an incorrect code version was inadvertently included in the original supplementary material. We have now re-uploaded the version used to produce the reported results.

The experiment configurations are organized as follows:

  • NeuroBOLT: configs/neurobolt_intra_scan.yaml and configs/neurobolt_inter_scan.yaml.
  • MI-NF: configs/minf_intra_scan.yaml.
  • Global reconstruction: configs/neurobolt_global_inter_scan.yaml, configs/cpd_global_inter_scan.yaml, and configs/sef_sleep_global_inter_scan.yaml.

The temporal-state implementation is provided in src/eeg2fmri/models/inter_scan.py. Specifically, TemporalStateBlock explicitly computes sk=Aˉksk−1+Bˉkxks_k=\bar A_k s_{k-1}+\bar B_kx_k and uk=Cˉksk+Dˉkxku_k=\bar C_ks_k+\bar D_kx_k. In the same file, BOLDSequenceSupervisor produces the matching BOLD-derived latent sequence, and state_supervision_loss applies sequence-level squared-error and cosine alignment, corresponding to Section 3.2 and Eq. (2).

ROI routing and the reported statistics. In src/eeg2fmri/models/multiscale_roi_moe.py, ROIMoEDecoder.forward computes routing logits from the input latent representation and ROI embeddings, then applies a softmax over experts. These per-input, per-ROI weights are the quantities to aggregate when reporting routing distributions.

Revision location (highlighted in blue): Appendix J.1 (p. 25), which clarifies the recurrent state formulation, BOLD-state alignment, and evaluation settings.

AuthorsOpenReview ↗

[Part 2/2] Response to Reviewer gpRP

Q2: What evidence distinguishes the contribution of EEG-derived scan-specific information from that of the training-set FC template?

A2: We thank the reviewer for this important question. We agree that the fixed FC template provides a population-level connectivity prior. To distinguish its contribution from EEG-derived information, we conducted three complementary analyses.

(1) Direct template-only baseline comparison.

Following the reviewer's suggestion, we directly evaluated the training-set FC template without EEG input.

EvaluationTemplate-only FC ↑Full-model FC ↑Relative gain
Intra-scan0.4070.772 ± 0.07689.8%
Cross-subject0.3450.583 ± 0.09669.1%

Relative gains were calculated using unrounded template-only values.

The full model outperforms the template-only baseline by 89.8% and 69.1% in the intra-scan and cross-subject settings, respectively. This directly demonstrates that the fixed FC template alone cannot explain the full model's connectivity prediction performance.

(2) Retraining ablations.

We further retrained the model after removing the FC template or randomly shuffling EEG–fMRI training pairs.

EvaluationMethodBOLD R ↑BOLD MSE ↓FC correlation ↑
Intra-scanFull model0.575 ± 0.1700.238 ± 0.1390.772 ± 0.076
Intra-scanWithout FC template0.249 ± 0.0191.028 ± 0.0180.157 ± 0.022
Intra-scanShuffled training pairs-0.001 ± 0.0171.099 ± 0.0020.092 ± 0.071
Cross-subjectFull model0.345 ± 0.0400.219 ± 0.0290.583 ± 0.096
Cross-subjectWithout FC template0.127 ± 0.0071.037 ± 0.0310.031 ± 0.014
Cross-subjectShuffled training pairs0.004 ± 0.0250.999 ± 0.0030.140 ± 0.133

Removing the FC template substantially degrades FC prediction, confirming its role as a structural prior. However, shuffling EEG–fMRI pairs also markedly reduces performance despite retaining the template, demonstrating the importance of meaningful EEG–fMRI correspondence.

(3) Test-time EEG replacement.

We kept the trained model, FC template, and target fMRI unchanged while replacing the correctly paired EEG with EEG from another subject or another scan of the same subject.

EvaluationEEG inputBOLD R ↑BOLD MSE ↓FC correlation ↑
Intra-scanCorrect EEG, all scans0.575 ± 0.1700.238 ± 0.1390.772 ± 0.076
Intra-scanOther-subject EEG-0.012 ± 0.0081.111 ± 0.0050.175 ± 0.031
Intra-scanCorrect EEG, matched subset0.229 ± 0.0070.994 ± 0.0070.089 ± 0.017
Intra-scanSame-subject, other-scan EEG0.045 ± 0.0141.152 ± 0.0130.087 ± 0.015
Cross-subjectCorrect EEG, all scans0.345 ± 0.0400.219 ± 0.0290.583 ± 0.096
Cross-subjectOther-subject EEG0.003 ± 0.0091.103 ± 0.0240.047 ± 0.017
Cross-subjectCorrect EEG, matched subset0.029 ± 0.0061.095 ± 0.063-0.346 ± 0.038
Cross-subjectSame-subject, other-scan EEG0.013 ± 0.0161.106 ± 0.063-0.348 ± 0.036

Replacing correctly paired EEG with other-subject EEG reduces FC correlation from 0.772 to 0.175 (77.4% reduction) in the intra-scan setting and from 0.583 to 0.047 (92.0% reduction) in the cross-subject setting, despite the unchanged FC template.

For same-subject, different-scan replacement, BOLD correlation decreases on the matched subsets, whereas FC correlation changes only slightly. Thus, these results support EEG-dependent connectivity prediction, although scan-to-scan FC specificity remains less conclusive.

Revision location (highlighted in blue): Appendix J.4 (pp. 27–28), including the template-only baseline comparison (Table A15), retraining ablations (Table A16), and test-time EEG replacement experiments (Table A17). The corresponding intra-scan EEG replacement results are also updated in Section 4.7, Table 4 (p. 9).

Thank you for the further clarifications and additional results. Most of my concerns have been addressed, and I have increased my score accordingly.

AuthorsOpenReview ↗

Response to Reviewer gpRP

We sincerely thank the reviewer for thoughtful feedback and for taking the time to reconsider our work. We greatly appreciate your constructive suggestions, which have helped us improve the clarity and rigor of the paper.

Reviewer_gpRPOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

E2fMoE reconstructs ROI-level BOLD time series from EEG using shared state-space temporal modeling, BOLD-supervised latent alignment, and ROI-conditioned mixture-of-experts decoding. A training-derived connectivity template and low-rank correction calibrate the predictions. Experiments on four simultaneous EEG–fMRI datasets report improvements in temporal reconstruction and functional-connectivity preservation, supported by component ablations and cross-subject evaluation.

Critical strengths
  1. Well-motivated architectural factorization. Separating shared temporal integration from region-specific decoding addresses delayed BOLD responses and regional heterogeneity. Ablations support the design: BOLD-state alignment improves temporal correlation from 0.548 to 0.575, while removing MoE reduces it to 0.392.

  2. Encouraging evaluation across complementary metrics and settings. The paper evaluates both regional trajectories and connectivity across four datasets. On cross-subject NeuroBOLT evaluation, temporal correlation improves from 0.323 to 0.345 and FC correlation from 0.532 to 0.583 over NeuroBOLT, supporting benefits beyond intra-scan prediction.

Critical weaknesses
  1. The physiological interpretation and connectivity attribution remain incomplete. BOLD-supervised alignment does not establish that latent states recover identifiable hemodynamic dynamics. Removing the FC template reduces FC from 0.772 to 0.612 but barely changes temporal correlation (0.575 to 0.568). Further controls are needed to distinguish individual EEG-dependent connectivity from population-level structure supplied by the template.

  2. Baseline configurations require clarification. Table A8 reports 0.432M parameters for LaBraM, versus 5.8M for the original LaBraM-Base. Model configurations, pretrained checkpoints, and parameter-count conventions should be specified to substantiate the SOTA claim. The related-work discussion should also acknowledge Qu et al., CVPR 2026, which addresses EEG-conditioned spatiotemporal fMRI reconstruction at cortical-vertex resolution.

Questions for authors
  1. How does the model compare with directly using the training-set mean FC template? Do shuffled EEG–target controls demonstrate connectivity information beyond this prior?
  2. What exact baseline configurations and pretrained checkpoints were used, and does Table A8 report total or trainable parameters?
  3. Is there direct evidence that the latent states capture physiologically meaningful response delays or time scales, beyond improving prediction?
5 comments
AuthorsOpenReview ↗

Part-1/3 — Q1–Q2: EEG-Dependent Connectivity and Baseline Parameters

We sincerely thank the reviewer for the thoughtful and constructive feedback, and for recognizing our architectural design and experimental results. We address the reviewer's comments point by point below:

Q1: Does the model capture EEG-dependent connectivity beyond the training-set mean FC template?

A1: We distinguish these contributions by keeping the trained model and training-set FC template fixed while replacing the correctly paired EEG with EEG from another subject.

DatasetEEG inputBOLD prediction R ↑FC correlation ↑
NeuroBOLT-RestCorrectly paired EEG0.575 ± 0.1700.772 ± 0.076
NeuroBOLT-RestCross-subject EEG0.025 ± 0.0100.228 ± 0.018

Cross-subject EEG substitution reduces BOLD prediction R from 0.575 to 0.025 and FC correlation from 0.772 to 0.228, despite retaining the same training-set FC template. This demonstrates that the shared template alone cannot explain the reconstruction performance: correctly paired EEG contributes information specific to the target scan, supporting both its temporal BOLD dynamics and connectivity pattern. The remaining FC correlation under mismatched EEG may reflect shared structure and model priors; it cannot be attributed exclusively to the FC template.

Revision location (highlighted in blue): Appendix J.4, Table A17 (p. 28), which reports the cross-subject EEG replacement experiment.

Q2: What baseline configurations and pretrained checkpoints were used, and how were parameters counted?

A2: Thank you for pointing out this ambiguity. In our original LaBraM baseline, the pretrained backbone was frozen, and only the regression head was trained. The 0.432M reported in Table A8 refers to the trainable regression-head parameters, rather than the total parameters of LaBraM-Base. We acknowledge that labeling this column simply as “Params” was ambiguous. We have clarified this distinction in the caption of Table A8, with the revision highlighted in blue.

Revision location (highlighted in blue): Appendix D, the LaBraM description (p. 18); the revised caption of Table A8 (p. 22); and Appendix J.1, “LaBraM parameter accounting” (p. 25).

AuthorsOpenReview ↗

Part-2/3 — Q3: Temporal Modeling Experiments

Q3: Is there direct evidence that the latent states capture physiologically meaningful hemodynamic delays or time scales?

A3: Thank you for this question. To directly investigate the model's ability to capture hemodynamically relevant delays and time scales, we conducted three complementary experiments: known-response recovery, EEG history-length analysis, and temporal occlusion.

(1) Known-response recovery. We generated synthetic targets using Gaussian response kernels with known delay and width parameters, enabling quantitative evaluation of temporal response recovery.

EncoderModel designationPrediction R ↑Delay error (s) ↓Response-width error (s) ↓
Explicit recurrent SSME2fMoE0.993 ± 0.0010.100 ± 0.0700.142 ± 0.051
GRUSimulation control0.998 ± 0.0000.017 ± 0.0370.057 ± 0.037
TransformerSimulation control0.994 ± 0.0010.083 ± 0.0590.248 ± 0.031

The recurrent SSM achieved a prediction correlation of 0.993, with a delay error of only 0.100 s and a response-width error of 0.142 s. These results demonstrate its ability to accurately recover temporal response characteristics under controlled conditions.

(2) EEG history length. We compared four EEG history lengths using identical eligible targets and data partitions.

DatasetEEG historyR ↑MSE ↓
NeuroBOLT-Rest4 s0.190 ± 0.0150.782 ± 0.008
NeuroBOLT-Rest8 s0.301 ± 0.0180.410 ± 0.013
NeuroBOLT-Rest16 s0.575 ± 0.1700.238 ± 0.139
NeuroBOLT-Rest32 s0.346 ± 0.0230.377 ± 0.018

Performance improved substantially as the EEG history increased from 4 s to 16 s, with the highest prediction correlation achieved at 16 s. Extending the history to 32 s did not yield further improvement. These results demonstrate that EEG-to-BOLD prediction depends on a specific range of preceding neural activity, consistent with the delayed temporal dynamics of the hemodynamic response.

(3) Temporal occlusion. We further examined the contribution of different temporal segments by independently occluding four intervals within the 16-second EEG input while keeping the trained model unchanged.

DatasetOccluded interval before the targetR ↑MSE ↓
NeuroBOLT-RestNone0.575 ± 0.1700.238 ± 0.139
NeuroBOLT-Rest16–12 s0.310 ± 0.0200.744 ± 0.195
NeuroBOLT-Rest12–8 s0.297 ± 0.0210.733 ± 0.175
NeuroBOLT-Rest8–4 s0.340 ± 0.0110.500 ± 0.019
NeuroBOLT-Rest4–0 s0.076 ± 0.1260.370 ± 0.239

Occluding any temporal segment degraded prediction performance, with the largest reduction occurring when the most recent 4 s were removed. This confirms that the model leverages temporal information distributed across the EEG history and is particularly sensitive to recent neural activity.

Together, these three experiments provide complementary evidence for the model's ability to capture hemodynamically relevant temporal dynamics. The known-response simulation demonstrates accurate recovery of response delays and widths, the history-length analysis identifies the importance of temporal context, and the occlusion experiment verifies the contribution of temporally localized EEG information.

Revision location (highlighted in blue): Appendix J.3, Tables A12–A14 (p. 26-27), covering known-response recovery, EEG history length, and temporal occlusion, respectively.

AuthorsOpenReview ↗

Part-3/3 — Q4: Comparison with Qu et al.

Q4: How does E2fMoE differ from Qu et al. (CVPR 2026)?

A4: Thank you for bringing this relevant work to our attention. We appreciate the opportunity to clarify its relationship to our study. We have added a citation to Qu et al. [1] and a brief discussion in Section 2 (Related Work). Both studies explore EEG-to-fMRI reconstruction, with complementary emphases, as summarized below.

AspectQu et al.E2fMoE
Reconstruction targetHigh-resolution cortical-vertex fMRI sequencesROI-level BOLD signals and inter-regional connectivity
Modeling approachEEG-conditioned diffusion Transformer with iterative denoisingDirect prediction through latent representations and ROI-conditioned mixture-of-experts decoding
Regional modelingSpatiotemporal generation of detailed cortical activityShared and specialized experts with input-dependent ROI routing
Additional objectiveTemporal continuity and measurement-consistent intermediate-frame reconstructionRegional prediction fidelity and FC-aware calibration using a training-set template
Evaluation focusReconstruction fidelity, temporal consistency, and downstream visual decoding on CineBrainROI-wise reconstruction and connectivity across the datasets considered in our paper

Qu et al. provide a valuable approach to high-resolution fMRI sequence reconstruction and intermediate-frame completion. Our study focuses on shared latent representations and ROI-dependent prediction functions, using expert routing and connectivity-aware calibration. We hope the added discussion helps clarify how these complementary directions contribute to EEG-to-fMRI reconstruction.

[1] Wanying Qu, Jianxiong Gao, Wei Wang, and Yanwei Fu. Modeling spatiotemporal neural frames for high resolution brain dynamic. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

Revision location (highlighted in blue): Section 2, Related Work, “EEG-to-fMRI reconstruction models” (p. 3); References (p. 13); and Appendix J.5 (p. 28), which provides the detailed comparison.

Reviewer_gWXSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

E2fMoE predicts regional and global BOLD values from EEG using a multi-scale encoder, ROI queries and a mixture of shared and specialized experts. A residual head and FC-aware mixing refine the predictions. The manuscript motivates the intermediate representation as a recurrent latent BOLD state, aligned with a BOLD-derived state during training. The release makes several details concrete: the BOLD branch is an autoencoder with a reconstruction loss, its state is detached in the alignment loss, and the module called HemoSSMBlock mixes pooled scale tokens with gated convolutions. The paper reports four datasets and architectural ablations. On the inter-subject NeuroBOLT comparison, overall R is 0.345 versus 0.323 for NeuroBOLT and FC correlation is 0.583 versus 0.532 (Table A2).

Critical strengths
  1. A sensible shared representation with region-dependent decoding

The combination of shared EEG information and region-dependent decoding is well motivated. Different targets need not use the same mixture of features, and the paper examines more than a single full-model ablation. Expert count, residual decoding, state mixing, FC templates and template shuffling are tested, including Table A9 (p. 22). The held-out-subject comparison also improves R, MSE and FC across the six displayed NeuroBOLT ROIs. These experiments provide a useful starting point for understanding why the complete predictor works, even though they do not yet validate its hemodynamic interpretation.

  1. The release resolves important training and evaluation ambiguities

The BOLD branch is explicitly trained to reconstruct observed targets, and the EEG state is aligned to a detached target state (neurohemo_ssm_moe.py, lines 134–157 and 296–308). This supplies an information-preserving objective missing from the manuscript description. The training runner builds the FC prior from its training loader and selects checkpoints using validation metrics. Split utilities keep a person's scans together, and the within-scan loader includes temporal gaps. The bootstrap script resamples whole scans, which is a useful attempt to respect dependence within recordings. These details deserve credit in the assessment.

Critical weaknesses
  1. The released implementation does not establish the stated latent state-space mechanism

Section 3.2 defines a state for each EEG token, updates it through input-dependent recurrent transition matrices, and describes an explicit memory for delayed BOLD responses (p. 4). In the release, ScalePatchEncoder averages over channels and patch positions to return one vector per temporal scale (multiscale_roi_patch.py, lines 42–51). These vectors are stacked along a scale axis, and HemoSSMBlock applies a gated depthwise convolution along that axis (neurohemo_ssm_moe.py, lines 13–27 and 264–275). Thus the released mixer operates on the three scale summaries, rather than the token-by-token state trajectory defined in the paper.

The BOLD supervisor also differs from Eq. (2): its decoder reconstructs ROI values, and supervision combines detached-state MSE with target reconstruction. The reconstruction term anchors the BOLD branch to observed ROI values. Whether the EEG latent follows slow hemodynamic dynamics still requires a temporal test. I need to know which implementation produced the reported tables and what the latent-state ablation actually removes. If this release is the experimental model, the paper should describe the scale mixer and its training objective accurately, then support the hemodynamic interpretation with a temporal diagnostic. If another recurrent model was used, that version is needed to assess the stated contribution.

  1. The FC results and uncertainty still do not have a reproducible statistical interpretation

The code specifies a seven-output NeuroBOLT configuration and an overall FC score based on upper triangles of correlation matrices across prediction samples. It explicitly returns NaN when fewer than two usable edges exist (roi_metrics.py, lines 70–85). With only the two displayed MI-NF ROIs, there is one undirected edge, so this implementation cannot produce the nontrivial FC correlations in Table 1. It also does not supply the ROI-specific FC calculation needed for the individual entries in Tables 1 and A2. Additional evaluated ROIs or a separate computation may explain the numbers, but that computation remains to be identified.

bootstrap_final_fusion_ci.py groups windows by scan (lines 40–57, 83–90), then concatenates resampled scans and evaluates one fixed set of predictions. It does not implement the paired 203 scan-ROI comparison in Appendix E2, or cluster repeated scans by participant. That distinction matters for the reported significance. Table A2 also states that overall standard deviations are averages of ROI standard deviations, rather than variability computed from each run's aggregate score. The split-generating utilities are helpful, but the XLSX consumed by the advertised full-test runner is absent. Exact split assignments, the table-specific FC evaluator and paired participant-level uncertainty are needed to establish the meaning and consistency of the reported gains.

Questions for authors
  1. Which model produced Tables 1 and A2: the token-wise recurrent state model in Section 3.2 or the released convolution over pooled scale tokens? Please map the claimed state sequence and Eq. (2) to the actual tensors and losses. A temporal response or memory-ablation result would help establish whether the state has the proposed hemodynamic role.

  2. What exact ROI set and evaluator produce the per-ROI FC numbers, especially those for MI-NF? The released overall metric returns NaN for a two-ROI input. Please provide a worked example showing the matrix dimensions, observation axis and resulting FC score, and explain whether FC is calculated separately within scans or after pooling them.

  3. Can you provide the split XLSX actually used, the paired baseline predictions and the script that generated Appendix E2? Please compute uncertainty by resampling participants with their scans and ROIs kept together, and compute overall seed variability from per-run aggregate scores. Do the Table A2 gains survive those changes?

5 comments
AuthorsOpenReview ↗

[Part-1/3] Response to reviewer gWXS

We sincerely thank the reviewer for the thorough and constructive evaluation, and for recognizing the strengths of our method and experiments. We address the reviewer's concerns point by point below:

Q1:Which model produced Tables 1 and A2, and how does its implementation align with the claimed state-space mechanism and Eq. (2)?

A1: We apologize that an incorrect code version was included in the original release. We have now re-uploaded the token-wise recurrent implementation used to produce Tables 1 and A2.

Tables 1 and A2 were produced by E2fMoE using configs/neurobolt_intra_scan.yaml and configs/neurobolt_inter_scan.yaml, respectively. The MI-NF results in Table 1 use configs/minf_intra_scan.yaml.

In src/eeg2fmri/models/inter_scan.py, TemporalStateBlock processes 64 chronological EEG tokens and explicitly computes

sk=Aˉksk−1+Bˉkxk,uk=Cˉksk+Dˉkxk,s_k = \bar{A}_k s_{k-1} + \bar{B}_k x_k, \qquad u_k = \bar{C}_k s_k + \bar{D}_k x_k,

producing state and output sequences of shape B×64×128B \times 64 \times 128.

BOLDSequenceSupervisor generates a matching BOLD-derived state sequence, and state_supervision_loss applies sequence-level squared-error and cosine alignment with gradients through both branches, without detached-state supervision or ROI reconstruction loss. These operations correspond to Section 3.2 and Eq. (2).

Revision location (highlighted in blue): Appendix J.1 (p. 24), which clarifies the recurrent state updates, sequence-level supervision, and evaluation settings.

Q2: How are FC metrics computed, including the special case of the two-ROI MI-NF dataset?

A2: Thank you for requesting this clarification. For each scan ss, we compute the reference and predicted FC matrices from the ROI BOLD time series:

Fij(s)=Pearson⁡(Y:,i(s),Y:,j(s)),F^ij(s)=Pearson⁡(Y^:,i(s),Y^:,j(s)).F_{ij}^{(s)} = \operatorname{Pearson}(Y_{:,i}^{(s)}, Y_{:,j}^{(s)}), \qquad \hat{F}_{ij}^{(s)} = \operatorname{Pearson}(\hat{Y}_{:,i}^{(s)}, \hat{Y}_{:,j}^{(s)}).

For multiple ROIs, Overall FC compares unique off-diagonal connections:

FCoverall(s)=Pearson⁡({Fij(s)}i<j,{F^ij(s)}i<j).\mathrm{FC}_{\mathrm{overall}}^{(s)} = \operatorname{Pearson} \left( \lbrace F_{ij}^{(s)}\rbrace_{i < j}, \lbrace \hat{F}_{ij}^{(s)}\rbrace_{i < j} \right).

Per-ROI FC compares the connectivity profile of ROI ii:

FCi(s)=Pearson⁡({Fij(s)}j≠i,{F^ij(s)}j≠i).\mathrm{FC}_{i}^{(s)} = \operatorname{Pearson} \left( \lbrace F_{ij}^{(s)}\rbrace_{j\ne i}, \lbrace \hat{F}_{ij}^{(s)}\rbrace_{j\ne i} \right).

These scores are computed within each scan and then aggregated.

For two-ROI MI-NF, there is only one unique connection, F12(s)F_{12}^{(s)}. A valid alternative is to compare this connection across scans, which is the approach adopted in our paper:

FCMI-NF=Pearson⁡([F^12(1),…,F^12(S)],[F12(1),…,F12(S)]).\mathrm{FC}_{\mathrm{MI\text{-}NF}} = \operatorname{Pearson} \left( [\hat{F}_{12}^{(1)},\ldots,\hat{F}_{12}^{(S)}], [F_{12}^{(1)},\ldots,F_{12}^{(S)}] \right).

Revision location (highlighted in blue): Appendix C, Eqs. (14)–(15) and the MI-NF cross-scan FC definition (p. 18).

Q3: What does the latent-state ablation actually remove?

A3: The latent-state ablation removes the BOLD-derived supervision branch (EBE_B and DBD_B) and sets λstate=0\lambda_{\mathrm{state}}=0. The EEG encoder, state-space mixer, ROI-conditioned MoE, and FC calibration remain unchanged. This ablation isolates the contribution of BOLD-state alignment while preserving the EEG latent representation and recurrent memory.

Revision location (highlighted in blue): Appendix J.1, “Latent-state ablation” (p. 25), and the revised caption of Table A9 (p. 23).

AuthorsOpenReview ↗

[Part-2/3] Response to reviewer gWXS

Q4: What evidence supports the claimed hemodynamic dynamics?

A4: Thank you for this question. To directly investigate the model's ability to capture hemodynamically relevant delays and time scales, we conducted three complementary experiments: known-response recovery, EEG history-length analysis, and temporal occlusion.

(1) Known-response recovery. We generated synthetic targets using Gaussian response kernels with known delay and width parameters, enabling quantitative evaluation of temporal response recovery.

DatasetEncoderModel designationPrediction R ↑Delay error (s) ↓Response-width error (s) ↓
Known-response simulationExplicit recurrent SSMSupplementary SSM; not Full E2fMoE0.993 ± 0.0010.100 ± 0.0700.142 ± 0.051
Known-response simulationGRUSimulation control0.998 ± 0000.017 ± 0.0370.057 ± 0.037
Known-response simulationTransformerSimulation control0.994 ± 0.0010.083 ± 0.0590.248 ± 0.031

The recurrent SSM achieved a prediction correlation of 0.993, with a delay error of 0.100 s and a response-width error of 0.142 s. These results demonstrate its ability to accurately recover temporal response characteristics under controlled conditions.

(2) EEG history length. We compared four EEG history lengths using identical eligible targets and data partitions.

DatasetEEG historyR ↑MSE ↓
NeuroBOLT-Rest4 s0.190 ± 0.0150.782 ± 0.008
NeuroBOLT-Rest8 s0.301 ± 0.0180.410 ± 0.013
NeuroBOLT-Rest16 s0.575 ± 0.1700.238 ± 0.139
NeuroBOLT-Rest32 s0.346 ± 0.0230.377 ± 0.018

Performance improved substantially as the EEG history increased from 4 s to 16 s, with the highest prediction correlation achieved at 16 s. Extending the history to 32 s did not yield further improvement. These results demonstrate that EEG-to-BOLD prediction depends on a specific range of preceding neural activity, consistent with the delayed temporal dynamics of the hemodynamic response.

(3) Temporal occlusion. We kept the trained model unchanged and independently occluded four segments of its 16-second EEG input. The unoccluded condition corresponds to the supplementary Full reference.

DatasetOccluded interval before the targetR ↑MSE ↓
NeuroBOLT-RestNone0.575 ± 0.1700.238 ± 0.139
NeuroBOLT-Rest16–12 s0.310 ± 0.0200.744 ± 0.195
NeuroBOLT-Rest12–8 s0.297 ± 0.0210.733 ± 0.175
NeuroBOLT-Rest8–4 s0.340 ± 0.0110.500 ± 0.019
NeuroBOLT-Rest4–0 s0.076 ± 0.1260.370 ± 0.239

Occluding any temporal segment degraded prediction performance, with the largest reduction occurring when the most recent 4 s were removed. This confirms that the model leverages temporal information distributed across the EEG history and is particularly sensitive to recent neural activity.

Together, these three experiments provide complementary evidence for the model's ability to capture hemodynamically relevant temporal dynamics. The known-response simulation demonstrates accurate recovery of response delays and widths, the history-length analysis identifies the importance of temporal context, and the occlusion experiment verifies the contribution of temporally localized EEG information. Collectively, these findings support the effectiveness of temporal modeling in EEG-to-fMRI reconstruction.

Revision location (highlighted in blue): Appendix J.3, Tables A12–A14 (p. 26-27), covering known-response recovery, EEG history length, and temporal occlusion, respectively.

AuthorsOpenReview ↗

[Part-3/3] Response to reviewer gWXS

Q5: How were the 203 paired scan-ROI comparisons in Appendix E2 performed, and how is statistical significance assessed using participant-level resampling?

A5: Thank you for highlighting the importance of participant-level statistical evaluation. We conducted a paired participant-level bootstrap analysis, resampling participants while preserving their associated scans and ROIs. The evaluation included 22 participants in the intra-scan setting and 4 participants in the inter-scan setting. Identical bootstrap samples were used across methods to ensure paired comparisons.

(1) Intra-scan evaluation (22 participants).

ModelR ↑MSE ↓FC ↑
E2fMoE0.575 [0.533, 0.617]0.238 [0.219, 0.257]0.772 [0.712, 0.826]
NeuroBOLT0.502 [0.459, 0.547]0.247 [0.226, 0.270]0.576 [0.498, 0.642]
UnEBOLT0.406 [0.358, 0.457]0.306 [0.283, 0.330]0.378 [0.307, 0.454]

Paired performance differences:

ComparisonΔR [95% CI]ΔMSE [95% CI]ΔFC [95% CI]
E2fMoE vs. NeuroBOLT+0.073 [0.046, 0.102]+0.009 [−0.003, 0.020]+0.196 [0.128, 0.265]
E2fMoE vs. UnEBOLT+0.169 [0.140, 0.197]+0.068 [0.059, 0.078]+0.394 [0.333, 0.455]

E2fMoE achieved the best performance across all three metrics. The paired bootstrap confidence intervals show improvements over NeuroBOLT in R and FC, and over UnEBOLT in all three metrics.

(2) Inter-scan evaluation (4 participants).

ModelR ↑MSE ↓FC ↑
E2fMoE0.345 [0.285, 0.405]0.219 [0.163, 0.277]0.583 [0.413, 0.838]
NeuroBOLT0.323 [0.269, 0.358]0.232 [0.191, 0.290]0.532 [0.422, 0.691]
UnEBOLT0.046 [0.005, 0.104]0.261 [0.198, 0.324]0.240 [0.025, 0.448]

Paired performance differences:

ComparisonΔR [95% CI]ΔMSE [95% CI]ΔFC [95% CI]
E2fMoE vs. NeuroBOLT+0.022 [−0.048, 0.063]+0.013 [−0.010, 0.036]+0.051 [−0.009, 0.142]
E2fMoE vs. UnEBOLT+0.299 [0.254, 0.341]+0.042 [0.035, 0.049]+0.343 [0.197, 0.489]

E2fMoE maintained the highest performance across all three metrics in the inter-scan setting. The paired bootstrap analysis shows improvements over UnEBOLT across R, MSE, and FC. Relative to NeuroBOLT, E2fMoE also achieved higher point estimates across all metrics.

All reported intervals are 95% participant-level bootstrap confidence intervals. Performance differences are defined as E2fMoE minus the baseline for R and FC, and baseline minus E2fMoE for MSE, so positive values consistently indicate improvements by E2fMoE.

Revision location (highlighted in blue): Appendix E.2 and the revised captions of Figure A1 and Table A5 (p. 20); the participant-level bootstrap results and paired differences are reported in Appendix J.2, Tables A10–A11 (p. 25-26).

Q6: Can the exact subject-level and scan-level data splits?

A6: Yes. We have included the exact subject-level and scan-level split assignments in the Data_Splits/ directory of the newly uploaded supplementary material. These files identify the subjects and scans assigned to each partition, making the training, validation, and test assignments explicit and allowing the evaluation splits to be reproduced.

Revision location (highlighted in blue): Appendix J.1, “Data partitions” (p. 25), which points to the exact subject-level and scan-level assignments in the supplementary material.

Q7: How does correcting the overall standard deviation calculation affect Table A2, and do the reported improvements remain statistically significant?

A7: We apologize for the ambiguity in the original caption of Table A2. In our actual calculation, we first averaged the six ROI scores within each seed and then computed the sample standard deviation across the five seed-level aggregate scores. We did not average the ROI-wise standard deviations. We have corrected the caption to accurately describe this procedure; the reported values remain unchanged.

Revision location (highlighted in blue): The revised caption of Table A2 (p. 19). Participant-level statistical comparisons are provided in Appendix J.2, Tables A10–A11 (p. 25-26).

Comments on the paper

2 comments

Dear Reviewers,

Thank you again for your reviews. The authors have posted their rebuttal. If you haven't already, please take a look and let them know whether it addresses your concerns. If anything is still unclear, feel free to ask follow-up questions while the discussion period is open.

Even a short note (e.g., whether you are keeping or updating your score, and why) would help both the authors and the final decision.

Thanks for your time!

Best, AC

AuthorsOpenReview ↗

Global Response and Summary of Manuscript Revisions

Dear Reviewers, AC, SAC, and PC,

We sincerely thank Reviewers gWXS, gpRP, and y6Vm for their careful reading and constructive feedback. We appreciate their recognition of the motivation for combining shared EEG representations with region-specific decoding (gWXS, gpRP), the evaluation across datasets and prediction settings (gpRP, y6Vm), and the component ablations (gWXS, y6Vm).

We particularly appreciate the discussion of how the implementation relates to the manuscript and what the experiments establish about EEG-dependent connectivity. All revisions have been marked in blue in the revised paper. We summarize the revisions below:

We clarified the model implementation and provided additional reproducibility details.

  • Following gWXS and y6Vm, we corrected the supplementary code release and identified the configurations used for Tables 1 and A2. We also mapped the recurrent updates over EEG tokens and the sequence-level BOLD supervision to Section 3.2 and Eq. (2). (Appendix J.1, p. 25.)
  • Following gWXS and y6Vm, we clarified that the latent-state ablation removes BOLD-derived supervision while retaining the EEG recurrent states. (Table A9, p. 23, and Appendix J.1, p. 25.)
  • Following gWXS, we clarified the within-scan FC definitions for multiple ROIs and the cross-scan calculation for the single MI-NF edge. We also documented how overall standard deviations are computed from seed-level scores and provided the subject and scan assignments in the supplementary material. (Appendix C, p. 18, Table A2, p. 19, and Appendix J.1, p. 25.)
  • Following gpRP, we clarified that LaBraM’s 0.432M trainable parameters belong to the regression head and that the pretrained backbone was frozen. (Appendix D, p. 18, Table A8, p. 22, and Appendix J.1, p. 25.)

We added controls for the contribution of EEG to connectivity prediction.

  • Following gpRP and y6Vm, we evaluated the training-set FC template alone. FC correlation increased from 0.407 with the template to 0.772 with the full model intra-scan, and from 0.345 to 0.583 cross-subject. We also retrained the model after removing the template or shuffling EEG–fMRI pairs. (Appendix J.4, Tables A15–A16, p. 27.)
  • Following gpRP and y6Vm, we replaced the paired EEG at test time while holding the trained model, FC template, and target fMRI fixed. Another participant’s EEG reduced FC correlation to 0.175 intra-scan and 0.047 cross-subject. EEG from another scan of the same participant produced little change in FC on matched subsets. These results support EEG-dependent connectivity prediction. (Section 4.7, Table 4, p. 9, and Appendix J.4, Table A17, p. 28.)

We added temporal and statistical analyses.

  • Following gWXS and gpRP, we tested four EEG history lengths and occluded individual segments of the 16-second input. The 16-second history yielded the highest prediction correlation, and occluding the most recent four seconds produced the largest decrease. A separate simulation tested recovery of known response delays and widths using a supplementary recurrent SSM. (Section 4.7, pp. 9–10, and Appendix J.3, Tables A12–A14, pp. 26–27.)
  • Following gWXS, we added paired participant-bootstrap intervals that keep each participant’s scans and ROIs together. Against NeuroBOLT, the intra-scan intervals favored E2fMoE for prediction correlation and FC. The interval for intra-scan MSE and all three inter-scan intervals included zero. (Appendix E.2, p. 20, and Appendix J.2, Tables A10–A11, pp. 25–26.)

We expanded the related-work discussion.

  • Following gpRP, we cited Qu et al. (CVPR 2026) and compared their cortical-vertex fMRI sequence reconstruction with our ROI-level BOLD and connectivity prediction. (Section 2, p. 3, and Appendix J.5, Table A18, p. 28.)

We thank the reviewers and chairs again for their time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.