acceptodds
Under review as a conference paper at ICLR 2027

Rhythm Flow: Bridging the Gap Between Functional Fluctuations and Neural Spikes Through Oscillatory Synchronization

Abstract

Understanding the complex relationship between spatially coarse millisecond-scale neural oscillations and fine-resolution yet slow hemodynamic fluctuations remains a fundamental challenge in multimodal neuroimaging. Although recent generative models attempt to synthesize functional magnetic resonance imaging (fMRI) from concurrent electroencephalography (EEG), current `black-box' architectures fail to model intrinsic neurovascular coupling mechanisms, yielding anatomically plausible but functionally incoherent fMRI data. To bridge this gap, we introduce Rhythm Flow, a biologically-principled generative framework in which one oscillatory-synchronization mechanism governs both the EEG encoder and the residual dynamics it drives. Our approach decomposes the generated latent space into three components: a time-invariant anatomy term, a low-rank EEG residual produced by a rhythm-gated spiking encoder (named as VibeSync), and a stochastic correction flow. The correction flow uses the same oscillator-and-spiking computation (VibeSync) to capture population-level dynamics that the EEG residual leaves unexplained. To test whether the predictions depend on the specific EEG input, we swap the input EEG, which changes only the EEG residual, and check without retraining whether temporally matched EEG explains the observed fMRI better than temporally shifted or cross-subject controls. Across two simultaneously recorded EEG–fMRI cohorts, temporally matched EEG outperforms both controls, and without the correction flow, the EEG residual is positive for nine of ten held-out subjects under five-fold subject cross-validation (one-sided Wilcoxon ), indicating that the EEG specificity generalizes to unseen subjects. On 1,152 subjects of the Healthy Brain Network, Rhythm Flow further outperforms all compared models in both representational similarity (RSA-ROI) and voxel fidelity. Beyond empirical gains, Rhythm Flow turns EEG reliance into a quantity measured directly on a trained model, making the claim that a generative EEG-to-fMRI model has learned physiological coupling falsifiable.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spike Neural Network, fMRI, EEG, Generative Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AMAzOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors present an approach to analyse EEG-fMRI data that integrates a number of neurophysiological insights and appears to yield better performance in some experiments. The work is embedded well in the existing literature and the experiments compare with some other approaches. Some more clarity on the experiments and notation could help to strengthen the manuscript.

Critical strengths

Great that there seems to be error bars given in some of the results (e.g. in table 1) but it could be explained better what the metrics here refer to.

The model appears to integrate a number of interesting angles on neural and hemodynamic coupling.

Critical weaknesses

Fig 1 is great and helpful for providing a broad overview of the approach, but some of the basic assumptions appear a bit strong and could be explained better. The fact that spiking activity is reflected EEG or fMRI in a way that can be / is captured with such a spatial resolution that it requires the proposed model for instance sounds physiologically implausible. The authors allude to empirical evidence, but to support the claims made in the experimental section these connections could be elaborated on a bit more.

Another strong assumption is that of spatio-temporal coupling and how it factorizes. There is evidence that the coupling is not as simple as a static spatial pattern would suggest, as used in the model.

The metrics could be explained better, for instance how exactly the MSE/PSNR are computed.

Comparisons with other models would be needed. Table 3 is helpful, but the variance across methods and approaches makes it difficult to assess which part of the approach yields an advantage. Most importantly, error bars must be provided for all metrics, maybe the authors could consider dropping some metrics in favour of error bars for the most relevant metric.

It would be helpful to provide more explanations on the exact preprocessing steps and how they relate to the difference in performance. Often in these kind of studies, preprocessing explains most of the advantages. It would be important that the authors clearly state that all preprocessing steps were kept constant across methods.

The approach assumes a static (across space and time) hemodynamic response function. Given the complexity of the model, it seems reasonable to assume (and there is evidence for that from several decades of research on hemodynamic coupling) that the model should maybe account for more complex hemodynamic coupling.

The cross-validation procedure could maybe run a full leave one/n-subject(s) out analysis and report variances across folds. It wasn’t clear whether that was done.

Questions for authors

were all preprocessing steps kept the same across all baseline methods?

Reviewer_Eo7NOpenReview ↗
Rating
3
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the conditional generation of 4D fMRI from EEG and structural MRI (sMRI). The authors identify an “identifiability failure” when both modalities are fed into a single flow-matching generator: the model can rely on the strong anatomical prior provided by sMRI while ignoring EEG, and the architecture itself offers no guarantee that EEG is used. To address this issue, they propose Rhythm Flow.

Generally speaking, this paper addresses an important and often overlooked question in EEG-to-fMRI generation: how to ensure that a generative model actually uses the neural dynamics captured by EEG. The three-branch decomposition and EEG-swapping evaluation offer interesting contributions, and the experiments suggest that the approach has potential.

Critical strengths
  • The paper addresses a clear and practically relevant problem. A central challenge in EEG-to-fMRI generation is whether the model actually extracts information from EEG that corresponds to BOLD dynamics, rather than relying primarily on anatomical priors or learning average spatiotemporal patterns from the training set. Establishing this distinction is a worthwhile research goal that goes beyond improving reconstruction accuracy.
  • The method is clearly formulated and introduces novel ideas. The experiments also cover datasets of different sizes and acquisition conditions.
Critical weaknesses
  • The physiological claims need stronger support. The paper does not show whether VibeSync’s learned coupling graph and phase states reflect actual EEG synchronization patterns. The rhythm-gated spiking mechanism is evaluated mainly through downstream performance. Hence, using analyses of spike activity, phase synchronization, or rhythm gating would strengthen its interpretation. The authors should also distinguish the physiological motivation of the EEG encoder from its architectural reuse in the correction flow, which is driven by generated states rather than measured EEG.

  • The ablations do not isolate component contributions within the final architecture. Most ablations in Table 2 use a single-flow generator, whereas Rhythm Flow uses a three-branch architecture. For example, the improvement in HBN subject-generalization RSA-ROI from 0.421 for RSPC to 0.469 for Rhythm Flow cannot be attributed to the encoder alone. Controlled ablations within the final framework should compare RSPC with Kuramoto, remove the rhythm gate and zero reference separately, and examine different values of K.

  • The lack of temporal alignment in HBN limits the conclusions. Because HBN EEG and fMRI were not acquired simultaneously, these experiments cannot establish whether the model recovers BOLD dynamics corresponding to a particular EEG segment. Do different EEG windows from the same subject produce meaningful changes in the generated dynamics? Comparisons across subjects alone cannot adequately separate EEG specificity from individual differences.

  • Temporal dynamics are not evaluated directly enough. Although functional fluctuations are central to the method, the main evaluation relies on MSE, PSNR, and RSA-ROI. These metrics may favor static or average spatial structure and do not establish temporal fidelity. Direct evaluation on simultaneously acquired EEG-fMRI data would provide stronger evidence that the model recovers EEG-related BOLD dynamics.

Questions for authors
  • Given VibeSync’s coupled oscillators, spiking integration, and recurrent computation, please report parameter counts, training time, and inference time against standard Transformer and CNN encoders.

  • Please clarify how EEG downsampling and token aggregation preserve frequency information, accommodate different TRs, and account for the hemodynamic lag.

  • Reimplementing NeuroBOLT on HBN is a useful addition. However, comparisons with other published methods largely rely on results obtained with different datasets and protocols. More comparisons under matched data splits and evaluation settings would strengthen the results.

Reviewer_WGJZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Rhythm Flow, which generates subject-space 4D fMRI from EEG and a T1 scan. The VAE latent of the fMRI window is split into three additive parts: Anatomy term A. A time-invariant term predicted from the sMRI alone. EEG residual D. A zero-mean, low-rank term produced from EEG by an encoder called VibeSync. VibeSync combines spherical Kuramoto-style phase coupling over a learned electrode graph with a rhythm-gated LIF spiking integrator and zero-referenced tokens. D’s mode coefficients are integrated over time with a cumulative sum. Correction flow CθC_\theta. A stochastic flow-matching term that sees neither modality. It is trained on a stop-gradient remainder R⋆−sg(D)R^\star - \mathrm{sg}(D).

Given that CθC_\theta never sees EEG, swapping the EEG under shared noise changes only D. The authors use this as a retraining-free test of EEG specificity against within-run (other window, same run) and cross-subject controls. The same score also serves as a training loss (LEEG\mathcal{L}_{EEG}) and as the checkpoint-selection criterion. The results come from two settings: simultaneous EEG-fMRI and HBN. Rhythm Flow reports the best RSA-ROI, MSE and PSNR among the baselines.

Critical strengths

The evaluation protocol asks the right question, and the authors report unflattering results. Swapping the EEG at inference under shared noise, with within-run and cross-subject nulls and per-subject permutation tests, is the kind of falsification test EEG-to-fMRI work usually lacks. The authors also ran a five-fold subject cross-validation with sealed test subjects. They report results that hurt their own case: the full model is not significant out of fold, RSA-ROI is undefined for 39 of 92 HBN subjects, and the comparison against the mean template has p=0.16p = 0.16. This transparency is the main reason the paper is informative.

The decomposition is simple, and it makes attribution clean. Regressing the temporal mean from sMRI and restricting the generator to a zero-mean residual removes the need to re-synthesize anatomy from noise. The orthogonality argument (Appendix C.2) is correct and easy to verify. The stop-gradient on D prevents the correction flow from cancelling the EEG term, so any EEG dependence must sit in D. The idea is not deep, but it is a reasonable design choice that others could reuse.

Critical weaknesses
  1. The evidence for EEG specificity is statistically fragile, practically negligible, and does not support the coupling claim.

(a) The effect size is tiny.

  • Matched-EEG scores ρ\rho (cosine between predicted and true frame-to-frame latent changes) are on the order of 10−310^{-3} (Fig. 2). Advantages over the nulls are 0.20.2-1.8×10−31.8 \times 10^{-3} (Table 1).
  • For scale: the per-frame latent has 32×10×10×6=19,20032 \times 10 \times 10 \times 6 = 19{,}200 dimensions. Two unrelated isotropic vectors in that space have a cosine with standard deviation of about 1/19200≈0.0071/\sqrt{19200} \approx 0.007.
  • Averaging over frames and windows makes the difference detectable. But the generated dynamics are still essentially orthogonal to the observed ones.
  • No variance-explained figure is given for D against R⋆R^\star. Without one, “EEG specificity” says little about how much of the BOLD signal the model actually captures.

(b) The only clean test does not favor the proposed model. Appendix E is the only analysis with sealed test subjects.

  • The full model’s mean within-run advantage there is +0.01×10−3+0.01 \times 10^{-3} at the selected checkpoint (p=0.065p = 0.065) and +0.21×10−3+0.21 \times 10^{-3} at the final one (p=0.053p = 0.053).
  • Significance appears only for no flow, a separately trained A + D model. Its out-of-fold RSA-ROI is 0.023 at the selected checkpoint.
  • At the selected checkpoint, the full model’s RSA-ROI (0.135) does not exceed no EEG (0.138), which has no EEG pathway at all.
  • So the model that is detectably EEG-specific does not produce usable dynamics, and the model that produces usable dynamics is not detectably EEG-specific.

(c) The headline results appear to be selected on the subjects they are tested on. As I read Sec. 2.4 and 3.1, checkpoints are chosen by the EEG-conditioning gap on the validation subjects.

  • Tables 1 and 5 then report permutation tests on those same subjects. That is selection on the test statistic.
  • The Bonferroni argument in Appendix D does not correct for selection over epochs.
  • Even taken at face value, only 3 of the 5 validation subjects separate from the nulls (xp102, xp109, xp216).

(d) The test establishes dependence on matched EEG, not physiological coupling.

  • Anything shared between simultaneously recorded EEG and fMRI would pass it. Head motion, residual ballistocardiogram or gradient artifacts, and respiration all qualify.
  • No motion regression or lag analysis is reported. If the effect is neurovascular, specificity should peak near the hemodynamic delay and fall off at zero or large lags.
  • The fixed lag of 3 TR is also physiologically inconsistent across cohorts: it means 6s for ds002336 and 3s for ds002338.
  • The abstract’s claim that the method makes “learned physiological coupling” falsifiable is therefore not delivered.

(e) The paired test does not need the factorization.

  • Sec. 1 argues that a single conditional flow can only be checked “indirectly by retraining an EEG-ablated variant”. That is not the case: one can swap the EEG at inference with fixed noise in any conditional generator.
  • Running the same paired test on the single-flow baselines would show whether the factorization buys any specificity. It is not reported.
  1. HBN cannot carry the generation-quality claims, and the comparisons there are weak.

(a) The HBN setup cannot test coupling, and temporal generalization tests something else.

  • EEG and fMRI come from separate sessions, so HBN cannot test coupling. On HBN, LEEG\mathcal{L}_{EEG} with a cross-subject control trains D to prefer a subject’s own EEG over others’. That is subject fingerprinting, not dynamics.
  • One 30-second EEG segment conditions both windows of a subject. “Temporal generalization” therefore feeds the model exactly the inputs it was trained on and asks for a different fMRI window. That measures within-subject stationarity or memorization.
  • The obvious baseline is missing: return the subject’s own training-window fMRI.

(b) A constant template comes close on every metric. A population mean template ignores every input and returns the same output for every subject. Yet:

  • On subject-generalization RSA-ROI it scores 0.413, against 0.469±0.0360.469 \pm 0.036 for Rhythm Flow. Its MSE is 0.0109 against 0.0104, and its PSNR is 37.4 against 37.5.
  • On the 53 subjects where RSA-ROI is defined, the means are 0.73 versus 0.69 with Wilcoxon p=0.16p = 0.16.
  • On the other 39 of 92 unseen subjects, Rhythm Flow’s output is too static for the metric to be defined.
  • “Outperforms all compared models” is not supported. Also, the text says Rhythm Flow beats the template on 33/53 subjects, while Fig. 3c says 31/53.

(c) RSA-ROI largely rewards a static map.

  • Regional means are not removed, and the regions are a fixed grid rather than an atlas.
  • So the region-by-region cosine matrix is close to all-ones, and its structure is driven largely by each region’s mean-normalized fluctuation amplitude. That is a tSNR-like map that is largely shared across subjects.
  • This is consistent with two observations: the template’s strong score, and the eeg only single-flow model (sMRI zeroed) beating smri only on unseen subjects (0.377 vs 0.264).

(d) The ablations do not test the proposed model.

  • All ablations are on single-flow variants, not on Rhythm Flow, and each is a single run. Rhythm Flow’s own seed SD (0.036) is larger than several of the gaps being interpreted, e.g. no kuramoto 0.458 vs RSPC 0.421.
  • "no kuramoto" actually beats RSPC on subject generalization.
  • "no rhythm" scores 0.078, far below a constant template. That looks like a failed training run rather than evidence that the rhythm gate is “essential”.
  • None of VibeSync’s components is ablated inside Rhythm Flow. Neither are the stop-gradient, LEEG\mathcal{L}_{EEG}, or zero-referencing.

Taken together: the architecture is plausible and the evaluation idea is good. But when the authors’ own falsification test is applied, the measured EEG effect is close to nil. The HBN gains are within reach of a template that ignores both inputs.

Questions for authors
  1. Were early stopping and checkpoint selection for Tables 1 and 5 done on the same validation subjects whose permutation tests are reported? If so, please re-run with selection and testing on disjoint subjects, as in Appendix E, for both cohorts.
  2. What fraction of the variance of R⋆R^\star (or of ΔTR⋆\Delta_T R^\star) does D explain on held-out subjects? Please also report raw ρ+\rho^+ alongside a reference scale for unrelated vectors in this latent.
  3. Does the matched-EEG advantage survive two controls? Regressing motion parameters or framewise displacement out of the target. A sweep of the EEG-fMRI lag, from 0 to about 10 s, showing where specificity peaks.
  4. What is the cross-subject null exactly? Sec. 2.4 describes a task-, run- and window-matched segment, but Table 1 and Appendix D describe “up to 32 same-task windows of other subjects”. If the null is not window-matched, block timing in the neurofeedback runs could drive part of the within- vs cross-subject difference.
  5. What does the paired test give for the single-flow baselines under shared noise?
  6. HBN temporal and joint generalization: what does the “subject’s own training-window fMRI” baseline score? How are undefined RSA-ROI values handled for the baselines (e.g., the template on the 39 subjects)? What does RSA-ROI look like with regional means removed, on an anatomical atlas?
  7. Within Rhythm Flow, how does VibeSync compare with a standard EEG encoder (e.g., band power plus a temporal CNN or transformer)? What happens without the stop-gradient and without LEEG\mathcal{L}_{EEG}? How many seeds were run for the single-flow variants?
  8. ds002336 has ten subjects (Appendix E), but the main split uses 6 + 2. What happened to the other two? Which checkpoint is used for no eeg in Appendix E, given that it cannot be selected by the EEG gap?
  9. What in VibeSync is new relative to Dan & Wu (2026), Miyato et al. (2025) and Yan et al. (2025)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.