- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces BrainWM, a multimodal model for representing, forecasting, and simulating whole-brain sMRI+fMRI dynamics. It consists of a structure–function VAE, a flow-matching transformer, and a latent Brain Action Model, trained across three phases. Experiments cover competitive downstream performance and forecasting/simulation capabilities.
Critical strengths
-
The paper evaluates representation learning, within-run forecasting, rest-to-task generation, and longitudinal anatomical changes within a common framework. These cover a wide breadth of tasks that are not commonly covered but has the possibility to be important in the future.
-
Beyond temporal correlation, the authors assess connectivity, spectral properties, variance, and individual specificity, providing a more informative assessment of the generated brain dynamics compared to what reconstruction quality alone could tell.
Critical weaknesses
-
For the downstream results in table 1, the best BrainWM variant is inconsistent. In other words, sometimes the phase 2 and/or phase 3 training harms the model performance possibly due to the representation being worse. Besides for downstream task evaluation, benefits of BAM is also questionable for forecasting. In the real setting, Table 5 show that BAM is beneficial only for the short horizon (h32), and Table 23 show BAM is not beneficial for all lengths.
-
Results in table 5 and 23 suggest that the BrainWM performs almost the same even when the stimulus is zeroed out. This suggests that the model recieves almost no information from the content of the stimulus itself, and the model may be relying only on information such as the initial state or the text description, instead of reading the ongoing stimulus. Interestingly, the no-BAM model actually loses performance when the stimulus is zeroed out; it may be due to how the stimulus is constructed for the no-BAM model, or it may suggest that the current way of supplying the information of the stimulus through BAM is unable to properly pass on the information, although I am not completely sure about this part. Nonetheless, the fact that a "world model for brain dynamics" does not care about the actual content of the stimulus is a huge red flag, and must be addressed for the proposed method to truely be a functioning world model.
Questions for authors
-
As a follow-up to weakness #1, BAM seems help in table 5 but not for table 23 in the true data h32 setting. Could the authors provide explanations for this difference?
-
As a follow-up to weakness #2, could the authors provide additional ablation to locate the source of forcasting performance when the stimulus content does not seem to be the core factor?
-
Could the authors compare BAM against direct conditioning for rest-to-task simulation, including correct, incorrect, and absent task descriptions? The forecasting ablations do not establish whether BAM improves conditional simulation.
-
Does the MONAI forecasting baseline receive the same per-frame stimuli and other information as BrainWM?