- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates when model complexity benefits cross-subject fMRI prediction. Across three datasets, the authors show that pretrained stimulus representations improve prediction accuracy, while simple linear autoregressive models often outperform complex nonlinear models when recent BOLD history is available. These findings highlight the importance of strong linear baselines and suggest that increased model complexity does not necessarily lead to better performance.
Critical strengths
• The paper addresses an important question regarding when increased model complexity is beneficial for cross-subject fMRI prediction.
• The regional analyses and controlled hemodynamic simulations provide useful neuroscientific insights.
Critical weaknesses
• The performance results were given in Figures and graphs without exact numbers, which may limit follow-up studies for fair comparison.
• The number of samples is limited; BOLD Moments has 10 and Algonauts has only 4 samples. Small subject cohorts and limited statistical uncertainty analyses weaken the generalizability of some conclusions.Moreover, the substantial overlap of stimuli between training and test subjects limits conclusions about generalization to unseen stimuli.
• Several nonlinear models are substantially adapted or reimplemented, raising concerns about whether the comparisons fairly reflect their original capabilities.
• The paper reads more like a technical report than a research paper, as it primarily focuses on benchmarking existing models and analyzing their performance under different settings. While the extensive experiments provide useful insights, the methodological novelty and conceptual contributions are somewhat limited.
Questions for authors
• Could the authors evaluate the original pretrained models or provide stronger evidence that the benchmark adaptations preserve their intended capabilities?
• How well do the findings generalize to entirely unseen stimuli? Could stimulus overlap favor simpler models that exploit stimulus-specific temporal patterns?
• How much of RR-AR's performance can be explained by temporal correlation alone, without stimulus information?
• Could the authors extend their simulations to nonlinear neural dynamics to determine whether the advantage of linear prediction persists?
5 comments
Response to official implementations
General Question 3 — Official model implementations (W3 / Q1)
Summary. Do the benchmark adaptations fairly reflect the original models, and what happens when official implementations are evaluated?
Response.
(1) Why adapt the models for this benchmark? We appreciate the reviewer's concern about the influence of implementation choices. Our original benchmark was designed to compare downstream predictors under common stimulus inputs and matched BOLD-history availability. For MIRAGE, we therefore retained the released downstream network while replacing its Qwen features with the common features used by the other predictors. For BrainVista, whose official code was unavailable when we implemented the benchmark, we implemented the model from the published description and integrated it into the same evaluation framework.
(2) Additional runs using the authors' released implementations. For MIRAGE, we restore native Qwen3-Omni representations and released model components, train the downstream predictor from scratch on training subjects, and average their readout heads for unseen subjects. For BrainVista, we replace our paper-based implementation with the released model components and official training procedure. Both evaluations retain our cross-subject protocol.
The additional BrainVista results cover all three datasets, and MIRAGE covers HCP and BOLD Moments. MIRAGE on Algonauts would require more than two weeks of feature extraction on our available hardware, exceeding the revision window.
(3) How do the official-code models compare with RR-AR? The tables compare manuscript (M) and official-code (O) results against RR-AR. Rows show H (TRs); entries are mean (subject SD). MIRAGE is stimulus-only, so its score repeats across H; a dash indicates an unavailable result.
HCP
| H | RR-AR | BrainVista M | BrainVista O | MIRAGE M | MIRAGE O |
|---|---|---|---|---|---|
| 1 | .685(.032) | .618(.035) | .551(.028) | .267(.049) | .047(.013) |
| 2 | .632(.031) | .563(.034) | .506(.029) | .267(.049) | .047(.013) |
| 4 | .541(.033) | .472(.034) | .424(.031) | .267(.049) | .047(.013) |
| 8 | .444(.033) | .377(.030) | .332(.032) | .267(.049) | .047(.013) |
| 16 | .377(.034) | .310(.025) | .264(.030) | .267(.049) | .047(.013) |
| 32 | .336(.037) | .270(.024) | .224(.031) | .267(.049) | .047(.013) |
| 96 | .304(.045) | .241(.031) | .188(.032) | .267(.049) | .047(.013) |
| Never | .294(.049) | .217(.031) | .178(.032) | .267(.049) | .047(.013) |
BOLD Moments
| H | RR-AR | BrainVista M | BrainVista O | MIRAGE M | MIRAGE O |
|---|---|---|---|---|---|
| 1 | .408(.039) | .305(.043) | .320(.028) | .090(.008) | .104(.012) |
| 2 | .329(.032) | .227(.036) | .244(.024) | .090(.008) | .104(.012) |
| 4 | .254(.021) | .156(.025) | .173(.016) | .090(.008) | .104(.012) |
| 8 | .193(.015) | .109(.020) | .127(.011) | .090(.008) | .104(.012) |
| 16 | .147(.010) | .082(.014) | .097(.007) | .090(.008) | .104(.012) |
| 32 | .110(.008) | .058(.010) | .079(.008) | .090(.008) | .104(.012) |
| 96 | .073(.008) | .029(.008) | .064(.008) | .090(.008) | .104(.012) |
| Never | .054(.009) | .002(.001) | .058(.008) | .090(.008) | .104(.012) |
Algonauts
| H | RR-AR | BrainVista M | BrainVista O | MIRAGE M | MIRAGE O |
|---|---|---|---|---|---|
| 1 | .514(.033) | .438(.028) | .424(.030) | .240(.011) | — |
| 2 | .449(.024) | .356(.015) | .353(.021) | .240(.011) | — |
| 4 | .368(.016) | .259(.008) | .265(.012) | .240(.011) | — |
| 8 | .308(.013) | .193(.007) | .201(.010) | .240(.011) | — |
| 16 | .268(.011) | .152(.008) | .161(.011) | .240(.011) | — |
| 32 | .241(.011) | .125(.011) | .140(.014) | .240(.011) | — |
| 96 | .222(.011) | .096(.016) | .127(.018) | .240(.011) | — |
| Never | .212(.012) | .022(.008) | .136(.020) | .240(.011) | — |
Comparison with RR-AR. The official-code reruns retain RR-AR's advantage over BrainVista on HCP and Algonauts. On BOLD Moments, RR-AR performs better at short intervals, while the two models become close at long intervals and nearly match without refresh (0.054 versus 0.058). Official-code MIRAGE remains well below RR-AR on HCP; on BOLD Moments, it is close to RR-AR at (0.104 versus 0.110) and performs better at and without refresh. Overall, RR-AR remains a strong reference when recent measured history is available, while the benefit of nonlinear stimulus-based prediction becomes more apparent as that history grows distant.
An HCP audit found no scoring or duplicated-offset error. Head averaging reduced accuracy in a training diagnostic, but scores were low with both readouts; it is therefore only a possible contributor to the low score (Appendix K.4).
Implementation and evaluation details, aggregation procedures, and complete results are provided in Appendices B and K.4.
Response to ML contribution and the role of history
General Question 4 — Contribution to machine learning (W4)
Summary. What empirical and conceptual contribution does the study make beyond benchmarking existing models?
Response. Thank you for pointing this out. ICLR’s reviewer guidelines explicitly recognize new, relevant, and impactful knowledge, including empirical findings, as a contribution. Methodological novelty is therefore not the only form of contribution recognized by ICLR.
Our study contributes new knowledge through three empirical insights:
- Representation quality and predictor complexity provide different benefits. Pretrained stimulus representations generally improve prediction, while increasing downstream complexity does not consistently improve performance under matched inputs.
- The value of model complexity depends on available history. Recent BOLD supports strong linear prediction, while stimulus input becomes more valuable as that history grows distant. History-only controls and the information analysis in Appendix H characterize these contributions.
- The observation process shapes predictability. Paired simulations show that slower hemodynamic responses can prolong history-based predictability, including under nonlinear neural dynamics (Appendices F–G).
This contribution is consistent with empirical representation studies published at ICLR:
- Oota et al., Multi-modal Brain Encoding Models for Multi-modal Stimuli (ICLR 2025) compare pretrained representations using ridge regression and modality-removal analyses.
- Oota et al., Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain) (ICLR 2025) examine instruction-dependent representations using linear encoding and variance partitioning.
Our study complements these representation studies by examining when additional predictor complexity improves BOLD prediction under matched stimulus representations and history availability.
General Question 5 — Predictive contribution of BOLD history (Q3)
Summary. How much of RR-AR's prediction accuracy is supported by BOLD history without stimulus input?
Response.
We refit RR-AR with stimulus inputs set to zero during training, validation, and testing, keeping the same BOLD history, subjects, and evaluation targets. This history-only control measures how much predictive accuracy is supported by past BOLD alone. Figure 12 and Appendix K (Numerical Results) of the revised manuscript provide the complete curves and numerical results. The tables below reproduce mean temporal correlations at representative intervals.
HCP
| Inputs / H | 1 | 2 | 4 | 8 | 16 | 32 | 96 | Never |
|---|---|---|---|---|---|---|---|---|
| History + stimulus | 0.685 | 0.632 | 0.541 | 0.444 | 0.377 | 0.336 | 0.304 | 0.294 |
| History only | 0.682 | 0.624 | 0.517 | 0.392 | 0.294 | 0.222 | 0.149 | 0.120 |
BOLD Moments
| Inputs / H | 1 | 2 | 4 | 8 | 16 | 32 | 96 | Never |
|---|---|---|---|---|---|---|---|---|
| History + stimulus | 0.408 | 0.329 | 0.254 | 0.193 | 0.147 | 0.110 | 0.073 | 0.054 |
| History only | 0.419 | 0.327 | 0.240 | 0.176 | 0.132 | 0.095 | 0.052 | 0.027 |
Algonauts
| Inputs / H | 1 | 2 | 4 | 8 | 16 | 32 | 96 | Never |
|---|---|---|---|---|---|---|---|---|
| History + stimulus | 0.514 | 0.449 | 0.368 | 0.308 | 0.268 | 0.241 | 0.222 | 0.212 |
| History only | 0.507 | 0.426 | 0.319 | 0.234 | 0.172 | 0.122 | 0.073 | 0.035 |
At the shortest intervals, history-only RR-AR performs close to the full-input model across all three datasets. As increases, history-only accuracy declines, while the full-input model retains higher accuracy. The growing benefit of adding stimulus is especially apparent in HCP and Algonauts; the difference is smaller in BOLD Moments.
Thus, recent BOLD history accounts for much of RR-AR's short-interval predictive accuracy, while stimulus input becomes more valuable as that history grows distant.
Response to numerical evidence and generalization
General Question 1 — Complete numerical results (W1)
Response.
We have added the complete numerical results corresponding to the reported figures to Appendix K (Numerical Results) of the revised manuscript.
General Question 2 — Statistical evidence and generalization (W2 / Q2)
Response.
The independent units for statistical inference are held-out subjects: 32 HCP, 10 BOLD Moments, and four Algonauts participants. Prediction time points describe evaluation size but do not increase the independent subject count (Appendix A.8). HCP provides the larger cross-subject evaluation alongside the two smaller cohorts.
Appendix I now reports subject-level paired model comparisons with pointwise 95% confidence intervals and Holm-adjusted p-values, alongside analyses of the paper's twelve main conclusions. For example, in the shared-input benchmark, RR-AR outperforms MLP and GRU at H=1,2,4,8 in all three datasets after correction.
Generalization to unseen stimuli. We added a stimulus-disjoint HCP evaluation: models are trained on Movie 1/2, selected on Movie 3, and tested on Movie 4, with repeated clips excluded across partitions. We test both previously seen subjects and entirely held-out subjects. The comparisons include RR-AR, history-only ridge, RR-stim, MLP, GRU, BrainVista, TRIBE v2, and both shared-input and official-code MIRAGE configurations.
History-conditioned models use preceding available BOLD observations, whereas stimulus-only models receive no measured BOLD. Representative mean correlations for held-out subjects viewing unseen stimuli are shown below. Stimulus-only scores are independent of H.
HCP: unseen subjects and unseen stimuli
Hist. = history-only ridge; BV = BrainVista; S/O = shared-input/official-code. BV and TRIBE v2 use shared inputs.
At short H, recent BOLD history supports high accuracy: history-only ridge matches or exceeds full-input RR-AR, while RR-AR matches or outperforms BrainVista and outperforms MLP and GRU. At long H, history-only accuracy declines and adding stimulus improves prediction; RR-AR remains stronger than BrainVista, MLP, and GRU. Thus, the main patterns—strong linear prediction with recent history and increasing benefit from stimulus as history becomes distant—persist on unseen stimuli in both subject cohorts. Appendix J of the revised manuscript provides the complete curves (Figure 19), numerical results (Tables 22–23), familiar-stimulus comparison, and implementation and statistical details.