acceptodds
Withdrawn from ICLR 2027

From -Level to BOLD-Level: Modeling Neural Dynamics for Visual Brain Encoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing visual brain encoding models typically predict GLM-estimated responses, which summarize each stimulus-evoked BOLD time course as a single condition-wise amplitude. However, fMRI is acquired as time-resolved BOLD signals, and compressing these signals into a single amplitude does not retain the temporal information needed to characterize response timing and evolution. We propose , a stimulus-aligned BOLD encoding framework that directly predicts stimulus-aligned BOLD responses by decomposing each condition into a temporal response profile and a signed voxel-wise spatial response pattern, modeling the two factors with asymmetric dual-branch diffusion, and recombining them under direct BOLD-level supervision. Across four NSD subjects, BOLDSyner achieves the state-of-the-art prediction performance at both the and BOLD levels, increasing -level correlation from 0.687 to 0.739 and BOLD-level correlation from 0.157 to 0.610. Compared with -level signals, the predicted BOLD responses recover category-dependent temporal effects observed in measured responses ( across 120 subject–ROI–category cells), while preserving the category selectivity and semantic relationships captured by -level signals. Collectively, these results imply the effectiveness and necessity of mechanistically grounded, time-resolved BOLD signal prediction.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Visual Brain Encoding, fMRI, BOLD Response Prediction, Temporal-Spatial Modeling, Diffusion Models, Neural Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wUEDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Overall, the paper addresses a meaningful problem by extending visual brain encoding to time-resolved BOLD prediction. However, the mechanistic interpretation of the predicted temporal responses remains limited, and the lack of representative response trajectories and component-wise ablation studies weakens the evidence for the proposed framework's effectiveness and methodological novelty.

Critical strengths

The work extends visual brain encoding from static β responses to stimulus-aligned BOLD time-series prediction. It further develops a joint generative modeling framework based on temporal–spatial decomposition to model stimulus-evoked responses.

The problem has clear scientific significance. Extending visual brain encoding from static β responses to stimulus-aligned BOLD responses enables the analysis of response latency, width, and category-dependent temporal differences, which cannot be fully characterized by conventional β-level prediction.

Critical weaknesses

Limited mechanistic interpretation of neural dynamics. The predicted responses consist of fixed six-time-point BOLD windows, reflecting hemodynamic responses rather than neural activity dynamics directly. Although recovering category-dependent temporal effects is valuable, these results do not establish that the model captures the underlying mechanisms of neural dynamics. Furthermore, the main text primarily presents average BOLD response changes over a short post-stimulus window, without showing representative measured-versus-predicted response trajectories. This limits the evidence supporting the temporal branch's ability to characterize how responses evolve over time and whether different visual categories exhibit distinct temporal response profiles.

Insufficient validation of methodological novelty and effectiveness. The proposed framework relies on a conventional diffusion modeling technique, but the paper does not sufficiently demonstrate the specific advantages of its proposed design for the target problem. In particular, the lack of component-wise ablation studies makes it difficult to determine whether the temporal–spatial decomposition, asymmetric dual-branch design, and direct BOLD-level supervision each contribute to the reported improvements. Consequently, the experimental evidence does not adequately establish the effectiveness of the proposed components or demonstrate their advantages over simpler alternative designs.

Questions for authors

See the Weaknesses

Reviewer_mWGYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Visual brain encoding (image → fMRI response) typically predicts the β values estimated by GLM: each stimulus and each voxel has only one amplitude, and the temporal information is completely suppressed. This paper proposes to instead predict the "stimulus-aligned BOLD time window" (6 TR bins after the stimulus), and proposes a generative model BOLDSyner to do this. The authors claim that it achieves SOTA at both the β level and the BOLD level, and the predicted BOLD retains the category-related temporal effects.

Critical strengths
  1. Moving from condition-wise β to time-resolved, stimulus-aligned responses is a sensible direction for image-to-fMRI models. The paper shows that this can be done without losing β-level accuracy: BOLDSyner's V^\hat V correlates with GLM β at least as well as the baselines do. If the evaluation concerns below are resolved, this is a useful setting for the community.

  2. The ROI- and distance-aware noise covariance gives a clear gain over isotropic noise in the ablation (0.262 → 0.625 BOLD correlation), suggesting that structured priors over voxels matter for generative encoding models.

Critical weaknesses
  1. My main concern is the absence of a baseline. MindSimulator and SynBrain are trained only on β and never see the BOLD target, so a jump from 0.157 to 0.610 mainly reflects training on a different target, not a better model. The obvious control is absent: a deterministic regressor (ridge or MLP from the same CLIP features) trained on the same (U,V)(U, V) factors and recombined with the same operator RR. Without it, the paper cannot show that the dual-branch diffusion contributes anything beyond the target construction.

  2. By Eq. (22), Bc=VcUc⊤B_c = V_c U_c^\top: for each image every voxel shares one temporal profile, up to scale and sign. Predicted "category-dependent temporal effects" in an ROI can therefore only arise from whole-brain profiles that differ across images, weighted by each ROI's category selectivity. They cannot come from ROI-specific dynamics.

    The "measured" profiles are also hard to interpret. With NSD's 4-s SOA, a 4–12 s window overlaps the following 2–3 trials and no deconvolution is applied. Finally, the 0.922 temporal-profile correlation needs a canonical-HRF reference: a 6-bin profile dominated by the hemodynamic shape will correlate highly with almost any HRF-like prediction.

Questions for authors
  1. What exactly is YY in the "Main BOLD-level" evaluation: raw repetition-averaged windows, or the FIR/rank-1 reconstruction? Is correlation computed across voxels, time bins, images, or all of them pooled? How are β-only baselines converted to BOLD in that column?

  2. What does a deterministic baseline (ridge or MLP from CLIP features to (U,V)(U, V), recombined with RR) achieve on Table 1? What does the diffusion add over it?

  3. Given Bc=VcUc⊤B_c = V_c U_c^\top, can the model produce different temporal shapes in two ROIs for the same image? If not, please restate the claims in Sec. 4.2 accordingly. How large are the 0.922 and r = 0.866 numbers when the predicted profile is replaced by a canonical HRF scaled by the predicted V^\hat V?

  4. How do you handle overlap with subsequent trials (4-s SOA) in the 8-s windows? Why does the window start at ≈4 s rather than at onset, given that this omits the rising phase?

Reviewer_eKzqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BOLDSyner, a visual encoding model that predicts stimulus-aligned BOLD responses: 6 TRs after image onset compared to Betas estimates. From NSD's raw fMRI time series, the authors derive two targets per image: one short response time course (6 time points) and one voxel map of response strengths across voxels. Two diffusion models conditioned on CLIP image features generate these two targets, against the measured BOLD response. The method is tested on 4 NSD subjects and the authors report higher beta-level and BOLD-level accuyracy compared to the baseline models, preserved category selectivity and concept-level representational similarity.

Critical strengths

Strengths of the paper:

  1. Meaningful change of target: Instead of one predicting beta maps, the model predicts a time-resolved response: a voxel map combined with a per-image time course over a 6-TR window. This is a natural next step for encoding models, and the voxel map stays directly comparable with beta analyses.
  2. Extensive evaluations: Four subjects with per-subject results, temporal, category and semantic analyses, ablations, and code.
Critical weaknesses

Some weaknesses of the paper, mostly regarding the baseline comparisons:

  1. The main claim is unclear because it does not test the baselines in the same conditions. The model's output is a voxel map multiplied by one time course per image (a 6-TR window in which all voxels share the same shape). Yet all baselines (Linear, MindSimulator, SynBrain) were trained on beta maps and are given one fixed HRF, so the BOLD-level gain (from 0.157 to 0.610) mostly shows that a model trained on the target beats models that were not. A fair comparison needs models that predict the BOLD signal directly, such as encoding models trained on BOLD time series (e.g., TRIBE, d'Ascoli et al., 2025; TRIBE v2, d'Ascoli et al., 2026), or at least a ridge/FIR encoding model trained on the same windows from the same CLIP features. A second simple comparison would be a predicted beta map combined with voxel-specific HRFs, which gives the same kind of output with a time profile per voxel. Without these, the gain may come from predicting the right temporal shape rather than from the proposed model.
  2. The timing results are over-interpreted. In the authors' own model, each voxel has one fixed response shape for all images, and an image's time course is simply the average shape of the voxels it activates most. The category timing effects therefore reflect which voxels respond, not changes in timing, and a beta map combined with voxel-specific HRFs would likely show the same effects; the paper only compares against beta with one fixed HRF. The ROI time-course correlation (r=0.92) also lacks a reference, since any HRF-shaped curve correlates highly, and each window also contains responses to the next images (NSD shows a new image every 4 s).
Questions for authors

Some clarifications:

  • Could you clarify, in the Main BOLD-level, how are the baselines' predictions obtained?
  • What is the linear basline?
  • The targets are built from windows cut after each image onset rather than from a deconvolution. How is the overlap with responses to the following trials handled (NSD shows a new image every 4 s)?
  • Was the rank-one fit restricted to training images?
  • Regarding the metrics: how are MSE and correlation computed (per voxel, per image, or pooled)?
  • It is not clear why the authors do not directly predict voxel or ROI specific time courses, rather than a single time course?
  • Why use two separate diffusion branches for the time course and the voxel map, rather than a single joint model as in recent approaches such as ReDi (Kouzelis et al., 2025), which generates two coupled representations in one diffusion model with a shared timestep and balances their sizes?
  • The paper states that 'stimulus-conditioned prediction of such time-resolved BOLD responses remains largely unexplored in visual brain encoding', but naturalistic encoding models such as Wen et al. (2018, cited) and TRIBE already predict BOLD time courses. Could you qualify this claim and clarify what your formulation adds?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.