acceptodds
Withdrawn from ICLR 2027

BrainSteer: Probing and Reducing the Neural Evidence Utilization Gap in Single-Trial fMRI Time-Series-to-Image Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Does equivalence under a trained neural readout imply equivalence for generative reconstruction? We introduce BrainSteer, a diagnostic and closed-loop steering framework on frozen DynaDiff. BrainSteer decomposes six-TR evidence into a retained component visible to the trained temporal readout and an Exact-Null component annihilated by it, then injects TRAIN-centered evidence through independent zero-preserving, state-conditioned paths. It adds 947,264 trainable parameters (0.947M). On Shared1000, four-subject PixCorr improves from 0.21766 to 0.24590 and SSIM from 0.30864 to 0.35637; non-development relative gains are 12.51% and 14.32%. On frozen VAL64, CORRECT exceeds SHUFFLED structurally in every subject, and five paired ablations favor Full on both structural metrics. Changing Exact-Null identity changes reconstruction while preserving the frozen readout. Time-Only retains about 0.5% of Full structural specificity; the rollout-initial hidden summary preserves most specificity but remains below Full feedback. A 20-state audit finds lower target predicted- error under matched evidence throughout. Matched Exact-Null evidence reduces target-relative error, but neither null-only nor coherent retained/null donor transfer reliably recovers donor identity under fixed recipient Official conditioning. Perceptual/semantic metrics do not improve uniformly. For this studied readout, readout-equivalent evidence need not be reconstruction-equivalent when an independent zero-preserving evidence path is available; its residual utility is context-dependent.

Primary area:
applications to neuroscience & cognitive science
Keywords:
representation learning, fMRI-to-image reconstruction, brain decoding, generative models, diffusion models, neural representation geometry, mechanistic analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WeE8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper builds on DynaDiff, which decodes from the raw single-trial fMRI time series instead of the GLM preprocessing used by most other methods. DynaDiff maps each time point to a latent vector and learns one weight per time point to combine them into a single weighted average. The method adds conditioning on the individual time points, while DynaDiff conditions only on their weighted average. This leads to some improvement in PixCorr and SSIM over DynaDiff.

Critical strengths
  • The paper works with the fMRI time series instead of the usual GLM betas and explores better use of its temporal information, a relatively under-explored research direction.
  • The paper shows some improvement in pixel correlation and SSIM over the method it builds on.
Critical weaknesses
  • The paper is very hard to follow. It uses complex terminology that I find unnecessary, does not add much substance and hinders clarity.
  • The paper only compares against the model it builds on. This is very strange given the multiple works addressing this problem. One might argue that those works report results on averaged fMRI. However nothing prevents running inference on single-trial fMRI or asking the authors of those papers for such results.
  • The generalizability of the results to other methods is questionable as the approach is very technical and tailored to DynaDiff.
  • The work improves only pixel correlation and SSIM. More recent works report much better results on these metrics and take a more methodological approach to capturing the layout and structure of the images measured by these low-level metrics. The work does not compare to such works or discuss them. Additionally the capacity of diffusion guidance approaches might be limited for low-level metrics due to the nature of diffusion models.
  • The paper uses "Official" to denote DynaDiff results. This is a misuse of the term. "Official" would refer to a base model released by the authors of the dataset examined.
Questions for authors

I have no further questions.

Reviewer_MoXBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BrainSteer, which improves fMRI-driven image reconstruction by decomposing fMRI time series into retained and null-space components relative to an existing temporal readout, and introducing both components into a frozen diffusion model through a state-conditioned controller. Experiments show improvements in PixCorr and SSIM on Shared1000. Ablation and identity intervention experiments further validate the contribution of the null-space component to structural reconstruction. However, perceptual and semantic metrics do not improve consistently, and the effect of null-space information also appears to depend on the original generation conditioning.

Critical strengths
  1. The paper formulates neural evidence utilization as a sufficiency problem relative to a specific temporal readout and tests it by changing the stimulus identity of the null-space component and observing the resulting reconstruction changes. This moves beyond simply comparing reconstruction performance and provides an operational approach to analyzing information underutilized by existing decoders.

  2. The Correct–Shuffled comparisons, component ablations, and identity interventions provide complementary evidence. Structural metrics improve consistently across multiple subjects, and the authors' interpretations of the perceptual metrics and donor-transfer results are generally consistent with what the experiments support.

Critical weaknesses

The paper's main contribution remains a methodological validation within a specific model. The null space is defined only relative to a fixed temporal readout, while the additional controller has direct access to temporal information not retained by the original readout. The experiments demonstrate that this information can improve reconstruction through an additional pathway, but it remains unclear to what extent the scientific contribution goes beyond directly reusing information discarded by lossy temporal aggregation. This limits the conceptual novelty of the paper's contribution to the study of neural evidence utilization.

Questions for authors
  1. How does the paper's core contribution differ from simply feeding information discarded during temporal aggregation into an additional trainable controller?

  2. Which results provide further support for the proposed neural evidence utilization gap, beyond demonstrating that additional access to the original temporal information benefits a specific reconstruction model?

Reviewer_qMaROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrainSteer investigates whether fMRI features discarded by a frozen temporal readout remain useful for image reconstruction. It decomposes features into retained and exact-null components and injects them into a frozen diffusion model through a lightweight controller. Experiments report improved pixel-level fidelity and stimulus-dependent effects, although perceptual and semantic improvements are not consistent.

Critical strengths
  1. The operator-relative decomposition provides a precise way to identify features invisible to the frozen temporal readout. The paired identity interventions, independently trained ablations, and matched generation randomness are useful controls for examining how the additional evidence affects reconstruction.

  2. The structural improvements are consistent across the four evaluated subjects, including the three non-development subjects. On Shared1000, mean PixCorr increases from 0.2177 to 0.2459 and SSIM from 0.3086 to 0.3564, while the baseline remains frozen. The paper also reports negative results on perceptual metrics and donor-identity transfer rather than presenting universal improvement.

Critical weaknesses
  1. The specificity experiments do not fully distinguish additional visual information from learned compatibility checking. The training objective explicitly encourages shuffled evidence to reproduce the Official output and suppresses shuffled gates. Consequently, Correct–Shuffled differences may partly reflect detecting consistency with the existing condition. The identity-factorial experiment establishes sensitivity to null-branch identity, but does not establish whether that branch supplies information unavailable from the retained representation. Even redundant evidence could produce this pattern when mismatched. I would find a control replacing the true null component with one predicted solely from retained/Official features particularly informative.

  2. Important evaluation details remain insufficiently clear. Section 4.1 names TRAIN512, VAL64, and Shared1000 without clearly specifying their construction or stimulus-level overlap, including repeated presentations of the same image. This is important for interpreting held-out specificity. There is also a substantial unexplained difference in absolute PixCorr: Appendix Table 5 reports approximately 0.6606 for matched VAL64 evidence, whereas Table 1 reports 0.2345 on non-development Shared1000. Different evaluation sets can yield different scores, but the manuscript should clarify whether selection, preprocessing, or metric computation contributes to this gap. Inconsistent numerical precision further complicates comparison across sections.

Questions for authors
  1. Does true Exact-Null evidence outperform a null-component prediction derived only from retained/Official features, using the same controller architecture and training budget? This would help distinguish additional reconstruction information from compatibility checking.

  2. Could you specify how TRAIN512, VAL64, and Shared1000 are constructed and separated at the image-identity level, and explain the large difference between the absolute PixCorr values in Tables 1 and 5?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.