acceptodds
Withdrawn from ICLR 2027

CortexFlow: Topology-Aware Spectral Decoding with Confidence-Adaptive Diffusion for fMRI-to-Image Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Most fMRI-to-image reconstruction methods, despite advances in generative modeling, begin by representing spatially organized cortical activity as a flat vector. This overlooks the way visual cortex actually works. Cortical activity is organized across functionally specialized regions and spread over a folded cortical surface, where neighboring locations respond continuously. Beyond that, the relative importance of low-level structure and high-level semantics varies across scenes. Natural landscapes may depend more on spatial layout and texture, whereas scenes depicting events or activities may require a stronger emphasis on object identities and their interactions. A fixed diffusion schedule and guidance strength ignore this variability. We propose CortexFlow, which builds a within-hemisphere graph over the fsaverage cortical surface, extracts its Laplacian eigenmodes, and integrates branch-specific conditional flows in these spectral coordinates to incorporate cortical topology into both the VGG structural branch and the CLIP semantic branch. Two confidence gates estimate the feature-level agreement of the respective predictions. Structural confidence adapts the diffusion start, while semantic confidence controls the strength of semantic guidance. CortexFlow achieves state-of-the-art reconstruction performance on NSD across a broad range of low- and high-level metrics, with particularly strong structure-sensitive performance in SSIM. Cortical projections of topology-induced updates further provide a model-level interpretation of the decoded representations.

Primary area:
applications to neuroscience & cognitive science
Keywords:
fMRI-to-image reconstruction, neural decoding, diffusion models, cortical topology

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Jx1iOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CortexFlow, an fMRI-to-image reconstruction framework designed to address two main issues in existing methods: the flattening of cortical spatial structure into 1D vectors, and the use of fixed diffusion hyper-parameters across all brain responses. Experimental results show that CortexFlow achieves competitive reconstruction performance on NSD across a broad range of low and high level metrics, with particularly strong structure-sensitive performance in SSIM.

Critical strengths
  1. Across the four evaluated subjects, the method shows clear improvements on structural metrics like SSIM (reaching 0.502) and competitive numbers on high-level identification metrics compared to published baselines.

  2. Rather than using a fixed noise level and fixed guidance for every test sample, using estimated feature agreement to pick the starting step and CFG scale is well-motivated and makes intuitive sense.

Critical weaknesses
  1. The continuous spectral flow matching on the graph Laplacian is presented as a central contribution of the paper. However, the ablation in Table 2 only tests "Baseline" vs. "Baseline + topology". There is no comparison against straightforward graph neural networks (GCN/GAT) on the mesh, simple diffusion smoothing, or even a basic MLP over surface coordinates. As a result, it is impossible to tell if the complex ODE integration is necessary or if a simpler graph operator would achieve the same performance.

  2. The overall system is very fragmented. The structural and semantic branches are trained separately as disjoint jobs. The confidence gates are trained on a separate held-out split with a separate loss. At test time, one branch runs 1,800 steps of DIP optimization, while the other runs an ODE flow and a 38-step diffusion process. This multi-stage design requires significant tuning, makes error propagation hard to track, and feels like an accumulation of loose heuristics rather than a unified framework.

  3. My main concern about soundness is whether the heavy mathematical machinery of continuous spectral flow is actually doing what the authors claim. The model maps voxels to a template surface, truncates down to 128 low-frequency modes, integrates an ODE with 16 steps, pools back to 128 discrete functional clusters, and adds a residual MLP update. The paper does not provide an ablation comparing this continuous flow against much simpler alternatives, such as basic graph convolutions or standard spatial smoothing on the surface. Without that baseline, it is unclear if the gains come from the continuous spectral ODE or just from having an extra set of trainable residual parameters.

Questions for authors
  1. Why choosing a continuous heat-process flow matching ODE over standard graph convolutions (such as ChebNet or GCN) on the surface mesh? Did you test any simpler graph baselines, and how do their reconstruction metrics compare?

  2. How long does inference take for a single image? Running 16 RK4 steps for the ODE plus 1,800 optimization iterations for DIP and multiple diffusion steps seems very slow compared to direct feed-forward decoders.

  3. Adding the confidence gates only gives a small improvement over the topology baseline (PixCorr goes from 0.387 to 0.392, SSIM from 0.491 to 0.502). Given that training these gates requires a dedicated dataset split and extra network heads, is this modest gain worth the added complexity?

Reviewer_qymvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduced CortexFlow, a framework for fMRI to image reconstruction, that incorporates cortical topology into visual feature decoding and adapts diffusion generation according to feature reliability.

The method consist of two primary contributions. First, topology-conditioned cortical flows operate in a lower dimensional spectral representation, learning branch specific refinements for VGG and CLIP feature decodings. Second, two confidence predictors (MLPs) estimate the reliability of decoded features and adapt diffusion starting noise level and CFG strength.

Empirical study on NSD report strong quantitative performance across metrics against baselines. Ablations demonstrate that improvements are seen from both topology aware decoding and confidence adaption. Qualitative results show reconstructions, as well as presenting model failure cases.

Critical strengths
  1. Strong empirical performance - CortexFlow achieves competitive results against a broad range of metrics. Component ablations show topology improves SSIM, with additional gains from confidence adaption.

  2. Well motivated study - Incorporation of spatial cortical activity is sensible for fMRI decoding. Current methods typically utilise flattened neural responses, discarding anatomical structure.

  3. Empirical / implementation - The paper includes ablations, sensitive analysis, qualitative results, failure cases, and cortical visualisations. Implementation details for reproducibility are also included.

Critical weaknesses

Major:

  1. The primary concern is whether the proposed conditional spectral flow is necessary to achieve the reported improvements. The topological module is shown to lead to improved quantitative metrics, however the isolated components (spectral representations, learned flow dynamics, additional network params/adaptor etc) are not ablated. For example, a FM objective is use to learn mappings from Gaussian states toward cortical spectral representations (x2) conditioned on functional data. However, these cortical representations are directly computable from input fMRI measurements. Consequently, it is unclear whether the improvements arise from the flow matching formulation, or simply from incorporating spatial/cortical information. A useful missing baseline is therefore to directly inject the computed cortical spectral representation, without learning a conditional flow. Since topology conditioned flow is presented as a central methdological contribution, this is a significant limitation

  2. Similarly, the confidence-adaptation ablation does not establish whether a response-specific adaptation is required over a fixed hyperparam section. The motivation to align confidence with feature agreements to improve diffusion start states and CFG strengths is intuitive, however the experimental evidence does not support its necessity. The main ablation compares the adaptive approach against a fixed diffusion start index of 14 and strength 5. However, it does not demonstrate the the adaptive method outperforms a tuned fixed configuration As the incremental gains from confidence adaptation are modest (improved SSIM 0.011), comparison against the strongest fixed baseline is necessary to justify the second contribution.

Minor:

  1. Possibly limited methodological novelty. Each component builds on existing approaches, e.g. graph Laplacians, FM dynamics, diffusion hyperparam adaption etc. The contribution primarily lies in how these are combined for fMRI reconstruction. This in itself is not a limitation (combining different methodologies for improved downstream tasks is in itself a novelty), however, it makes the aforementioned major weaknesses particular important.

  2. Computational considerations - The methodology has two trained spectral flows each required ODE integration. Computational runtime information at runtime would help to determine the trade offs with performance and model complexity.

Questions for authors
  1. Since the cortical spectral target is directly computable from the fMRI input, how does the proposed conditional flow compare with directly injecting these spectral features? i.e does continuous flow matching yield an improvement beyond a simpler mechanism.

  2. Does the confidence-adaptive procedure outperform the best fixed diffusion start index and CFG strength selected using the same validation protocol? And can it be established if this comes from adaptation of diffusion start vs CFG strength.

Reviewer_uYBfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CortexFlow proposes two additions to the prior Brain-IT framework for fMRI-to-Image reconstruction:

  1. Topology-aware spectral processing that incorporates cortical surface geometry into visual decoding.

  2. Confidence-adaptive diffusion that adjusts structural preservation and semantic guidance based on estimated prediction reliability.

Although not explicitly mentioned in the paper, the method heavily builds on Brain-IT's functional voxel grouping mechanism, its dual-branch VGG/CLIP reconstruction architecture, as well as its DIP-based structural reconstruction which is then fed as an initial guess into the semantic-guided diffusion.

The authors report improvements on the NSD dataset.

CortexFlow requires anatomical alignment of all brains prior to training and inference. It therefore performs a pre-processing step of warping/transforming all the fMRI scans of the NSD dataset (of all subjects --both Train Sets and Test sets) to the same coordinate system.

Critical strengths
  1. The Topology-aware spectral processing, which incorporates cortical surface geometry into functional brain representations.

  2. The Confidence-adaptive diffusion, which adjusts image generation according to the estimated reliability of structural and semantic predictions.

Critical weaknesses

(1) Flawed experimental comparisons and exaggerated claims:

  • QUANTITATIVE COMPARISONS: The Brain-IT results reported in Table 1 (page 8) are inconsistent with and significantly worse than the original results reported in the Brain-IT paper [ICLR 2026]. This is true for all 8 full-data metrics reported in the current paper. The paper does not disclose nor explain why it reports wrong values for Brain-IT.

In fact, when using the correct original Brain-IT reported results, CortexFlow's PixCorr improvement significantly shrinks – from the reported gap of 0.026 to only 0.006, and its SSIM improvement shrinks from 0.050 to 0.016. Moreover, when the correct Brain-IT values are used, CortexFlow performs worse than Brain-IT on AlexNet-5 and EfficientNet. Discrepancies exist also in the one-hour transfer results as well.

Importantly, a close look at Tables 1&2 shows that the falsely-reported Brain-IT scores in Table1 are identical to the authors' own "Baseline" ablation in Table2. Although CortexFlow heavily builds on Brain-IT, this “Baseline” ablation is NOT identical to Brain-IT at all, for 2 main reasons: (i) CortexFlow uses pre-alignment of all fMRIs, by transforming them into a shared cortical space. Such registration introduces interpolation errors, distortions and inaccuracies in the input fMRIs (as acknowledged by the authors), whereas Brain-IT does not require any anatomical alignment and can use the original fMRIs without any distortions. (ii) CortexFlow uses different basic training losses in its 2 branches. These significant differences may explain the significant gap between the author’s Baseline's inferior performance compared to the original reported performance of Brain-IT.

Table 1 must be corrected using the original published Brain-IT results, and the exaggerated claims about CortexFlow's state-of-the-art superiority (including those in the Abstract), must be toned down accordingly.

  • QUALITATIVE COMPARISONS: The visual qualitative comparison to prior methods is very limited, and is hidden in Appendix D (Figure 5). The few (cherry-picked?) results displayed there appear to show inferior visual quality compared to Brain-IT in most of the shown images. Why not show a visual comparison on many more test images out of the 1000? Comparisons on hundreds of test images appear in previous papers.

(2) Insufficient acknowledgment of Brain-IT; Insufficient separation between the contributions of the current paper and those of prior work:

CortexFlow relies heavily on Brain-IT's architecture, including its functional grouping into 128 voxel clusters with 512-dimensional tokens, its dual VGG/CLIP decoding branches, its DIP-based structural reconstruction, and its use to initialize the subsequent semantic-guided diffusion. These are all fundamental contributions of Brain-IT, not of CortexFlow.

Yet, Brain-IT is barely mentioned in the paper (only in half a sentence in Line 102 in the Related Work section, and then compared against in Table 2 on Page 8 using flawed (much worse) reported results). Brain-IT is not acknowledged at all in Section 3 (Method), where much of Brain-IT’s architecture is described as if invented by the current paper. The additions and novelty of the current paper beyond Brain-IT are the topology-aware spectral processing, and the confidence-adaptive diffusion. It is a pity that the authors did not bother to separate between their own contributions (which are potentially interesting on their own) and those they borrowed from prior work.

In light of that, Sections 2 and 3 must undergo a MAJOR rewrite, to explicitly acknowledge and clearly separate between Brain-IT’s contributions and inherited components, and CortexFlow's actual contributions. Any reuse of Brain-IT's published code within CortexFlow’s code should also be acknowledged.

Questions for authors
  1. Why do the Brain-IT results in Table 1 differ from the originally published results, and why are they identical to CortexFlow's Baseline Ablation in Table 2?
  2. Why do you say that the evaluation protocols of prior methods are different then yours? (Lines 363-365). All previous papers evaluate on the same 4 NSD subjects and on the same 1,000 shared test images, making the authors' statement regarding differing evaluation protocols unclear.
  3. Can the authors quantify the amount of error/inaccuracies introduced into the fMRIs by the fsaverage cortical surface alignment? One possible way to quantify this is by transforming an fMRI scan to the shared cortical space, then transform it back to its original space, and compute the difference between the original and transformed fMRI. How much relative error is introduced by this warping process? Quantifying this error is important, since – unlike CortexFlow -- prior fMRI decoding methods do not require any anatomical alignment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.