acceptodds
Under review as a conference paper at ICLR 2027

pRF-Guide: Measured Receptive Fields Guide fMRI-to-Image Reconstruction

Abstract

Learned neural readouts can infer spatial assignments from fMRI, leaving unclear whether independently measured retinotopy contributes additional information. We introduce pRF-Guide, which reweights response-dependent attention with quality-weighted population receptive-field footprints. Matched training controls separate the effect of a structured prior from that of its correct voxel correspondence. Across four NSD participants and three training seeds, true geometry improves reconstructed-image spatial cosine by 0.0039 over a retrained shuffled prior, with favorable participant means throughout. A separately refitted single-participant study supports the contribution of joint footprint–scale correspondence, while fixed-weight interventions on pRF centers and widths further link measured geometry to reconstruction outcomes. Measured retinotopic correspondence contributes information beyond learned spatial capacity, but does not uniformly improve every reconstruction component or metric.

Primary area:
applications to neuroscience & cognitive science
Keywords:
fMRI-to-image reconstruction, neural decoding, population receptive fields, retinotopy, geometry-guided attention, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_2FVvOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose using measured population receptive-field maps as an attention bias in an fMRI-to-image decoder, to improve the spatial structure of reconstructed images.

Critical strengths
  • The idea itself is simple and can be theoretically incorporated into fMRI-to-image decoders (probably more relevant with limited data)
  • The paper is honest and reports negative results.
Critical weaknesses
  • Underperforms prior methods: The paper proposes a method to improve fMRI-to-image decoding, but it underperforms prior methods on most metrics. It is worse on every metric in Table 3 than MindEye, the method it is built on.
  • Insufficient evidence of usefulness: Overall, the evidence does not establish that the proposed method, or incorporating pRF information, is useful.
  • Small, non-significant main results: In the main results (Table 2), the reported differences are very small and not statistically significant, or significant at all for that matter.
  • Unclear role of the bias: The learned attention scores can override the pRF-based bias, so it is unclear how much the bias actually shapes the final results.
  • Unjustified metrics: The proposed Spatial and Center metrics are not sufficiently justified. It is unclear why feature energy in a deep network layer is a good indicator of spatial accuracy. Decades of research in computer vision and human vision have produced established shape and position metrics, such as mask IoU/Dice, boundary F-score and contour distances, and the COCO annotations needed to compute them were available.
  • Necessity of pRF measurements: It is unclear whether pRF-like information could be estimated directly from the training data, and whether separate pRF measurements are needed at all.
Questions for authors

I have no questions.

Reviewer_951LOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes pRF-Guide, the paper adds population receptive fields (pRFs) into attention-based fMRI-to-image reconstruction. The method uses pRF as spatial attention biases. Experiments on NSD compare true pRF geometry against learned attention and shuffled pRF priors.

The paper addresses an interesting question and provides useful controlled experiments. However, the proposed mechanism appears largely incremental relative to existing receptive-field-based approaches, and its advantages over simpler spatial priors remain unestablished. More importantly, the additional gain from correct pRF correspondence is small, the claimed spatial alignment is not directly validated.

Overall, I find the current evidence insufficient to establish a sufficiently strong methodological or empirical contribution.

Critical strengths
  • The paper investigates a meaningful question -- adds retinotopic pRF information into brain decoding image reconstruction.
  • The retrained shuffled-pRF control is a useful design for separating the contribution of correct voxel correspondence from that of a structured spatial prior.
Critical weaknesses
  1. Limited novelty and insufficient justification of the proposed formulation. Using receptive-field geometry to constrain visual representations and reconstruction has been explored in prior work, including fwRF (St-Yves and Naselaris, 2018) and pRF-based visual reconstruction (Senden et al., 2019). The main methodological contribution here appears to be incorporating Gaussian pRF footprints as an additive attention bias. However, the paper does not establish why this formulation is preferable to simpler alternatives, such as pRF-weighted pooling, fixed Gaussian attention, or directly concat pRF coordinates into voxel embeddings.

  2. The contribution of correct pRF correspondence is relatively small. In Table 10, shuffled pRFs improve spatial cosine by 0.00696 over learned attention, whereas correct pRF correspondence provides an additional improvement of only 0.00390. Thus, approximately 64% of the total improvement is already obtained with an incorrect voxel-to-pRF assignment. While the consistent direction of the true-versus-shuffled improvements is encouraging, these results suggest that much of the benefit may come from a generic structured spatial prior rather than accurate retinotopic correspondence.

  3. Lack of direct validation of retinotopic correspondence. The main spatial metrics are computed from reconstructed-image features rather than the model's internal voxel-to-location assignments. Improved spatial cosine or centroid error does not necessarily indicate that the learned attention follows measured retinotopy. Since the method explicitly biases attention using measured pRFs, a direct analysis of the resulting attention maps would be valuable.

  4. Inconsistent improvements in low-level reconstruction quality. In Table 4, the geometry-free low-level branch achieves higher GridSSIM than the full model (0.3351 vs. 0.3263), with larger structural losses reported in peripheral regions. This contrasts with the improvements in SwAV-based spatial metrics. The paper does not sufficiently explain why measured spatial information improves feature-level alignment while degrading local RGB structure.

References:

  • St-Yves, G., and Naselaris, T. (2018). The feature-weighted receptive field: an interpretable encoding model for complex feature spaces. NeuroImage.
  • Senden, M., et al. (2019). Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception. Brain Structure and Function.
Questions for authors
  1. Can simpler spatial priors, such as Gaussian-weighted pooling or fixed spatial attention, reproduce the reported improvements?
  2. Do the learned attention maps exhibit retinotopic alignment beyond what is directly imposed by the measured pRF bias?
  3. What explains the degradation in local GridSSIM when geometry is introduced into the low-level reconstruction branch?
Reviewer_kMmUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose a receptive field biased attention in the fMRI to image decoding task. During the biasing stage, the weighting factor for each voxel is determined using the pRF R2R^2 and (presumably during image viewing?) voxel noise ceiling.

The authors demonstrate improved image decoding results on both high-level semantic metrics and low-level visual metrics.

Generally the paper is quite difficult to read. Their use of scientific & technical terms does not match what I would expect from my understanding of the field.

Critical strengths
  1. Population receptive fields have been often overlooked in the modern/recent fMRI to image decoding literature, and since they can be derived from a localizer experiment, or from data in event-driven fMRI designs, it makes sense to utilize this information as an additional constraint.

  2. (unclear due to ambiguous language usage) The authors seem to use both permutation tests and bootstrap tests as validation for statistical significance. I think efforts to characterize the statistical significance of methods in general are worth a positive evaluation.

Critical weaknesses
  1. The terminology used in this paper is very non-standard, many grammatical issues. Genuinely one of the most difficult to understand papers in my batch. Here are a few examples:

    • Line 15, unclear what correct voxel correspondence means. In general the abstract is effectively word spaghetti to me.
    • Line 43, what does this test refer to here?
    • Line 47, what is True, shuffled, and absent geometry
    • Line 52 generally word salad here
    • Line 103 what is a cell
  2. The most serious issue, in my perspective is this work does not address why decoding images from passive fMRI viewing is a scientifically insightful task. What does this work teach us about the brain? In my view, nothing beyond what we currently know. What does this work teach us about the machine learning? In my view, again nothing beyond what we currently know. So in my view, this is scientifically dubious, especially when such powerful naturalistic image priors are combined with brain information, the fact that high quality images can be decoded from the brain does not yield any scientific insight.

Questions for authors
  1. Details about fMRI pre-processing needs to be provided. Can you explain if data is used from a subject-specific volume? Or just the cortical surface? Is it remapped to a standard template?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.