acceptodds
Under review as a conference paper at ICLR 2027

Align, Then Adapt: Few-Shot fMRI Visual Reconstruction for New Subjects

10 views

Abstract

Reconstructing visual images from fMRI data is a central but challenging problem in neuroscience. Despite recent progress, current methods fall short when data and computation are limited—precisely the conditions under which this task is most critical. We introduce Adapter Alignment, a staged, architecture-agnostic training procedure for adapting a pre-trained reconstruction model to a new subject with little data. Rather than letting a shared representation space emerge implicitly, lightweight subject-specific adapters are first explicitly aligned to a pre-trained reference subject on a small set of shared calibration images, and the full pipeline is then fine-tuned while preserving this alignment. The resulting representations not only place different subjects in a shared space but also align them semantically. Our results demonstrate significant gains in low-data regimes, improving reconstruction metrics by up to 8%, and pre-diffusion retrieval shows that these gains reflect additional image-specific information in the fMRI embeddings rather than the generative prior. We also propose a coverage-based algorithm to select representative calibration images. Using both techniques together, we surpass baseline models fine-tuned with one hour of data using only 24 minutes of subject-specific data. The procedure is effective across architectures, reference subjects, and datasets, including adapting 3T subjects to a 7T reference.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Computational Neuroscience, fMRI, Neuroimaging, Diffusion, CLIP, alignment, neuroAI

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_X4TnOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents a method for explicit fmri alignment before doing fmri to seen image decoding. They first learn a subject-specific transform of fmri voxels designed for a single reference subject and optimized for seen image decoding. Then subsequent subjects have their fmri voxels projected onto this latent space while the pipeline adapts to cross subject data. The initial reference subject's space creates an alignment anchor point. They show that this improves performance across metrics in low data regimes.

Critical strengths

The paper is well motivated and shows improvements against baselines in low data regimes. They show that it works well at only 1 hour of scan time and can likely improve interpretability by forcing the models to explicitly align in the latent space.

Critical weaknesses

On full data the method only matches baselines like MindEye1. The alignment only helps in low data regimes and the gains in reconstruction performance close at higher data.

I would like to see more interpretability analyses for the claims around "allows for more precise post-hoc analysis after extracting embeddings from the common space" and "the resulting representations not only place different subjects in a shared space but also align them semantically" . The comparisons with baselines in appendix K and L only uses the low data versions. I would like to see some comparisons with the full models to see how much the injected inductive bias from explicit subject alignment helps at scale.

Questions for authors

How are the MindEye2 baselines calculated? The original ME2 pipeline uses multi subject training and single subject finetuning for low data results. Do the baselines also use multi subject pretrain or only single subject? The captions imply that subject 1 was used as pretrain and finetunes were from there.

1 comment

Thank you again for your time and for the detailed review you provided on our paper. As the discussion period is ongoing, we wanted to check in and see if you have any follow-up questions or if there is any part of our response we could clarify further for you. We are ready to provide any additional information that might be helpful.

Reviewer_ziAFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AAMax, a method for adapting fMRI-to-image reconstruction models to new subjects with limited data. After a reference subject pretraining, it aligns a new subject’s adapter to reference embeddings using an alignment objective in embedding space, then fine-tunes the reconstruction pipeline en-to-end while retaining the alignment loss. Furthermore, a complementary algorithm selects the best calibration images for this transfer learning on the minimal set of images. Experiments on the NSD dataset split show improvements over matched MindEye1/MindEye2 fine-tuning baselines (although using a different data split), supported by retrieval, alignment, and cross-dataset analyses.

Critical strengths
  1. Consistent improvements in limited-data transfer-learning. AAMax improves semantic evaluation metrics over baseline fine-tuning across three target subjects (Table 2). Improvements also hold with MindEye2 and alternative reference subjects, without adding inference components. This supports its practical usefulness, although improvement of structural metrics such as PixCorr and SSIM is not significant for the main evaluations.

  2. Evidence of improved representations beyond image reconstruction. Pre-diffusion image retrieval improves on both backbones, while the paper also demonstrates better cross-subject alignment on images excluded from calibration, including analyses of independently defined functional regions. Although limited to one subject pair, these results strengthen the evidence that AAMax improves stimulus-specific representations, beyond final image reconstruction quality alone.

Critical weaknesses
  • Competitive evaluation and architectural coverage are insufficient for the broader claims. The contemporary adaptation methods MindTuner (AAAI 2025) and Duala (CVPR 2026) directly address limited-data adaptation within the MindEye2 family, yet are neither cited nor compared against. Moreover, the recent image reconstruction method Brain-IT (ICLR 2026) is also absent, although it directly regards new-subject transfer learning using very limited data. Its relevance extends to the architecture invariance claim, which is currently supported only by MindEye1/MindEye2 experiments, which share similar architecture in the relevant components. The MindAligner comparison is valuable but shows mixed standalone results and promising combination benefits for only one subject pair, without uncertainty estimates in Table 3. Appendix G’s Ferrante et al. and MindBridge-inspired variants do not reproduce those complete methods. Controlled evidence of competitive performance or added value over a strong contemporary method is needed to establish the contribution’s practical significance.
  • The benefit of staged adaptation is not isolated. After reference subject pretraining, AAMax performs alignment-only adapter training followed by end-to-end fine tuning. The missing control retains the same reference checkpoint, losses, and trainable components but begins joint end-to-end adaptation immediately, without the alignment-only warm-up. A second control of alignment initialization followed by task-only fine-tuning further would isolate the benefit of retaining alignment.
Questions for authors
  1. Competitive validation and generality: Can AAMax+IS demonstrate a reliable advantage over, or added value on top of, strong contemporary adaptation methods under comparable conditions? MindTuner/Duala are relevant within the MindEye family, while Brain-IT provides an important alternative architecture and transfer procedure, which could show the effectiveness and generalization of AAMax to different architectures, as claimed in the paper.
  2. Contribution of staged training: After identical reference pretraining, how does AAMax compare with (a) immediate joint adaptation using both the task and alignment losses, and (b) alignment-only initialization followed by fine-tuning without the alignment loss?
  3. Acquisition-budget clarification: For the 100, 150, and 250 images experiments, how many raw presentations are used? Also, are repetitions averaged, and which sessions supply them? What data inform preprocessing, alignment, and fine-tuning, counting reused responses only once? If possible, please clarify how does this relate to MindEye2’s chronological first-session protocol.
3 comments

Thank you again for your time and for the detailed review of our paper. As the discussion period is still ongoing, we wanted to follow up and ask whether you have any additional questions or if there are any points in our response that we could clarify further. We would be happy to provide any additional information that might be helpful.

Dear Reviewer,

As we approach the end of the discussion phase, we would be extremely grateful if you could take the time to go through our rebuttal and let us know whether your concerns have been addressed. Thank you for your time and consideration.

Thank you for your time and effort in carefully addressing my concerns, and I apologize for my late response. The new evaluations and the updated manuscript greatly alleviate my initial concerns, and I believe the revised manuscript much more clearly explains the method’s role and applicability relative to other adaptation and reconstruction methods. I believe the revised submission is a valuable contribution to the field and have therefore raised my rating. For the final version, I would however recommend including in the main text a more explicit discussion about how your AA integrates with reconstruction and adaption methods more conceptually (for its usability in future methods), clarifying where it can be applied directly and where modifications would be needed.

Reviewer_uzTSOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adapter Alignment (AA), a two-stage training strategy for few-shot fMRI visual reconstruction. It first aligns subject-specific adapters to a pretrained reference subject using shared calibration images, followed by end-to-end fine-tuning with an alignment loss. The authors further introduce a greedy image selection algorithm to improve data efficiency. Experiments on NSD, THINGS-fMRI, and BOLD5000 demonstrate improvements in low-data reconstruction, with reduced training time and subject-specific data requirements.

Critical strengths
  1. The paper addresses an important and practical problem in fMRI visual reconstruction, particularly the high cost of collecting subject-specific training data.
  2. The proposed method achieves promising results in low-data settings. The experiments cover different subjects, datasets, reconstruction architectures, and training budgets.
  3. The experiments are comprehensive, including ablation studies, and cross-subject representation analysis.
Critical weaknesses
  1. The interpretation of the shared representation space needs further discussion. In Fig. 1, the near-perfect cross-subject similarity is achieved on images used for adapter alignment. Although Appendix K provides additional evidence on held-out images and functional brain regions, it remains unclear to what extent the learned representations reflect intrinsic shared neural structure rather than the imposed alignment objective.
  2. Insufficient comparisons with related methods. Although the authors provide extensive experiments, comprehensive comparisons with existing few-shot adaptation methods for fMRI decoding are still lacking. For example, a unified evaluation protocol that pretrains on N−1 subjects and fine-tunes on a new subject would help demonstrate the advantages of the proposed method over existing methods.
  3. Some experimental settings need clarification. Section 4.2 and Appendix F state that 40 eigenvector dimensions are used for the image selection results in Table 2. However, the Subject 2 results in Table 2 match the 20-dimensional setting in Appendix Table 12, rather than the 40-dimensional setting. The authors should clarify this inconsistency.
Questions for authors

See the weaknesses. Besides, the presentation could be improved. Although the paper provides extensive methodological details, theoretical analysis, and experimental results, the overall organization is somewhat confusing, making it difficult to follow the main ideas and contributions. I suggest adding an overview figure in the Introduction to illustrate the motivation and key findings, and reorganizing the paper to improve its clarity and readability.

3 comments

Thank you again for your time and for the detailed review you provided on our paper. As the discussion period is ongoing, we wanted to check in and see if you have any follow-up questions or if there is any part of our response we could clarify further for you. We are ready to provide any additional information that might be helpful.

Thanks to the author's reply, my problem was basically solved, so I decided to raise my score. However, I would still recommend that the authors include the overview figure in the Introduction to illustrate the motivation and key findings, and reorganize the paper to improve its clarity and readability.

We sincerely thank the reviewer for acknowledging our rebuttal and increasing their score. We will definitely retain the overview figure in the Introduction of the final version of the manuscript and incorporate structural re-organizations to improve clarity and readability

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.