acceptodds
Under review as a conference paper at ICLR 2027

Beyond Alignment: Test-Time Neural Consistency for Cross-Subject Visual Decoding

Abstract

Cross-subject visual decoding aims to reuse pretrained brain decoders across individuals with limited subject-specific data. Existing approaches mainly address functional alignment, enabling neural representations to transfer across subjects, but they do not explicitly constrain the subsequent generative trajectory to remain faithful to the neural evidence observed at test time. We propose , a training-free inference framework that reintroduces complementary neural and structural evidence into two stages of cross-subject generation. During diffusion-prior sampling, feeds the observed source-subject fMRI back into the latent trajectory through a frozen learned neural forward operator. During image enhancement, uses the fMRI-derived low-level estimate as a reliability-weighted multi-scale structural reference. A key challenge arises in MSC: unlike physical inverse problems with known forward operators, the visual-to-fMRI mapping is only available through an imperfect learned surrogate. We therefore use weak step-wise guidance together with latent refinement rather than exact measurement fitting. Our analysis shows that such refinement simultaneously reduces latent estimation error and propagates surrogate-model residuals, inducing an implicit spectral regularization that preferentially corrects well-observed directions while limiting error amplification in weakly constrained modes. On the Natural Scenes Dataset, DS-DC consistently improves a pretrained MindAligner decoder across directed cross-subject transfers without updating its parameters. MSC reduces source-fMRI measurement MSE by %, while the full framework improves all reported low- and high-level decoding metrics. These results show that test-time consistency provides a principled complement to cross-subject alignment, while learned neural constraints are most effective when enforced finitely rather than aggressively.

Primary area:
applications to neuroscience & cognitive science
Keywords:
fMRI visual decoding, cross-subject decoding, measurement-space consistency, neural forward modeling, structural guidance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_siooOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a test-time training-free method that improves cross-subject visual decoding consistency through two modules: MSC and QSC. Specifically, MSC predicts the source-subject fMRI from the current clean latent estimate to provide gradient-based guidance; QSC is applied during the image enhancement stage to pull the VGG features closer to the fMRI-derived blurry image.

Critical strengths
  1. Clear motivation. Existing representation alignment methods focus on transferring representations across subjects but do not explicitly ensure whether the generated samples are faithful to the observed fMRI. This is intuitive and clearly articulated.
  2. The proposed method is training-free and potentially applicable to other visual decoding frameworks.
Critical weaknesses
  1. The cross-subject setting assumes the source subject has quite limited training data. Yet MSC needs a subject-specific encoding model FsF_s for this subject. If the encoding models (NeuroFlow and SynBrain) were trained on the source subject’s full 40 sessions, then DS-DC uses far more data than the MindAligner baseline (only one session). This could undermine the fairness of the comparison.
  2. Several prior studies have explored ideas similar to MSC. Brain-optimized inference improves reconstructions of fMRI brain activity [1] uses encoding models to optimize reconstruction results; while BrainDiVE [2] guides diffusion process with encoder gradients. The paper should discuss and compare these methods.
  3. Compared to using MSC alone, the full DS-DC performed worse on 6 out of 8 metrics. This suggests that QSC may primarily acts as a regulator that balances low-level visual fidelity with high-level semantic quality rather than "complementary roles".
  4. The theory adds little novelty and does not cite relevant classical literature.

[1] Kneeland, Reese, et al. "Brain-optimized inference improves reconstructions of fMRI brain activity." arXiv preprint (2023).

[2] Luo, Andrew, et al. "Brain diffusion for visual exploration: Cortical discovery using large scale generative models." Advances in Neural Information Processing Systems 36 (2023): 75740–75781.

Questions for authors
  1. For each source subject, what training data were used to train NeuroFlow and SynBrain?
  2. MindEye2/MindAligner already use blurred reconstructed images as the starting point for img2img. A simple baseline would be to adjust the img2img strength or the low-level mixing weights within the existing pipeline. Could the authors compare QSC against these alternatives?
3 comments

W1 and Q1:

Thank you for highlighting this data-budget concern. The original NeuroFlow and SynBrain operators followed their respective full-data NSD protocols, which report 9,000 training images per subject, for subjects 1, 2, 5, and 7, with the 1,000 shared test images held out. We acknowledge that these operators used more new-subject supervision than the one-session MindAligner baseline.

To address this, we retrained both operators using exactly the same new-subject and known-subject training data as the corresponding MindAligner configuration. For each evaluated direction, the new subject is restricted to the same single session, while the known subject uses the same full training split already available to the baseline. Forward-model training and alignment reuse these training sets, introducing no additional paired fMRI–image samples. We report the revised few-shot results separately from the original full-data results and use the matched-budget comparisons to assess performance under limited-data cross-subject decoding.

Under the matched data budget, DS-DC retains improvements in pixel and structural fidelity across all four subject groups. The full-data operator remains stronger on some higher-level feature measures, potentially benefiting from broader subject-specific stimulus–response coverage. These results show that useful reconstruction gains persist without additional paired training data, although the trade-off across metrics changes.

DS-DC performance with full-data and matched-budget NeuroFlow operators.

Subj.MethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
S1MindAligner0.22250.41700.87830.92680.83940.81830.81520.4606
+ DS-DC (full-data)0.23960.42070.89360.93750.85480.83430.79560.4519
+ DS-DC (few-shot)0.24330.44220.89370.93130.85130.81970.79120.4701
S2MindAligner0.21630.42920.87650.93640.83840.83140.80210.4515
+ DS-DC (full-data)0.22050.42400.88010.93700.86070.84170.79030.4475
+ DS-DC (few-shot)0.22470.44650.88110.92980.85160.82800.78590.4669
S5MindAligner0.19610.41120.84370.91990.84540.83200.79470.4518
+ DS-DC (full-data)0.21910.41510.85620.92970.87430.85910.76870.4369
+ DS-DC (few-shot)0.22400.43900.86010.92130.87020.84410.76430.4545
S7MindAligner0.18710.40960.81660.88340.80060.77710.84500.4850
+ DS-DC (full-data)0.19160.41260.82170.89190.81210.78700.83760.4790
+ DS-DC (few-shot)0.19460.43620.82620.88000.81080.77470.82820.4943
AllMindAligner0.20550.41670.85380.91660.83090.81470.81420.4623
+ DS-DC (full-data)0.21770.41810.86290.92400.85050.83060.79800.4538
+ DS-DC (few-shot)0.22170.44100.86530.91560.84600.81660.79240.4715

Full-data denotes the original NeuroFlow operator; few-shot denotes the retrained operator using the same new-subject and known-subject training data as the corresponding MindAligner baseline. Best results within each subject group are shown in bold.

W2:

Thank you for highlighting these studies. We will expand Related Work to clarify their connections to MSC.

BOI [1] selects and iteratively regenerates candidates using encoder scores, whereas MSC directly refines latents through measurement-residual gradients. We evaluated BOI-inspired reranking and iterative search on MindAligner in both 1→2 and 2→1 directions, with a random-candidate control. The BOI-inspired variants achieve higher PixCorr and SSIM, while MSC—including its few-shot version—performs better on all six image-feature metrics, demonstrating different reconstruction trade-offs.

BrainDiVE [2] uses encoder gradients to generate images that maximize predicted activation in selected cortical regions, revealing their visual preferences. MSC instead seeks agreement with the measured response to a specific stimulus. We did not include BrainDiVE as a direct reconstruction plug-in because this would require changing its original objective. Its encoder-gradient guidance nevertheless provides important methodological inspiration for MSC.

Comparison with BOI-inspired search on MindAligner.

DirectionMethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
1→2MindAligner0.22140.41430.87830.92350.83820.82470.80930.4598
+ MSC (full-data)0.23170.40000.90520.94430.85120.84210.77890.4394
+ MSC (few-shot)0.23480.42370.90640.93810.85020.82480.77720.4585
B1: Random candidates0.26310.46830.76210.81120.76170.76630.86470.5374
B2: BOI-inspired reranking0.26030.46730.78090.84390.78150.77270.85300.5264
B3: BOI-inspired iterative search0.25790.47000.77900.84180.77980.77440.85270.5316
2→1MindAligner0.21570.42750.87560.93800.83510.82860.81060.4562
+ MSC (full-data)0.21670.40290.88480.94210.85800.84450.78540.4386
+ MSC (few-shot)0.22170.42820.88810.93990.85460.82920.77970.4563
B1: Random candidates0.23140.43500.74070.81150.76380.75390.87070.5513
B2: BOI-inspired reranking0.23050.43540.76300.83720.77230.76650.86370.5419
B3: BOI-inspired iterative search0.22930.43830.76320.83280.77640.76640.86370.5451

All methods use MindAligner as the decoding backbone. B1 is the random-candidate control; B2 and B3 are BOI-inspired adaptations using one-round reranking and two-round iterative search, respectively. Best results within each direction are shown in bold.

[1] Kneeland, Reese, et al. "Brain-optimized inference improves reconstructions of fMRI brain activity." arXiv preprint (2023).

[2] Luo, Andrew, et al. "Brain diffusion for visual exploration: Cortical discovery using large scale generative models." Advances in Neural Information Processing Systems 36 (2023): 75740–75781.

W3:

Thank you for this observation. We agree that the ablation results support interpreting QSC as a structural regularizer. Adding QSC improves pixel and structural fidelity, but reduces several feature-based metrics relative to MSC alone. We will revise the “complementary roles” description to explicitly acknowledge this structure–feature fidelity trade-off and retain separate MSC-only and full DS-DC results.

W4:

Thank you for this comment. We will strengthen the connection to classical iterative regularization and cite the relevant literature.

The specific value of our analysis is to characterize the competing effects of refinement with a learned neural forward model. Our finite-step decomposition separates initial-error contraction, forward-residual-induced bias, and nonlinear accumulation. In particular, a nonzero measurement gradient can persist even at the true stimulus representation, so further loss reduction need not improve reconstruction. This connects the PLR iteration count to a concrete correction–bias trade-off and motivates finite refinement.

Q2:

Thank you for suggesting these comparisons. We evaluated them on 1,000 MindAligner test stimuli in the 1→2 direction. The baseline corresponds to an effective img2img strength of 0.48 and a final blurry-image weight of 0.25. We varied strength from 0.2 to 0.8 and blending weight from 0 to 0.5, both in increments of 0.1, changing one parameter at a time. The table also includes Direct-VGG and QSC without reliability weighting to examine the guidance design.

In the reported results, QSC alone achieves higher PixCorr and SSIM than every tested img2img strength. Blending at 0.5 yields higher PixCorr, but QSC provides comparable SSIM with better scores on all six feature metrics. QSC also achieves higher SSIM and better scores on most feature metrics than Direct-VGG and QSC without reliability weighting. These comparisons support selectively using reference features: local retrieval accommodates spatial offsets, while reliability weighting reduces the influence of uncertain matches. This offers finer structural guidance than global strength or blending adjustments. Complete results are provided in the supplementary PDF.

QSC ablations and comparison with simpler structural guidance on MindAligner (1→2, 1,000 test stimuli).

MethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
MindAligner0.22140.41430.87830.92350.83820.82470.8093‾\underline{0.8093}0.4598
+ QSC0.22320.4327‾\underline{0.4327}0.86930.91870.84110.8218‾\underline{0.8218}0.82050.4679‾\underline{0.4679}
+ QSC w/o reliability0.22510.40910.86990.91610.83480.80740.82740.4816
+ Direct-VGG0.22550.40890.87020.91640.83430.80860.82780.4827
Img2img strength = 0.20.21890.38370.8817‾\underline{0.8817}0.9283‾\underline{0.9283}0.84700.81140.82440.4762
Img2img strength = 0.70.21920.40110.86000.90400.80630.79730.83510.4886
Blending weight = 0.50.25990.42960.86520.91670.83410.80220.82430.4829
+ DS-DC0.2399‾\underline{0.2399}0.44060.89920.93220.8466‾\underline{0.8466}0.82150.78750.4683

Best results in each column are shown in bold, and second-best results are underlined.

Reviewer_YvJzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Dual-Space Data Consistency (DS-DC), a training-free inference framework for cross-subject fMRI visual decoding. It introduces two components: Measurement-Space Consistency (MSC), which uses a pretrained neural forward model to guide diffusion sampling, and Query-Guided Structural Consistency (QSC), which improves reconstruction through multi-scale structural guidance. The authors also provide theoretical analysis of finite-step measurement refinement. Experiments on NSD across 12 cross-subject transfer settings demonstrate improvements over the pretrained MindAligner baseline.

Critical strengths
  1. The paper addresses the limitation of existing cross-subject alignment methods by introducing neural measurement consistency during inference, providing a reasonable motivation for improving reconstruction fidelity.

  2. The experiments include component-wise ablations, measurement-space evaluations, and theoretical analysis, offering useful insights into the complementary roles of MSC and QSC.

Critical weaknesses
  1. Although DS-DC does not update the pretrained decoder, MSC relies on an independently pretrained, subject-specific neural forward model. The additional training data required for this model are not clearly quantified or incorporated into the cross-subject data budget. This raises concerns about whether the framework genuinely preserves the limited-data advantage of cross-subject decoding.

  2. The reported 24.45% measurement MSE reduction is evaluated using the same forward operator employed for guidance. Under independent cross-operator evaluation, the improvements decrease substantially to approximately 1–2%. Moreover, the final reconstruction gains are relatively modest (e.g., SSIM improves only from 0.417 to 0.418), and adding QSC reduces several high-level metrics compared with MSC alone. The experiments are also restricted to the MindAligner backbone, limiting evidence of generalizability.

Questions for authors
  1. How much subject-specific paired fMRI–image data are required to pretrain the neural forward operators? How does this additional supervision affect the fairness of comparisons under limited-data cross-subject decoding?

  2. Given the substantially smaller improvements under cross-operator evaluation (Table 3), how can the authors demonstrate that MSC improves genuine neural consistency rather than primarily optimizing the particular learned forward operator?

  3. Can the authors provide experiments on additional decoding backbones and report the inference-time computational overhead? Furthermore, does increasing PLR refinement beyond the evaluated range eventually degrade reconstruction quality, as suggested by the theoretical analysis?

4 comments

W1 and Q1:

Thank you for raising this concern. The original NeuroFlow and SynBrain operators used full-data NSD training splits comprising 9,000 images per subject for subjects 1, 2, 5, and 7, with the 1,000 shared evaluation images held out. We acknowledge that this introduced additional new-subject supervision relative to the one-session MindAligner baseline.

We therefore retrained both operators using exactly the same new-subject and known-subject training sets as the corresponding MindAligner configuration: one session from the new subject and the same full training split from the known subject. Operator training and alignment reuse these pairs, so their combined supervision introduces no additional distinct fMRI–image training pairs. Evaluation stimuli remain excluded. With matched-budget NeuroFlow, DS-DC improves pixel and structural fidelity over MindAligner in all four subject-group averages, supporting useful reconstruction gains while preserving the baseline’s limited new-subject data budget.

To avoid duplicating tables within the response limit, please see the summary comparisons in our responses to V6Ed (W2 and Q2) and sioo (W1 and Q1). Complete experimental results are provided in the revised PDF.

W2:

Thank you for raising these concerns. We address the concern about operator-dependent measurement gains in our response to Q2, together with additional independent-operator evaluation and response-mismatch controls.

Under the revised matched-budget setting, DS-DC achieves a mean SSIM of 0.4410, compared with 0.4167 for MindAligner across 12 directions. Adding QSC to MSC improves PixCorr and SSIM in every direction, while the other six metrics decline on average. We therefore describe QSC as a structural regularizer that trades some feature fidelity for stronger spatial agreement, and report the component results separately.

We also evaluated MSC, QSC, and DS-DC on the frozen Duala backbone using few-shot NeuroFlow for MSC. DS-DC improves all eight macro-averaged reconstruction metrics over Duala, extending the reconstruction evidence beyond MindAligner. Subject-wise averages appear below; complete per-direction results will be included in the revised appendix.

Component evaluation on the Duala backbone.

Subj.MethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
S1Duala0.26230.44150.91000.94500.85840.83110.77970.4580
+ MSC0.26720.42740.92540.95600.87480.84590.75790.4409
+ QSC0.2678‾\underline{0.2678}0.45710.90120.93650.85170.82760.79190.4694
+ DS-DC0.27260.4441‾\underline{0.4441}0.9193‾\underline{0.9193}0.9496‾\underline{0.9496}0.8716‾\underline{0.8716}0.8444‾\underline{0.8444}0.7695‾\underline{0.7695}0.4512‾\underline{0.4512}
S2Duala0.23850.4486‾\underline{0.4486}0.89090.93890.83760.82770.79760.4658
+ MSC0.24130.43280.90840.95220.87110.84790.76330.4437
+ QSC0.2439‾\underline{0.2439}0.46190.88030.93230.83820.82150.80870.4764
+ DS-DC0.24630.44830.9024‾\underline{0.9024}0.9460‾\underline{0.9460}0.8702‾\underline{0.8702}0.8455‾\underline{0.8455}0.7737‾\underline{0.7737}0.4528‾\underline{0.4528}
S5Duala0.21230.43430.86650.92760.85810.82000.78470.4614
+ MSC0.2280‾\underline{0.2280}0.42130.89030.94480.90060.86570.72370.4255
+ QSC0.21750.44980.85710.91770.85720.81960.79590.4723
+ DS-DC0.23300.4387‾\underline{0.4387}0.8837‾\underline{0.8837}0.9387‾\underline{0.9387}0.8979‾\underline{0.8979}0.8617‾\underline{0.8617}0.7359‾\underline{0.7359}0.4360‾\underline{0.4360}
S7Duala0.20360.42640.8551‾\underline{0.8551}0.9087‾\underline{0.9087}0.82830.79150.8083‾\underline{0.8083}0.4778
+ MSC0.20780.41580.85820.90890.8273‾\underline{0.8273}0.79830.80040.4719
+ QSC0.2084‾\underline{0.2084}0.44400.84510.89900.82360.78820.81640.4839
+ DS-DC0.21280.4336‾\underline{0.4336}0.85020.90190.82420.7980‾\underline{0.7980}0.80880.4775‾\underline{0.4775}
AllDuala0.22920.43770.88060.93010.84560.81760.79260.4658
+ MSC0.2361‾\underline{0.2361}0.42430.89560.94050.86850.83950.76130.4455
+ QSC0.23440.45320.87090.92140.84270.81420.80320.4755
+ DS-DC0.24120.4412‾\underline{0.4412}0.8889‾\underline{0.8889}0.9341‾\underline{0.9341}0.8660‾\underline{0.8660}0.8374‾\underline{0.8374}0.7720‾\underline{0.7720}0.4544‾\underline{0.4544}

Best results within each group are shown in bold, and second-best results are underlined.

Q2:

Thank you for raising this concern. We report additional validation in the 2→1 direction using a frozen AlexNet-based fMRI encoding model from BERG [1], exclusively for evaluation on its 515-stimulus NSD test partition. NeuroFlow-guided MSC reduces independently evaluated MSE by 3.71% relative to MindAligner and improves spatial response correlation.

To test response specificity, we keep the decoder input, generation settings, and random seed fixed, replacing only MSC’s measurement target with another stimulus’s response from the same participant, with the same repetition count and a similar response norm. All conditions are evaluated against the correct stimulus’s measured, repeat-averaged fMRI response.

Independent BERG evaluation in the 2→1 direction.

MethodMSE ↓Spatial Pearson ↑
MindAligner0.3990220.420016
+ MSC with mismatched response0.3866820.450566
+ MSC with correct response0.3842320.454535

The same frozen BERG encoding model is used across all conditions and does not participate in MSC guidance. Best results are shown in bold.

Correct-response MSC outperforms both the baseline and mismatched guidance on both metrics, supporting a response-specific benefit beyond the guidance operator in this setting.

[1] Gifford, Alessandro Thomas, et al. "A resource of encoding models for in silico neuroscience." 9th Annual Conference on Cognitive Computational Neuroscience.

Q3:

Thank you for these questions. Additional-backbone results are reported in our response to W2. We additionally report the parameter overhead introduced by MSC, QSC, and DS-DC on Duala.

We evaluated PLR up to K = 64, beyond the original maximum of K = 10, on 500 MindAligner test stimuli in the 1→2 direction. We used the retrained, matched-budget NeuroFlow operator and disabled SMG and QSC to isolate PLR. Increasing K from 10 to 64 reduces measurement MSE from 0.4071 to 0.3387.

Final reconstruction quality largely saturates, with no statistically reliable deterioration from K = 10 to 64. Our local analysis motivates finite refinement by identifying the risk of amplifying forward-model residuals; it does not guarantee deterioration at a specific iteration count. These results support finite PLR as a practical choice, since additional iterations increase cost without consistent reconstruction gains.

Parameter overhead on Duala.

VariantCounted parameters (M)Increase over baselineTime (s/image)Peak GPU memory (GiB)
Duala2231.98—8.8928.75
+ MSC2868.5228.52%38.1231.26
+ QSC2233.710.08%12.3333.17
+ DS-DC2870.2528.60%41.7533.15

Effect of PLR depth on MindAligner (1→2, 500 test stimuli).

PLR steps KNeuroFlow MSE ↓PixCorr ↑SSIM ↑CLIP ↑
0—0.21300.42060.8265
10.50590.22830.41150.8478
20.48620.22850.41160.8490
40.45610.22840.41130.8458
80.41940.22820.41140.8487
100.40710.22820.41130.8488
160.38360.22810.41120.8493
320.35720.22810.41120.8447
640.33870.22830.41090.8482
Reviewer_YvJzOpenReview ↗

Follow-up on the Rebuttal

Thank you for the detailed responses and additional experiments. The authors have adequately addressed all my concerns. I have no further questions and am satisfied with the revisions.

Thank you very much for your thoughtful feedback and constructive suggestions throughout the review process. Your comments encouraged us to examine our work from different perspectives and motivated us to conduct additional experiments and analyses. We are grateful for the opportunity to clarify and improve our work through this discussion, and we sincerely appreciate your time and consideration.

Reviewer_V6EdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DS-DC, an inference-time framework for cross-subject visual decoding. It combines measurement-consistency guidance through a frozen neural forward model with structural guidance from an fMRI-derived blurry image. Evaluated on 12 NSD transfer directions, it improves reconstruction metrics over frozen MindAligner.

Critical strengths

The framework improves all eight macro-averaged reconstruction metrics without updating the pretrained decoder. The component ablations help explain the complementary effects of measurement and structural guidance, and the cross-operator evaluation provides a useful check beyond the optimized surrogate.

Critical weaknesses
  1. The methodological contribution is not sufficiently established by the comparisons. MSC combines measurement-loss gradient guidance with finite latent refinement, while QSC applies local feature matching against a decoded structural reference. The paper acknowledges related consistency-guided generation and structural refinement approaches, but the experiments mainly compare against unguided MindAligner and subsets of the proposed method. These ablations establish that the components affect performance, but do not establish why the particular design is preferable to simpler alternatives. For example, QSC is not compared with direct multi-scale feature matching using the same blurry reference. I find the incremental contribution difficult to assess without such controls.

  2. The additional subject-specific supervision is insufficiently documented. Section 4.1 states that the forward operator’s pretraining supervision is treated separately from MindAligner’s alignment budget, but does not adequately specify its data volume, subject coverage, or separation from evaluation stimuli. Freezing this operator makes DS-DC training-free at inference, but does not eliminate its additional supervision requirements. This matters particularly for cross-subject decoding: the comparison does not establish whether the improvement remains under a matched total source-subject data budget, or how much subject-specific data are needed to obtain a useful forward operator.

Questions for authors
  1. Can you compare DS-DC against simpler guidance alternatives under matched inference budgets, particularly direct multi-scale structural matching without QSC’s local retrieval and reliability weighting? Does the proposed design retain a meaningful advantage?

  2. What exact subject-specific data were used to pretrain each forward operator, and how were evaluation stimuli excluded? Can you report performance when forward-model training and alignment share the same total source-subject data budget as the baseline?

3 comments

We thank all reviewers for your constructive feedback. We have revised the manuscript as follows:

  • Data-budget fairness: Clarified forward-operator training data and evaluation exclusions, and added comparisons between full-data and matched-budget few-shot settings (Sections 4.1–4.2; Tables 1–2).
  • Simpler alternatives: Added Direct-VGG, QSC without reliability weighting, img2img-strength and blending-weight controls, together with BOI-inspired search comparisons (Section 4.3; Appendices B.10–B.11).
  • Backbone generalization: Added MSC, QSC, and DS-DC experiments on the frozen Duala backbone across 12 transfer settings (Section 4.3; Table 5).
  • Component interpretation: Clarified QSC’s role as a structural regularizer and its trade-off between structural agreement and feature fidelity (Section 4.3).
  • Theory and related work: Added classical references on Landweber iteration and iterative regularization, and expanded the discussion of BOI and BrainDiVE (Section 3; Appendix A.3).

W1 and Q1:

Thank you for highlighting the need to clarify QSC’s advantage over simpler structural guidance. We evaluated four controls on 1,000 MindAligner test stimuli in the 1→2 direction: Direct-VGG, QSC w/o reliability weighting, img2img strength tuning, and low-level mixing weight tuning. Direct-VGG matches features at corresponding locations, while QSC w/o reliability weighting retains local retrieval but removes reliability weighting. These two structural controls use the same VGG layers, feature loss formulation, and denoising and guidance update counts.

QSC addresses both spatial imprecision and unreliable local information in the blurry reference. Local retrieval accommodates small spatial offsets, while reliability weighting limits the influence of uncertain matches. This allows structural guidance to depend on the quality of individual correspondences. Direct-VGG and QSC w/o reliability weighting perform similarly, whereas full QSC achieves higher SSIM and better scores on most feature metrics. This pattern supports the value of reliability-aware matching, while providing limited evidence for an advantage from local retrieval alone.

Img2img strength and blending adjust the reference’s overall influence without accounting for local match quality. The largest tested blending weight achieves higher PixCorr, but QSC provides comparable SSIM and better scores on all six feature metrics. This supports selective structural guidance as a way to preserve more feature fidelity while improving structural agreement.

QSC ablations and comparison with simpler structural guidance on MindAligner (1→2).

MethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
MindAligner0.22140.41430.87830.92350.83820.82470.8093‾\underline{0.8093}0.4598
+ QSC0.22320.4327‾\underline{0.4327}0.86930.91870.84110.8218‾\underline{0.8218}0.82050.4679‾\underline{0.4679}
+ QSC w/o reliability0.22510.40910.86990.91610.83480.80740.82740.4816
+ Direct-VGG0.22550.40890.87020.91640.83430.80860.82780.4827
Img2img strength = 0.20.21890.38370.8817‾\underline{0.8817}0.9283‾\underline{0.9283}0.84700.81140.82440.4762
Img2img strength = 0.70.21920.40110.86000.90400.80630.79730.83510.4886
Blending weight = 0.50.25990.42960.86520.91670.83410.80220.82430.4829
+ DS-DC0.2399‾\underline{0.2399}0.44060.89920.93220.8466‾\underline{0.8466}0.82150.78750.4683

Best results in each column are shown in bold, and second-best results are underlined.

For brevity, the table presents selected ablation and comparison results, with representative settings from the parameter sweeps. The complete results, including all tested img2img strengths and blending weights, are provided in the supplementary PDF.

W2 and Q2:

Thank you for highlighting the need to account explicitly for forward-model supervision. The original NeuroFlow and SynBrain operators followed their published full-data NSD protocols: 9,000 training images per subject for subjects 1, 2, 5, and 7, with the 1,000 shared evaluation images held out. These operators therefore used more new-subject supervision than the one-session MindAligner baseline.

We addressed this mismatch by retraining both operators using the same new-subject and known-subject training sets as the corresponding MindAligner configuration. The new subject contributes the same single session, and the known subject contributes the same full training split available to the baseline. Forward-model training and alignment reuse these pairs, so their combined set of distinct training samples is unchanged from the baseline. The shared evaluation images remain excluded from these training sets. We will revise Section 4.1 to clarify the training data, evaluation exclusions, and supervision budgets.

The results show that matched-budget DS-DC improves pixel and structural fidelity over MindAligner in all four subject-group averages. The few-shot setting achieves stronger pixel and structural agreement within the baseline’s data budget, while the full-data setting retains advantages on several higher-level feature measures. These results demonstrate useful reconstruction gains under matched supervision, with different trade-offs between structural and feature fidelity.

DS-DC performance with full-data and matched-budget NeuroFlow operators.

Subj.MethodPixCorr ↑SSIM ↑Alex(2) ↑Alex(5) ↑Incep. ↑CLIP ↑Eff. ↓SwAV ↓
S1MindAligner0.22250.41700.87830.92680.83940.81830.81520.4606
+ DS-DC (full-data)0.23960.42070.89360.93750.85480.83430.79560.4519
+ DS-DC (few-shot)0.24330.44220.89370.93130.85130.81970.79120.4701
S2MindAligner0.21630.42920.87650.93640.83840.83140.80210.4515
+ DS-DC (full-data)0.22050.42400.88010.93700.86070.84170.79030.4475
+ DS-DC (few-shot)0.22470.44650.88110.92980.85160.82800.78590.4669
S5MindAligner0.19610.41120.84370.91990.84540.83200.79470.4518
+ DS-DC (full-data)0.21910.41510.85620.92970.87430.85910.76870.4369
+ DS-DC (few-shot)0.22400.43900.86010.92130.87020.84410.76430.4545
S7MindAligner0.18710.40960.81660.88340.80060.77710.84500.4850
+ DS-DC (full-data)0.19160.41260.82170.89190.81210.78700.83760.4790
+ DS-DC (few-shot)0.19460.43620.82620.88000.81080.77470.82820.4943
AllMindAligner0.20550.41670.85380.91660.83090.81470.81420.4623
+ DS-DC (full-data)0.21770.41810.86290.92400.85050.83060.79800.4538
+ DS-DC (few-shot)0.22170.44100.86530.91560.84600.81660.79240.4715

Full-data denotes the original NeuroFlow operator; few-shot denotes the retrained operator using the same new-subject and known-subject training data as the corresponding MindAligner baseline. Best results within each subject group are shown in bold.

AuthorsOpenReview ↗

Follow-up on our responses

Dear Reviewer,

Thank you again for your thoughtful feedback. As the discussion period is nearing its end, we would appreciate your feedback on whether our responses have adequately addressed the concerns raised in your review. If any concerns remain unresolved, please let us know which points would benefit from further clarification. We would be happy to address them.

If our responses have resolved your main concerns, we would be grateful if you could reconsider your overall assessment and score accordingly.

Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.