- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper converts early, mid, and late visual brain region responses into tags with two-dimensional location encodings, enabling staged visual semantic supervised learning of shared representations. Then, the three-stage conditions are injected into different depths of the diffusion model to achieve zero-sample cross-subject reconstruction and few-sample adaptation.
Critical strengths
-
The protocol for simultaneously isolating the subject from the evaluation stimulus is clearly defined. Section 4.1 excludes shared1000 evaluation images and their corresponding brain responses from all source subjects, and few-sample fitting also excludes the target test image.
-
The magnitude of semantic improvement from zero-sample testing warrants further validation.
-
A relatively complete preliminary ablation is provided. This includes stage combination, a Flat model with parameter matching, positional encoding, alignment targets, and routing perturbations, and reports its own subject-by-subject results and cross-subject standard deviation.
Critical weaknesses
-
The Flat model in Section 4.3 does indeed match the input stage, module, and number of trainable parameters, and is trained independently. However, its approach involves first resizing the three-stage conditions and then directly averaging them, before feeding the same fusion condition into the entire depth; the main model retains three distinguishable information streams. Converting three signals with stage identities into a single mean is a many-to-one information mixing. Same parameters do not necessarily mean the same usable information. The increase in CLIP from 77.2% to 79.5% in Table 3 only demonstrates that "preserving branches and specifying routes" is superior to "averaging first and then broadcasting," but it cannot independently prove the correctness of the authors' chosen biological hierarchical correspondence. The truly crucial control should retain stage identities, canceling the preset hierarchy through splicing, independent projection, or learnable fully connected routing. The appendix probe map is also insufficient to compensate.
-
Equations 5-7 pass the frozen CLIP global image features through a trainable stage projection head and cosine-align it with the brain label mean. Target projection and brain mapping are trained simultaneously, and the three stages use the same global CLIP feature, without explicit cross-subject correspondence constraints. Considering the cosine term alone, ensuring that the target projections and mean brain labels of all stimuli converge to the same vector yields minimal loss. Constraining only the label mean does not guarantee that local labels have the same meaning across different subjects. Removing alignment from Table 4 leads to performance degradation, demonstrating the effectiveness of correct visual supervision, but it fails to distinguish between cross-subject normalization and ordinary auxiliary semantic regression. "Canonical" is a title-level concept, requiring at least direct evidence of cross-subject canonical retrieval, representational correspondence, and information retention.
-
The main text and Appendix A.1 perform maximum projection, smoothing, scaling, and pooling of brain region responses in each subject's native volume along the axis into scalars: 128 for early, 64 for mid-stage, and 32 for late. The coordinate anchor refers to the regular position of the final two-dimensional canvas, not the cortical surface, visual field, pRF, or common anatomical coordinates. The same two-dimensional index from different subjects need not have the same functional meaning; maximum projection along depth may also mix different cortical fragments. According to the mask-outside-zero re-maximization projection formula given in the appendix, as long as there are out-of-mask positions in a column, valid responses with all negative values may still be covered by zeros. The performance degradation after removing position encoding or using random coordinates only proves the usefulness of fixed-position indexes, not that these indexes constitute cross-subject functional anchors.
-
Table 1 lists the results for zero-sample, 250-sample, and full-fit tests. This method explicitly excludes shared1000-source training stimuli, which is an advantage; however, it is unclear whether this rule is equally applied to all comparison subjects. The inclusion of "250 images" does not automatically imply the same brain signal budget, as the number of trials and image selections may differ. Figure 3 compares this zero-sample method with the one-hour fit MindEye2 (the caption is disclosed), but this presentation cannot replace a quantitative comparison with a uniform budget.
-
The appendix does indeed report SD1.5, frozen modules, training steps, DDIM steps, and guidance intensity. However, the system includes a frozen text encoder and is guided by a classifier-free approach, but it doesn't specify whether training and inference use empty text, fixed cues, or text from other sources, nor does it fully define what information was removed from the conditional and unconditional branches. There is no evidence that the authors used test images for description, so this missing information cannot be escalated to a leak accusation. However, textual conditions are crucial inputs determining generative capacity and information budget and must be explicitly stated. Other important gaps include the complete structure of conditional branches, optimization of learning rate and scheduling, few-shot adaptation steps, and which parameters are updated in full adaptation.
-
The appendix reports the standard deviations of four participants, but cross-participant differences do not equate to training randomness or the selection variance of 250 adaptation images. The few-shot experiments primarily rely on a single non-zero budget LoRA design (K=250) to support parameter savings, but this alone cannot demonstrate practical personalized efficiency. The authors acknowledge that the evaluation was conducted only on four NSD participants with the same acquisition protocol. Under the strong claim of "transferable canonical cortical representations," the existing single-dataset results are insufficient to distinguish robust brain representation learning.
Questions for authors
-
How does the method, which preserves the splicing and fusion of three-stage identities, learnable routes, and six fixed permutations, perform after independent retraining? Is the so-called correct hierarchical correspondence still stable and optimal?
-
Are the representations of the same stimulus across subjects more similar, while still distinguishing different stimuli? Are ordinary CLIP-assisted regression or fixed target projection sufficient?
-
How sensitive is the method to projection axes, volume orientation, negative response handling, and the number of labels? Are signed mean, dual-channel (positive and negative), three-dimensional, or surface representations more robust?
-
Where are all textual conditions obtained, and how is CFG specifically implemented? Please provide the complete conditional network, optimization settings, adaptation configuration, and validation selection process.
-
Do hierarchical control and few-sample results hold true on multiple fit subsets? What are the learning curves and actual adaptation costs for different values?
2 comments
Thank you for the additional experiments. They strengthen the evidence for the proposed design and partially address my concerns. However, the contribution still appears primarily to be an architectural combination of CLIP-supervised stage representations and fixed stage-to-depth conditioning. The distinctive contribution of “canonical alignment” beyond shared semantic supervision remains unclear. I therefore remain unconvinced that the methodological advance is sufficiently substantial and maintain my original rating.
Sincerely looking forward to the further discussions
Dear Reviewer FsdC,
As the discussion deadline is approaching, we kindly remind that we have answered all your concerns carefully with additional experiments and clarification.
Could you please check our response and consider improving the rating if we successfully addressed you concerns?
Any feedback will be greatly appreciated and we are glad to provide any additional clarifications that you may need to bring a fruitful discussion.
Warm regards,
The authors of paper 12590