- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies differences in visual representations between the hemispheres using fMRI responses to natural images. The authors introduce HemiPatchGate, which separately encodes responses from each hemisphere and uses gating weights to combine their representations for predicting CLIP image features at different patch locations. They evaluate the method on four NSD subjects and use hemisphere input zeroing and retraining controls to examine the gating readout. They recover the known contralateral pattern and report that greater coverage by people and animals is associated with a shift toward the right hemisphere branch. They also find that local foreground coverage is associated with a stronger shift toward ipsilateral weighting near the vertical midline than elsewhere.
Critical strengths
-
Interesting motivation. Developing interpretable models to investigate differences in visual representations between the hemispheres is an interesting research direction.
-
Clear presentation. The paper is well written and clearly motivates its research question.
Critical weaknesses
- The methodological contribution is not sufficiently justified.
There are significant flaws in the motivation and ablations of the proposed methods. The justification for the specific architecture over simpler alternatives is lacking, making the claim that it constitutes a contribution unconvincing. Training a separate decoder for each hemisphere is a simple baseline that, in my opinion, is cleaner and allows better disentanglement for this research question (see Question 1). It is unclear why the proposed complex method is needed. Moreover, the current method might entangle multiple factors influencing the presented results. The interpretation of gating weights also appears stronger than the evidence supports. These weights do not necessarily quantify the relative strength of information available from each hemisphere and could be affected by differences in signal quality or optimization (Questions 2). Residual prediction experiments would provide a much stronger and more informative analysis, explicitly showing what each hemisphere contributes beyond the other (Questions 3).
- Unclear and limited scientific contribution.
Contralateral organization is well established, and it is unclear why so much space is devoted to recovering this known pattern. The animacy finding needs to be distinguished from the established right hemisphere bias for faces, which are included in the animate content analyzed here. Likewise, the foreground related ipsilateral shift near the vertical midline could reflect known receptive field overlap and greater contributions from object responsive areas with broader receptive fields (Question 5). The paper examines only a small number of possible effects and presents limited quantitative and qualitative results characterizing them. Overall, the paper does not clearly establish what new understanding these findings provide beyond existing knowledge.
Overall Assessment. I consider this paper well below the acceptance threshold for the conference. Addressing these critical weaknesses would require significant changes to the paper that are beyond the scope of rebuttal in my opinion.
Questions for authors
-
Comparison with separate decoders. Why use the proposed architecture rather than train a separate decoder for each hemisphere? This seems a simpler and potentially stronger baseline. What additional insight does gating provide?
-
Interpretation of gating magnitude. How do gate weights quantify the relative strength of hemispheric information? Could small differences in signal quality or optimization lead one branch to receive most of the weight, without a correspondingly large difference in available information?
-
Residual prediction. Why not train a decoder on one hemisphere and then measure what the other hemisphere can additionally predict about its residual errors? This could clarify the additional predictive information contributed by each hemisphere.
-
Signal quality and dataset biases. Can you analyze differences in signal to noise ratio between hemispheres and across image categories, including animate versus other images? How do you distinguish their influence on decoder performance and gating from the representational differences of interest?
-
Relation to established findings. How do the reported effects extend known right hemisphere face selectivity (Kanwisher et al., 1997) and ipsilateral receptive field coverage near the vertical meridian and in higher visual areas (Tootell et al., 1998; Dumoulin and Wandell, 2008)? Analyses separating face content from other animate content, and accounting for regions with different receptive field properties, are needed to clarify the additional scientific insight.