- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors ask whether a decoder trained on participant X during on EEG data related to a vision perception task, could generalize and classify participant Y visual imagery without any labels. Two very complex tasks together (cross-subject and cross-task). They explore this concept on 3 main datasets (Gao, THINGS-EEG2 and Wadia). In the first case, they rely only on re-normalization of the signal and show clearly that no more complex method clearly beat this baseline. Data support transfer to OVI (Object visual imagery), partially for FVI and fails on AVI (animals) - the negative control.
On Wadia at al they explore shared representations between perception adn iamgery and finally propose a pipeline that reconstruct images from EEG data in partially brain-guided manner.
Critical strengths
Interesting problem tackled here, and a good set of controls and null distribution that makes clear the interpretation of results.
Particularly interesting is reporting the negative results about learned alignments or more sophisticated architectures that are not able to beat a z-scored feature alpha readout. I really liked it.
Critical weaknesses
Despite the interesting idea, I think there are some weaknesses that could mine the main claims of the paper.
-
I'm not convinced that the paradigm itself could separate imagery from perceptual carryover. Each trial shows an image for 4 s, then fixation cross or blank, than ask to "recall" or imagine the same thing for another 4 s. Neural activity could sustain and memorize the representations of the previously seen image and therefore models might not be truly capturing perceptual -> imagery. This is not fault of the authors of course and cannot be changed, so please explain me if I'm wrong in this interpretation or it is fair to say that is difficult to separate these contributions in the current data. If so, I would suggest the authors to tone down some of the claims of the paper.
-
Each class is a specific image, it might be the case that is what is transferred is mostly low-level? The 10 features are probably a low-resolution map that could capture low-level proprierty of the images like luminance or shapes that are unique to that stimulus (class) and sustained into the imagery window. Could authors check whether the low and high level statistics of the data correlates somehow with the classes themselves? In other words, can I just identify the class from this low-level features? Are those encoded in the brain? or something more abstract?
-
Authors are honest about their results, and it seems that only OVI is surviving correction. They are saying 2 out 3 (which is already small), but in this scenario I think some of the claims should be toned down.
-
In terms of overall novelty, I haven't found anything surprisingly novel in terms of problem tackled, methods or analysis. There is one important exception, which I liked, that is the set of controls and negative findings.
-
This is possibly just a matter of taste, but I found this paper a bit scattered. There is one problem, but the text put together multiple disjoint contributions: EEG cross-subject imagery transfer, then a re-analysis of single-neurons dataset and the peception only reconstruction approach. I wasn't really able to find a coherent story among all of these three.
Questions for authors
-
Can you rule out carryover? For example, does the transfer pattern decay with time since image offset in the way a WM or aftereffect account predicts? Is there any dataset with cue-only imagery, where the item is revealed at the cue and not shown immediately before, on which this can be tested? The planned Alljoined replication is perception-only as far as I know, so how does it address this?
-
Do pairwise transfer accuracies correlate with low-level image differences such as luminance, contrast, spatial centroid or extent? What do the SVM weights look like topographically, and are they lateralised?
-
Why is AVI a valid specificity control rather than a low-power condition? What is its detection floor relative to the OVI effect?
-
Has HINT been compared against a noise-strength or guidance-scale baseline on the same frozen VAR? Moreover, what is the role of HINT in this paper?