acceptodds
Withdrawn from ICLR 2027

Brain Decoding: Toward Real-Time Readout of Visual Imagery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Decoding visual mental imagery typically requires subject-specific calibration, raising the question of whether imagery can be decoded from representations shared across individuals. We show that a decoder trained on perception EEG from 21 participants transfers to imagery in a previously unseen participant, with no labelled imagery trials from the target participant. Per-participant standardisation is sufficient, while learned alignment provides no additional benefit. Transfer is confined to a pre-specified posterior-alpha readout, increases with the number of source participants, and survives leakage controls. On an independent dataset, cross-subject transfer replicates for perception, whereas perception-to-imagery transfer does not, highlighting state generalisation as an open problem. At the single-neuron level, the same perception-to-imagery transfer under the identity map emerges in human ventral temporal cortex. Optimal-transport analysis of EEG further suggests that the perception-to-imagery transformation is better described as translation and rescaling of a common representation than as a change in neural code. Finally, conditioning a frozen next-scale autoregressive image model on brain-predicted token prefixes enables real-time reconstruction of perceived images, while imagery reconstruction remains limited primarily by neural decoding accuracy. Together, these results show that cross-subject transfer can replace subject-specific calibration in imagery decoding and provide evidence for a common visual representation underlying perception and imagery.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, mental imagery, cross-subject transfer, brain decoding, neural representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_As22OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors ask whether a decoder trained on participant X during on EEG data related to a vision perception task, could generalize and classify participant Y visual imagery without any labels. Two very complex tasks together (cross-subject and cross-task). They explore this concept on 3 main datasets (Gao, THINGS-EEG2 and Wadia). In the first case, they rely only on re-normalization of the signal and show clearly that no more complex method clearly beat this baseline. Data support transfer to OVI (Object visual imagery), partially for FVI and fails on AVI (animals) - the negative control.

On Wadia at al they explore shared representations between perception adn iamgery and finally propose a pipeline that reconstruct images from EEG data in partially brain-guided manner.

Critical strengths

Interesting problem tackled here, and a good set of controls and null distribution that makes clear the interpretation of results.

Particularly interesting is reporting the negative results about learned alignments or more sophisticated architectures that are not able to beat a z-scored feature alpha readout. I really liked it.

Critical weaknesses

Despite the interesting idea, I think there are some weaknesses that could mine the main claims of the paper.

  1. I'm not convinced that the paradigm itself could separate imagery from perceptual carryover. Each trial shows an image for 4 s, then fixation cross or blank, than ask to "recall" or imagine the same thing for another 4 s. Neural activity could sustain and memorize the representations of the previously seen image and therefore models might not be truly capturing perceptual -> imagery. This is not fault of the authors of course and cannot be changed, so please explain me if I'm wrong in this interpretation or it is fair to say that is difficult to separate these contributions in the current data. If so, I would suggest the authors to tone down some of the claims of the paper.

  2. Each class is a specific image, it might be the case that is what is transferred is mostly low-level? The 10 features are probably a low-resolution map that could capture low-level proprierty of the images like luminance or shapes that are unique to that stimulus (class) and sustained into the imagery window. Could authors check whether the low and high level statistics of the data correlates somehow with the classes themselves? In other words, can I just identify the class from this low-level features? Are those encoded in the brain? or something more abstract?

  3. Authors are honest about their results, and it seems that only OVI is surviving correction. They are saying 2 out 3 (which is already small), but in this scenario I think some of the claims should be toned down.

  4. In terms of overall novelty, I haven't found anything surprisingly novel in terms of problem tackled, methods or analysis. There is one important exception, which I liked, that is the set of controls and negative findings.

  5. This is possibly just a matter of taste, but I found this paper a bit scattered. There is one problem, but the text put together multiple disjoint contributions: EEG cross-subject imagery transfer, then a re-analysis of single-neurons dataset and the peception only reconstruction approach. I wasn't really able to find a coherent story among all of these three.

Questions for authors
  1. Can you rule out carryover? For example, does the transfer pattern decay with time since image offset in the way a WM or aftereffect account predicts? Is there any dataset with cue-only imagery, where the item is revealed at the cue and not shown immediately before, on which this can be tested? The planned Alljoined replication is perception-only as far as I know, so how does it address this?

  2. Do pairwise transfer accuracies correlate with low-level image differences such as luminance, contrast, spatial centroid or extent? What do the SVM weights look like topographically, and are they lateralised?

  3. Why is AVI a valid specificity control rather than a low-power condition? What is its detection floor relative to the OVI effect?

  4. Has HINT been compared against a noise-strength or guidance-scale baseline on the same frozen VAR? Moreover, what is the role of HINT in this paper?

Reviewer_eML6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether a decoder trained on other participants' perception EEG can classify a new participant's visual imagery without any labelled data from that participant, and finds modest transfer with a simple posterior alpha-power readout. It complements this with a perception-to-imagery analysis of published single-neuron recordings and with an EEG-to-image reconstruction pipeline built on a frozen next-scale autoregressive generator (VAR).

Critical strengths
  1. The controls behind the EEG transfer result are careful for this literature. A negative control is designated in advance, temporal leakage is measured, standardisation is estimated on never-scored epochs and also causally, ocular components are removed, and the class correspondence and presentation order are controlled. Together these make the object-set effect credible as a real, label-free, cross-participant signal, even though it is small.

  2. The paper makes a useful observation about EEG-to-image evaluation: a flat colour beats all published methods on PixCorr and SSIM, and IS/FID/KID score a permuted, wrong-image prefix as well as the correct one.

Critical weaknesses
  1. The decoded signal is not shown to be imagery. Each class is one fixed image and each category set is a separate run. The transferred pattern is already present in the delay before the imagery cue, drops to chance while the image is on screen, and peaks with the post-stimulus alpha rebound. It is therefore equally consistent with an offset, afterimage or working-memory signal from the image just seen. The authors acknowledge this in the limitations, yet the title and abstract claim that cross-subject transfer can replace calibration in imagery decoding and that perception and imagery share a common representation.

  2. The effect sizes do not support the paper's framing. Only the object set survives multiple-comparison correction. The single-trial effect is +1.9 pp in pairwise classification, and that number comes from a single seed on the pre-exclusion data. Imagery "reconstruction" reduces to four-way selection that is correct 40% of the time. Calling this a "real-time readout of visual imagery" overstates the evidence considerably.

  3. The supporting analyses do not support the general claims. The optimal-transport residual between perception and imagery is just as large on the negative control, so it measures a change of state rather than shared content. The single-neuron result rests on four patients, an uncorrected p = 0.03, and neurons pre-selected for the very property being tested; cross-patient transfer is null.

Questions for authors
  1. Does the post-cue signal carry information beyond the pre-cue delay pattern? For example, does transfer survive after regressing out the target's delay-period pattern, or on a dataset in which the imagined item is cued without having just been shown?

  2. What is HINT in Table 2? Does partial prefix forcing help over other conditioning mechanisms, and does it help at all when the prefix comes from imagery rather than perception?

Reviewer_7u5wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper first asks whether visual imagery can be decoded in a new participant without any calibration data from them. Using a public EEG dataset in which participants view and then imagine a small set of known images (each class is a single fixed image), the authors train a simple linear classifier on alpha-band power at posterior electrodes from 21 participants viewing the images, and test it on a held-out participant imagining them. Transfer is above chance for one image set (objects, OVI; about 55% vs 50% in pairwise tests), weaker and not significant after correction for the second set (figures, FVI), and absent for the last set (animals, AVI), which was used as a pre-designated negative control. A series of controls supports the object result, and show that more complex EEG pipelines do not beat the simple classifier. The paper also includes three additional analyses: it re-analyses human single-neuron recordings, describes how perception and imagery features differ, and presents a fast EEG-to-image reconstruction pipeline evaluated on perception data (THINGS-EEG2).

Critical strengths

The main strengths of the paper are as follows:

  1. The main analysis is extremely rigorous: The analyses were pre-registered, with a negative control fixed in advance (the animal images). All results were validated against a chance level estimated by shuffling labels, and corrected for all the comparisons the authors ran. The authors also checked that the effect is not explained by eye movements, by the order in which images were presented, or by information leaking from the test data. This makes the results credible, including the null ones.
  2. Even though some results are negative, they remain informative: Interestingly, a simple classifier on 10 alpha-power features is not beaten by many more complex alternatives, including deep networks, Riemannian methods and a contrastive EEG–image encoder, which is a useful reference point for imagery decoding. The reconstruction section also shows that standard metrics can be misleading: a single flat-colour image beats published methods on two common metrics (PixCorr and SSIM), and distribution metrics (FID, KID) cannot tell a wrong reconstruction from a correct one.
Critical weaknesses

Main weaknesses are listed below:

  1. Regarding the results, the claim of the paper are probably too strong: Only one category of images present a positive results. Because each class is one fixed image, the classifier may pick up low-level differences between the specific images (brightness, contrast) rather than imagery content. This result would probably stronger with more image diversity. Figure 2 also shows that the signal is already present during the mask, before the imagery cue, so the effect may be a carry-over of perceiving the image (memory, attention) rather than imagery itself, as the authors acknowledge in their Limitations. Finally, the headline numbers average 10 test trials of the same image, which requires knowing which trials belong together; on single trials the classification result is about 52%. The title ("real-time readout of visual imagery") and the conclusion of "a common visual representation" are probably too strong.

  2. The paper is globally hard to follow and tries to do too much: It combines very different studies for different type of experiments (EEG transfer, a single-neuron re-analysis, an optimal-transport analysis and an image-reconstruction pipeline), which makes the general objective of the paper slightly confusing. The secondary results add little to the main claim of the paper: the reconstruction model is trained per subject on perception data, and for imagery it simply renders the image picked by the classifier; the single-neuron analysis uses four patients and neurons that were pre-selected for responding during imagery. Focusing on the EEG transfer result would make the contribution much clearer.

Questions for authors

Clarification:

  1. The main numbers average 10 test trials of the same image, which requires knowing which trials belong to the same class, information a real-time system would not have. Could you report single-trial and causal (real-time) results for all three image sets, and make them the headline numbers?

Suggestions:

  1. Replicate the transfer on another public perception–imagery EEG dataset (e.g., Wilson et al., 2023, or Chang et al., 2025, both cited), ideally with several images per class, and consider refocusing the paper solely on calibration-free transfer.
  2. In the dataset, each image is shown for 4 s and participants then imagine that same image after a short mask, so the imagery period directly follows perception of the imagined item. Figure 2 shows the signal is already present before the imagery cue. Is this design suitable for separating imagery from carry-over of perception? A design in which the item to imagine is not the one just seen, such as a 2-back task or a retro-cue design (two images shown, then a cue indicating which one to imagine), would separate the two. Could the authors discuss this limitation more explicitly and explain how it constrains their conclusions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.