acceptodds
Under review as a conference paper at ICLR 2027

Learning Brain-Aligned Visual Representation with Adaptive Temporal Recalibration for EEG-Based Visual Decoding

Abstract

Decoding visual stimuli from brain activity is a central task in understanding visual perception and advancing BCI applications. Existing methods typically align EEG signals with frozen pretrained visual embeddings and process temporal activity uniformly. This paradigm suffers from two limitations: a representational mismatch between machine vision and human perception, and temporal sparsity of decodable visual information in EEG. Together, these yield poorly aligned decoders that struggle with zero-shot generalization. To address these challenges, we propose BEAT, a Brain-aligned visual Encoder with Adaptive Temporal recalibration. To mitigate the representational mismatch, we introduce a dual-alignment strategy, integrating multi-teacher visual distillation with cross-modal contrastive learning. Together, these objectives guide the visual encoder to learn semantically meaningful and neurally aligned representations. To address temporal sparsity, an adaptive temporal recalibration module emphasizes visually informative signal intervals while suppressing noisy ones via temporal contextualization. This lightweight calibrator could be readily applied to various backbones in a plug-and-play manner. Experiments demonstrate state-of-the-art zero-shot brain-to-image retrieval, while reconstruction results further validate the utility of the learned representations.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, visual decoding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_E34FOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BEAT, a framework for EEG-based visual decoding that addresses the mismatch between visual and neural representations and the non-uniform distribution of temporal EEG information. It introduces a Brain-aligned Visual Encoder (BAE) to learn neural-compatible visual representations through multi-teacher distillation and contrastive learning, and an Adaptive Temporal Recalibration (ATR) module to emphasize informative EEG intervals. Experiments on THINGS-EEG and THINGS-MEG demonstrate competitive zero-shot visual decoding performance.

Critical strengths
  1. The proposed BAE and ATR modules address two distinct challenges in EEG-based visual decoding. Their integration enables the simultaneous adaptation of visual representations and temporal EEG features, providing a coherent framework for cross-modal learning.

  2. The authors provide extensive ablation studies across five EEG backbones, along with analyses of visual distillation, temporal attention, and representation dimensionality.

Critical weaknesses
  1. Although BEAT combines multi-teacher distillation, cross-modal contrastive learning, and Transformer-based temporal modeling, these techniques are well established. The authors should clarify the methodological novelty beyond integrating existing components.

  2. The claim that BAE learns brain-aligned visual representations is mainly supported by retrieval accuracy, UMAP, and PCA analyses, which do not directly establish neural consistency. The authors should provide stronger evidence to validate this claim.

  3. Although BEAT achieves the best average performance, it underperforms some baselines on several subjects. The authors should analyze these inconsistencies, particularly in the inter-subject setting.

  4. The authors should further analyze subject-level failure cases to identify the limitations of BEAT in cross-subject generalization.

Questions for authors

In addition to the concerns listed in the Weaknesses section, I have the following questions:

  1. ATR independently models temporal dependencies within each EEG channel. The authors should clarify whether ignoring cross-channel interactions limits its ability to capture spatially distributed neural responses.

  2. The contrastive and distillation losses are assigned equal weights. The authors should investigate how different loss weights affect decoding performance.

Reviewer_AeXcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents BEAT, a framework for EEG-based image retrieval and reconstruction. Rather than mapping EEG signals to a fixed visual embedding, it jointly learns a visual target space using EEG–image contrastive learning and distillation from two pretrained visual teachers. An adaptive temporal recalibration module processes the EEG before encoding, with the aim of emphasising informative time intervals. The method achieves strong retrieval results on THINGS-EEG, with additional experiments on THINGS-MEG.

Critical strengths

The strongest point is the retrieval performance. BEAT reaches 81.1% intra-subject Top-1 accuracy on THINGS-EEG, compared with 75.7% for HVF, and also improves leave-one-subject-out retrieval. These are meaningful gains on the main task and provide a substantive reason to consider acceptance.

The benefits also extend across five EEG backbones, with additional support from the MEG experiments. This makes the approach more useful than an improvement tied to one particular encoder. Learning the visual target alongside the EEG representation appears to be a promising direction that other neural decoding methods could build on.

Critical weaknesses

How much of the improvement remains when the visual student is trained without distillation, or when both branches use the same teacher? These controls seem important given that single-teacher distillation already reaches about 78.8% Top-1 accuracy.

Could the authors clarify how checkpoints and hyperparameters are selected, especially for leave-one-subject-out evaluation? I would also welcome a repetition-count analysis and a clearer account of whether reconstruction baselines use the same generation and evaluation settings.

Questions for authors

Were the baseline results reproduced under the same preprocessing, trial averaging, and evaluation settings as BEAT? Please also clarify how validation data were constructed and checkpoints selected, particularly for leave-one-subject-out evaluation. These details would affect my confidence in the reported gains.

Do you have results for a trainable visual student without distillation, and for two branches supervised by the same teacher? These controls would help determine whether the gains specifically support multi-teacher supervision or mainly reflect visual adaptation and branch averaging. I regard these as checks on the claimed contribution, rather than suggestions for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.