acceptodds
Under review as a conference paper at ICLR 2027

BraVista: A Visual Language Framework for Multi-Task EEG Decoding

Abstract

A key challenge in electroencephalography (EEG) decoding is learning representations that generalize across diverse cognitive tasks, subjects, and recording conditions. While EEG provides a non-invasive window into brain activity across a wide range of applications, its datasets are often limited in scale and highly heterogeneous, making multi-task learning difficult. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continual post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without an additional large-scale EEG self-supervised pretraining stage. We evaluate BraVista across four EEG datasets spanning multiple BCI tasks and problem settings, and use this setup to identify the factors that enable effective multi-task EEG learning. First, we show that the right choice of EEG-to-image representation is critical for adapting general-domain VLMs to EEG. Second, we demonstrate that multi-task learning improves decoding performance in low-data regimes, indicating effective knowledge sharing across tasks. Third, a controlled comparison with the same vision encoder and task-specific classifier heads shows that the language model contributes beyond visual feature extraction. Finally, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise and task-specific degradation when individual frequency bands are masked, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these results establish BraVista as a framework for multi-task EEG decoding via visual representations. Our findings show that visual representations can serve as a viable interface for multi-task EEG learning when paired with appropriate representation and post-training strategies, highlighting the potential of general-domain foundation models as a scalable basis for EEG signal decoding.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Brain Foundation Model, EEG Decoding, Large Vision Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GFNSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BraVista, a vision-language framework for unified multi-task EEG decoding. It transforms multichannel EEG signals into structured time–frequency images and combines them with task-specific textual instructions for vision-language model adaptation. Experiments on four EEG datasets demonstrate competitive performance across diverse decoding tasks, supported by representation comparisons, multi-task learning analyses, and noise robustness experiments.

Critical strengths
  1. Meaningful research motivation. The paper explores an interesting direction for EEG decoding by leveraging general-purpose vision-language models instead of relying entirely on EEG-specific foundation models.

  2. Unified multi-task framework. BraVista supports abnormality detection, emotion recognition, sleep staging, and cognitive workload classification within a single instruction-conditioned architecture, demonstrating its applicability across heterogeneous EEG tasks.

  3. Potential benefits for data-limited tasks. The experiments indicate that multi-task training can improve performance on relatively small EEG datasets, suggesting that sharing information across tasks may benefit EEG decoding.

Critical weaknesses
  1. The authors should further discuss the rationale for converting EEG signals into structured images and whether this transformation adequately preserves the underlying neural information.

  2. BraVista arranges channel-wise spectrograms into a structured grid, but the spatial relationships between EEG electrodes may not be adequately preserved. The authors should investigate the sensitivity of decoding performance to channel ordering and grid arrangement.

  3. Simplifying instructions has little impact on performance, while removing label options causes a substantial decline. The authors should clarify whether the model genuinely understands task instructions or mainly relies on label options for prediction.

Questions for authors
  1. The authors attribute improvements on SEED and EEGMAT to knowledge sharing across tasks. However, these gains may also result from increased training data or additional optimization. Further experiments are needed to distinguish genuine cross-task knowledge transfer from the benefits of larger training sets.

  2. The authors should further analyze subject-level failure cases and performance variability to clarify the limitations of the proposed framework.

  3. There are also some minor issues that could be improved, such as the small font size in some figures, which affects readability.

Reviewer_tVV1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BraVista, a multi-task EEG decoding framework that interfaces multichannel EEG signals with a general-domain Vision-Language Model (Qwen3-VL-2B). Instead of pretraining a specialized neural tokenizer on massive unlabeled EEG corpora, BraVista converts multi-channel EEG segments into grid-structured Short-Time Fourier Transform spectrogram images and pairs them with task-specific natural language prompts. By jointly fine-tuning the vision encoder and applying LoRA to the LLM decoder across four EEG datasets (HMC, SEED, EEGMAT, TUAB), BraVista achieves competitive multi-task classification performance and demonstrates good low-data transferability and neurophysiologically consistent frequency sensitivity.

Critical strengths
  1. Bypassing massive EEG self-supervised pretraining and discrete neural tokenizers by mapping multichannel STFTs into a window-aligned image grid is practical, reproducible, and effectively leverages off-the-shelf VLM inductive biases.

  2. Joint training across heterogeneous datasets (varying from 4 to 62 channels and 4s to 30s durations) yields substantial gains on small-scale datasets without degrading performance on larger corpora.

Critical weaknesses
  1. The pipeline is a direct application of visual instruction tuning (Qwen3-VL + STFT grid + LoRA SFT). Converting EEG to spectrograms for vision backbones and applying VLMs to imaged time-series are both well-established in prior literature; the technical contribution is primarily empirical engineering.

  2. Despite using a 2B–7B autoregressive VLM, the output is strictly constrained to generating a single class token inside <class> tags. As shown in Fig. A.7, scaling the LLM from 2B to 7B brings almost zero gain, and removing candidate labels from the prompt causes performance to collapse (Fig. A.5).

  3. Flattening up to 62 channels into an arbitrary 2D image grid discards 3D scalp geometry and incurs massive visual token overhead compared to lightweight 1D/spatiotemporal EEG architectures, yet no computational efficiency or latency comparison is provided.

  4. All experimental results are presented solely as bar charts without exact numerical tables in either the main text or appendix, making precise benchmarking difficult.

Questions for authors

Please refer to the above Weaknesses.

Reviewer_r9icOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BraVista converts multichannel EEG into grids of spectrograms and fine-tunes a pretrained vision–language model to predict task labels from these images and textual instructions. A single model handles sleep staging, emotion recognition, cognitive workload, and abnormality detection. The paper also examines the choice of visual representation, the contribution of the language decoder, and the benefits of multi-task training.

Critical strengths

The strongest aspect is the practical demonstration that one general-domain VLM can handle four quite different EEG tasks without additional EEG-specific self-supervised pretraining. BraVista improves over the reported NeuroLM results on three datasets and remains competitive on TUAB. I find this useful: it offers an alternative to building and pretraining a dedicated EEG model, while retaining a single model for several tasks.

The experimental analysis is another strength. The authors compare visual representations, single-task and multi-task training, and different adaptation strategies. The gains on smaller datasets are particularly encouraging: on EEGMAT, balanced accuracy rises from 61.28% for the tuned single-task model to 71.26% with multi-task training. The comparison using the same vision encoder with classifier heads also provides a useful starting point for assessing what the language decoder adds.

Critical weaknesses

My main concern is that the evidence does not yet establish the specific value of language conditioning. Replacing the decoder with linear heads changes both model capacity and the way features are processed. This shows that the complete VLM performs better than that particular baseline, but does not isolate the contribution of pretrained language knowledge. Similarly, near-zero accuracy when the correct labels are removed from the prompt is unsurprising and says little about task understanding. A stronger nonlinear classifier and a control using task identifiers rather than descriptive instructions would make this central claim more convincing.

The scope of the generalisation claim also needs care. Each task corresponds to a different dataset, with its own channel layout, image dimensions, and label set. The model can therefore recognise which dataset it is processing without necessarily learning a flexible instruction-conditioned task representation. Removing the dataset name from the prompt does not resolve this, since the visual layout and available labels still identify the task. The results support joint learning across four known datasets, but broader claims about generalisation across tasks and recording conditions go beyond what is directly tested.

Questions for authors

How are datasets sampled during multi-task training, given their very different sizes? Are the single-task, multi-task, and vision-only comparisons matched in their exposure to each dataset and selected using comparable validation procedures? This would help assess the source of the low-data gains.

Do you have results using a stronger nonlinear classification head, or replacing descriptive instructions with simple task identifiers? These controls could materially change my assessment of whether the language component provides a distinctive benefit beyond additional modelling capacity and task selection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.