What You Think Is What You Plot? A Benchmark for Decoding Visualization Intent from EEG
Abstract
Non-invasive Brain-Computer Interfaces (BCIs) have shown promise in decoding simple commands. Their capacity to interpret complex cognitive intents remains largely unexplored. A critical application is direct visualization generation, where a system can render charts simply by reading an analyst's mind. We introduce EEG2VIS, a novel task for decoding executable visualization (VIS) programs directly from non-invasive electroencephalography (EEG) signals. This task enables a “what you think is what you plot” paradigm for data exploration. Decoding visualization intent from EEG is intrinsically difficult due to the compositional nature of analytical queries and the low signal-to-noise ratio of scalp EEG. To facilitate research in this direction, we construct EEG2VIS-Bench, the first benchmark dataset comprising 9,531 rigorously aligned trials from 23 domain experts (46 human hours) under a three-stage cognitive paradigm. We evaluate a spectrum of baselines, ranging from end-to-end foundation models to LLM-aligned teacher-student architectures and a neuro-symbolic noisy-channel decoder. Extensive experiments show that EEG signals provide statistically significant information for coarse-grained intent, achieving up to 91.09% accuracy on chart type recognition. Accurately decoding deep compositional logic, such as data filtering conditions and sorting rules, remains a fundamental bottleneck, with independent clause-set accuracy dropping to 75.30%. We also demonstrate that current high full-query exact-match scores (62.59%) heavily rely on retrieval cache mechanisms (routing 58.05% of queries) rather than pure neural decoding.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.