acceptodds
Under review as a conference paper at ICLR 2027

Decode, Then Average: Real-Time Visual EEG Decoding from Single Trials

Abstract

Decoding perceived images from non-invasive electroencephalography (EEG) has progressed rapidly in recent years. This progress relies on averaging, a practice inherited from decades of neurophysiology. An individual EEG trial is dominated by noise, so decoders are trained on averages of repeated trials of each image, and at test time every available trial of a test image is averaged into one clean query that is decoded once. We argue that the single trial, not the average, should be the default operating point. We first build a decoder for this regime, with every design choice, from training data to hyperparameters, made for single trials. On THINGS-EEG2, the standard benchmark for this task, where a decoder must identify which of 200 candidate images evoked a response, it attains top-1 accuracy from one trial, more than twice the single-trial accuracy of the strongest published method. From this base, the decoder scores each trial separately and averages the resulting decisions, a readout we call *decode-then-average*, in contrast to the standard *average-then-decode*. Aggregating decisions rather than signals offers three advantages. (1) **Data scale**: training on trials rather than on their averages preserves the training set at its full recorded size. (2) **EEG response timing**: averaging superimposes responses whose latencies differ from trial to trial, whereas per-trial decisions preserve each response's own timing. (3) **Real-time decoding**: the number of trials becomes a decision made during inference. A patience stop during real-time decoding, halting presentation once the answer stops changing, matches the 80-trial accuracy of the strongest published method from trials per image, a reduction in presentations. The averaged decisions reach with all 80 trials and exceed the accuracy of every published method, under either readout, at every trial count. Real-time decoding also exposes a new property of the stimuli: each image has a stable *decodability*, from under four trials for the most decodable to nearly twice as many for the least. Decodability reflects how crowded an image's neighborhood is among the 200 candidates, not its pixels, and a new subject's per-image trial counts can be predicted in advance. Single-trial EEG decoding is therefore more tractable than existing results indicate, and treating it as the default turns decoding into a real-time process with a measurable, predictable cost.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Visual Decoding, Brain Decoding, Brain-Computer Interface

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vaYVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates single-trial EEG-based visual decoding and compares two aggregation strategies: Average-Then-Decode (ATD), which averages EEG responses before decoding, and Decode-Then-Average (DTA), which decodes individual trials and subsequently aggregates their predictions. The authors develop a single-trial decoder by combining and optimizing existing components, including multi-subject training, visual representation alignment, regularization, and test-time calibration. They further introduce a patience-based adaptive stopping strategy to reduce the number of trials required for reliable decoding and analyze factors affecting the decodability of different visual stimuli. Experiments on THINGS-EEG2 demonstrate substantial improvements over existing approaches.

Critical strengths
  1. The paper investigates an important setup of existing EEG decoding methods, using averaged EEG responses or use single trail EEG's prediction average. This is valuable for developing more practical and efficient brain-computer interfaces.
  2. Comprehensive empirical evaluation on a series of decoding components. The authors reproduce and evaluate several representative EEG-to-image decoding components under different trial budgets. The experiments provide useful insights.
  3. The proposed system achieves substantial performance improvements on THINGS-EEG2. The adaptive stopping strategy is also practically useful.
Critical weaknesses
  1. Limited methodological novelty and insufficient understanding of the underlying mechanisms. The proposed approach primarily consists of systematically combining and tuning existing techniques. Although such an empirical investigation can be valuable, much of the paper focuses on comparing different configurations and identifying effective combinations, with limited analysis of the mechanisms underlying the observed improvements.

In particular, the paper does not sufficiently explain why DTA outperforms ATD in some settings but performs worse in others. Figure 2 shows that DTA degrades performance, whereas it improves performance when combined with the proposed decoder. The authors mainly attribute this difference to single-trial decoding quality, but why?

  1. Limited generalization across datasets, experimental settings, and decoding tasks. All major experiments are conducted on THINGS-EEG2 and focus exclusively on visual image retrieval. Although THINGS-EEG2 is a widely used benchmark, its experimental protocol involves a fixed candidate image set and a very short presentation time. It remains unclear whether the observed advantages of DTA generalize beyond this specific setting. The authors are encouraged to evaluate the approach on additional datasets, such as CrossPT, and under more diverse experimental settings, including cross-subject and cross-session decoding. Moreover, other EEG decoding tasks, such as emotion recognition (DEAP and SEED series), would help establish whether the benefits of DTA reflect general properties of EEG decoding or are specific to visual image retrieval.

  2. Insufficiently controlled comparisons between DTA and ATD. Some comparisons are not fair. For example, Table 2 compares the proposed DTA model against Brain-HIVE using ATD, making it difficult to determine whether the improvements come from the aggregation strategy or differences in model architecture and training. A more controlled analysis would strengthen the paper.

Additionally, the proposed patience-based stopping strategy is primarily compared against fixed trial budgets. Comparisons with alternative adaptive stopping approaches, such as confidence-based or score-margin-based stopping, would help determine whether the improvements are specific to the proposed strategy.

  1. Several writing or minor experimental issues. The paper contains several writing issues. (1) Fonts in Figure 6 is too small. (2) Figure 4b states that patience stopping matches Brain-HIVE's accuracy of 76.6% with n=80, whereas Table 5 reports a reproduced accuracy of 76.22%. (3) The line in Figure 3b is not clear, which channel correspond to each line?

  2. Some claim not sufficiently supported. Section 4.2 argues that decodability is a property of the image rather than the subject. This is an unexpected claim and may need more evidence. The authors should investigate whether the observed decodability patterns remain stable under different candidate-set sizes, candidate compositions, and visual embedding models.

Questions for authors

Q1. Under what conditions does DTA help versus hurt (Figure 2)? What is the core mechanism that makes DTA better? (instead of attributing it qualitatively to single-trail decoding quality)

Q2. Does the experimental result limit in Things-EEG?

Q3. Is the image-level decodability (Section 4.2) stable across candidate-set sizes, compositions, and CLIP backbones? Does it transfer to across subjects/sessions?

Reviewer_TsVgOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that single trials, not 80-trial averages, should be the default operating point for EEG-to-image decoding on THINGS-EEG2. Six published decoders are re-run from their released code at one and at 80 trials. A decoder built from known parts and trained on single trials reaches 27.39% top-1 from one trial on a pre-registered frozen split. Averaging its per-trial scores (decode-then-average) gives 94.50% at 80 trials, which the authors credit to training-set size and trial-to-trial latency jitter. A patience stopping rule and a per-image decodability analysis follow.

Critical strengths
  1. The baseline audit is careful and useful by itself. All six methods run from their own code with three seeds, land within a point of published numbers, and real defects are documented, such as a NICE checkpoint restore that loads nothing and BraVL's 5.82% being a single-trial SVM score.
  2. The evaluation discipline is rare here. The single-trial headline is pre-registered (27.36% development, 27.39% frozen) and the readout contrasts were fixed in advance, with a failed contrast reported. These two points are why I lean towards acceptance.
Critical weaknesses
  1. The real-time headline does not add up. Patience 2 cannot stop before trial 2, so a mean of 2.6 trials lets at most 30% of streams reach trial 4. A stream that stops at trial 2 or 3 is right only if its two-trial decision is right, which the fixed-count curve in Figure 4b puts near 42%. That caps accuracy near 72%, below the 76.6% claimed. Sections 4.1 and 4.2 also give an 18% saving and 5.1 trials for patience 3, while Figure 4b plots patience 3 near 4 trials and 85%. The 2.6-trial and 96.8% figures in the abstract need to be recomputed.
  2. The timing mechanism is not isolated. Table 2 compares the proposed DTA pipeline with Brain-HIVE's ATD pipeline, so model and readout change together. The 9.9 ms jitter is estimated from single-trial cross-correlation, which Appendix K itself finds unreliable (0.27 agreement between channel halves), so part of it may be estimation noise.
Questions for authors
  1. Clarification: how is "trials per image" computed for the patience curve? For patience 2 and 3, please give the mean stopping trial, the share of streams stopping at trials 2 and 3, and accuracy, reconciled with Section 4.2.
  2. Experiment: the injected-jitter test on one model read out both ways (Config A under ATD and DTA).
  3. Clarification: configuration selection used run 3 of the test sessions, so the same 200 images. Does the single-trial result hold with selection on held-out training images?
  4. Future work: a second dataset would support the generality claims.
Reviewer_pc8pOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper states that EEG image decoding should use the single trial as its training data, and it presents a decoder built for single trials that reaches 27.39% top-1 accuracy on THINGS-EEG2 (200-way), more than double the best published single-trial result. Instead of averaging signals before decoding, it decodes each trial separately and averages the decisions ("decode-then-average"). This keeps the full training set, and matches the best published 80-trial accuracy using only 2.6 trials per image and reaches 94.50% with all 80 trials.

Critical strengths

Strong empirical performance under the DTA pipeline. The paper reports high decoding accuracy on THINGS-EEG2.

Critical weaknesses

Limited novelty and weak mechanistic analysis. The work mainly combines existing choices for training, aggregation, and stopping, and the empirical gains are not matched by explanatory insight. Section 3 offers no convincing analysis of why DTA beats signal averaging first.. Section 4 largely reports stopping performance and correlations, which do not adequately support the broader claims about DTA’s advantages or its relationship to image difficulty. As a result, the scientific contribution beyond improved performance by a different pipeline is not clear.

Poor writing and organization substantially hinder assessment. The main paper is difficult to understand, particularly in Sections 3 and 4. The manuscript needs a complete rewrite over the methodology part (Section 2) and clearer explanations of what each experiment establishes (Section 3,4). These presentation problems make it difficult to assess which components produce the improvements and whether the evidence supports the central claims.

Questions for authors
  • **Section 3.1 ** Each averaged training example contains information from four raw trials, so equal example counts do not represent equal recording budgets. Does the larger single-trial condition use exactly the same underlying recordings and image coverage as the averaged condition? What conclusion holds under this matched comparison, and how does the training analysis explain the advantage of DTA over ATD at inference?

  • **Section 3.2, Figure 3b ** Do the displayed peaks correspond to stimulus-related neural responses carrying information that distinguishes images? What evidence indicates that their timing differences reflect latency jitter rather than noise, and that averaging these responses removes information useful for decoding?

  • **Section 3.2, Table 2 ** Since the DTA and ATD conditions use different decoders, how can their different sensitivity to imposed jitter be attributed to aggregation order rather than decoder robustness? Which controls support the proposed explanation that DTA preserves information lost through signal averaging?

  • **Figure 4b ** Which components and recording budgets are held fixed when comparing the proposed DTA pipeline with ATD baselines? Are these comparisons intended to establish the performance of complete systems or the advantage of aggregation order itself? Furthermore, what evidence establishes a specific adaptive stopping advantage for DTA, given that ATD could also update its prediction after each additional trial?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.