- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates single-trial EEG-based visual decoding and compares two aggregation strategies: Average-Then-Decode (ATD), which averages EEG responses before decoding, and Decode-Then-Average (DTA), which decodes individual trials and subsequently aggregates their predictions. The authors develop a single-trial decoder by combining and optimizing existing components, including multi-subject training, visual representation alignment, regularization, and test-time calibration. They further introduce a patience-based adaptive stopping strategy to reduce the number of trials required for reliable decoding and analyze factors affecting the decodability of different visual stimuli. Experiments on THINGS-EEG2 demonstrate substantial improvements over existing approaches.
Critical strengths
- The paper investigates an important setup of existing EEG decoding methods, using averaged EEG responses or use single trail EEG's prediction average. This is valuable for developing more practical and efficient brain-computer interfaces.
- Comprehensive empirical evaluation on a series of decoding components. The authors reproduce and evaluate several representative EEG-to-image decoding components under different trial budgets. The experiments provide useful insights.
- The proposed system achieves substantial performance improvements on THINGS-EEG2. The adaptive stopping strategy is also practically useful.
Critical weaknesses
- Limited methodological novelty and insufficient understanding of the underlying mechanisms. The proposed approach primarily consists of systematically combining and tuning existing techniques. Although such an empirical investigation can be valuable, much of the paper focuses on comparing different configurations and identifying effective combinations, with limited analysis of the mechanisms underlying the observed improvements.
In particular, the paper does not sufficiently explain why DTA outperforms ATD in some settings but performs worse in others. Figure 2 shows that DTA degrades performance, whereas it improves performance when combined with the proposed decoder. The authors mainly attribute this difference to single-trial decoding quality, but why?
-
Limited generalization across datasets, experimental settings, and decoding tasks. All major experiments are conducted on THINGS-EEG2 and focus exclusively on visual image retrieval. Although THINGS-EEG2 is a widely used benchmark, its experimental protocol involves a fixed candidate image set and a very short presentation time. It remains unclear whether the observed advantages of DTA generalize beyond this specific setting. The authors are encouraged to evaluate the approach on additional datasets, such as CrossPT, and under more diverse experimental settings, including cross-subject and cross-session decoding. Moreover, other EEG decoding tasks, such as emotion recognition (DEAP and SEED series), would help establish whether the benefits of DTA reflect general properties of EEG decoding or are specific to visual image retrieval.
-
Insufficiently controlled comparisons between DTA and ATD. Some comparisons are not fair. For example, Table 2 compares the proposed DTA model against Brain-HIVE using ATD, making it difficult to determine whether the improvements come from the aggregation strategy or differences in model architecture and training. A more controlled analysis would strengthen the paper.
Additionally, the proposed patience-based stopping strategy is primarily compared against fixed trial budgets. Comparisons with alternative adaptive stopping approaches, such as confidence-based or score-margin-based stopping, would help determine whether the improvements are specific to the proposed strategy.
-
Several writing or minor experimental issues. The paper contains several writing issues. (1) Fonts in Figure 6 is too small. (2) Figure 4b states that patience stopping matches Brain-HIVE's accuracy of 76.6% with n=80, whereas Table 5 reports a reproduced accuracy of 76.22%. (3) The line in Figure 3b is not clear, which channel correspond to each line?
-
Some claim not sufficiently supported. Section 4.2 argues that decodability is a property of the image rather than the subject. This is an unexpected claim and may need more evidence. The authors should investigate whether the observed decodability patterns remain stable under different candidate-set sizes, candidate compositions, and visual embedding models.
Questions for authors
Q1. Under what conditions does DTA help versus hurt (Figure 2)? What is the core mechanism that makes DTA better? (instead of attributing it qualitatively to single-trail decoding quality)
Q2. Does the experimental result limit in Things-EEG?
Q3. Is the image-level decodability (Section 4.2) stable across candidate-set sizes, compositions, and CLIP backbones? Does it transfer to across subjects/sessions?