- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces TRACE (Temporal Resolution-Aware Complementary EEG Decoding), a pipeline to decode visual images from EEG signals. It revisits each stage of the common EEG-to-Image pipeline, including the encoder, the EEG-image alignment, and image generation module. It shows that the information in visual EEG responses is concentrated locally (models trained on specific time windows only generalize to closely temporal windows and not further ones). Interestingly, an MLP model applied to EEG timepoints outperforms previous convolutional, transformer, and large EEG models. The second contribution is on alignment between EEG and visual features. The paper shows aligning EEG features separately with high-level and low-level visual features can improve the decoding performance. Finally, instead of using a single-level conditioning signal for image reconstruction, it uses both high- and low-level features for conditioning: semantic features initialize the coarse generation stage, while structural features guide subsequent refinement.
Critical strengths
- The paper has a logical flow and is generally easy to follow. The design choices (such as the choice of encoder and the dual-space alignment) proposed are investigated for what makes them better than other options. Therefore, it can provide insight to other researchers on what should be optimized to improve the performance.
Critical weaknesses
- One of the weaknesses of the paper is that almost every metric value has been reported as a single number without any standard deviation values. A single number cannot tell whether the method outperforms the others significantly or is the difference only related to a lucky seed or a few outlier high-performing seeds.
- The second weakness is the limited data the paper uses. The only dataset everything is built on is THINGS-EEG2. Claims such as the ones presented in figures 4b and 4c can be specific to this dataset. If the authors claim for example localized temporal information in brain-responses to visual images, it should be verified with other datasets as well. If the authors' claims are limited to only THINGS-EEG2, my concern shifts towards applicability of the proposed method to other scenarios.
Questions for authors
-
For the hierarchical visual targets, could the authors clarify why they selected layers 2 and 3 for low-level features? Is this optimized somehow? Also, works like Du et al. (2026) [Deep Models, Shallow Alignment: Uncovering the Granularity Mismatch in Neural Decoding] suggests extracting visual features from the intermediate layers result in better decoding performance. I'd like to know the authors opinion on whether fusion is better than optimizing the extracted layer depth.
-
I'm not sure if I understood why the augmented visual features are summed to get the target embeddings and if the augmentation is needed at all. Did the authors perform any ablation on the augmentations used?
-
Why did the authors only normalized visual features and not EEG features? This is mentioned in the paper but not supported.
-
In the capacity sweep experiment, why did the authors only tested different widths and not depth, given that deeper networks usually learn better representations than wider ones?
-
Could the authors elaborate on the two control experiments used to test the effect of temporal resolution on the EEG encoder (two TSConv-inspired forms of temporal coarsening)? Specifically could you provide the details of the network used for each test and compare it with the original TSConv network? A pytorch pseudocode or mathematical explanation would be helpful. Also, if the only goal is to coarsen the temporal resolution, why didn't you simply downsample or low-pass filtered the input signal?
-
I'm not sure if I understood the motivation for "temporal alignment and shift sensitivity experiment". Could you please explain how it contributes to the claims and design choices of the paper?
2 comments
Q3. Normalization. Image-only normalization applies to the training loss; retrieval uses cosine scores, so both sides are normalized (Eq. 4 now shows the EEG denominator). With a fixed logit scale, the EEG norm acts as a per-sample confidence during training. At retrieval, normalizing EEG changes fusion Top-1 by only 0.1 point (A.2).
Q4. Depth. The capacity analysis (Sec. 4.2, A.6) asks whether encoder size explains the ranking, not which architecture is largest. Both ends of depth are covered. On the deeper side, Table 4 already includes deeper encoders, and all fall below EEGProject in fused Top-1: TSConv (79.3%), ATM with an iTransformer (81.2%), and LaBraM with a 12-layer Transformer (23.5%), versus 84.2%. On the shallow side, the Linear control (Table 14) removes the hidden residual layer and changes Top-1 by only 1.6 points, whereas input smoothing at an identical parameter count costs 12.9 points (now in Sec. 4.2). Together with the 110-fold width sweep (Fig. 4a), neither width nor depth explains the ranking.
Q5. Temporal controls. B.8 now describes the three controls; their code is in the updated supplementary material. Both requested operations are included: smoothing is a 204-ms low-pass filter before an unchanged EEGProject, and 40-ms weight sharing equals block-average downsampling to 25 Hz. TSConv's pooling window/stride goes from 204/20 to 20/4 ms (S8, five seeds):
| Encoder | Smoothing | Weight grid | Top-1 |
|---|---|---|---|
| EEGProject (TRACE) | none | 4 ms | 84.20±0.91 |
| EEGProject | none | 40 ms (weight sharing) | 72.30±2.17 |
| EEGProject | 204 ms (input low-pass) | 4 ms | 71.30±3.05 |
| TSConv (original) | 204 ms | 20 ms | 79.30±1.68 |
| TSConv | 20 ms | 4 ms | 84.60±1.39 |
Full retrieval and reconstruction results are in Table 16, and the S5/S9 replication in Table 19.
Our question, however, concerns encoder design, not signal quality: as defined in Sec. 3.1 (L191–192), fine-grained temporal encoding is the encoder's ability to exploit fine temporal variation in a fixed input. All encoders get the same 17×250 input, and TSConv's own pooling already does a 204-ms average and 5× subsampling internally. The controls are therefore bidirectional on the same input: coarsening EEGProject's aggregation lowers Top-1, and refining TSConv's raises it. Input downsampling or filtering can only remove information and cannot produce the second direction: the bottleneck is how the encoder aggregates time, not what the signal contains.
Q6. Motivation for the shift experiment. Our main claim is that EEGProject performs best because it keeps fine, tens-of-ms temporal resolution. The temporal interventions in Sec. 4.2 test this on the input side: coarsening the input or the encoder lowers accuracy. The shift experiment tests it on the trained model. If EEGProject relies on precise timing, shifting the test input by a few ms should hurt it more than the other encoders. That is the result: EEGProject is the best encoder at zero shift and degrades fastest, and TSConv overtakes it on most reconstruction metrics at about 8–12 ms (A.12).
As noted in L405–411, this result has another possible explanation. EEGProject has a separate weight for each time point, so it would be shift-sensitive even if the EEG carried no fine-timing information. Jitter training separates the two explanations. If the sensitivity came only from the weights, the same model trained with jitter should become shift-tolerant at little cost. If decoding depends on precise timing, shift tolerance should cost clean accuracy. The second is what happens (S8, fused Top-1, %):
| Training jitter | Clean Top-1 (5 seeds) | Test shift 0 → 24 ms (1 run) |
|---|---|---|
| none | 85.6±1.14 | 88.00 → 3.00 |
| ±8 ms | 83.7±0.84 | 86.00 → 8.75 |
| ±20 ms | 79.4±1.75 | 78.50 → 49.50 |
| ±40 ms | 67.1±2.92 | 67.50 → 67.75 |
| ±80 ms | 56.3±2.68 | 56.50 → 52.25 |
With ±40 ms jitter, accuracy is flat across test shifts, but clean Top-1 falls from 85.6 to 67.1% (Tables 24–25).
The experiment contributes to the paper in three ways. It supports the claim that fine timing carries decodable information with the architecture and parameter count held fixed. It explains why we do not add shift invariance to EEGProject, such as jitter augmentation or temporal pooling: invariance costs accuracy. And EEGProject stays ahead of TSConv at every jitter level (Table 23), so its lead does not come from position-specific weights.
We hope these answers resolve your concerns about variability, data scope, and the design choices you asked about. If anything remains unclear or you have further questions, we would be glad to discuss them at any time during the discussion period. If your concerns are resolved, we would be grateful if you would consider raising your score; your assessment matters a great deal to us.
Thank you for the questions. Line numbers refer to the submitted version; changes are in blue, and Appendix B is new.
W1. Variability. TRACE retrieval results (Tables 2, 6) are five-seed means. The ten-subject mean is 77.5±0.45% Top-1 and 96.2±0.19% Top-5 (sample SD), and per-subject Top-1 SDs range from 0.74 to 2.26 points. S8 analyses report SDs (Tables 14/16/17/20/24). Table 5 now reports three Stage-2 seeds per routing row (e.g., CLIP 0.8445±0.0017), and the main new controls (B.1, B.3, B.4, B.7) use 3–5 seeds. Against Brain-HIVE's published scores, TRACE has the highest average retrieval (higher Top-1 on 6/10 subjects), and its reconstruction gains are large and consistent: six of seven metrics improve on all ten subjects (A.4).
W2. Datasets. We transferred TRACE to Alljoined-1.6M, an independently recorded dataset (consumer-grade 32-channel EEG, lower SNR), on S01 and S14 (fused Top-1, %, three seeds):
TRACE nearly doubles NeuroBridge on S14 and on S01 beats every method in ProCA's Table II. More importantly for your concern, the temporal finding itself transfers: coarsening EEGProject with 204-ms input smoothing lowers Top-1 on both subjects, and the same TSConv refinement raises it on both (B.7, Table 34). Together with the S5/S9 replication (A.8), the claim is not specific to one dataset or subject.
Q1. Layer choice. Layers 2–3 are RN50's mid-level stages, and layer 4 feeds the H embedding, so the two targets cover different depths. To test whether a better single depth could replace fusion, we kept the full model and replaced only the L target with each single layer (S8, five seeds; new B.4, Table 30):
In line with Du et al., intermediate-layer targets decode well: layer 3 alone (73.9%) already exceeds the final embedding (71.2%). But the best single depth stays 10.3 pp below fusion (84.2%), and fusing H with any single layer improves on that layer alone. Optimizing depth therefore does not replace fusion. Among the tested targets, layer2+3 is the best both alone and fused. This is consistent with the capacity control in B.3: a capacity-matched L+L reaches 74.1%, versus 83.2% for a smaller H+L. Sec. 3.2 now discusses Du et al. and points to this control.
Q2. Augmentation pooling. Features of four level-specific augmentations are averaged with equal weights (Eq. 3). As described in Sec. 3.2 (L222–224), this keeps level-stable information. The relevant controls (S8, Top-1, %):
Removing pooling lowers Top-1 by 8.2 pp, and adding further transforms also hurts, so more augmentation is not simply better. The new test-only control fixes the five trained models and changes only the test gallery, isolating test-time pooling.