- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes PPA-SAT, a two-level alignment framework for EEG-to-image retrieval. PPA uses EEG-conditioned attention to combine five perceptual visual representations for each EEG–image pair. SAT introduces subject-specific temperature offsets to calibrate contrastive learning across source subjects.The method is evaluated on THINGS-EEG under intra-subject and leave-one-subject-out (LOSO) settings. The paper reports 91.7% Top-1 intra-subject accuracy and 45.8% Top-1 LOSO accuracy, with large improvements over the listed baselines. Overall, PPA introduces a meaningful idea with some scientific novelty, and the reported experiments provide encouraging results. However, several serious concerns remain regarding implementation consistency, evaluation validity, and experimental evidence. These issues significantly weaken confidence in the reported improvements and the paper's main conclusions.
Critical strengths
-
Clear and meaningful problem formulation. The paper identifies two relevant sources of heterogeneity in EEG–visual alignment: trial-level differences in visual information and subject-level differences in neural separability. The motivation is clear, and the two-level formulation gives a reasonable distinction between what visual information should be aligned and how the alignment should be optimized.
-
A meaningful and well-motivated extension of perceptual-prior alignment. PPA has a thoughtful design beyond the fixed visual mean fusion used in NeuroBridge. In Eqs. 3–7, the attention weights depend jointly on the current EEG query and candidate image. This allows the same candidate image to have different effective representations for different EEG queries. More importantly, PPA changes the retrieval scoring function itself, rather than simply producing another fixed visual embedding. The residual formulation in Eq. 6 also provides a controlled way to combine the uniform visual prior with EEG-conditioned attention. This is a concrete and meaningful methodological contribution.
-
A simple and reasonable design for source-subject calibration. SAT introduces a lightweight temperature parameterization with a shared global component and source-subject-specific offsets (Sec. 3.4, Eq. 9). This is a reasonable way to account for subject differences during joint contrastive training, without introducing separate subject-specific encoders. The distinction between source-subject training calibration and unseen-subject inference is also clearly defined (Sec. 3.4, lines 223–225).
-
Relevant experimental settings and useful component comparisons. The paper reports results for all ten subjects under both intra-subject and LOSO protocols, including Top-1 and Top-5 accuracy (Tables 1–2). These two settings are important for evaluating EEG–visual alignment, especially cross-subject generalization. The experiments also compare intermediate representations, perceptual-prior fusion, SAT, and their combinations (Tables 3–5). In particular, the comparison between uniform fusion, residual fusion, and EEG-conditioned attention is directly relevant to the proposed PPA mechanism.
Critical weaknesses
-
Inconsistency between the proposed PPA scoring function and the submitted implementation. Sec. 3.3 (Eqs. 3–7) defines PPA as a query-conditioned scorer, and the paper states that ground-truth identity is not used during inference (p. 4, lines 194–196). However, the submitted training and evaluation code constructs fused image representations using their correctly paired EEG, rather than the current EEG query for every candidate. During evaluation, this uses known test EEG–image pairings in candidate construction. The training contrastive scores also differ from the proposed pairwise formulation. The supplementary package includes another analysis implementation with the correct query-conditioned scorer. It is therefore important to clarify which implementation produced Tables 1–2 and the ablations. This difference could affect how the reported retrieval performance should be interpreted.
-
The checkpoint selection procedure needs further clarification. The submitted training code evaluates the test set after each epoch and uses test performance for best-checkpoint selection and early stopping. The provided LOSO script also selects checkpoints based on test Top-1 accuracy. In this implementation, the held-out subject is excluded from gradient-based training, but its test performance still influences model selection. This may introduce selection bias if the reported results follow this procedure. The paper does not clearly describe how the final checkpoints for Tables 1–2 were selected (Secs. 4.1–4.4). Clarifying this procedure would help establish the reliability of the reported generalization results.
-
SAT implementation and ablation need further clarification. The image-to-EEG temperature scaling in the submitted code differs from Eq. 13 (Sec. 3.5). When source subjects have different temperatures, the two formulations define different contrastive objectives. Also, the provided SAT ablation changes both the learnability of the global temperature and the introduction of subject-specific offsets. Thus, Table 3 does not clearly isolate the contribution of subject-specific calibration. A comparison with a learnable global temperature without subject offsets would help clarify the independent benefit of SAT.
-
The ablation studies could provide stronger evidence for the proposed mechanisms. Table 3 presents a component ablation, but its baseline and intermediate-only results exactly match the reported NeuroBridge [1] and Shallow Alignment [2] results in Table 2. It would be helpful to clarify whether all configurations were evaluated under matched training and evaluation settings. There is also an unclear relationship between Tables 4 and 5. The "All five candidates" result in Table 4 (79.20% Top-1) matches "Residual Mean + Entropy" in Table 5, rather than the full PPA result (91.70%). This makes it unclear whether Table 4 evaluates prior removal from the full PPA model. Furthermore, Appendix Table 6 reports normalized attention entropy close to 1. This does not contradict PPA, but more direct evidence would help explain how query-dependent visual weighting contributes to the large improvement, beyond a general conditional matching effect.
-
Experimental comparability and reporting could be more complete. Sec. 4.2 states an EEG window of 0–250 ms, while the provided code uses 250 samples with a default sampling rate of 250 Hz, corresponding to around 1 second. The actual experimental window should be clarified. The SoTA comparison could also be more complete. Table 1 uses the 82.6% single-layer result from Shallow Alignment [2], which also reports 91.3% Top-1 using a stronger multi-layer configuration. The closely related SAMGA [3] is not discussed. These methods may use different protocols, but discussing them would help clarify the scope of the reported improvements and the novelty of PPA-SAT. Finally, Table 1 reports 91.70%/99.00%, while Appendix Table 6 reports 91.35%/96.70% under the stated entropy coefficient. This difference needs explanation. Multi-seed statistics for the main results are also not clearly reported. Some presentation issues remain, including the unfinished baseline-verification statement in Sec. 4.3 and the placement of Table 6 within the References.
Questions for authors
-
Could the authors clarify which scoring function was used for Tables 1–2 and the main ablations? Were candidate representations computed using the current EEG query as defined in Eqs. 3–7? Also, how does the training implementation relate to the proposed pairwise contrastive objective?
-
How were the final checkpoints selected? Did the reported experiments use test-set performance for checkpoint selection or early stopping? If so, could the authors provide results with an independent validation and model-selection protocol?
-
Could the authors clarify the difference between Eq. 13 and the submitted SAT implementation? Also, does SAT still provide additional gains over a learnable global temperature without subject-specific offsets?
-
Could the authors clarify the experimental configurations used in Tables 3–5, especially the difference between the five-prior result in Table 4 and the full PPA result in Table 5? What direct evidence supports the contribution of query-conditioned visual weighting to the reported improvements?
1. Corrected scorer
We thank the reviewer for identifying a consequential problem. Constructing a candidate using its paired EEG and then scoring other queries against it leaks privileged pairing information and is invalid for query-conditioned retrieval. The submitted implementation also did not implement the stated pairwise contrastive objective. The original headline results (91.70% intra-subject; 45.80% inter-subject) are superseded; we cannot reliably map old tables to specific scripts.
The revised query-conditioned alternatives use exact pairwise scoring: weights for query i and candidate j depend only on that pair. The cosine numerator is the weighted sum of query–view dot products; the denominator is sqrt(w_ij^T G_j w_ij), where G_j is the 5×5 Gram matrix of the candidate's projected views. The same score matrix is used in both directional losses. Uniform PPA (lambda=0) uses fixed gallery embeddings. The shared-projector experiment also encodes the gallery independently of EEG. Sections 3.3, 3.6 and Figure 1 are revised.
2. Checkpoint selection and independent validation
We acknowledge that the main tables use test-based checkpoint selection. The three-seed study selects by test Top-1 with early stopping (patience 15 intra-subject; 20 inter-subject); shared-projector scripts select lexicographically by (Top-1, Top-5). Test labels therefore affect inter-subject selection, and results may be optimistic.
Appendix A.1 reports a separate three-seed intra-subject experiment: 200 training concepts are held out only for validation and 1,454 are used for fitting. Single-view accuracy is 69.58/93.47% (Top-1/Top-5), versus 77.83/96.15% for uniform PPA, an 8.25-point Top-1 gain. Residual query-conditioned variants reach 76.05% (without entropy) and 75.67% (with entropy); attention-only reaches 68.30%. For source-validation inter-subject selection, only subject 8 is complete. Its four-view global/SAT Top-1 is 23.50/24.50% (reference scale) and 18.50/21.00% (reciprocal scale); these are single-fold observations, not a ten-fold result. Source-only preprocessing remains unaudited.
3. SAT formulation and global-temperature control
We agree that the directional SAT formulations differ. Let m_ij be EEG i–image j similarity and c_s the subject-specific logit scale. Transposing the scaled EEG-to-image matrix gives image-to-EEG logits c_{s_j}m_ji, whereas independently scaling rows after transposition gives c_{s_i}m_ji. These coincide for a global scale, but not generally for SAT. Sections 3.4–3.6 distinguish reciprocal-softplus temperature and reference logit-scale parameterizations and specify both directional objectives.
Table 4 compares learnable global temperature with SAT. For uniform PPA, results are 26.95±0.75/52.92±0.54% without SAT and 26.43±0.40/51.90±0.48% with SAT (Top-1/Top-5). In matched four-view TSConv runs, global/SAT results are 31.90/61.15% and 31.85/60.50%. Appendix A.2 finds no consistent advantage from a high-learning-rate offset. We therefore do not claim an independent SAT gain.
4. Controlled ablations and attention
The former 79.20% all-five-prior and 91.70% headline results are not interpretable controlled comparisons; their historical configuration mapping is unresolved and they are superseded. Table 3 reports uniform, residual and attention-only settings, entropy coefficients and three-seed variability. Uniform PPA reaches 88.65% Top-1, versus 83.67% for both residual variants and 77.93% for attention-only. Attention-only normalized entropy is about 0.99, and the residual gate remains near initialization. Evidence favors uniform aggregation.
Table 5 adds a capacity-matched shared-projector view-count study. Original/four/five-view Top-1 is 75.25/83.20/83.35% intra-subject and 28.20/31.90/31.95% inter-subject. Five views improve over the original by 8.10 and 3.75 points; the fifth view adds little over four transformed views. We do not infer a separate-projector advantage from different settings.
5. Protocol, related work and reporting
We corrected the EEG interval to 250 samples at 250 Hz (approximately 0–1,000 ms) and clarify the 17-channel EEGProject and 63-channel TSConv configurations, fixed 200-image gallery and query averaging. Tables 1–2 distinguish reproduced NeuroBridge and single-layer Shallow Alignment from published results and report all subjects. Stronger multi-layer Shallow Alignment and SAMGA are discussed, not presented as matched baselines.
The five-view inter-subject result is 31.95/61.75% (Top-1/Top-5), from a one-seed shared-projector TSConv configuration; 88.65/98.48% intra-subject is a three-seed separately projected result. We report settings and seed counts separately from the four-view global/SAT comparison. We corrected the entropy-table discrepancy, moved tables out of References, narrowed novelty and performance claims, and state remaining limitations.