- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes RNSC for cross-subject EEG-to-image retrieval. The framework combines a multi-level visual reference (UMVA), an EEG encoder (STAR-EEG), training-time subject normalization (TC-SAW), and sequential test-time calibration (SPPC). The main idea is to learn EEG representations in subject-standardized coordinates and estimate the corresponding statistics for unseen subjects using source statistics and past unlabeled target EEG. On THINGS-EEG2, the authors report 93.67% intra-subject Top-1 and 48.10% inter-subject Top-1 under 200-way retrieval. The paper also provides component ablations, latent geometry analysis, and cold-start experiments.
While the unified coordinate-system view could be useful for this research area, and the paper reports very high intra- and inter-subject retrieval accuracy, several important concerns remain regarding evaluation validity, performance attribution, and scientific novelty. These issues substantially weaken the evidence supporting the main claims and the overall contribution of the paper.
Critical strengths
-
The component ablations are reasonably well structured. Table 5 uses a 2×2 comparison of TC-SAW and SPPC under the same STAR-EEG representation. The authors also report paired results across multiple runs. This is useful for separating the effects of training-time normalization and deployment-time calibration, rather than only comparing the full system against previous methods. (Sec. 4.5, Table 5, Appendix Tables 13 and 15)
-
The cold-start analysis provides some useful insights. Figure 3 and Appendix Table 21 show how the benefit of the source prior changes as target observations accumulate. The authors also examine different prior strengths and report that an overly strong prior can hurt performance. These analyses help explain when sequential calibration is useful and where its gains become limited. (Sec. 4.6, Fig. 3, Appendix Tables 21–22)
Critical weaknesses
-
Checkpoint selection and evaluation validity. The supplementary implementation appears to select the best checkpoint using Top-1 accuracy on the final test set. If this is the code used for the reported experiments, it would raise a substantial concern about test-set independence, especially for the leave-one-subject-out evaluation. It also seems inconsistent with the model selection description in Sec. 4.1. If the reported results were obtained using this procedure, the performance estimates may be optimistic. The SPPC ablation could also be affected, since the selected checkpoints are evaluated under different calibration settings. This issue needs to be clarified before the main results can be considered reliable. (Sec. 4.1, Tables 1–2 and 5, Appendix Table 13, supplementary implementation)
-
The reported performance gains are not clearly attributed to the proposed components. Table 3 shows that EEGProject with raw-mean aggregation already achieves 92.05% intra-subject Top-1. Under the same encoded-mean aggregation, STAR-EEG improves Top-1 from 92.81% to 93.67%, a gain of only 0.86 percentage points. Similarly, Table 5 shows that the complete TC-SAW+SPPC configuration improves inter-subject Top-1 from 46.00% to 48.10% compared with the configuration without TC-SAW and using History-only calibration. These improvements are positive, but they do not clearly explain the much larger performance gap against published inter-subject methods, such as SAMGA (Jiang et al., 2026) in Table 2. Since the previous methods follow their original protocols, I think stronger matched comparisons are needed to clarify where the reported improvements actually come from. (Sec. 4.3–4.5, Tables 1–3 and 5)
-
The motivation and additional scientific insight are not sufficiently convincing. In the Introduction, the authors describe two main directions of prior work: representation learning and handling subject variation. However, it is not clearly explained why treating these two aspects separately is insufficient, or what specific problem requires them to be coordinated in a unified coordinate system. The distinction between the two directions is also somewhat simplified. For example, SAMGA involves both multi-level visual supervision and subject-aware visual routing, rather than fitting only into the second direction. More importantly, although the proposed combination is reasonable, Table 5 mainly supports the individual benefits of TC-SAW and SPPC, with limited evidence for a distinct coordination effect. I think the paper needs a clearer motivation and stronger scientific insight beyond combining existing techniques. (Sec. 1–3, Table 5)
-
The cold-start evaluation protocol and robustness need further clarification. The main evaluation uses repeated-trial aggregation, while sequential calibration operates over 200 concept-level queries. The exact preprocessing information available from the unseen subject is not sufficiently clear, especially whether target-subject statistics are used before the first query. Also, the large early SPPC gains are reported under a fixed query order, while the overall gain over 200 queries is only about 1.2 percentage points. More evidence is needed to understand how robust these results are under different query orders and initialization conditions. (Sec. 4.1, Sec. 4.6, Fig. 3, Appendix Table 6 and Table 21, supplementary preprocessing)
Questions for authors
-
Could the authors clarify how the final checkpoints were selected? The supplementary implementation appears to use test Top-1 accuracy for checkpoint selection. Were the main results obtained using this procedure? Could the authors provide results using an independent validation protocol without using the final test labels for model selection?
-
What mainly explains the very high retrieval accuracy? Table 3 shows that STAR-EEG improves only 0.86 percentage points over EEGProject under the same encoded-mean aggregation. Could the authors better separate the contributions of the visual representation, EEG encoder, and training protocol? More closely matched baselines would help clarify the large differences against previous methods.
-
Why is a unified coordinate system needed, and what new insight does it provide? The Introduction describes representation learning and handling subject variation as two main directions of prior work, but it is not clear how this motivates the need for a unified coordinate system. Could the authors further explain what specific limitations arise when these two aspects are handled separately, why the proposed coordination is necessary, and what new scientific insight it provides beyond combining existing techniques?
-
Could the authors clarify the cold-start information budget and robustness? How many EEG repetitions are used per query, and are target-subject statistics available during preprocessing before the first query? Also, do the SPPC gains remain stable under different query orders and a fixed Source-only normalization baseline?
Reference
Jiang, L., She, Q., Xu, J., Xu, H., Wu, D., and Kuang, Z. (2026). Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval. arXiv preprint arXiv:2604.17782.
9 comments
Q3. Why a unified coordinate system is necessary and what scientific insight it provides
3.Why is a unified coordinate system needed, and what new insight does it provide? The Introduction describes representation learning and handling subject variation as two main directions of prior work, but it is not clear how this motivates the need for a unified coordinate system. Could the authors further explain what specific limitations arise when these two aspects are handled separately, why the proposed coordination is necessary, and what new scientific insight it provides beyond combining existing techniques?
The need for a unified coordinate system arises because source representation learning and unseen-subject calibration act on the same EEG–visual retrieval geometry. TC-SAW standardizes trial-level EEG latents using source-subject statistics inside contrastive training. The shared encoder is thus optimized to preserve stimulus-discriminative information under reduced subject-specific statistical variation, rather than relying solely on subject-dependent latent moments.
At deployment, an unseen subject has no directly available matching latent statistics. SPPC combines the source prior with strictly past unlabeled target queries, estimating the subject's normalization coordinates under predict-before-update, without updating the encoder or accessing future queries.
Crucially, SPPC does not invert TC-SAW or restore raw features; it maps target latents into coordinates compatible with those used to train the retrieval model. Deployment-only calibration cannot change an encoder learned without training-time standardization, whereas training-time standardization alone cannot supply an unseen subject's latent statistics. This is the concrete limitation addressed by coordinating the two stages.
This emphasis on sequentially available statistics also complements the batch-oriented structural expert/PoE discussed in SATTC’s own Limitations (Huang and Zhu, 2026): RNSC makes each prediction before incorporating the current query into the calibration state. Consistent with their complementary roles, Table 5 shows 46.90% → 48.10% Top-1 when adding TC-SAW under the same SPPC setting, and 46.89% → 48.10% when replacing History-only with SPPC under TC-SAW training.
As a separate-dataset deployment check, Alljoined-1.6M LOSO yields +0.592 pp SPPC-over-History-only across 60 folds at shared, test-selected best checkpoints (mirK AQ(a)). This provides additional evidence of deployment calibration on a second dataset; the checkpoint-selection protocol is addressed in PRNL Q1.
Alternative training-time normalization strategies. To test whether the underlying principle extends beyond one source-statistics estimator, we compared None, Source-subject (the paper's TC-SAW implementation), and Global (one pooled set of statistics from all nine source subjects in each batch) during training. At evaluation, all three retain the same online diagonal SAW/SPPC (κ=8), encoded-mean queries, and cosine retrieval. With five seeds × ten LOSO target subjects, the common epoch-20 results are:
| Training-time normalization | Top-1 (%) | Top-5 (%) |
|---|---|---|
| None | 43.25 | 75.58 |
| Source-subject (TC-SAW) | 44.21 | 75.93 |
| Global (pooled source statistics) | 45.39 | 76.47 |
Global improves Top-1 over None by +2.14 pp and over Source-subject by +1.18 pp, with subject-level paired 95% CIs of [+0.65, +3.63] pp and [+0.02, +2.34] pp, respectively. Together, these results show that the choice of source-side normalization coordinates matters, and that training-time latent standardization has substantial room for further development. In particular, the strong pooled-global result suggests that the RNSC perspective can accommodate a family of training-time estimators, motivating hierarchical and shrinkage-based alternatives alongside subject-conditioned statistics. The comparison uses shared subjects, seeds, and evaluation settings at epoch 20.
The paired seed-99 analysis (see ieJB Q2) further distinguishes raw scale changes from stimulus-relevant geometry: the same-stimulus versus different-stimulus cosine margin rises from 0.0111 to 0.4661 after online SAW/SPPC. This measures the combined deployment transformation, not TC-SAW alone. The scientific insight is that training-time latent normalization shapes what the encoder learns, while causal deployment calibration determines how an unseen subject enters that learned standardized retrieval space. We will articulate this specific training–deployment connection in the Introduction.
Q4. Cold-start information budget, preprocessing, query order, and Source-only baseline
4.Could the authors clarify the cold-start information budget and robustness? How many EEG repetitions are used per query, and are target-subject statistics available during preprocessing before the first query? Also, do the SPPC gains remain stable under different query orders and a fixed Source-only normalization baseline?
We distinguish three information sources: (i) the 80 EEG repetitions of the current image, encoded individually and averaged into a single query; (ii) upstream session-wise MVNN whitening from the UBP preprocessing pipeline used in our experiments, whose whitening matrix is fitted on each participant's training EEG partition only (sigma_tot = sigma_part[1]) and then applied to that participant's training and test EEG; and (iii) SPPC's sequential, unlabeled target-test-query latent history.
Importantly, the UBP whitening matrix is not fitted using target test EEG covariance or test labels. For the held-out participant, the upstream whitening can use that participant's training-split EEG as preprocessing data, but the learned encoder is trained only on source subjects. This preprocessing does not contradict our methodological contribution: TC-SAW operates on learned EEG latents during source training, whereas SPPC calibrates latent moments using only past target-test queries.
Before the first retrieval query, SPPC initializes from the source prior and has no target-test-query latent history; each query is retrieved before its latent representation updates the online state. Separately, upstream input whitening uses the held-out participant’s training EEG partition as described above.
Our earlier five random orderings × five training seeds × ten held-out subjects provide a matched-checkpoint analysis of causal calibration:
| Query regime | History-only Top-1 | SPPC Top-1 | Difference |
|---|---|---|---|
| Original order, all 200 | 46.89 | 48.10 | +1.21 pp |
| Random orders, first 8 | 15.80 | 23.80 | +8.00 pp |
| Random orders, first 16 | 29.90 | 31.35 | +1.45 pp |
Early-history composition and order. We additionally compare three configurations of the first 32 target queries, reporting the Top-1 gain of SPPC over History-only at different query horizons:
| First-32-query configuration | First 8 gain | First 32 gain | All 200 gain |
|---|---|---|---|
| Original set, original order | +18.00 pp | +5.81 pp | +1.21 pp |
| Same set, shuffled internally | +15.50 pp | +5.31 pp | +1.13 pp |
| Random set | +13.00 pp | +1.56 pp | +0.30 pp |
Retaining the same first-32 set while shuffling its internal order largely preserves the gain (+5.81 → +5.31 pp over the first 32; +1.21 → +1.13 pp over all 200). Changing the early set itself has a larger effect (+1.56 pp over the first 32; +0.30 pp over all 200). These results suggest that which observations enter the early calibration history matters, beyond history length alone. The two analyses probe different order/composition perturbations and their prefix gains should not be pooled.
A fixed Source-only normalization reference obtains 20.42% Top-1 over 200 queries, regardless of order. For the earlier five-permutation analysis, the first-eight-query paired, subject-level 95% CI is [+3.33, +12.67] pp. Overall, these comparisons identify causal cold-start calibration as SPPC's main strength and show that its effectiveness depends on the information contained in the early target history, not only the number of accumulated queries.
A fixed source prior can be especially useful before reliable target statistics emerge, while the optimal balance may evolve as evidence accumulates. These observations also motivate two future directions: reliability-aware online moment fusion, which adjusts source-versus-target weighting according to the uncertainty and stability of sequential target estimates; and adaptive switching, which progressively shifts from source-prior-dominated to target-history-dominated calibration when the available evidence becomes sufficiently reliable. Both would retain the predict-before-update information constraint; they are prospective extensions rather than mechanisms evaluated in this work.
We appreciate the reviewer’s detailed scrutiny of checkpoint selection, component attribution, and the cold-start information budget. These are important questions for evaluating the validity and interpretation of our results. We address each point below through explicit protocol descriptions, empirical comparisons, and their interpretation.
Q1. Checkpoint selection and evaluation validity
We agree that independent checkpoint selection is important for evaluation validity. The original checkpoints were selected using target-test Top-1; the 27-study protocol audit and common-epoch comparisons quantify and explain the reasons for and effects of checkpoint selection.
(1) Checkpoint practices in 27 related studies. Based on reported methods and publicly available implementations:
Examples: NICE selects by validation loss; SATTC by source-validation Top-5; public EEGiT cross-subject code reuses its test loader for validation; Shallow Alignment tracks test loss/Top-1; SAMGA reports its final retrieval metrics on the test set.
(2) Validation mismatch. THINGS-EEG2 has 16,540 training images (4 EEG repetitions each) versus 200 test images (80 repetitions each). Our encoded-mean aggregator encodes repetitions individually before averaging, and test concepts also differ. Source-only validation is independent but does not exactly match the final 80-repetition, 200-way test condition.
(3) Checkpoint audit and fixed-epoch evaluation. We audited 300 run–fold records. The original
best.ptcheckpoints were selected by held-out target-test Top-1. To quantify performance under a common training budget, we report epoch 50 for intra-subject retrieval and epoch 20 for inter-subject LOSO:Calibration alternatives within each run/fold share one checkpoint. Epochs 50/20 were chosen after inspecting historical trajectories; these are descriptive common-epoch comparisons, not prospectively selected independent-validation results. Separate 60-epoch intra-subject runs are labeled accordingly.
In summary, the 27-study audit documents protocol heterogeneity; the 4-vs-80 repetition mismatch explains the gap between source-validation and test conditions; and the fixed-epoch comparison demonstrates the model's performance under a common training budget. We will state the checkpoint rule once in the evaluation protocol and label both sets of reported results accordingly.
Q2. Attribution of the high retrieval accuracy
As summarized in the General Response, visual representation quality contributes materially to retrieval performance alongside calibration. The epoch-50 intra-subject visual comparisons (66Jr Q3(b)) yield 92.15% UMVA versus 89.93% parameter-matched independent projection and 86.02% single L28. For EEG encoders at epoch 50, STAR-EEG = 91.94% and EEGProject = 90.92%.
At epoch 50, encoded-mean yields 91.94% versus 90.93% for raw-mean. Table 5 separately reports 48.10% for full RNSC versus 46.00% for no TC-SAW + History-only, under the original test-selected checkpoint rule.
The separate 60-fold Alljoined LOSO comparison (mirK AQ(a)) yields SPPC 16.025% versus History-only 15.433% Top-1 (+0.592 pp) at the same checkpoints selected using target-test Top-1. This provides additional deployment-calibration evidence under the reported selection rule; PRNL Q1 describes the checkpoint-selection protocol. The source-side normalization strategy comparison in PRNL Q3 further examines the training component of the framework.