acceptodds
Under review as a conference paper at ICLR 2027

PairCal: Training-Free Paired-Support Calibration for Cross-Subject EEG-to-Image Retrieval

Abstract

Cross-subject EEG-to-image retrieval remains highly challenging due to substantial inter-subject variability. Existing methods primarily address this issue by learning subject-agnostic representations for cross-modal alignment. In contrast, few studies have explored how to adapt pretrained retrieval models to unseen subjects using limited examples. We propose PairCal, a training-free calibration framework for adapting retrieval to unseen subjects with only a small set of paired EEG–image support examples. PairCal calibrates both EEG queries and image candidates through two complementary components. Support-Guided Calibration (SGC) uses EEG–image residuals from the support set to derive query-adaptive cross-modal corrections. Meta-Distractor Calibration (MDC) constructs pseudo-unseen queries from the same support set via a leave-one-out procedure to estimate and suppress candidate-specific hubness bias. Together, these components enable effective few-shot adaptation to unseen subjects without updating the pretrained encoders. Extensive experiments across held-out subjects, encoder architectures, training schemes, and support-set sizes demonstrate consistent retrieval gains.

Primary area:
applications to neuroscience & cognitive science
Keywords:
brain decoding, EEG

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rt1SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PairCal, a training-free calibration framework for few-shot cross-subject EEG-to-image retrieval. It consists of two components: Support-Guided Calibration (SGC), which uses paired EEG–image examples to correct query representations through residual kernel regression, and Meta-Distractor Calibration (MDC), which uses leave-one-out reference queries to estimate and suppress candidate-specific attraction. Without updating the pretrained encoders, PairCal achieves improved retrieval performance on the THINGS-EEG dataset.

Critical strengths
  1. PairCal addresses the important challenge of adapting pretrained EEG-to-image retrieval models to unseen subjects using limited paired support data, without updating model parameters or accessing other target test queries.

  2. PairCal achieves substantial retrieval improvements across held-out subjects, EEG backbones, and support-set sizes. Component ablations and candidate-level analyses provide evidence supporting the effectiveness of the proposed calibration framework.

Critical weaknesses
  1. SGC lacks direct quantitative analysis of EEG–visual representation alignment before and after calibration. Although existing analyses suggest that MDC mitigates candidate-side bias, they do not sufficiently examine whether it may over-penalize correct candidates with high attraction scores.

  2. SGC closely resembles existing residual kernel regression methods, while MDC shares similarities with established retrieval calibration approaches. More targeted comparisons are needed to establish the distinct contributions of the proposed mechanisms. Furthermore, the evaluation primarily relies on trial-averaged EEG signals and a fixed-size candidate gallery, leaving robustness under more challenging retrieval conditions insufficiently explored.

Questions for authors
  1. Is there more direct evidence supporting the relationship between local similarity in EEG representations and the transferability of cross-modal residuals? Can the authors further demonstrate that SGC improves the alignment between paired EEG and visual representations?

  2. Although existing experiments suggest that MDC can reduce spurious candidate attraction, its impact on correct matches remains unclear. Could MDC excessively penalize correct candidates with high attraction scores?

3.The paper shows that LOO-based reference queries achieve higher correlations with offline candidate-attraction statistics. Does this advantage translate into improved retrieval performance? Could the authors compare different reference-query construction strategies to better isolate the contribution of the LOO mechanism?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors

Response to Reviewer rt1S

We thank the insightful review and would like to address the issue by points:

W1 / Q1. Residual transfer and EEG–visual alignment

Residual transfer. We evaluated EEG-conditioned residual prediction on held-out support pairs. For each held-out pair, we compared the EEG-conditioned prediction with a mean-residual baseline fitted to the remaining support pairs. The leave-one-out residual-prediction skill relative to this baseline was positive at all five support sizes, averaging 0.0830 across budgets, with positive average skill in all 10 subjects. These results support the effectiveness of EEG-conditioned residual prediction on held-out pairs and motivate a more detailed analysis of how residual transferability relates to local EEG similarity.

To directly quantify this relationship, we conducted a controlled single-donor transfer analysis using the original frozen embeddings and support draws. For each held-out support pair ii, we transferred another support sample's residual Rj=Ij−BjR_j = I_j - B_j and measured the resulting paired-alignment gain:

gij=cos⁡(normalize⁡(Bi+Rj),Ii)−cos⁡(Bi,Ii),i≠j.g_{ij} = \cos\left(\operatorname{normalize}(B_i + R_j), I_i\right) - \cos(B_i, I_i), \qquad i \ne j.

For each fixed recipient, we divided donors into low-, medium-, and high-similarity tertiles based on their pre-calibration EEG cosine similarity to that recipient, thereby controlling for recipient difficulty across donor groups. We additionally performed 20 image-pair derangements within each recipient's remaining support set, excluding its own pair while preserving the EEG representations and fit-set mean residual. This allowed us to isolate the contribution of correct EEG–image correspondence at different levels of local EEG similarity. Specifically, we defined the controlled interaction as

D=(gtrue−gshuffle)high−(gtrue−gshuffle)low.D = (g^{\mathrm{true}} - g^{\mathrm{shuffle}})_{\mathrm{high}} - (g^{\mathrm{true}} - g^{\mathrm{shuffle}})_{\mathrm{low}}.

Here, gtrueg^{\mathrm{true}} and gshuffleg^{\mathrm{shuffle}} represent the alignment gains obtained with correctly paired and shuffled residuals, respectively. A positive DD indicates a stronger pairing-specific transfer advantage among EEG-similar donors. We averaged checkpoint and support-seed repetitions within each subject and estimated confidence intervals using 10,000 paired subject-bootstrap resamples.

Direct single-donor transfer analysis. Values are differences in cosine-similarity gains. Brackets denote pointwise 95% subject-bootstrap confidence intervals.

Support budget KKTrue transfer: high−low gain [95% CI]Controlled interaction DD [95% CI]Subjects with D>0D > 0
10−0.0024 [−0.0124, +0.0072]+0.0798 [+0.0724, +0.0875]10/10
20−0.0069 [−0.0126, −0.0005]+0.0729 [+0.0658, +0.0803]10/10
50−0.0043 [−0.0088, +0.0001]+0.0763 [+0.0686, +0.0834]10/10
100−0.0051 [−0.0102, +0.0004]+0.0767 [+0.0695, +0.0833]10/10
200−0.0035 [−0.0083, +0.0015]+0.0777 [+0.0701, +0.0848]10/10
Across-budget average¹−0.0044 [−0.0095, +0.0009]+0.0767 [+0.0703, +0.0827]10/10

¹ Budgets were averaged within each subject before cross-subject inference.

The controlled interaction was consistently positive across all five support budgets, with all 10 subjects showing positive differences at every budget. The across-budget average reached +0.0767 [95% CI: +0.0703, +0.0827]. These results provide direct quantitative evidence that the pairing-specific contribution of residual transfer depends on local EEG similarity: correctly paired residuals offer a greater advantage over shuffled residuals when donors are more similar to the recipient in EEG space. This finding supports the role of local EEG structure in guiding residual transfer, consistent with the EEG-conditioned residual prediction employed by SGC.

Paired alignment. We further evaluated how SGC affects EEG–visual alignment on the test set. Specifically, we measured the cosine similarity between each test EEG embedding and its matching image embedding before and after SGC. We also assessed match specificity, defined as the similarity to the matching image minus the mean similarity to all nonmatching images for the same query.

KKPaired cosine after SGCΔPaired cosineΔMatch specificity
100.3703+0.0319+0.0180
200.3871+0.0487+0.0271
500.4058+0.0674+0.0418
1000.4154+0.0770+0.0488
2000.4305+0.0921+0.0611
AuthorsOpenReview ↗

Official Comment by Authors

The frozen model achieved a mean paired cosine similarity of 0.3384. After SGC, paired cosine similarity improved consistently across all support sizes, reaching 0.4305 at K=200K=200. Match specificity also increased at every budget, with the improvement growing from +0.0180 at K=10K=10 to +0.0611 at K=200K=200. These consistent gains demonstrate that SGC strengthens alignment between EEG embeddings and their corresponding visual representations while improving their relative similarity to matching versus nonmatching images. Together, the residual-prediction, similarity-stratified transfer, and paired-alignment results provide complementary evidence that SGC effectively exploits EEG-conditioned residual information to improve EEG–visual alignment.

W1 / Q2. Over-penalization of high-attraction correct candidates

We grouped gallery images into attraction-score quintiles within each run and assigned each query to the group of its ground-truth image. A harm is a correct SGC Top-1 result that becomes incorrect after applying MDC, and a rescue is the reverse transition.

KKOverall harm (%)Overall rescue (%)Overall ΔTop-1 (pp)Highest-attraction 20% ΔTop-1 (pp)
102.543.53+0.99−5.19
202.183.85+1.67−4.78
502.034.14+2.11−4.53
1002.125.42+3.31−2.97
2002.747.20+4.46−4.19

The reviewer's concern is borne out in the highest-attraction group. Its Top-1 accuracy decreased at every tested KK. Among queries that SGC originally retrieved correctly, the highest-attraction group had a harm rate 8.99 to 16.28 percentage points higher than the lowest-attraction group. Across all queries, however, MDC corrected more errors than it introduced at every support size, yielding positive overall Top-1 gains.

The penalty weight governs the size of this effect. When the correct image has a higher attraction score than a competing image, increasing λλ reduces the correct image's score margin over that competitor. We will add the subgroup results and discuss this trade-off in the revision.

W2 / Q3. Methodological contributions and the retrieval benefit of LOO

SGC builds on residual kernel regression by applying it to paired EEG–image embeddings while retaining the frozen EEG query as an anchor. The comparisons with Mean Residual, Similarity-Weighted Residual, and Direct KRR in Table 3 examine the role of the residual estimator and the frozen query anchor. MDC uses the same form of top-kk candidate correction as NNN. Our additional design is to construct the reference bank by calibrating each support EEG with the other support pairs.

We added final retrieval comparisons using Raw, Full, and LOO reference banks. All methods shared the same SGC query scores, support samples, checkpoints, and candidate galleries across 450 matched evaluation settings. Raw uses the original support EEG embeddings. Full calibrates each reference using all support pairs. LOO excludes the reference's own pair during calibration. Hyperparameters were selected using source-domain validation episodes.

KKSGCSGC + NNN (Raw)SGC + NNN (Full)PairCal (LOO)
1024.99 / 53.7425.60 / 55.2225.70 / 55.0225.98 / 55.62
2026.79 / 56.7327.72 / 58.2528.05 / 58.1328.46 / 59.24
5029.48 / 61.2930.50 / 62.1331.27 / 63.2531.59 / 63.44
10031.78 / 63.0431.93 / 63.3134.61 / 65.8635.08 / 66.50
20033.44 / 65.5533.86 / 65.7036.93 / 68.7837.90 / 69.81

Entries report Top-1 / Top-5 accuracy (%). In a separate ablation, we fixed the SGC queries, scoring rule, neighborhood size, and penalty weight. At K=200K=200, replacing the Full bank with the LOO bank increased mean Top-1 by 1.23 percentage points, with improvements in all 10 subjects. This measures the contribution of the reference construction directly on retrieval performance.

AuthorsOpenReview ↗

Official Comment by Authors

W2. Scope of the retrieval evaluation

We thank the reviewer for the constructive suggestion regarding evaluation under more challenging retrieval conditions. We agree that robustness to single-trial EEG signals and larger candidate galleries represents an important direction. Nevertheless, we would like to emphasize that our current experimental design follows established practices in EEG-based visual decoding and retrieval, rather than introducing evaluation simplifications specific to our method.

First, regarding trial-averaged EEG signals, averaging repeated EEG recordings corresponding to the same visual stimulus is a well-established practice for improving the signal-to-noise ratio in EEG-based visual decoding. This procedure has been adopted in representative studies, including UBP [1], CognitionCapturer [2], UMind [3], and Img2EEG [4]. In particular, UBP [1] employs trial averaging to improve EEG signal quality, resulting in 16,540 training examples and 200 testing examples per participant. By following this established preprocessing protocol, our evaluation remains consistent with existing methods and focuses on the effectiveness of cross-subject calibration rather than differences in EEG measurement noise.

Second, regarding the fixed candidate gallery, the 200-image gallery is directly determined by the official test partition of THINGS-EEG2 [5], comprising 200 unseen object concepts with one test image per concept. Importantly, we evaluate retrieval against the complete official test gallery rather than a manually selected subset. This 200-way evaluation protocol has been widely adopted in representative studies, including NICE [6], ATM [7], UBP [1], NICE-LLM [8], SATTC [9], and Hierarchical Visual Embeddings [10]. In particular, SATTC [9], which is closely related to our work in investigating cross-subject EEG-to-image retrieval calibration, also reports results under the 200-way leave-one-subject-out evaluation protocol.

Therefore, our use of trial-averaged EEG and the complete 200-image test gallery reflects established benchmark conventions, enabling meaningful and reproducible comparisons with prior research.

We acknowledge that these standard settings do not exhaustively characterize robustness under noisier single-trial inputs or substantially expanded candidate galleries. We will clarify this distinction in the manuscript.

References

  1. Wu, Haitao, et al. "Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2025.
  2. Zhang, Kaifan, et al. "CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 13. 2025.
  3. Xu, Chengjian, et al. "UMind: A Unified Multitask Network for Zero-Shot M/EEG Visual Decoding." Neural Networks 205 (2027): 109314. Published online in 2026.
  4. Lu, Zitong, and Julie Golomb. "Img2EEG: A Scalable and Interpretable Encoding Framework for Simulating Human EEG Responses to Visual Inputs." bioRxiv preprint, 2026. DOI: 10.64898/2026.09.16.751610.
  5. Gifford, Alessandro T., et al. "A Large and Rich EEG Dataset for Modeling Human Visual Object Recognition." NeuroImage 264 (2022): 119754.
  6. Song, Yonghao, et al. "Decoding Natural Images from EEG for Object Recognition." The Twelfth International Conference on Learning Representations (ICLR). 2024.
  7. Li, Dongyang, et al. "Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion." Advances in Neural Information Processing Systems 37 (NeurIPS). 2024.
  8. Song, Yonghao, et al. "Recognizing Natural Images From EEG With Language-Guided Contrastive Learning." IEEE Transactions on Neural Networks and Learning Systems 36.9 (2025): 15896–15910.
  9. Huang, Qunjie, and Weina Zhu. "SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2026.
  10. Zheng, Jiawen, et al. "Learning Brain Representation with Hierarchical Visual Embeddings." International Conference on Learning Representations (ICLR). 2026.
AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer,

We hope everything is going well for you. We are truly grateful for your thoughtful review and constructive comments on our work. We sincerely hope that our responses have helped address your concerns and would appreciate hearing whether any points still require clarification. If you have any further questions or suggestions, we would be delighted to discuss them with you. Thank you again for your valuable feedback and for the time and effort you have devoted to evaluating our manuscript.

Reviewer_ehSLOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PairCal, a training-free framework for few-shot cross-subject EEG-to-image retrieval. It uses paired EEG–image support examples from an unseen subject while keeping the pretrained encoders frozen. PairCal consists of two components: Support-Guided Calibration (SGC), which uses kernel ridge regression to estimate query-dependent residual corrections, and Meta-Distractor Calibration (MDC), which constructs leave-one-out support references to estimate and suppress candidate-specific attraction. Experiments on THINGS-EEG2 show that PairCal improves Top-1 accuracy from 20.50% to 37.90% with 200 support pairs. The paper also provides component ablations and mechanism analyses. Overall, the paper addresses a meaningful problem and reports encouraging empirical results. However, the methodological novelty remains relatively limited, and the additional benefit of the proposed LOO reference construction is not sufficiently isolated. These concerns weaken the paper's overall contribution despite its positive results.

Critical strengths
  1. A clearly defined and useful adaptation setting. The paper studies inductive few-shot adaptation to unseen subjects without updating pretrained encoders or accessing other target test queries. The support images are disjoint from the evaluation gallery. This is a meaningful setting for cross-subject EEG retrieval, and its difference from transductive calibration is clearly explained in Sections 3.1 and 4.5.

  2. Consistent empirical improvements with relevant ablations. Table 1 shows that PairCal improves Top-1 accuracy from 20.50% to 37.90% at K=200, outperforming Orthogonal Procrustes (32.12%). Table 2 further shows that SGC reaches 33.44%, with MDC providing another 4.46 percentage points. Table 3 includes useful comparisons with Mean Residual, Similarity-Weighted Residual, and Direct KRR. The evaluation also covers multiple checkpoints, support seeds, and four EEG backbones (Appendix A.7).

  3. Useful mechanism analyses beyond accuracy comparisons. Tables 4 and 6–7 examine whether the candidate-attraction score predicts false positives and whether MDC reduces their concentration. Table 8 compares reference constructions using an offline oracle, where LOO achieves a correlation of 0.604 versus 0.462 for full-support references at K=200. Table 9 further examines candidate-specific penalty assignment through permutation controls. These analyses provide useful evidence for the proposed mechanism.

Critical weaknesses
  1. The methodological novelty over existing approaches needs stronger justification. SGC (Eqs. 1–4) uses residual kernel ridge regression while preserving the frozen query representation, closely related to existing kernel adaptation methods such as ProKeR (Bendou et al., CVPR 2025), although the tasks differ. MDC (Eqs. 7–8) subtracts a Top-k reference-similarity penalty from candidate scores, following the same basic normalization form as NNN (Chowdhury et al., EMNLP 2024). The paired-support setting and LOO reference construction are meaningful differences. However, the additional methodological insight beyond adapting and combining these existing techniques needs to be more clearly established.

  2. The specific benefit of LOO reference construction is not sufficiently isolated. Table 8 shows that LOO references correlate better with the offline oracle than raw or full-support references, but does not compare their final retrieval accuracy. Table 9 confirms the importance of correct candidate-penalty assignment, but does not establish the additional gain specifically from LOO over standard NNN-style correction. Since MDC adds 4.46 percentage points over SGC at K=200 (Table 2), a matched Top-1/Top-5 comparison using different reference banks with the same SGC queries is important to support this contribution.

  3. Some experimental settings and reported results need clarification. Section 4.1 reports λ=0.55, which is absent from the hyperparameter grid in Appendix A.2. The SATTC Top-5 result at K=200 also differs between Table 1 (50.91%) and Appendix A.6 (49.87%). The exact definition of the offline oracle statistic in Table 8 and the construction of source-validation episodes could also be clearer. These issues make the experimental protocol harder to verify, particularly without public implementation code.

  4. The broader generalization evidence remains limited. Most results are based on THINGS-EEG2 with a 200-image gallery and 80 averaged EEG repetitions per test image. Although four EEG backbones are evaluated, they share the same main benchmark. The MEG extension (Appendix A.8) shows smaller gains, with Top-1 improving only from 5.00% to 5.96% at K=200. Further validation would help establish the broader applicability of PairCal.

Minor weakness. Figure 2 has dense visual elements and small labels, while the qualitative examples in Figure 5 are difficult to inspect. Both figures could benefit from further polishing.

Questions for authors
  1. Could the authors further clarify the methodological differences between PairCal and existing approaches such as ProKeR and NNN? What is the main additional insight provided by the paired-support LOO construction?

  2. Could the authors compare raw-support, full-support, and LOO reference banks on final Top-1/Top-5 accuracy, using the same SGC-calibrated queries and matched hyperparameter-selection rules? Subject-wise paired improvements of PairCal over SGC, with uncertainty estimates, would also be helpful.

  3. Could the authors provide the exact definition of the offline oracle attraction statistic in Table 8 and clarify how the source-validation episodes are constructed?

  4. Could the authors explain how λ=0.55 was selected when it is absent from the reported search grid? Also, which SATTC Top-5 result at K=200 is correct, 50.91% or 49.87%?

References

[1] Bendou, Y., Ouasfi, A., Gripon, V., and Boukhayma, A. ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models. CVPR, 2025.

[2] Chowdhury, N., Wang, F., Shenoy, S., Kiela, D., Schwettmann, S., and Thrush, T. Nearest Neighbor Normalization Improves Multimodal Retrieval. EMNLP, 2024.

6 comments
AuthorsOpenReview ↗

Official Comment by Authors

Response to Reviewer ehSL

We thank the insightful review and would like to address the issue by points:

W1 / Q1. Relationship to ProKeR and NNN

Thank you for highlighting these connections.

The additional methodological insight concerns how to estimate candidate attraction after adapting queries using the same small paired support set. SGC instantiates residual adaptation in the shared EEG–image embedding space, allowing calibration to transfer to gallery images absent from the support set. This adaptation also changes the query representations on which candidate attraction should be estimated. Raw support EEGs do not reflect this transformation, while support queries calibrated using their own pairs contain information unavailable for an unseen query.

This second issue can be made explicit algebraically. Let A=(G+ρI)−1(V−B)A=(G+\rho I)^{-1}(V-B). For the setting γ=1\gamma=1, the unnormalized full-support predictions satisfy

B+GA=V−ρA.B+GA=V-\rho A.

For nonsingular GG, these predictions approach the paired image embeddings as ρ→0\rho\to0. Thus, full-support references can reflect in-sample reconstruction rather than the behavior of independently calibrated EEG queries. PairCal addresses this by constructing each reference with its own pair excluded and using these references for candidate calibration. This provides a way to reuse scarce paired support for both operations while removing direct self-pair influence. The resulting reference bank remains an approximation, since each reference uses K−1K-1 pairs whereas a test query uses KK.

Our existing experiments provide evidence for this design. In Table 8, at K=200K=200, the correlation with the offline test-query oracle attraction statistic is 0.132 for raw support, 0.462 for full-support calibration, and 0.604 for LOO references; the same ordering holds at K=50K=50 and 100100. The oracle is used exclusively for diagnosis. Table 2 further shows that adding MDC to SGC increases Top-1 accuracy from 33.44% to 37.90%, while Table 4 shows that false-positive concentration H10H_{10} decreases from 0.358 to 0.312.

These results support the value of constructing candidate-calibration references that account for query adaptation while excluding each reference’s own paired target. We will foreground this methodological motivation, bring the reference-bank comparison into the main discussion, and explicitly distinguish the established KRR and normalization operations from the paired-support construction and its empirical validation.

AuthorsOpenReview ↗

Official Comment by Authors

W2 / Q2. Retrieval comparison of Raw, Full, and LOO reference banks

We evaluated three reference banks with the same SGC query scores, support samples, checkpoints, and candidate galleries across 450 matched evaluation settings. Raw uses the original support EEG embeddings. Full calibrates each support EEG using all support pairs, including its own pair. LOO excludes that pair when calibrating each reference EEG. Hyperparameters were selected using source-domain validation episodes.

KKSGCSGC + NNN (Raw)SGC + NNN (Full)PairCal (LOO)
1024.99 / 53.7425.60 / 55.2225.70 / 55.0225.98 / 55.62
2026.79 / 56.7327.72 / 58.2528.05 / 58.1328.46 / 59.24
5029.48 / 61.2930.50 / 62.1331.27 / 63.2531.59 / 63.44
10031.78 / 63.0431.93 / 63.3134.61 / 65.8635.08 / 66.50
20033.44 / 65.5533.86 / 65.7036.93 / 68.7837.90 / 69.81

Entries report Top-1 / Top-5 accuracy (%). PairCal has higher point estimates than both reference-bank baselines at all five support sizes. In a separate ablation, we held the SGC queries, scoring rule, neighborhood size, and penalty weight fixed and changed only the reference bank. At K=200K=200, replacing Full with LOO increased mean Top-1 accuracy by 1.23 percentage points, with improvements in all 10 subjects.

We will add these reference-bank comparisons to the revised manuscript and cite Chowdhury et al. [1] when introducing Nearest Neighbor Normalization (NNN).

We also computed subject-level paired improvements of PairCal over SGC. At each support size, we compared the two methods using the same subject, checkpoint, support-set seed, and test gallery. We averaged the nine paired differences within each subject (three checkpoints and three support-set seeds). We estimated 95% confidence intervals for the overall mean improvements by bootstrapping the paired differences at the subject level. Checkpoints and support-set seeds were treated as within-subject repetitions. We will report the mean improvement for each of the 10 subjects in the revised manuscript.

At K=10K=10, PairCal improved mean Top-1 accuracy by 0.99 percentage points (95% CI [0.47, 1.48]), with gains in 8 of 10 subjects. Mean Top-5 accuracy improved by 1.88 percentage points (95% CI [1.14, 2.64]), with gains in all 10 subjects. The intervals in the table quantify uncertainty in the final retrieval gains.

KKΔTop-1 (pp) [95% CI]ΔTop-5 (pp) [95% CI]Subjects with higher Top-1Subjects with higher Top-5
10+0.99 [0.47, 1.48]+1.88 [1.14, 2.64]8/1010/10
20+1.67 [1.04, 2.33]+2.51 [1.51, 3.54]10/1010/10
50+2.11 [1.42, 2.86]+2.14 [1.75, 2.59]10/1010/10
100+3.31 [2.53, 4.27]+3.46 [2.83, 4.14]10/1010/10
200+4.46 [3.42, 5.57]+4.26 [3.80, 4.69]10/1010/10

W3 / Q3. Offline oracle and source-validation episodes

Offline oracle. We apologize for the ambiguity. For each candidate image gⱼ, we compute its cosine similarities to all 200 SGC-calibrated, L2-normalized test EEG queries. We exclude its ground-truth paired query and define the offline oracle attraction statistic as the mean of the largest k_{eff}=min(k,199) remaining similarities. Here, k is the MDC neighborhood size fixed for that run, not the support budget K. Table 8 reports the Spearman correlation across the 200 candidates between this statistic and each support-derived attraction estimate. Ground-truth correspondence is used only to exclude the positive pair; consequently, this is an offline diagnostic and is never used for prediction or hyperparameter selection.

AuthorsOpenReview ↗

Official Comment by Authors

W3 / Q4. MDC coefficient and SATTC Top-5 result

MDC coefficient. The experiments used λMDC=0.5\lambda_{\mathrm{MDC}} = 0.5. The value of 0.55 in the original Section 4.1 was a typographical error and has been corrected. Following Appendix A.2, we fixed the selected SGC configuration and used source-validation scores to select one penalty weight shared across the five support budgets and a separate neighborhood size kk for each budget. This procedure selected λMDC=0.5\lambda_{\mathrm{MDC}} = 0.5 in every outer fold.

SATTC result. The correct SATTC Top-5 accuracy at K=200K=200 is 49.87%, as reported in Appendix A.6. We will correct the corresponding entry in Table 1 from 50.91% to 49.87%.

W4. Generalization evidence

Dataset Generalization. Our evaluation follows prior EEG/MEG decoding work (e.g., NICE [2], ATM [3], CognitionCapturer [4], UBP [5]), which also primarily use THINGS-EEG and THINGS-MEG [6, 7] as standard benchmarks with disjoint train/test concepts and a well-defined 200-way zero-shot retrieval protocol, enabling fair comparison across methods. Importantly, the Fusion Prior is pretrained on ImageNet-1k (∼1.3M images), so the visual module already sees much more diverse imagery than THINGS, which mitigates overfitting to a single stimulus set. While other datasets such as EEGCVPR40 [8, 9] exist, recent work has pointed out potential confounds from block designs and temporal autocorrelations that can inflate decoding performance [10, 11], so we deliberately focus on THINGS-style paradigms that better control for these issues. We agree that evaluating on more naturalistic datasets would further strengthen the generalization claim; if the schedule allows, we will add an additional benchmark in the revision, and otherwise regard this as an important direction for follow-up work.

Minor concern. Readability of Figures 2 and 5

Response. Thank you for your valuable feedback. We acknowledge that the visual presentation of Figures 2 and 5 can be further improved. In the revised version, we will optimize the layout and increase the label sizes in Figure 2 to enhance readability. For Figure 5, we will enlarge the qualitative examples and improve their visual presentation to facilitate closer inspection. These refinements will make both figures clearer and more accessible.

References

  1. Chowdhury, N., Wang, F., Shenoy, S., Kiela, D., Schwettmann, S., and Thrush, T. Nearest Neighbor Normalization Improves Multimodal Retrieval. EMNLP, 2024.
  2. Song, Yonghao, et al. "Decoding Natural Images from EEG for Object Recognition." The Twelfth International Conference on Learning Representations.
  3. Li, Dongyang, et al. "Visual decoding and reconstruction via EEG embeddings with guided diffusion." Proceedings of the 38th International Conference on Neural Information Processing Systems. 2024.
  4. Zhang, Kaifan, et al. "CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 13. 2025.
  5. Wu, Haitao, et al. "Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
  6. Gifford, Alessandro T., et al. "A large and rich EEG dataset for modeling human visual object recognition." NeuroImage 264 (2022): 119754.
  7. Hebart, Martin N., et al. "THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior." Elife 12 (2023): e82580.
  8. Palazzo, Simone, et al. "Decoding brain representations by multimodal learning of neural activity and visual features." IEEE Transactions on Pattern Analysis and Machine Intelligence 43.11 (2020): 3833-3849.
  9. Spampinato, Concetto, et al. "Deep learning human mind for automated visual classification." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  10. Xu, Xiran, et al. "Beware of overestimated decoding performance arising from temporal autocorrelations in electroencephalogram signals." arXiv preprint arXiv:2405.17024 (2024).
  11. Li, Ren, et al. "The perils and pitfalls of block design for EEG classification experiments." IEEE Transactions on Pattern Analysis and Machine Intelligence 43.1 (2020): 316-333.
Reviewer_Mi42OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PairCal calibrates frozen EEG–image retrieval using paired target-subject support. SGC predicts query residuals, while MDC corrects candidate attraction using leave-one-out support references.

Critical strengths

The inductive protocol is carefully defined. Gains over the frozen model and Procrustes are substantial, and the repeated checkpoints, support draws, and candidate-permutation controls strengthen the evidence.

Critical weaknesses
  1. MDC lacks a direct comparison with an established candidate correction under the same support resources. SATTC is explicitly transductive, while the paired-support baselines mainly adapt queries. The component and permutation controls show that MDC helps, but leave its advantage over existing candidate normalization unclear. Comparing SGC+MDC with SGC plus one suitable correction would address the main contribution question.

  2. The existing repetitions should support paired subject-level intervals for the main retrieval differences. These matter especially for MDC's 0.99-point Top-1 increment at K=10K=10; the reported mechanism intervals answer a different question.

  3. Table 2 needs to define the MDC-only reference bank and query representation. Removing SGC can change both.

  4. The “universal distractors” interpretation is stronger than the within-subject evidence. The later description of a support-derived attraction proxy is more appropriate.

Questions for authors

How does SGC+MDC compare with a closely related candidate correction using the same support data and source-only tuning?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors

Response to Reviewer Mi42

We thank the insightful review and would like to address the issue by points:

W1 / Q1. Comparison with established candidate correction

W1. MDC lacks a direct comparison with an established candidate correction under the same support resources. SATTC is explicitly transductive, while the paired-support baselines mainly adapt queries. The component and permutation controls show that MDC helps, but leave its advantage over existing candidate normalization unclear. Comparing SGC+MDC with SGC plus one suitable correction would address the main contribution question.

Q1. How does SGC+MDC compare with a closely related candidate correction using the same support data and source-only tuning?

Response. We added two Nearest Neighbor Normalization (NNN) baselines using the same SGC query embeddings as PairCal. NNN (Raw) uses the original support EEG embeddings as its reference bank, while NNN (Full) calibrates each reference EEG embedding using all support pairs, including its own pair. Across 450 matched evaluation settings, all three methods used the same support samples, checkpoints, and candidate galleries. Hyperparameters were selected using source-domain validation episodes.

KKSGC + NNN (Raw)SGC + NNN (Full)PairCal (SGC + MDC)
1025.60 / 55.22 / 0.397225.70 / 55.02 / 0.396325.98 / 55.62 / 0.3999
2027.72 / 58.25 / 0.420828.05 / 58.13 / 0.421828.46 / 59.24 / 0.4290
5030.50 / 62.13 / 0.451531.27 / 63.25 / 0.460731.59 / 63.44 / 0.4637
10031.93 / 63.31 / 0.465434.61 / 65.86 / 0.490135.08 / 66.50 / 0.4947
20033.86 / 65.70 / 0.484536.93 / 68.78 / 0.514437.90 / 69.81 / 0.5236

Each entry reports Top-1 (%), Top-5 (%), and MRR, in that order. PairCal yields higher point estimates than both NNN baselines on all three metrics across the five support budgets.

MDC and NNN use the same form of candidate correction, subtracting a scaled mean of the top-kk reference similarities. To isolate the effect of reference-bank construction, we also conducted a separate ablation that held the SGC queries, scoring rule, neighborhood size, and penalty weight fixed while replacing the full-support reference bank with the leave-one-out (LOO) reference bank.

We will include these comparisons in the revised manuscript.

W2. Paired subject-level confidence intervals for retrieval gains

The existing repetitions should support paired subject-level intervals for the main retrieval differences. These matter especially for MDC's 0.99-point Top-1 increment at K=10K=10; the reported mechanism intervals answer a different question.

Response. We used the existing repetitions to compute paired subject-level 95% confidence intervals for the retrieval gains from adding MDC to SGC. We paired the two methods under the same checkpoints and support draws, averaged the differences within each subject, and computed the intervals across subjects.

At K=10K=10, adding MDC improved mean Top-1 accuracy by 0.99 percentage points (95% CI [0.47, 1.48]), with improvements in 8 of 10 subjects. This interval supports a positive mean gain even at the smallest support budget.

KKΔTop-1 (pp) [95% CI]ΔTop-5 (pp) [95% CI]ΔMRR [95% CI]Subjects with higher Top-1Subjects with higher Top-5
10+0.99 [0.47, 1.48]+1.88 [1.14, 2.64]+0.0125 [0.0079, 0.0171]8/1010/10
20+1.67 [1.04, 2.33]+2.51 [1.51, 3.54]+0.0194 [0.0134, 0.0253]10/1010/10
50+2.11 [1.42, 2.86]+2.14 [1.75, 2.59]+0.0213 [0.0169, 0.0262]10/1010/10
100+3.31 [2.53, 4.27]+3.46 [2.83, 4.14]+0.0312 [0.0249, 0.0384]10/1010/10
200+4.46 [3.42, 5.57]+4.26 [3.80, 4.69]+0.0419 [0.0348, 0.0497]10/1010/10

All differences are SGC + MDC minus SGC. Top-1 and Top-5 differences are in percentage points (pp), and MRR differences are on the original [0, 1] scale. We will add these intervals to the retrieval results and report them separately from the mechanism analysis.

AuthorsOpenReview ↗

Official Comment by Authors

W3. Query representation and reference bank in the MDC-only ablation

Table 2 needs to define the MDC-only reference bank and query representation. Removing SGC can change both.

Response. In the submitted Table 2, the MDC-only row uses frozen test EEG embeddings before SGC calibration, while its reference bank contains support EEG embeddings calibrated by leave-one-out SGC. This setting removes SGC from the test-query path and retains it in the reference-bank construction.

We also evaluated an ablation that removes SGC from both paths. It uses frozen test EEG embeddings as queries and raw support EEG embeddings as the reference bank. MDC estimates candidate attraction directly from the similarities between these raw support EEG embeddings and the gallery images, then applies the candidate penalty to the frozen query–image scores.

KKTop-1 (%)Top-5 (%)MRR
1024.8453.410.3837
2025.9654.560.3958
5026.2255.160.3994
10026.2854.910.3991
20026.1854.910.3983

The new ablation covers 450 runs across 10 subjects, 3 checkpoints, 5 support budgets, and 3 support draws. In the revised Table 2, we will explicitly distinguish MDC with a frozen query and LOO-SGC bank from MDC with a frozen query and raw support bank.

W4. Interpretation of candidate attraction

Response. We agree that “universal distractors” overstates the within-subject evidence. In the revised manuscript, we will describe these candidates as images that repeatedly attract queries within a target subject and refer to the MDC score as a support-derived proxy for candidate attraction.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer,

We hope everything is going well for you. We are truly grateful for your thoughtful review and constructive comments on our work. We sincerely hope that our responses have helped address your concerns and would appreciate hearing whether any points still require clarification. If you have any further questions or suggestions, we would be delighted to discuss them with you. Thank you again for your valuable feedback and for the time and effort you have devoted to evaluating our manuscript.

The authors addressed my main concerns with the added NNN baselines, confidence intervals, and Table 2 clarifications, so I raise my rating to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.