- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RPA is a small trainable adapter on the frozen patch tokens of OpenCLIP ViT-H/14 layer 11: a residual convolutional bottleneck followed by average pooling and a linear map, trained contrastively with the UBP EEGProjection encoder. It reports 95.4% within-subject and 35.5% leave-one-subject-out top-1 on THINGS-EEG2, and 65.2% and 6.7% on THINGS-MEG. Supporting experiments cover layer sweeps over seven visual encoders, token pooling and masking, linear attribute probes, other EEG encoders and a reconstruction pipeline.
Critical strengths
- The controlled comparison at one layer is clear and the effect is large: CLS token 69.5%, patch mean 77.0%, pointwise RPA 93.0%, local RPA 95.4%, and every other adapter in Table 9, several of them much larger, does worse. This is the result the method stands on.
- The layer sweep is useful beyond this paper. Every encoder peaks before its final layer for both EEG and MEG, and Table 14 shows that layer choice can flip encoder rankings, a caution for anyone comparing visual backbones here.
Critical weaknesses
- Configuration was selected on the test set. Appendix D states that layer 11 was chosen because it gave the highest test top-1 after a full depth sweep, the MEG projection size of 64 is the best of eight test-evaluated settings (Table 20), and the adapter and bottleneck comparisons are also test numbers. The 3% validation split only picks checkpoints. With a 200-image gallery and margins such as 6.7% against 5.4% for MEG cross-subject, the headline numbers are optimistic.
- The state-of-the-art table uses published numbers from pipelines with different channels, windows and protocols, so only Table 3 is controlled. There the CLS and patch-mean controls get a linear head while RPA gets a nonlinear one, so the claim that the CLS token adds little is not separated from adapter capacity. The text also needs a careful edit (typos, unnumbered appendix references).
Questions for authors
- Clarification: what do the headline numbers become if the layer, time window, adapter and MEG projection size are selected on a validation gallery drawn from training concepts?
- Experiment: a capacity-matched nonlinear head on the CLS token and on the patch mean.
- Clarification: please tabulate single-trial accuracy (Figure 5D) for RPA and the CLS baseline.
- Experiment: re-run the two strongest baselines inside your pipeline with the same channels and window.
4 comments
Follow-up on Our Rebuttal
Dear Reviewer K57u,
We sincerely appreciate your careful review and constructive suggestions. We hope our additional experiments and explanations have addressed your concerns, and we would welcome your feedback on our response. We would be glad to clarify any remaining points.
Regards, Authors
Response to Reviewer K57u
Q4. Matched baseline evaluation
We reran Shallow Alignment and BlurPerception using the same EEG channels, temporal windows, splits, repetition averaging, and validation-based checkpoint selection as RPA.
| Method | Within Top-1 (%) | LOSO Top-1 (%) |
|---|---|---|
| Shallow Alignment | 84.32 ± 0.38 | 23.32 ± 0.13 |
| BlurPerception | 76.13 ± 0.92 | 18.28 ± 0.48 |
| RPA | 95.50 ± 0.25 | 35.47 ± 0.33 |
RPA exceeds Shallow Alignment by 11.18/12.15 points within-subject/LOSO and also outperforms BlurPerception. Shallow Alignment uses the same CLIP-H visual backbone, while BlurPerception retains its original RN50 backbone. We could not locate a public HCF implementation for a comparable rerun.
We also quantified the channel-selection effect: using 17 posterior channels instead of all 63 improves RPA's within-subject/LOSO Top-1 from 87.58/32.53% to 95.50/35.47%. The baseline reruns and channel comparison appear in Appendix L (Tables 23–24).
Presentation
We corrected typos, notation inconsistencies, and appendix references throughout the revised manuscript.
Thank you again for your helpful comments. We hope the added experiments and clarifications address your main concerns, and we welcome any further questions.
Further Follow-Up on Our Rebuttal
Dear Reviewer K57u,
Thank you for your careful assessment and insightful suggestions. We have incorporated additional experiments and revised the manuscript to better address the issues you identified. We hope these updates provide a clearer picture of our contributions and would greatly value your perspective on the revised work.
We remain happy to provide further clarification or discuss any outstanding concerns.
Best regards,
The Authors
Response to Reviewer K57u
Thank you for recognizing the strength of our controlled adapter comparison and the usefulness of the visual-layer analysis. We appreciate your constructive suggestions. All additional experiments and clarifications have been integrated into the revised manuscript, with changes highlighted in blue.
Q1. Validation-based model selection
We clarified the original model-selection procedure and conducted expanded EEG and MEG evaluations, selecting configurations using validation rather than test accuracy.
We corrected the ambiguous wording in Appendices A and D. The original layer search evaluated a subset of candidate blocks, and the visual layer, temporal window, adapter configuration, and MEG projection width were selected using validation. For EEG, each run holds out 3% of training samples for validation. Checkpoints are selected by validation retrieval Top-1, and the expanded layer sweep selects the block with the highest mean validation Top-1.
EEG visual-layer selection. Appendix K (Table 22 and Figure 14) provides the validation-selected results and the corresponding validation/test accuracy curves across evaluated blocks.
Figure 14 shows that validation and test performance both favor intermediate visual layers, although the exact maxima may differ. In particular, CLIP-H under LOSO selects block 11 by validation, even though block 12 achieves a slightly higher test accuracy (36.12% versus 35.47%). We report the validation-selected block 11 result, rather than the test maximum.
MEG projection-width selection. We additionally evaluated projection widths from 32 to 160, together with a direct 271-sensor control. Appendix O (Table 28) reports the complete validation and test results. The comparison below includes the original width and the widths selected in the additional sweep.
The additional MEG sweep shows a broad validation plateau among evaluated widths K≥64. Validation selects K=160 within subject and K=96 under LOSO. The K=64 entries above come from this additional sweep and are distinct from the original K=64 benchmark results.
The original benchmarks remain reported separately (EEG: 95.4%/35.5%; MEG: 65.2%/6.7%, within-subject/LOSO). The additional validation-selected results retain comparable performance and support the intermediate-layer finding without selecting configurations by test accuracy.
Q2. Capacity-matched nonlinear controls
We added the requested nonlinear CLS and patch-mean baselines, together with a per-token MLP control and CLS fusion, under matched model capacity and training settings. The MLP controls match RPA's parameter count within 0.023%.
Applying the same MLP before pooling rather than after pooling improves Top-1 by 17.42/9.57 points within-subject/LOSO. RPA provides a further 3.60/1.02-point improvement over the per-token MLP.
Adding CLS to RPA produces no consistent mean improvement across the two regimes. These results strengthen the original controlled adapter comparison (Appendix M, Tables 25–26).
Q3. Single-trial retrieval
We added the requested single-trial results for RPA and the capacity-matched nonlinear CLS control, alongside their 80-repetition results.
RPA outperforms the CLS control by 10.04 points within subject and 1.58 points under LOSO on individual trials. Its advantage therefore persists without test-time repetition averaging, although both methods benefit substantially from averaging. Training remains unchanged (Appendix N, Table 27).