- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This study proposed HarmTSFormer for cross-subject SSVEP decoding at unseen stimulus frequencies. The model propose a Dual-Branch Harmonic Encoder (DBHE) to encode candidate harmonic evidence with shared parameters and an Evidence-Preserving Aggregation Module (EPAA) to fuse learned and reference-matching scores, and was evaluated on three public datasets without target-user calibration. Although extensive evaluations and ablations have been performed, the study lacks convincing evidence that the modest gains over simpler baselines justify the added architectural and training complexity.
Critical strengths
The authors combine explicit reference matching with learned representations for unseen-frequency decoding and evaluate the method across three datasets through a number of comparative experiments, ablations, and sensitivity analyses.
Critical weaknesses
The paper is difficult to follow, particularly regarding the motivation, technical details, scientific contribution, and justification for the necessity of its components.
Notably, the method relies heavily on handcrafted features and hyperparameters but lacks sufficient EEG signal-processing rationale and theoretical justification for their design and combination.
Questions for authors
Although the paper evaluates alternative frequency splits, broader testing is needed, including repeated random frequency splits with results averaged across repetitions.
More informative visualizations and interpretation analyses are needed to clarify what the model learns and substantiate the role of each component.
Compared with the candidate-conditioned Transformer and CNN baselines, the proposed architecture provides only modest accuracy improvements, whose statistical significance remains unclear, leaving insufficient evidence that these gains justify the added complexity.
The practical relevance remains unclear. Given the focus on cross-subject and cross-frequency generalization, evaluating transfer between datasets with different stimulus frequencies is needed to address the concern that the method may be overfitting to the dataset due to complexity.
The paper should provide more complete preprocessing details
5 comments
Response to Reviewer 2SB9 (Part 2/2)
[Q4] Are the gains over the candidate-conditioned Transformer and CNN statistically significant, and how do they compare with the additional computational cost?
[A4]
Yes. HarmTSFormer gains 0.44–0.86 percentage points in mean accuracy over CCT and 0.39–1.96 over HCM. We used two-sided paired t-tests on participant accuracies and applied Holm correction across all 18 comparisons. All 18 remained significant (adjusted p ≤ 0.0063). Table 17(a) gives means and SDs across training runs. Table 17(b) gives paired differences and pointwise 95% confidence intervals. The following two comparisons are for Benchmark at 0.6 s.
| Comparator | Accuracy gain | 95% confidence interval | Holm-adjusted p |
|---|---|---|---|
| CCT | +0.86 pp | [0.40, 1.32] pp | 0.0063 |
| HCM | +1.96 pp | [0.92, 3.00] pp | 0.0063 |
Section 3.3 compares HarmTSFormer with CCT and HCM on accuracy and inference time (see Appendix M.3, Table 23, for the full comparison). We timed single-trial inference on the same host with matched inputs, 40 candidates and batch size one. The table gives median latency including preprocessing and scoring. Training time is reported per final source fit.
| Method | Benchmark 0.6 s accuracy | Training time per final fit | Inference time per trial |
|---|---|---|---|
| HCM | 47.80% | 10.8 min | 23.6 ms |
| CCT | 48.90% | 19.2 min | 44.1 ms |
| HarmTSFormer | 49.76% | 38.4 min | 67.4 ms |
On this workload, HarmTSFormer takes 1.53 times the inference time of CCT and 2.86 times that of HCM. The corresponding training-time ratios are 2.00 and 3.56 per final source fit (Appendices H.3 and M.3).
[Q5] Can you evaluate transfer between datasets with different stimulus frequencies to assess practical relevance and generalization?
[A5]
Yes. We fit on Benchmark participants S1–S16 and evaluated the frozen decoder on Wearable wet-electrode participants S17–S54 using eight common channels. Wearable’s 12 target frequencies are distinct from Benchmark’s 40 frequencies and lie within the source range. We used only source data for fitting and selection, then scored all 12 target candidates (Section 3.3, Table L.2, and Appendices L.4–L.5).
| Method | 0.4 s accuracy (%) | 0.6 s accuracy (%) | 0.8 s accuracy (%) |
|---|---|---|---|
| FBCCA-H3 | 32.24 | 47.65 | 58.11 |
| FBCCA-H5 | 15.44 | 38.90 | 56.07 |
| FBCCA/tlCCA + MLP | 31.78 ± 0.33 | 46.54 ± 0.42 | 57.79 ± 0.33 |
| HarmTSFormer | 33.64 ± 0.38 | 47.93 ± 0.65 | 59.08 ± 0.40 |
-
The FBCCA-H3 and FBCCA-H5 entries reproduce their mean accuracies on Wearable from Table 2. For the two fitted models, we report mean accuracy and SD across three runs.
-
HarmTSFormer has the highest mean accuracy at all three windows. After Holm correction across nine comparisons, the gains were significant at 0.4 and 0.6 s against independent fusion and at all three windows against FBCCA-H5. Against FBCCA-H3, mean accuracy was higher by 1.40, 0.28, and 0.96 percentage points at 0.4, 0.6, and 0.8 s, respectively. None of these three differences reached significance after Holm correction (Table E.1). The experiment demonstrates offline reuse of the frozen source model across participants, recording conditions, and stimulus grids without target-user calibration.
[Q6] Please provide more complete preprocessing details.
[A6]
Section 3.2 and Appendix D.4 specify the shared preprocessing. We select channels from the released 250-Hz arrays and crop each trial. Benchmark and BETA use Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2. Wearable and Benchmark-to-Wearable transfer use POz, PO3, PO4, PO5, PO6, Oz, O1, and O2, in that order. Table D.1 gives the starts relative to the beginning of the stored array.
| Dataset | Start within the stored trial array | Zero-based start index |
|---|---|---|
| Benchmark | 0.64 s | 160 |
| BETA | 0.50 s | 125 |
| Wearable wet | 0.64 s | 160 |
The 0.4-, 0.6-, and 0.8-s windows contain 100, 150, and 200 samples. Each channel receives a 6–90 Hz Butterworth bandpass with design-order parameter 4 using second-order sections and forward–backward filtering. Endpoint handling uses odd extension of the cropped window. We then subtract each channel’s window mean and divide by its population SD plus . The first cropped sample defines the reference-time origin. Appendix A.2 describes the subsequent F2 subband processing.
Dear Reviewer,
We hope that our responses have thoroughly addressed your questions. As the discussion phase comes to a close, we eagerly await your feedback and would appreciate knowing if you have any remaining concerns we can address. We would be grateful if you find our revisions satisfactory and consider raising your evaluation of our paper.
Thank you once again for the time and effort you have dedicated to reviewing our manuscript.
Best regards,
The HarmTSFormer Authors
Response to Reviewer 2SB9 (Part 1/2)
[Q1] The motivation, technical details, scientific contribution, and necessity of the components are difficult to follow. What EEG signal-processing rationale supports the features and model design?
[A1]
Design and contribution. A candidate’s frequency and phase specify its harmonic reference. Separate encoders process H1–H3 and H4/H5. Each branch uses the mean representation of the candidates to update each one. EPAM combines the learned score with reference-matching strength and harmonic log power (Sections 2.2–2.3).
EEG features and parameter choices. Real and imaginary spectral responses retain phase relative to the reference, and log power summarizes amplitude. Single-harmonic matching measures both matching strength and spatial structure across channels. Subband features combine local responses with joint multiharmonic matching. Appendix A.2 explains the spectral offsets and subbands, Table 5 distinguishes fixed cue scales from learned weights and cutoffs, and Appendix D.3 describes source-only selection. With other cues fixed, increasing one reliability cue can only maintain or raise the H4/H5 trial gate (Equation 3). Removing only the F2 encoder tokens leaves the explicit F2 score and both residual paths in place. Accuracy at 0.6 s fell by 1.56 points on Benchmark, 0.92 on BETA, and 0.63 on Wearable (Table 22).
[Q2] Can the evaluation include repeated random frequency splits, with results averaged across repetitions?
[A2]
Yes. We ran five repetitions of randomized five-fold frequency holdouts across all three datasets and observation windows. Averaged over the five repetitions, HarmTSFormer outperformed independent fusion in all nine conditions and the designated comparator in eight. In each repetition we held out each of the 38 interior Benchmark/BETA frequencies or all 12 Wearable frequencies once. The Benchmark/BETA endpoints stayed in the fitting set. All methods in Table F.1 used the same splits and scored the full candidate set.
We fixed one prespecified training seed across repetitions. Each outer fold refit the trainable stages, scoring heads, source templates, and fitted statistics on the permitted source data. Within each repetition, we pooled the five holdouts per participant and averaged participants equally. Table F.1 reports means and sample SDs across repetitions, including the differences computed within each repetition.
The designated comparators are SRRNet for Benchmark 0.4 s and BETA 0.4/0.6 s, tlCCA-1 for Benchmark 0.6/0.8 s and BETA 0.8 s, FBCCA-H3 for Wearable 0.4/0.6 s, and LINEAR for Wearable 0.8 s.
The SDs quantify sensitivity to frequency partitioning under the fixed training seed. Tables F.2–F.4 give the individual repetitions, including the reversals on Wearable (Appendix F.6).
[Q3] Can you provide more informative visualizations and interpretation analyses to clarify what the model learns and the roles of its components?
[A3]
Figure 3(a) compares the low-order and combined DBHE representation readouts. The combined readout has higher accuracy in all nine conditions. Figure 3(b) scans perturbation frequency for targets at 9, 11, and 13 Hz. Its vertical axis gives the original minus perturbed correct-target score margin. Low-order margin drops concentrate near H1–H3, and the high-order readout also responds near H4/H5 (Section 3.5 and Appendix K).
Table O.1 reports a separate test of accuracy. We subtract one target harmonic at a time by fitting a sine–cosine pair to each EEG channel and subtracting the fitted component. We then recompute the scores with the model weights fixed. Removing H3 on Wearable at 0.6 s lowered full-model accuracy by 13.19 percentage points.
Figure 4 on page 10 gives accuracy for each readout. Table O.2 gives the proportions of predictions corrected or made incorrect by fusion. On Benchmark at 0.6 s, corrections account for 8.80% of evaluation weight and newly introduced errors for 7.71% relative to the learned readout, giving a net gain of 1.09 points.