- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks how much the adaptation head matters when a pretrained EEG foundation model is used for event detection. SPAN maps backbone tokens to physical time and decodes a set of intervals. It uses a multiscale memory, anchor-conditioned queries, a duration-scaled Gaussian attention bias, denoising queries and an auxiliary dense head. The baseline is dense per-token fine-tuning (DENSE-FT: a two-layer MLP per token plus tuned post-processing). Pairs share checkpoint, data, adapter and scorer. The study covers six backbones, four tasks (seizures, motor imagery, artifacts, arousals) and three seeds. SPAN wins all 72 pairs on event F1 at IoU>0.5, by 4.5-8.3 points per task (Table 1). It adds a DANCE-style plain set decoder (Table 2), a component ablation (Table 3) and boundary and fragmentation analyses (Section 5.2).
Critical strengths
-
The paired design is careful, and I enjoyed reading it. Each pair shares checkpoint, data, adapter, training caps and scorer (L062-064, L730-731), and all 72 differences stay positive when the dense checkpoint is picked by event F1 instead (Table 15). A useful warning for per-second evaluation: the dense head has higher per-second AUPRC in 21 of 24 backbone-task means, but finds fewer events (L352-354, Table 10).
-
The paper shows where SPAN fails and why. On TUAR, onset error rises for every backbone and fragmentation reaches 70.5-77.8% (Tables 6-7). Appendix D traces this to duplicates (best-aligned candidate 0.43 vs. 0.46 s, Table 13), and Appendix H tests suppression. L482-484 also say frozen encoders and random initialisation were not tested.
Critical weaknesses
-
Dense baseline without temporal context. Because of this, the comparison doesn't separate out set prediction. Each backbone encodes 10-s windows that are joined in time (L122-123), and DENSE-FT applies a two-layer MLP to each token (L130). Apart from post-hoc smoothing and merging, nothing in it works across windows. SPAN's decoder has four Transformer layers of width 256 that attend over the whole 60-s or 300-s chunk (L199-203, L747, Table 5). So the gain in Table 1 mixes two changes. One is predicting intervals as a set. The other is a learned temporal head with chunk-level context and more parameters. This matters most when events are longer than a window. TUSZ references have a median of 34.9 s (Table 11), and on TUSZ the dense output has a median signed onset error of +7.50 s (L895-896). The paper frames the contrast as set prediction versus "per-window classifiers" (abstract L014-016). But dense segmentation with temporal context, like the U-Time family it cites (L094), is also a standard way to decode events. A dense baseline with the same multiscale memory and a Transformer of similar size, with per-token outputs and the same post-processing, would separate the two. Head parameter counts for both methods would help too. I could only find the 0.2 M of the pyramid (L1078).
-
Attribution. The claims go further than the controls. The abstract says the gains "come from SPAN's combined design rather than set prediction alone" (L026-027) and that the ablation "traces the gain to duration-dependent attention locality and query denoising" (L027-028). The plain set decoder is the authors' own DANCE-style implementation (L971-972). It has half the width and half the heads (128-d and 4 heads vs. 256-d and 8, Table 17). On arousal it was also trained under a shorter cap than the primary pairs (L1024-1025). There it gets 5.7 and 3.9 F1, far below even DENSE-FT (24.9 and 24.0, Table 2). This looks like a training failure, not a limit of set prediction, so the +24.4 gain on that task says little. The ablation (Table 3) tests only two tasks, and on arousal every interval includes zero. Nothing in Table 3 explains the 5.5-point arousal gain. The anchor-conditioned queries, the iterative refinement and the auxiliary dense head are not ablated. The last one gives SPAN the same dense supervision DENSE-FT gets. I'd ask for a width- and budget-matched plain decoder, an ablation of the auxiliary dense head, and an abstract limited to what Tables 2-3 show.
A few smaller points. The training caps don't match. Appendix B gives 12 h for TUSZ and arousal (L730), but Appendix F refers to "the 23 h cap of the primary LaBraM and REVE pairs" (L1025). On TUAR, SPAN's precision falls from 38.8% to 25.1%, and false positives go up from 2,238 to 8,293 per 24 h (L822-825). I couldn't find a false-alarm rate for TUSZ, which seizure-detection work usually reports. The text says Table 8 adds a false-positive rate, but the table doesn't show one (L808-809). So "a strong, broadly validated default" (L032-033) is stronger than the TUAR results support. Typos: "We transfers" (L106), "sill" (L431).
Questions for authors
- How does SPAN compare on TUSZ and arousal with a dense head that has chunk-level temporal context and a size similar to SPAN's decoder? What are the head parameter counts of DENSE-FT, the plain set decoder and SPAN?
- What happens to SPAN without the auxiliary dense head?
- Which training cap did the primary arousal pairs use, 12 h or 23 h? With the same cap and width as SPAN, is the plain set decoder on arousal still below DENSE-FT?
- What are the false alarms per 24 h on TUSZ for both methods at the selected operating points?
2 comments
PART 2/2
Q3: Training caps and the matched plain set decoder
Response: Dense-FT and SPAN share the same cap in all 72 primary pairs. On arousal task, Appendix F was correct, Appendix B misstated the caps, and we have corrected it (Initial runs were scheduled with a conservative 23 h wall-clock limit, this was reduced to 12 h for left runs after observing consistent early stopping within 12 h). Based on the log, table below reports completed Dense-FT and SPAN runs on arousal for LaBraM and REVE:
| Runs | Cap | Stopped by | Optimizer steps | Training time |
|---|---|---|---|---|
| Dense-FT and SPAN (12) | 23 h | early stopping, 12/12 | 123k--232k | all within 12 h (max 11.4 h) |
| Component ablation (18) | 23 h | early stopping, 18/18 | 137k--260k | 17/18 within 12 h, except REVE denoising seed 7: 12.7 h |
| Plain set decoder (6) | 12 h | early stopping, 6/6 | 109k--410k | all within 12 h (max 10.8 h) |
All 12 Dense-FT and SPAN runs reached within 12 h, after 123--232k optimizer steps. 17/18 ablation runs early stopped within 12 h, after 137--260k optimizer steps, only REVE denoising seed 7 early-stopped at 12.7 h.
To discover the gap between SPAN and plain set decoder, we retrained the plain set decoder at SPAN's width (256-d, 8 heads) and the same training caps, for LaBraM and REVE with three seeds each. Table below reports event F1 at IoU 0.5:
| Task | Dense-FT | Plain set (paper) | Plain set, matched width and cap | SPAN | SPAN matched plain [95% CI] |
|---|---|---|---|---|---|
| TUSZ | 45.2 | 49.1 | 50.4 | 55.7 | +5.3 [+1.4, +9.0], 5/6 pairs |
| MI | 23.2 | 16.7 | 19.2 | 27.1 | +7.9 [+5.6, +9.7], 6/6 pairs |
| TUAR | 13.5 | 10.4 | 10.2 | 18.9 | +8.7 [+8.1, +9.4], 6/6 pairs |
| Arousal | 24.5 | 4.8 | 4.7 | 29.2 | +24.6 [+22.0, +26.8], 6/6 pairs |
With SPAN's width and cap, the plain set decoder is still far below Dense-FT on arousal. From the results, matching width and budget changes the plain decoder by at most 2.5 points, so neither explains SPAN's advantage. On arousal, where the gap is largest, when we also replace DANCE's matching loss with SPAN's one in both matching and training, the SPAN remains 2.6 points higher [+0.0, +4.8], 5/6 pairs. Because pure IoU loss from DANCE has zero gradient when predicted and reference intervals do not overlap, the plain set decoder fails to localize short, sparse arousals. Equipping the plain decoder with SPAN's L1 + dIoU objective resolves this optimization bottleneck.
Q4: False alarms on TUSZ
Response: We have updated Table 8 to include false positives per 24 h at both overlap thresholds across all tasks. On TUSZ, at the default operating points, both methods produce comparable false-alarm rates:
| TUSZ, false positives per 24 h | Dense-FT | SPAN |
|---|---|---|
| IoU 0.3 | 39 | 45 |
| IoU 0.5 | 47 | 49 |
Because the default operating point maximizes validation event F1 rather than targeting a clinical false-alarm burden, we also reselected the threshold of every run under fixed false-alarm limits (Section 5.2; Appendix E, Table 18 and Figure 4). Under FP per hour on TUSZ, SPAN achieves 31.8% recall and 42.0% event F1 at compared to 15.1% recall and 21.2% event F1 for Dense-FT. On TUAR, held to Dense-FT's own FP rate, SPAN is 1.0 point lower there. From practitioner's perspective, SPAN is the better choice for events that rarely overlap within a class like seizures, motor imagery and arousals, while Dense-FT is the safer choice for densely overlapping events like artifacts under a tight false-alarm restriction.
We hope these new results and revisions fully address your concerns. Please let us know if any further clarification would be helpful during the discussion period!
PART 1/2
Dear Reviewer eien,
Thank you for your thorough review, recognizing both our paired design and our diagnostic analyses of duplicate predictions on TUAR, and we really appreciate that you enjoy reading it.
We have uploaded a revised manuscript (changes highlighted in blue) and run additional experiments to address your concerns directly below. In brief:
W1 / Q1: Dense baseline with temporal context
Response: To separate the effect of chunk-level temporal context from set prediction, we implemented Dense-Ctx. Dense-Ctx inserts SPAN's multiscale pyramid and four Transformer decoder layers (width 256, 8 heads, feed-forward 512) between the adapter and Dense-FT's per-token MLP. Each token attends to the same chunk as SPAN and to the multiscale memory. Targets, loss, checkpoint selection, training caps and the post-processing search are Dense-FT's. We also evaluated Dense-Ctx (threshold only), which sweeps only the confidence threshold like SPAN, without tuning smoothing windows, merge gaps, or minimum durations. Due to limited computation resources, we report event F1 at IoU > 0.5 for LaBraM and REVE with one seed 37 below:
Adding chunk-level multiscale temporal context improves Dense-FT, while SPAN remains ahead in all four comparisons (+0.9 to +2.0 points). Moreover, without post-processing, Dense-Ctx falls 14.4 points below SPAN on TUSZ, because SPAN outputs intervals directly while a dense head depends on the smoothing, merge-gap and minimum-duration search to convert probabilities into events.
Below are head parameters, excluding the adapter projection that all heads share (0.05--0.53 M by backbone).
Dense-Ctx (3.50 M) closely matches SPAN (3.60--3.66 M), and the width-matched 256-d plain set decoder (4.36 M) is slightly larger than SPAN.
W2 / Q2: Attribution and the auxiliary dense head
Response: We agree and have tightened the abstract to reflect what Tables 2 and 3 show.
To evaluate the auxiliary dense head, we trained SPAN without the auxiliary dense head on LaBraM and REVE across all four tasks on one seed 37 due to limited computation resources. The dense head reads the adapter tokens and feeds nothing into the decoder, so this changes only the training. Table below reports Event F1 at IoU > 0.5:
We use the auxiliary dense head to strengthen the features the decoder reads. One-to-one matching supervises only the few matched queries, while the dense head gives every adapter token a per-second target, these tokens form the decoder's memory and condition each query's initial content. From the results above, the auxiliary dense head is beneficial on TUSZ and MI, while having a neutral or mixed effect on TUAR and arousal, where set-level supervision alone is already dense. Moreover, even without the auxiliary dense head, SPAN outperforms Dense-FT in 7/8 comparisons.