- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies long-context EEG representation learning, focusing on how to retain information useful for downstream tasks amid extensive background activity. The authors propose SPARK, which combines multiresolution time–frequency features, gated recurrent memory, and importance-weighted compression and replay. The model first learns to accumulate information across chunks through self-supervised pretraining, then adapts its representations and controllers using a supervised utility objective. Experiments cover three recording-level classification datasets and short-window tasks involving clinical EEG, sleep, affect, and motor imagery.
Critical strengths
- The problem is practically meaningful, and the method is designed around it. Important information in long EEG recordings often appears in only a small number of segments. Retaining that information in a compressed representation is a worthwhile problem. The paper separates information accumulation across chunks from subsequent utility adaptation. The overall design has a clear motivation and goes beyond simply adding a classification module to an existing encoder.
- The controlled comparisons are valuable. Under the same pretraining data and update budget, Table 17 compares Stage 1 with Dense-Attn-120, which has a similar parameter count, and reports an 11.48-percentage-point improvement in CHB-MIT recording-level BAcc. Table 20 further introduces a CE control with shared auxiliary objectives, using the same adaptation data and update budget; full SPARK still performs 1.57 percentage points better. These comparisons help examine the contributions of recurrent memory and utility adaptation, respectively. Provided that the reported numbers can be verified, I consider them important evidence supporting the paper’s contributions.
Critical weaknesses
-
I cannot currently reconcile the reported CHB-MIT scores, which substantially affects my assessment. Section G.3 on page 31 reports 71 validation recordings from chb21–22 and 52 test recordings from chb23–24. However, the official file manifest contains only 64 and 31 original EDF files for these groups, respectively. There may be a reasonable explanation for the larger counts. For example, splitting the test EDF files into one-hour segments and retaining the final partial segments does yield 52 sub-recordings. However, I could not find an explanation of such processing in the paper. More importantly, the evaluation unit determines which metric values are attainable. As a simple example, ordinary accuracy evaluated on 10 samples can only take values of 0%, 10%, 20%, and so on. Although balanced accuracy is calculated differently, its possible values are also constrained by the numbers of positive and negative samples. If all original EDF files are retained, each EDF is treated as one evaluation sample, and any file containing a seizure annotation is labeled positive, the official manifest and annotations give positive/negative counts of 3/6 for chb23 and 12/10 for chb24. Following Equation (27), computing BAcc separately for each subject and then averaging equally across the two subjects gives: BAcc (%) = (5/12) × (20a + 10b + 5c + 6d). Here, a and b are the numbers of correctly classified positive and negative recordings for chb23, while c and d are the corresponding counts for chb24. All four must be integers within their valid ranges. The final score must therefore be an integer multiple of 5/12 (approximately 0.4167) percentage points, although not every multiple is attainable. For example, the two grid points surrounding 93.10% are 92.9167% and 93.3333%; neither rounds to 93.10% at two decimal places. I would like the authors to clarify the actual recording partition and labeling rules, and provide the confusion matrices for each test subject under all three seeds, so that readers can reproduce these scores using Equation (27).
-
The current experiments more directly support the usefulness of long-context pretraining than memory retention across complete long recordings. The model is pretrained on 120-second inputs, but downstream CHB-MIT evaluation uses independent 10-second windows, resets memory between windows, and averages the predicted probabilities to obtain recording-level scores. Thus, during evaluation, the model does not continuously update its memory throughout an entire long recording. This does not diminish the potential value of long-context pretraining, but two capabilities should be distinguished: learning better short-window representations from longer pretraining inputs, and retaining earlier important events through extended periods of background activity.
Questions for authors
What exactly does “recording” mean in Section G.3? Please explain how the 71 validation recordings and 52 test recordings are derived from the original EDF files, and provide the mapping from source EDF files to evaluation recordings, together with the recording-level labeling rules. This would help determine whether the discrepancy in recording counts simply reflects undocumented preprocessing. Could you verify the three CHB-MIT BAcc values for full SPARK in Table 19? Please provide TP, TN, FP, and FN for each of the two test subjects under each seed, along with the validation-selected thresholds. If the metric is not calculated by computing BAcc from hard predictions separately for each subject and then averaging equally across subjects, please explain how the implementation differs from Equation (27). This clarification is the most likely to change my current assessment. How were the 51,695 ten-second Siena windows generated? The EDF headers of the 41 official files indicate approximately 141.02 hours of data in total. Extracting complete, non-overlapping ten-second windows separately from each file yields 50,749 windows. Please clarify the window stride, handling of incomplete final windows, and whether repeated sampling was used. Which long-context capability is directly supported by the current experiments? Downstream evaluation uses independent ten-second windows and resets memory between them. An improvement in recording-level scores therefore does not, by itself, demonstrate retention of earlier important events throughout a complete long recording. Do any existing controls or analyses provide more direct evidence of this retention capability? If not, could you confirm that the current experiments primarily establish the transfer value of long-context pretraining? Do the CHB-MIT gains hold for both test subjects? Please report BAcc separately for chb23 and chb24 for Stage 1, the CE control, and full SPARK under the three existing seeds. I would like to understand whether full SPARK improves over both controls on both subjects, or whether the average gain is mainly contributed by one subject. This would affect my assessment of the robustness of the results.