acceptodds
Under review as a conference paper at ICLR 2027

NeuroDETR: Multiscale Discrete Spatiotemporal Tokenization for Asynchronous EEG Event Detection

Abstract

Asynchronous EEG event detection supports practical brain-computer interfaces by identifying event classes and temporal boundaries directly from continuous recordings without predefined event boundaries at inference time. The primary challenge lies in extracting event-related patterns embedded in ongoing background activity. To address this, we propose NeuroDETR, a discrete spatiotemporal multiscale framework that combines structured tokenization with event-aware feature refinement. Multiscale temporal codebooks capture brief responses and sustained dynamics, while Spatial-Pyramid VQ hierarchically encodes spatial structure across EEG channels. These complementary representations form a compact event memory for jointly classifying events and estimating their onsets and offsets. Before tokenization, we apply FOCAL, an event-aware sparse attention module that uses predicted event presence to guide temporal feature refinement and spatial aggregation, emphasizing likely event periods while preserving context across the recording. Experiments on eight EEG datasets that span diverse paradigms demonstrate state-of-the-art performance. These findings provide a basis for the development of BCI systems that recognize meaningful neural activity during continuous interaction and monitoring.

Primary area:
applications to neuroscience & cognitive science
Keywords:
asynchronous EEG event detection, brain-computer interfaces, discrete representation learning, multiscale spatiotemporal tokenization, vector quantization, event-aware sparse attention

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gbiCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a multi-scale discrete spatiotemporal framework named NeuroDETR, designed to address the problem of asynchronous event detection in continuous EEG signals. Unlike traditional methods relying on sliding-window classification or fixed-segment analysis, this approach employs a DETR-like set prediction paradigm, enabling the joint prediction of event categories, onset times, and offset times directly from continuous, unsegmented EEG recordings.

Critical strengths
  1. This paper focuses on asynchronous event detection in continuous EEG signals, enabling the joint identification of event categories and the localization of temporal boundaries; compared to methods relying on fixed-window classification, this approach better aligns with the detection requirements of continuous brain-computer interface applications.
  2. The paper conducts experiments on eight datasets covering a variety of EEG tasks, as well as differing event durations and frequencies, thereby facilitating an assessment of the method's applicability from multiple perspectives.
  3. In addition to comparisons with various temporal detection methods and DANCE, the authors examined the roles of quantization mechanisms, time scales, and spatial hierarchies, and analyzed event localization performance across different temporal IoU thresholds, providing preliminary evidence to understand the model's performance advantages.
Critical weaknesses
  1. NeuroDETR draws upon various components—including VQ, hierarchical spatial encoding, sparse attention, and DETR—yet the paper has not fully demonstrated the unique technical advantages of this combination over existing EEG event detection methods.
  2. Temporal quantization scales, spatial hierarchical partitioning, event-density-based guidance, and dual-branch fusion all involve manually designed choices, yet there is a lack of in-depth analysis regarding the rationale and applicability limits of these designs.
  3. Although the paper is set against the backdrop of continuous EEG monitoring, it primarily employs a windowed input approach for training and evaluation; consequently, it lacks sufficient discussion regarding the handling of event boundaries across windows, real-time inference latency, and stability in online scenarios.
Questions for authors
  1. What is the core original contribution of this paper—the discrete spatiotemporal representation, the event-guided mechanism, or the collaborative learning mechanism between the two? Specifically, where do the performance gains come from when compared to existing query-based EEG detection methods such as DANCE?
  2. The paper addresses this issue by employing probability thresholds and a Top-K minimum selection strategy; however, there is currently a lack of systematic analysis regarding this mechanism in scenarios characterized by low event prevalence, class imbalance, and weak events. In particular, while the authors attribute some performance gains to FOCAL's ability to handle low-prevalence events, existing ablation studies—which primarily compare results between the full module and the module with specific components removed—have not yet directly demonstrated a consistent improvement in recall for weak events.
  3. The paper does not adequately clarify whether future signals need to be buffered during deployment, how duplicate detections between adjacent windows are handled, or whether event merging rules across windows introduce additional latency. If the model requires full computation over extended time windows, then—even if offline localization performance is strong—one cannot directly infer its suitability for strict real-time scenarios.

If the author addresses the questions raised above, I will raise my score.

5 comments

Answer (1/3)

Thank you for your thoughtful review.

W1, Q1:

1. Core Original Contribution: Synergistic Event-Guided Discrete Spatiotemporal Memory Our core contribution is not a modular assembly of sparse attention, VQ, and DETR, but a co-adapted, event-guided discrete spatiotemporal memory mechanism for asynchronous EEG—where event-presence routing (FOCAL), multiscale temporal and hierarchical spatial VQ, and set-level decoding mutually reinforce one another to drive performance gains.

Unlike existing EEG tokenizers (LaBraM, NeuroLM, DeWave) that uniformly quantize pre-segmented trials for generic reconstruction, continuous asynchronous EEG is dominated by non-stationary background activity. Here, isolated components yield limited benefits; the performance leap stems directly from cross-module synergy:

  • FOCAL unlocks VQ codebook capacity via foreground enrichment: Applied alone, EMA codebooks waste capacity modeling repetitive background noise, yielding minimal standalone VQ gains. By concentrating local temporal refinement and spatial pooling onto candidate event intervals before tokenization, FOCAL shields EMA updates from background dominance so codebooks specialize in discriminative foreground prototypes.
  • Spatiotemporal VQ regularizes routed features into a noise-blocking bottleneck: Conversely, routing alone over continuous features still leaks residual background variance linearly into decoder memory. Multiscale temporal and residual spatial VQ map FOCAL-enriched dynamics onto a compact, norm-bounded prototype vocabulary, filtering out subtle background fluctuations and regularizing representations across subjects and paradigms.
  • Synergistic memory empowers NMS-free set decoding: Together, foreground-enriched discretization supplies clean temporal boundaries and regional topography to the DETR bipartite decoder, enabling accurate localization across two orders of magnitude in event duration. Consequently, combining FOCAL and spatiotemporal VQ delivers strong super-additive gains far exceeding the sum of their standalone improvements, confirming that our gains arise from tight functional coupling rather than additive stacking.

2. Unique Advantages & Source of Gains over DANCE

  • Comparison with dense detectors. NeuroDETR predicts event classes and intervals directly from a fixed set of queries, without non-maximum suppression (NMS). We tuned the confidence and class-wise NMS thresholds for each of the four dense detectors on validation data. At a temporal IoU threshold of 0.5, NeuroDETR achieves higher test event F1 than the strongest tuned dense detector on each of the eight datasets.
  • Over query-based DANCE: NeuroDETR shares the exact same SimpleConv backbone, DETR decoder, and auxiliary losses (Ldense\mathcal{L}_{\mathrm{dense}}, Lcons\mathcal{L}_{\mathrm{cons}}) with DANCE. With continuous features, enabling guidance raises mean event F1 from 40.95% to 43.49%, a gain of 2.54 points over the unguided variant. The guided continuous variant scores 1.08 points above DANCE (42.41%). The complete NeuroDETR model scores 46.70%, a gain of 4.29 points over DANCE, confirming improvements stem from our collaborative discrete memory.

W2: Rationale and Applicability Scope of Core Designs

Our designs reflect intrinsic EEG neurophysiology and asynchronous detection traits (non-stationary background, low SNR, unsegmented streams), using globally fixed hyperparameters grounded in physiological timescales, the 10–20 standard, and stable optimization:

  1. Multiscale Temporal VQ (St={4,16,64}\mathcal{S}_{\mathrm{t}}=\{4,16,64\}): EEG events span two orders of magnitude (0.2 s0.2\,\mathrm{s} transients to 63 s63\,\mathrm{s} seizures), defying single-scale tokenization. At 128 Hz128\,\mathrm{Hz}, strides match 31.2531.25, 125125, and 500 ms500\,\mathrm{ms} to preserve brief waveform details while aggregating long-range context. Limit: Sub-step micro-transients (≪31.25 ms\ll 31.25\,\mathrm{ms}) or ultra-slow drifts exceeding window WW fall outside this envelope.
  2. Spatial hierarchy and residual quantization. We group the available electrodes at three levels: one whole-head group, five coarse scalp-electrode groups (frontal, temporal, central, parietal, and occipital), and twelve fine clusters. At each finer level, we subtract the parent group's quantized representation before quantizing the child group's features. This lets the spatial memory represent both whole-head activity and regional differences. Limit: With few available channels, some regional groups may be empty or share the same electrodes as their parent groups, limiting the additional spatial information captured at finer levels and potentially reducing the benefit of hierarchical residual quantization.

The remaining component designs and deployment analysis continue in Answer (2/3).

Answer (2/3)

Continuing W2 from Answer (1/3):

  1. FOCAL Routing (τ=0.4,kmin⁡=1\tau=0.4, k_{\min}=1): Target events are often sparse and weak against irregular background. τ=0.4\tau=0.4 follows high-recall proposal filtering, retaining moderate-confidence windows over a hard 0.50.5 cutoff for local refinement, while kmin⁡=1k_{\min}=1 guarantees non-empty local attention (pure-background windows remain suppressed by decoder Class 0 without forced false alarms). Limit: Under high event occupancy, routing may saturate, causing FOCAL to approach standard local–axial attention.
  2. Temporal-First Fusion & Gate Init (g0=0.02g_0=0.02): EEG has millisecond temporal precision but coarse spatial resolution; asynchronous detection requires exact onset/offset anchoring while spatial topography aids class discrimination. Initializing g0=0.02g_0=0.02 prevents coarse-binned spatial updates from perturbing early boundary regression while keeping non-zero gradients. Limit: On boundary-free, purely spatial stationary tasks (e.g., resting-state classification), this may slow early spatial convergence and add temporal overhead.

W3, Q3: Thank you for raising this deployment question. For our cross-window boundary protocol and statistics (retaining events with both onset and offset fully within the window, strictly shared across all baselines at non-overlapping stride ΔW=W\Delta W = W), please see [Reviewer Woxn, W4 & Q4]. We provide a more in-depth discussion in Appendix B. Below we clarify buffering, cross-window handling, latency, and stability:

  • Offline evaluation protocol: Our NeuroDETR adopts an offline evaluation protocol in which continuous EEG is partitioned at a non-overlapping stride (ΔW=W\Delta W = W) and each window is inferred strictly independently—detecting events in the current window never buffers or accesses subsequent windows, relying solely on bidirectional context within the current WW-second window. Meanwhile, because adjacent test windows are disjoint under the complete-interval evaluation criterion, duplicate detections between adjacent windows naturally do not occur offline, requiring no cross-window event-merging post-processing.
  • Online streaming deployment (chunk buffering, overlap deduplication, and merging delay): When deploying this non-causal windowed model in continuous online scenarios, the operational rules are adapted as follows:

(i) Future signal buffering: Although awaiting subsequent windows is unnecessary, symmetric dilated convolutions, FOCAL axial attention, and set decoding rely on bidirectional intra-window context, so initial inference buffers the full WW-second window and thereafter updates incrementally at a sliding stride Δt<W\Delta t < W (introducing intra-window lookahead buffering).

(ii) Adjacent-window deduplication and merging delay: Introducing overlapping sliding windows (Δt<W\Delta t < W) to capture boundary-crossing events requires center-zone retention or 1D IoU/endpoint stitching to eliminate duplicate detections in overlap regions. Because each window outputs at most NeN_e sparse intervals, scalar interval matching takes only microseconds (≪0.1 ms\ll 0.1\,\mathrm{ms}, negligible compute latency); the sole additional wait arises when an ongoing event crossing the current window's right edge must await the next buffer step (Δt\Delta t) to finalize its offset, alongside potential sensitivity of heuristic stitching to boundary confidence jitter.

  • Full-window computation and strict real-time applicability boundary: We fully agree that offline localization gains cannot be directly extrapolated to zero-lookahead strict real-time control. Strict real-time responsiveness is bounded primarily by the data-acquisition buffering stride above. Accordingly, our current framework is directly suited for offline auto-annotation and trigger-free near-real-time monitoring tolerant of second-scale buffering (e.g., clinical seizure screening/alerting and passive state assessment), whereas evolving from heuristic cross-window stitching to causal masking and end-to-end streaming cross-window memory propagation is a core focus of our future work toward sub-second closed-loop control.

The efficiency discussion, stability analysis, and remaining responses continue in Answer (3/3).

Answer (3/3)

Continuing W3 and Q3 from Answer (2/3):

Measured Long-Window Latency & Strict Real-Time Scope We benchmarked all models across all eight full test sets (226.19 h226.19\,\mathrm{h} EEG, warmed up) on a single NVIDIA A100 GPU (Table R1), reporting total end-to-end Runtime (h), 8-dataset mean median single-window Latency (ms) at default WW, and compute per second of EEG trt=Latency/Wt_{\mathrm{rt}}=\mathrm{Latency}/W (ms/s):

Table R1: Inference efficiency across all eight test sets (226.19 h226.19\,\mathrm{h} EEG on NVIDIA A100).

MethodDOSEDEventNetActionFormerDyFADetDANCENeuroDETR
Runtime (h)0.160.150.160.190.330.26
Latency (ms)0.520.903.618.2336.2827.52
trtt_{\mathrm{rt}} (ms/s)0.040.060.250.561.030.81

Processing time. NeuroDETR processed 226.19 h of test EEG in 0.26 h on one NVIDIA A100. Its mean of the eight dataset-specific median processing times was 27.52 ms per acquired window, compared with 36.28 ms for DANCE. These measurements exclude signal acquisition and cross-window stitching.

Stability over long recordings. We consider sustained artifacts and gradual changes in electrode impedance potential sources of instability during long recordings. NeuroDETR processes each window independently and keeps its codebooks fixed at inference. These mechanisms prevent hidden-state errors from accumulating across windows and avoid drift caused by online codebook updates, which may help limit cumulative instability over long sessions. **Appendix B outlines how streaming evaluation could assess first-alert delay, time to finalize event offsets, and false alarms per hour.

Q2. Weak events, foreground prevalence, and class imbalance

FOCAL uses a class-agnostic target indicating whether a subwindow contains a foreground event. Temporal routing selects subwindows for local attention. Unselected subwindows pass through background normalization and still participate in axial attention within the same input. Spatial pooling uses continuous weights derived from predicted event probabilities. The decoder can assign every event query to background, including when a subwindow has been selected for refinement.

We report two additional analyses in this response. M5 removes the FOCAL module and its density supervision and replaces density-weighted spatial pooling with adaptive average pooling. The comparison with the full model measures the combined effect of these changes.

For each test subject, we ranked events by their RMS contrast with the flanking background within 2 s on either side. We used this contrast as a proxy for signal strength and evaluated recall on the lowest third. A correct detection requires the correct class and temporal IoU above 0.5.

Table R2. Event recall (%) on the lowest-contrast third of events for each test subject.

VariantBI2014bBNCI2015BNCI2020TUSZ
M5 (w/o FOCAL)54.4568.2036.1411.04
Full (NeuroDETR)56.3370.3941.8619.63

The full model improves recall on this subset in all four evaluated datasets. The largest increase is on TUSZ, from 11.04% with M5 to 19.63% with the full model, a gain of 8.59 percentage points.

  • Class-imbalanced detection (Table R3): On imbalanced P300 (BI2014b, BI2013a) and clinical seizures (TUSZ), we report equal-weighted foreground Macro-F1event\mathrm{Macro\text{-}F1}_{\mathrm{event}} (τIoU=0.5\tau_{\mathrm{IoU}}=0.5):

Table R3: Foreground class-wise Macro-F1event\mathrm{Macro\text{-}F1}_{\mathrm{event}} (%, τIoU=0.5\tau_{\mathrm{IoU}}=0.5) on imbalanced benchmarks.

VariantBI2014bBI2013aTUSZ
M5 (w/o FOCAL)27.4935.1610.27
Full (NeuroDETR)31.5738.5015.06

Compared with M5, the full model improves foreground macro-event F1 by 4.08 points on BI2014b, 3.34 points on BI2013a, and 4.79 points on TUSZ. This metric gives each foreground class equal weight.

Reviewer_U6EvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a EEG based model for simultaneous segmentation and classification that uses the DETR (Detection transformer) and dense segmentation prediction along with a complicated model design to achieve state-of-the-art performance on multiple datasets.

Critical strengths

DETR seems a logical inclusion for EEG segmentation tasks.

I appreciate the attempts to actually put the math formulations in the main body to follow the signal flow.

The results seem to be a meaningful gain over the recent DANCE method.

Critical weaknesses

It is not clear how hyperparameters were selected (hopefully on validation, but then they will vary by split). It's not clear if Table 4 details F1 in validation or test. The results on some datasets seem sensitive to the number of queries and the number of queries varies between datasets. Table 6 should show the number of classes. From what I gather it is BI2014b and BI2013a 2 class; BNCI2014-001 4 class; BNCI2015-001 2 class; HDOP 3; Broderick and TUSV 4 class.

Questions for authors

What do " 270 virtual channels" correspond to in (2)?

The notations of ht\mathbf{h}_t, $\mathcal{W}_n $, and nn in equation 3 are not explicitly defined, do they refer to a vector of the matrix, the set of windows, and the index of the signal?

What is the symbol at the beginning of equation 4 and used for mnm_n in (5)? I guess it is an Iverson bracket/ indicator function...

Why say TopK but then use kmin=1k_\text{min}=1?

Should line 205 start with B~n\tilde{\mathbf{B}}_n?

Why is the reshaping in (8) needed to be stated and why would Batch size be involved? To look across all windows I would assume that is within a signal not across different signals in a batch.

The presentation of VQ with commitment penalty is rather confusingly presented in (9). I would suggest putting an overall loss function rather than describing QQ separate from the commitment penalty. Note that the commitment penalty doesn't appear in the losses in Figure 2.

Line 247 what does 'a shared stem' mean and is this a common term?

2 comments

Answer (1/2)

Thank you for your careful review. Here are our answers, in order.

W1. Hyperparameter selection, Table 4, and query-count sensitivity

1. Hyperparameter settings and selection.

We use shared architectural settings, loss weights, and optimization schedules across all eight datasets, as listed in Appendix H.2. Input-window durations, batch sizes, and the event-query budget NeN_e are dataset-specific.

We selected NeN_e using validation performance. For each of the seven datasets evaluated with five-fold subject-wise cross-validation, we held out one fold for testing and reserved 10% of the remaining subjects for validation. Each candidate NeN_e therefore had five validation event-F1 scores, one per fold. We selected the candidate with the highest mean validation event F1 and used that same NeN_e in all five folds. For TUSZ, we selected NeN_e on the official development set.

2. Results reported in Table 4.

Table 4 reports test-set event F1 for every candidate NeN_e. Each entry is the mean over the five outer test folds for the seven cross-validation datasets, or the score on the official evaluation set for TUSZ. Boldface identifies the query count selected using validation performance and used in Table 1. The displayed test scores characterize sensitivity to the query budget. The selection criterion uses the validation scores described above.

3. Sensitivity to the number of queries.

The query count NeN_e sets an upper bound on the number of events predicted within one input window. Its effect on event F1 varies across datasets and can be non-monotonic.

We corrected the HDOP entry at Ne=24N_e=24 from 41.17% to 51.17% and updated the accompanying discussion in Appendix A.3. HDOP consequently spans 51.17% to 52.91% across the five query counts, a range of 1.74 percentage points. Other datasets show greater sensitivity. On BNCI2015, increasing NeN_e from 40 to 80 reduces event F1 from 67.77% to 64.12%, a decrease of 3.65 points. On BI2013a, the five settings span 52.46% to 56.50%, a range of 4.04 points. These results motivate selecting the query budget separately for each dataset using validation performance.

Changing NeN_e affects both prediction capacity and training supervision. Unmatched queries receive background labels, and the localization losses are normalized by NeN_e. These factors may contribute to the observed sensitivity by changing the balance between background classification and event localization.

W2. Number of event classes in Table 6

We added the number of foreground classes (KK) to the dataset statistics table, which was Table 6 in the original submission and is now Table 12 in Appendix E. BI2014b, BI2013a, BNCI2015, and BNCI2020 each have two foreground classes. BNCI2014 and TUSZ each have four, HDOP has three, and Broderick has one. In our Broderick evaluation, all word annotations share a single foreground label. The counts exclude the background class, indexed by 0 in all datasets.

Q1. Interpretation of the 270 virtual channels

Each virtual channel is a learned weighted combination of the available physical electrodes. The spatial mixing layer computes its weights using Fourier embeddings of the normalized electrode coordinates. The weights therefore depend on the input montage, while the output width remains fixed at 270. We clarified this operation in Section 3.2.

Q2. Notation in Eq. (3)

We define all three immediately after Eq. (3). Within one input signal, n∈0,…,Nw−1n\in\\{0,\ldots,N_w-1\\} indexes a temporal subwindow, Wn=nw+1,…,(n+1)w\mathcal W_n=\\{nw+1,\ldots,(n+1)w\\} contains its ww token indices, and ht∈RD\mathbf h_t\in\mathbb R^D is the feature vector at token position tt. These indices refer to the padded feature sequence, including the final subwindow when its length would otherwise be less than ww.

Answer (2/2)

Q3. Indicator notation in Eqs. (4) and (6)

Yes. It equals 1 when the enclosed condition holds and 0 otherwise. We now use 1[⋅]\mathbf 1[\cdot] consistently and define it at its first occurrence.

Q4. TopK selection with kmin⁡=1k_{\min}=1

kmin⁡k_{\min} specifies the minimum number of subwindows selected for local attention. With the default threshold of 0.4 and a minimum selection count of one, Eq. (6) selects every subwindow whose predicted event probability exceeds 0.4. If no subwindow exceeds the threshold, it selects the single highest-scoring subwindow. We state this fallback immediately after Eq. (6).

Q5–Q6. Input to axial attention and the role of the batch dimension in Eq. (8)

The input to axial attention is the output of the local operation in Eq. (7). We rewrote Eq. (8) for a single EEG input. For each fixed position within a subwindow, axial attention connects the features at that position across all subwindows of the same input. The resulting sequence has one row per subwindow and one column per feature dimension. Batching allows these sequences to be processed in parallel, with attention computed independently for each signal.

Q7. VQ commitment losses and the overall training objective

We revised Sections 3.4 and 3.6 to clarify how the quantizers are trained. Section 3.4 defines the straight-through quantization operator in Eq. (9). The codebooks are updated by exponential moving averages, while the commitment penalties encourage the encoder features to approach their assigned codewords.

Section 3.6 now defines both commitment losses explicitly. The temporal loss is averaged across scales. The spatial loss is computed over valid tokens from nonempty electrode groups within each level and then averaged across levels. Together with temporal reconstruction, these terms form the VQ objective:

Lvq=λvq,tLcommit(t)+λvq,sLcommit(s)+λrecLrec.\mathcal{L}_{\mathrm{vq}} = \lambda_{\mathrm{vq},\mathrm{t}}\mathcal{L}_{\mathrm{commit}}^{(\mathrm{t})} + \lambda_{\mathrm{vq},\mathrm{s}}\mathcal{L}_{\mathrm{commit}}^{(\mathrm{s})} + \lambda_{\mathrm{rec}}\mathcal{L}_{\mathrm{rec}}.

The revised caption for Figure 2(e) identifies the detection and auxiliary losses as the main event loss, Levent=Ldet+Laux\mathcal L_{\mathrm{event}}=\mathcal L_{\mathrm{det}}+\mathcal L_{\mathrm{aux}}, and points to the full training objective in Section 3.6. This objective combines event supervision, VQ training, and FOCAL density supervision:

Ltotal=Levent+Lvq+λdenLden.\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{event}} + \mathcal{L}_{\mathrm{vq}} + \lambda_{\mathrm{den}}\mathcal{L}_{\mathrm{den}}.

Q8. Meaning of the shared stem Section 3.4 now describes the shared encoder explicitly. We first average the available electrode signals within each nonempty group. We then apply the same temporal encoder, consisting of two strided convolutions with GELU, independently to each group signal. Its weights are shared across all groups at the whole-head, coarse-region, and fine-cluster levels.

Reviewer_WoxnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NeuroDETR for asynchronous EEG event detection, jointly predicting event classes and temporal boundaries from continuous EEG windows. The main additions to a query-based detection pipeline are multiscale temporal vector quantization, hierarchical spatial residual quantization, and FOCAL, which uses predicted event presence to guide local attention and spatial pooling. The resulting representations are fused into a memory for a DETR-style decoder. Evaluation covers eight datasets spanning five task domains, using subject-grouped five-fold cross-validation for seven datasets and the official split for TUSZ. The reported mean event F1 is 46.70, exceeding DANCE by 4.29 percentage points, with the highest event F1 on all eight datasets. Component ablations and a factorial study examine the contributions of quantization and event guidance.

Critical strengths
  1. The problem-to-design correspondence is clear and technically plausible. Continuous recordings contain substantial background activity, while relevant events vary widely in duration and spatial distribution. The proposed design maps these challenges to corresponding components.
  2. The individual ingredients, e.g., local and axial attention, vector quantization, spatial hierarchies, and DETR-style decoding, are established techniques. The main novelty lies in sharing event-presence estimates across temporal routing and spatial aggregation, then constructing a joint event memory from multiscale temporal codes and cross-scale residual spatial codes. This is a non-trivial extension of the continuous Perceiver-based representation used by DANCE.
  3. Table 1 reports the highest event F1 on all eight datasets, covering P300, motor imagery, motor execution, speech, and seizures; the average improvement over DANCE is 4.29 percentage points and reaches 8.99 points on BNCI2015. Subject-grouped evaluation on the seven cross-validation datasets strengthens the evidence for generalization to unseen subjects. Table 3 provides an overall useful factorial comparison.
Critical weaknesses
  1. The overall training objective is incomplete. Eq. 21,35 define only the event-supervision objective with LdetL_{\mathrm{det}}, LbndL_{\mathrm{bnd}}, LdenseL_{\mathrm{dense}}, and LconsL_{\mathrm{cons}}. Eq. 5 separately defines the FOCAL density loss LdenL_{\mathrm{den}}, Eq. 12 defines the VQ reconstruction loss LrecL_{\mathrm{rec}}, and Section 3.4 introduces commitment losses for the temporal and spatial quantizers. The paper never states an overall objective combining these terms with LeventL_{\mathrm{event}} and also provides no numerical values for λℓ1\lambda_{\ell 1}, λcls\lambda_{\mathrm{cls}}, λgiou\lambda_{\mathrm{giou}}, λbnd\lambda_{\mathrm{bnd}}, λdense\lambda_{\mathrm{dense}}, λcons\lambda_{\mathrm{cons}}, the commitment coefficient β\beta, or any coefficients used to weight the density, reconstruction, and commitment objectives.
  2. Comparisons with dense detector baselines are potentially unfair. Section 4.2 and Appendix D apply the same confidence threshold of 0.05 and class-wise NMS IoU threshold of 0.5 to EventNet, ActionFormer, DyFADet, and DOSED, without reporting per-model and per-dataset tuning on the validation subjects within each fold. Different dense detectors can have different confidence calibration and duplicate-prediction behavior, so a common operating point may place them at different precision–recall trade-offs.
  3. The ablations do not isolate the mechanisms within the major component bundles. Existing ablations establish that the major component bundles are useful, but do not isolate the mechanisms within those bundles. In particular, the VQ comparisons cannot determine whether the gains arise from hard discretization, information bottlenecking, or the accompanying reconstruction and commitment objectives. Similarly, the guidance ablation jointly changes temporal routing and spatial pooling, and the reported factorial interaction is not accompanied by routing-level diagnostics that explain its variation across datasets.
  4. Key architectural hyperparameters and window-level evaluation details are missing. Appendix F does not report the temporal scale set, codebook sizes, FOCAL window size, spatial temporal-bin count, or fusion memory length. Although Table 6 reports input-window lengths, the manuscript also does not specify the stride between input windows, the treatment of events crossing window boundaries, cross-window prediction merging or deduplication, or whether event matching is performed per window or after recording-level reconstruction. No source code is provided to resolve these omissions.
Questions for authors
  1. What is the complete training objective combining event supervision, density prediction, reconstruction, and commitment losses; what are all associated coefficients; and which terms are used in each ablation?
  2. Were confidence and NMS thresholds tuned separately for each dense baseline and dataset using validation data within each fold?
  3. Which parts of the VQ mechanism, including effects of hard discretization, compression, effective capacity, learned codewords, reconstruction, and commitment regularization, account for the gains, and what routing-level evidence explains the dataset-dependent quantization–guidance interactions?
  4. What are the missing architectural hyperparameters, and what are the input-window stride, cross-window event-handling and prediction-merging rules, and the level at which event matching is performed?
  5. Typo: Missing Appendix number in line 348
6 comments

Answer (1/3)

Thank you for your careful review.

W1 & Q1: Training Objective and Ablation Losses

The original manuscript introduced local objectives without an explicit joint objective. Section 3.6 now include the VQ loss Lvq\mathcal{L}_{\mathrm{vq}} and grouped auxiliary loss Laux\mathcal{L}_{\mathrm{aux}}:

Laux=λbndLbnd+λdenseLdense+λconsLcons.\mathcal{L}_{\mathrm{aux}} = \lambda_{\mathrm{bnd}}\mathcal{L}_{\mathrm{bnd}} + \lambda_{\mathrm{dense}}\mathcal{L}_{\mathrm{dense}} + \lambda_{\mathrm{cons}}\mathcal{L}_{\mathrm{cons}}.

Weights and hyperparameters are fixed across all eight datasets (Table R1). Table R2 specifies the losses in each ablation of main-paper Tables 2 and 3.

Table R1. Training weights and hyperparameters.

SymbolValue
λl1\lambda_{\mathrm{l1}}2.0
λcls\lambda_{\mathrm{cls}}1.0
λgiou\lambda_{\mathrm{giou}}5.0
λbnd\lambda_{\mathrm{bnd}}0.5
λdense\lambda_{\mathrm{dense}}1.0
λcons\lambda_{\mathrm{cons}}0.5
λden\lambda_{\mathrm{den}}0.5
λrec\lambda_{\mathrm{rec}}0.05
λvq,t\lambda_{\mathrm{vq},\mathrm t} , λvq,s\lambda_{\mathrm{vq},\mathrm s}0.05
β\beta0.25
λmcls\lambda_{\mathrm{mcls}} / σbnd\sigma_{\mathrm{bnd}} / ϵclamp\epsilon_{\mathrm{clamp}}1.01.0 / 22 / 10−810^{-8}

Table R2. Loss terms enabled in each ablation variant.

Ablation variantLevent\mathcal{L}_{\mathrm{event}}Lden\mathcal{L}_{\mathrm{den}}Lrec\mathcal{L}_{\mathrm{rec}}Lcommitt\mathcal{L}_{\mathrm{commit}}^{\mathrm{t}}Lcommits\mathcal{L}_{\mathrm{commit}}^{\mathrm{s}}Description
Full NeuroDETR✓✓✓✓✓Complete end-to-end training objective
Table 2 – M1 (without temporal VQ)✓✓××✓Replace the temporal branch with continuous features
Table 2 – M2 (without spatial VQ)✓✓✓✓×Replace the spatial branch with continuous features
Table 2 – M3 (single temporal scale)✓✓✓✓✓Retain only the finest temporal scale
Table 2 – M4 (single spatial hierarchy level)✓✓✓✓✓Retain only fine-grained electrode clusters
Table 2 – M5 (without FOCAL)✓×✓✓✓Remove the FOCAL module and event-density supervision
Table 3 – Quant ×, Guidance × / ✓✓✓×××Retain gϕg_\phi and Lden\mathcal{L}_{\mathrm{den}}; bypass both quantizers
Table 3 – Quant ✓, Guidance ×✓✓✓✓✓Retain gϕg_\phi and Lden\mathcal{L}_{\mathrm{den}}; use unguided routing and uniform pooling

W2 & Q2: Validation-Based Threshold Tuning

We retain the original confidence threshold (τconf=0.05\tau_{\mathrm{conf}}=0.05) and class-wise NMS threshold (τNMS=0.5\tau_{\mathrm{NMS}}=0.5) in Table 1 to compare the four dense detectors under a common post-processing rule across datasets. Because detectors can differ in confidence calibration and in the number of overlapping predictions they produce, we also evaluated how detector-specific threshold selection affects the comparison.

For each dense detector (DyFADet, ActionFormer, EventNet, and DOSED), we evaluated all 48 combinations of the following thresholds:

τconf∈{0,0.01,0.05,0.1,0.2,0.3,0.4,0.5},\tau_{\mathrm{conf}} \in \{0, 0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5\},

τNMS∈{0.1,0.3,0.5,0.7,0.9,1.0}.\tau_{\mathrm{NMS}} \in \{0.1, 0.3, 0.5, 0.7, 0.9, 1.0\}.

For each of the seven cross-validation datasets, we selected the threshold pair that maximized event F1 on the validation subjects of each fold and applied that pair to the corresponding held-out test fold. For TUSZ, we selected thresholds on the official development split and applied them to the official evaluation split. In both cases, a correct detection required a one-to-one match with the correct class and temporal IoU above 0.5.

Table R3. Test F1eventF1_{\mathrm{event}} (%) after validation-based threshold tuning for the four dense detectors. Scores are means over five test folds for the seven cross-validation datasets and a single evaluation score for TUSZ. The NeuroDETR row reproduces Table 1.

MethodBI2014bBI2013aBNCI2014BNCI2015BNCI2020HDOPBroderickTUSZ
DyFADet (tuned)35.2051.4821.1641.8625.0433.2436.622.87
ActionFormer (tuned)33.9250.3421.2832.0728.6926.0637.731.88
EventNet (tuned)32.0542.4238.6055.1637.5741.5826.9812.00
DOSED (tuned)42.6552.0636.4841.4233.3349.3238.022.58
NeuroDETR (ours)45.2356.5045.5567.7741.8552.9138.7625.06

Threshold selection materially affected some comparisons. On HDOP, tuning raised DOSED's test event F1 from 41.37% to 49.32%. NeuroDETR achieved higher test event F1 than the strongest tuned dense detector on each of the eight datasets. Appendix F.2 reports the full validation-tuned comparison.

Controlled submechanism ablations and codebook diagnostics follow in Answer (2/3).

Answer (2/3)

Following Answer (1/3), we examine the guidance and quantization submechanisms.

W3 & Q3: Factorized Ablations and Codebook Diagnostics

  1. Decoupling Temporal Routing and Spatial Pooling

Keeping the FOCAL backbone, density head gϕg_\phi, and supervision Lden\mathcal{L}_{\mathrm{den}} unchanged, Table R4 independently enables temporal event routing (Eq. 6) and spatial density-weighted pooling (Eq. 14).

Table R4. Temporal routing and spatial pooling ablation (F1eventF1_{\mathrm{event}}, %).

Temporal routingSpatial poolingBI2013aBNCI2015BNCI2020TUSZ
××51.1164.1537.2816.32
×✓51.8065.4538.0517.51
✓×53.1965.8139.3423.44
✓✓56.5067.7741.8525.06

Each pathway improves performance on all four datasets, and enabling both gives the best results.

  1. Factorized Isolation of VQ Submechanisms

With FOCAL enabled throughout, Table R5 evaluates continuous features (V0), reconstruction supervision (V1), a continuous low-rank bottleneck (V2), soft attention over learned prototypes without hard quantization (V3), and hard VQ with commitment but without reconstruction (V4). Full NeuroDETR combines hard VQ, commitment, and reconstruction.

Table R5. Factorized ablation of VQ mechanisms (F1eventF1_{\mathrm{event}}, %). FOCAL guidance is enabled in all variants.

Model variantDiscretization / bottleneck typeLrec\mathcal{L}_{\mathrm{rec}}Lcommit\mathcal{L}_{\mathrm{commit}}BI2013aBNCI2015BNCI2020TUSZ
V0: Continuous features———51.9965.3437.7920.14
V1: Continuous features + Lrec\mathcal{L}_{\mathrm{rec}}—✓—53.3566.9338.2420.64
V2: Continuous bottleneckLow-rank linear projection✓—53.7066.2137.7421.21
V3: Soft codebookSoft prototype attention✓—54.0167.3239.2222.46
V4: Hard VQ without Lrec\mathcal{L}_{\mathrm{rec}}Hard discrete quantization—✓54.7667.4839.1722.78
Full NeuroDETRHard discrete quantization✓✓56.5067.7741.8525.06
  • Reconstruction and continuous compression (V0 to V2). Reconstruction alone (V1) gives modest gains, whereas adding a continuous low-rank bottleneck (V2) produces mixed changes across datasets. Neither reconstruction nor linear compression alone accounts for the full gains.
  • Prototype learning, hard assignment, and reconstruction. Soft prototypes (V3) improve on the low-rank projection (V2) on all four datasets. Comparing V4 with Full holds hard assignment and encoder commitment fixed. Adding reconstruction of the pre-quantization features increases event F1 by 0.29–2.68 points. Comparing V3 with Full holds reconstruction fixed and measures the combined change to hard assignment and encoder commitment.
  1. Diagnosing Effective Codebook Capacity

Table R6. Temporal codebook active ratios and perplexities across eight datasets.

DatasetActive ratio (s=4/16/64s=4/16/64)Perplexity (s=4/16/64s=4/16/64)
BI2014b76.56% / 87.03% / 89.30%120.8 / 149.3 / 157.6
BI2013a99.38% / 99.22% / 95.86%169.9 / 175.1 / 151.1
BNCI2014100.00% / 100.00% / 96.88%161.6 / 157.0 / 132.8
BNCI2015100.00% / 100.00% / 99.77%187.1 / 182.1 / 146.4
BNCI2020100.00% / 99.92% / 99.45%178.1 / 176.8 / 127.7
HDOP98.98% / 95.31% / 77.58%156.6 / 150.3 / 112.0
Broderick99.92% / 99.84% / 99.77%199.3 / 203.9 / 178.4
TUSZ100.00% / 100.00% / 100.00%166.0 / 161.6 / 170.5

Table R6 reports temporal codebook active ratios and perplexities on all eight test sets. Perplexity, exp⁡(H)\exp(\mathcal{H}), measures effective capacity relative to the single-scale vocabulary size Vt=256V_t=256. Active ratios range from 76.56% to 100%, with most settings between 95% and 100%; perplexities range from 112.0 to 203.9, or 43.8%–79.6% of nominal capacity. On TUSZ, all 256 entries are active at each scale, with perplexities of 166.0, 161.6, and 170.5. The assignments therefore use the full vocabulary with unequal frequencies.

The routing diagnostics explaining cross-dataset interactions, followed by reproducibility and windowing details, are provided in Answer (3/3).

Answer (3/3)

Continuing from Answer (2/3), we explain routing differences and the evaluation protocol.

W3 & Q3 (continued): Routing diagnostics

Table R7 reports routing ratio ρroute\rho_{\mathrm{route}}, event-subwindow recall, foreground enrichment (FER: routed foreground density/global coverage), and the interaction Δint\Delta_{\mathrm{int}} from main-paper Table 3. With fixed τ=0.4\tau=0.4 and kmin⁡=1k_{\min}=1, routing ratios range from 10.28% to 87.47%, with a mean routing recall of 84.67% across the eight datasets.

  • Low/moderate duty cycles: On HDOP, BNCI2020, BNCI2014, and BNCI2015, routing enriches foreground by 1.69–5.09 times, reducing background dominance in EMA updates and amplifying VQ gains.
  • Dense streams: On Broderick and short-window BI2014b, foreground already dominates unguided codebooks. Routing thus provides largely additive benefits (Δint≈0\Delta_{\mathrm{int}}\approx0).
  • Special cases: On TUSZ, 10.28% of subwindows are routed, including 71.53% of event-containing subwindows. The routed subwindows have 1.63 times the overall foreground density, and the quantization–guidance interaction is +0.84 event-F1 points. For BI2013a, dense stimuli limit binary enrichment, but continuous dud_u still sharpens spatial soft pooling around phase-locked P300 peaks within time bins (Eq. 14), explaining its +4.04 interaction.

Table R7. Test-set routing diagnostics and interactions.

DatasetEvent coverage (%)Routing ratio ρroute\rho_{\mathrm{route}} (%)Routing recall (%)Foreground enrichment ratio (FER)Δint\Delta_{\mathrm{int}} (F1eventF1_{\mathrm{event}} points)
BI2014b4461.5988.461.45×-0.23
BI2013a8087.4795.321.09×+4.04
BNCI20145048.6275.031.69×+1.00
BNCI20154743.2481.551.90×+1.33
BNCI20202730.1884.572.95×+3.51
HDOP6.117.2383.525.09×+4.77
Broderick6983.1997.361.18×-0.01
TUSZ3.210.2871.531.63×+0.84

W4 & Q4: Architecture and windowing protocol

Architectural settings. Appendix H.2 now lists training coefficients, optimization settings, and hyperparameters for the backbone, FOCAL, temporal/spatial VQ, fusion, and decoder.

Window strides. Only TUSZ uses 50% overlapping training windows (W=256W=256 s, stride 128 s) to augment sparse seizures. Validation/test windows on all eight datasets are non-overlapping (stride=W\mathrm{stride}=W).

Table R8. Input lengths and strides.

DatasetWindow length WW (s)Training stride (s)Validation / test stride (s)
BI2014b, BNCI2014, Broderick888
BI2013a, BNCI2015, BNCI2020323232
HDOP999
TUSZ256128256

Boundary handling and matching. Asynchronous decoding identifies event classes and onset/offset times without external trial triggers. Following DANCE, we study interval localization for continuous monitoring rather than online recording-level stitching.

  • Valid targets: Only annotations whose onset and offset both fall entirely within the current window are valid targets. NeuroDETR and all baselines strictly share this criterion, avoiding artificial truncation boundaries in localization evaluation.
  • Clipping sensitivity. The proportion of test events crossing a window boundary ranges from 0.00% on HDOP to 33.33% on BNCI2014. In the clipped-target evaluation, each annotation that intersects a window retains its class and is clipped to that window. A crossing event can therefore contribute a target fragment to each adjacent window. Table R9 compares the two protocols.
  • Matching scope: Prediction/ground-truth matching is one-to-one within each window; no cross-window merging or deduplication is applied. For future streaming reconstruction, overlapping windows could pass active boundary queries forward as memory states for end-to-end event linking. Appendix B describes how a streaming extension could carry open event queries into subsequent windows and evaluate the linked detections at the recording level.

Table R9. Boundary-crossing proportions and clipping sensitivity (event F1, %; IoU>0.5\mathrm{IoU}>0.5).

Statistic / experimental settingBI2014bBI2013aBNCI2014BNCI2015BNCI2020HDOPBroderickTUSZ
Events crossing window boundaries (%)12.63.133.3315.3415.2802.7618.22
NeuroDETR (standard complete-event setting)45.2356.545.5567.7741.8552.9138.7625.06
NeuroDETR (boundary-clipped setting)42.4854.642.766.8140.7352.9138.6520.18

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response Summary

We thank the reviewers for their careful reading and constructive feedback. The reviews recognize the relevance of asynchronous EEG event detection, the evaluation across eight datasets and five task domains, and the gains in event-level performance. In response to their suggestions, we have added:

  • Validation-based tuning of confidence and NMS thresholds for each dense detector, using fold-specific validation data or the official TUSZ development split (Woxn).
  • Separate ablations of temporal routing and spatial pooling, VQ controls examining reconstruction, bottlenecks, and quantization, and codebook usage and routing diagnostics (Woxn, gbiC).
  • Recall on low-contrast events and macro-event F1 averaged across foreground classes, reported in our responses (gbiC).
  • Sensitivity to retaining and clipping boundary-crossing annotations, inference-efficiency measurements, and clarification of window buffering and proposed streaming extensions (Woxn, gbiC).
  • The full training objective and coefficients, active losses in each ablation, architectural hyperparameters, and clarification of query-count selection and the evaluation split used in Table 4 (Woxn, U6Ev).

We have submitted a revised manuscript incorporating these clarifications and additional experiments. We appreciate the Area Chair’s and reviewers’ time and consideration and welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.