- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates causal and irreversible command decoding from two-channel electrooculography (EOG), focusing on the challenge of distinguishing new gesture events from continuing evidence of previously detected events. The authors propose Action-Constrained Event Decoding (ACED), which explicitly models event ownership through two approaches: ACED-P, a learnable finite-state policy trained by exact marginalization over supervision-compatible action paths, and ACED-D, a rule-based decoder that defers ambiguous double/triple-blink classification until additional evidence becomes available.
The method is evaluated on a dataset containing 5 participants, 30 recording sessions, and 1,442 annotated gesture windows, using leave-one-wear-out (LOWO), leave-one-subject-out (LOSO), and public-dataset transfer experiments. Under identical classifier outputs, ACED-D improves the LOWO first-correct fraction from 0.641 to 0.715 compared with hysteresis, while ACED-P13 reduces false emissions at a comparable detection rate. The results demonstrate trade-offs between command recognition accuracy, false-trigger suppression, and decision latency, although neither proposed decoder consistently outperforms the baselines across all metrics.
Critical strengths
- Meaningful problem formulation: The paper explicitly addresses event ownership in continuous EOG command decoding, distinguishing new gestures from ongoing events. The proposed deferred-resolution mechanism improves consecutive-pair recognition from 0.303 to 0.478 compared with hysteresis on identical classifier outputs, demonstrating the practical relevance of this problem formulation.
- Rigorous and transparent experimental evaluation: The paper compares decoders using identical classifier outputs and includes LOWO/LOSO evaluations, false-trigger analysis, and event-time versus commit-time measurements. This controlled evaluation and the disclosure of unsuccessful results provide valuable insights into the trade-offs of causal command decoding, strengthening the credibility of the empirical analysis.
Critical weaknesses
- Limited evidence for the core methodological contribution. Although the paper emphasizes learnable event ownership through finite-state action-path marginalization, the best performance is achieved by the hand-designed ACED-D rather than the learned ACED-P. In Table 1, ACED-P18 even underperforms the hysteresis baseline, raising concerns about the effectiveness and necessity of the proposed learning approach.
- Inconsistent evaluation of irreversible command decoding. The primary results are evaluated using event time rather than actual command commitment time. As reported in Section 5.3, ACED-D's first-correct fraction drops from 0.715 to 0.542 when evaluated at commitment time, substantially weakening its claimed advantage for real-time command execution.
Questions for authors
- Evaluation Protocol: Why is 30-fold leave-one-wear-out (LOWO) selected as the primary evaluation protocol rather than using fewer folds with multiple held-out recordings? Given the substantial performance degradation under LOSO, which evaluation setting better reflects practical deployment?
- Comparison with Lightweight Neural Networks: Why does the paper not compare against lightweight temporal neural networks commonly used for continuous EMG/EEG signal recognition, such as sliding-window-based 1D-CNNs, TCNs, or GRUs? These methods can directly model temporal dependencies and produce continuous predictions. Could the authors clarify what advantages the proposed event ownership mechanism provides over such approaches?
- Effectiveness of the Core Learning Method: The main methodological contribution is finite-state action-path marginalization, yet the learned ACED-P performs worse than the hand-designed ACED-D, and ACED-P18 underperforms hysteresis in Table 1. What evidence demonstrates that the proposed learning-based formulation provides advantages over simpler rule-based decoding?
- Event Time versus Commitment Time: Section 5.3 reports that ACED-D's first-correct fraction decreases from 0.715 to 0.542 when evaluated at commitment time. Since commands are irreversible, why is event-time accuracy selected as the primary evaluation metric?
- Potential Evaluation Bias: Section 5.6 acknowledges 17 development iterations involving the outer LOWO evaluation results. Could the authors clarify how these iterations influenced the final decoder design and how potential adaptive overfitting was addressed?
- Cross-Subject Generalization: Under LOSO evaluation, ACED-D's first-correct fraction drops to 0.522, and outside emissions exceed the predefined guardrail. How do the authors interpret these results regarding the practical applicability of the proposed framework?
- Dataset Collection and Scale: The proposed dataset contains only five participants, 30 recording sessions, and approximately 2.54 hours of valid EOG signals. What were the main challenges in collecting and annotating the data, and why was the dataset limited to such a small number of participants? How might this limited scale affect the reliability and generalizability of the conclusions?