- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StimEM, a stimulus-conditioned foundation model for learning the representation of eye-tracking signals. The method first jointly trains an FSQ-based ocular tokenizer through waveform reconstruction and trial-level behavioral feature prediction. It then freezes the tokenizer, uses the discrete code as the masked modeling target, and pre-trains a stimulus-conditioned Transformer. The paper uses approximately 4,255 participants and 500 hours of eye-tracking data for pretraining and evaluates the model on six clinical classification tasks. Additionally, the authors conducted extensive ablation experiments to validate the effectiveness of the method.
Critical strengths
-
In the field of eye-tracking modeling, a pretraining corpus comprising approximately 500 hours of data and 4,255 participants is quite substantial. This in itself represents a significant contribution to the field.
-
The authors conducted experiments across multiple tasks, as well as thorough ablation studies, to validate the effectiveness of the method described in this paper.
Critical weaknesses
-
In Sec. 4.1, the authors explicitly state that the pretraining split and the downstream split are constructed independently. Appendix A.3 directly refers to the current protocol as the “transductive classification protocol.” In other words, although the downstream train, validation, and test sets are subject-disjoint internally, there is no guarantee that the downstream test subjects are excluded from the tokenizer training and masked pretraining. I would like the authors to first specify the exact number or proportion of test subjects from the six downstream tasks that were seen during pretraining.
-
In Section 4.4, the authors analyze data scaling using pretraining data fractions of 12.5%, 25%, 50%, and 100%. However, Appendix C.1 explicitly states that different configurations share the same frozen 12-layer FSQ tokenizer. Therefore, I would like the authors to clarify: How much pretraining data was used to train this tokenizer itself? If it was trained on 100% of the corpus, then even if the masked pretraining encoder only sees 12.5% of the data, its targets still come from a teacher/tokenizer that has already seen 100% of the data.
Since this is not my area of expertise, I hope AC will take my comments into consideration as appropriate.
Questions for authors
see weaknesses.
Since I'm not familiar with this topic, I will maintain my rating score.