- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies the problem of task-driven, parallel scanpath modelling, in models that use spatial fixation maps. The key question is how the spatial distribution for each fixation should be used to predict scanpath locations at training and inference. The core contribution is a set of analyses with a controlled set of models that re-use the same shared representations but differ in the way fixation coordinates are regressed or sampled. What they show is that by better using the spatial distribution during inference much of the gap with more complex models (like their RCSD).
However, at its current state, my overall assessment is that of a paper lacking both writing and experimental depth, and as such not meeting the bar for acceptance at ICLR: the contributions and motivations are not clear, the presentation is quite shallow and repetitive at times, the model names are hard to follow through the text, and while nicely controlled, the key experiments are quite simple in nature and performed only on the authors’ models, e.g. without trying to retrieve the same mechanism from the existing baselines.
Critical strengths
- The experiments comparing their model classes are clean and well-isolated.
- The evaluation protocol is careful, also with respect to image splits and avoidance of test-set leakage, and with re-training of the external baseline and SOTA models.
Critical weaknesses
-
The core contribution is unclear, and the writing itself does not help in highlighting it. The main result that the author show, i.e. that using spatial supervision in scanpath prediction is better than a regression head to coordinates that bypasses it, seems un-surprising. Models like DeepGaze and some of the reported baselines already operate similarly. While it is helpful to see the effect, this is shown only for the authors’ models, which are not SOTA on any metric. Does this affect also other models and could be used to improve them?
-
Despite the simplicity of its results, I found the paper quite hard to follow. Results are reported but not deeply interpreted. The central claims are restated multiple times in the abstract, introduction, discussion and conclusion but they are never fully fleshed out in terms of applications, consequences, effects on other models or impact to the broader field. The space used to repeat the claims could have been used e.g. to better motivate the paper, give definition for the metrics (for which also citations are missing), better interpret the results, and so on. The naming of the models is also initially quite hard to follow, as multiple models are present and similarly named. This is mostly evident in the abstract, which I think could and should probably be re-written to convey the main message of the paper without naming the different abbreviations several times. In comparing the models, and helping keeping track, a single table mapping each variant to its training losses, inference pathway and inputs would have helped a lot.
A more minor point is that the paper references an appendix, which I cannot see in the current submission pdf.
Questions for authors
See above point 1.