- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper uses phase correlation between consecutive screenshots to estimate motion when a GUI action log has a missing entry. Recorded actions are left unchanged. It reports lower tracking errors on synthetic sequences across five checkpoints, better trajectory-log reconstruction on GUI-Odyssey, and lower gesture-referenced displacement error on AITW.
Critical strengths
- The method changes only entries marked as missing. When no entry is marked missing, it follows exactly the same updates as the baseline. This is useful for repairing logs without changing recorded actions.
- The comparisons use the same checkpoints and missing-action masks. The burst tests keep the number of erased actions fixed, and the oracle replaces only missing actions. These controls make the comparisons easier to interpret. The authors also clearly state the limits of screening and the real-data metrics.
Critical weaknesses
- The new part of the method seems limited: it uses standard phase correlation and inserts the estimate only when a record is missing. The paper does not show how often this failure occurs in practice or explain how valid zero-motion records are kept separate from missing ones in Equation (1). Figure 3 and Appendix E measure errors against gesture logs. They do not directly show whether target positions are more accurate on real GUIs.
- The description is missing details needed to reproduce the experiments. Appendix A does not describe the trained frontend, how visual observations update the memory, the training settings, or data generation. Appendix B does not give exact error formulas, averaging rules, bootstrap resampling details, or Holm-adjusted p-values. Appendix E also omits crop and image-size settings, how RAFT flow is reduced to a displacement, and parameters for the registration baselines. These gaps make the reported results difficult to check.
Questions for authors
- Can you provide the tracker and data-generation settings, exact error formulas and recovery window, and sequence-level results for each checkpoint, together with the bootstrap and Holm calculations?
- How do you distinguish missing records from valid zero-motion records? Please give the image preprocessing and baseline settings, and explain what evidence links lower gesture-log error to more accurate viewport or target tracking.