- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses on graphical user interface (GUI) agents, and the authors introduce AsyPO: an asymmetric post-training methodology with segment-level reasoning guidance. To the best of my understanding, the proposed method applies GRPO to action tokens while using OPSD for segment-level reasoning guidance. The authors test their proposed method on two models (Qwen3-VL-4B-Thinking and Qwen3-VL-8B-Thinking) on two datasets (OSWorld and OmniACT), demonstrating improvements over the provided baselines.
Critical strengths
S1) Strong suite of baselines: I appreciate that the authors took the time to test a variety of baselines. In my opinion, these are pretty helpful for comparison
S2) Informative Appendix: I believe that the authros ran many helpful experiments to understand their method better, especially when looking at the various ablations provided in the appendix
Critical weaknesses
Critical
W1) No uncertainty reported: Unfortunately, the authors report no uncertainty or confidence in the main manuscript. In their defence, they provide it in the Appendix, but only for Table 7, which helps clarify the method. Their method performs better than SDAR, but there are several instances of overlapping standard deviations (I assume? I did not find a description). Naturally, I don't expect the authors to run their experiments multiple times, but I would at least encourage them to report, e.g. a bootstrapped confidence interval over the test set, as that can already give us an indication of how much spread there is in the results, and how statistically meaningful they are.
Soft
W2) Performance of the method seems very reliant on Hyperparameters: Looking more closely at the ablations in the Appendix, I see that performance on OmniACT can range from 33.1 to 38.2 (the reported score). In this case, most baselines outperform 33.1, except base and OPSD. To me, this suggests that performance is quite dependent on the right hyperparameter choice. This is not necessarily a bad thing, but I wonder if the same amount of effort went into finding good hyperparameters for the baselines, and how much variability there is in the results (re-emphasising the point above)
Questions for authors
Q1) Hyperparameters chosen for different baselines: How did you choose the baselines' hyperparameters to ensure a fair comparison with AsyPO?
Q2) Trajectory-Level Baselines for OmniACT: I might have missed something, but why are they not reported for OmniACT?