- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This article proposes an adaptive reinforcement learning training framework for GUI agents. The method combines intermediate checkpoints with a two-phase training strategy that gradually expands from local updates to full task replay. Experiments on MobileGym and CUA-Gym-Hub demonstrate significant improvements in training efficiency and task performance.
Critical strengths
The motivation of the paper is clear, the methods are simple, intuitive, and reasonable, and it has certain practical application value.
The overall experiment is relatively complete, reflecting significant cost reduction and performance improvement.
Critical weaknesses
Intermediate state training and checkpoint recovery have been studied in works such as PivotRL and BPO. However, the paper lacks direct comparisons with these highly relevant baselines.
The core design of the method does not seem to take advantage of the uniqueness of the GUI environment and appears to be directly applicable to a broader range of agent tasks. However, the paper lacks discussion on this point and does not explain why it specifically focuses on the GUI environment.
Questions for authors
See Weaknesses
We thank the reviewer for recognizing the practical value of our method and the improvements in training efficiency and task performance.
As the first author has recently left the institution and no longer has access to the original experimental infrastructure, we are currently unable to conduct additional experiments. We therefore address the concerns using existing evidence and clarify the scope of our work.
W1. Comparisons with PivotRL and BPO.
PivotRL. Its core approach focuses on selecting intermediate states using local reward variance and performing one-step RL, rather than full-trajectory training. We have already included a controlled comparison with this core mechanism in Section 4.2. Our Reward Variance baseline uses the same one-step RL procedure as Action Disagreement, differing only in pivot selection. On MobileGym, Action Disagreement achieves 48.83 vs. 45.70 Held-out Avg@4 under the same 8.2K budget; on CUA-Gym-Hub, the corresponding results are 52.08 vs. 47.50. This directly evaluates the effectiveness of our selection criterion relative to PivotRL-style reward-variance selection, although it is not a full reproduction of PivotRL.
BPO. While BPO also exploits intermediate states, its training strategy differs from ours. BPO selects high-entropy states from policy-generated trajectories and branches suffix rollouts for optimization. In contrast, our framework selects action-disagreement pivots from successful expert trajectories, first improves local decisions through one-step RL, and then progressively expands the rollout frontier using adaptive Local, Suffix, and Root training. Our focus is therefore not simply intermediate-state branching, but jointly determining where to learn and how far to roll out as training progresses.
W2. Why focus on GUI agents?
Our research was motivated by the practical application scenarios and research objectives of our institution, which primarily focus on GUI agents. We therefore designed and evaluated the framework specifically for multi-step GUI interaction tasks, where repeated environment interactions and intermediate-state recovery are important practical concerns.
Nevertheless, the underlying ideas of action-disagreement-based pivot selection and adaptive rollout allocation are not inherently restricted to GUI environments. We believe they could potentially extend to other interactive agent settings, which we consider a promising direction for future work.