- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces semantic rewards for reinforcement learning of GUI agents. The policy describes its next action in natural language, and a vision-language evaluator rewards actions that are functionally equivalent to the target. Its Self-Reward RFT method periodically updates the evaluator from the policy while keeping it frozen between updates. With 6,000 training samples, MAI-UI-2B improves from 56.7% to 59.0% on ScreenSpot-Pro click grounding. The results suggest that semantic feedback complements coordinate-based rewards, although the additional gain from periodic synchronization is small and the evaluation focuses on single-step grounding rather than end-to-end task success.
Critical strengths
- The paper is clear and easy to follow.
- The semantic reward method is simple and effective, improving performance across multiple GUI grounding benchmarks.
- The ablation studies and analyses are thorough.
Critical weaknesses
-
Self-Reward’s effect is unclear. Its gains over a fixed evaluator are only 0.1–0.3 points, with no paired multi-seed evidence that they exceed run-to-run variation. This weakens the paper’s central contribution.
-
The headline comparison is not fully matched. The 6k semantic result is compared mainly with coordinate-reward training on 2k samples. Without a comparable larger-data baseline, it is unclear how much of the gain comes from the reward design.
Questions for authors
Introduction
- The motivation for Self-Reward is unclear. Could a value model, such as the critic used in PPO, achieve similar results?
Presentation
- Using AI to generate diagrams is understandable in the coding-agent era, but the authors should refine the output. Several labels in Figure 1 overflow their boxes. That said, the figure communicates the idea clearly at a glance.
Experiments
- The authors acknowledge that Self-Reward brings only small gains. Could these gains be due to random variation rather than the mechanism itself?
- Were the selected AndroidControl trajectories cleaned? To my knowledge, this dataset contains many cases where the instruction and ground truth are misaligned.
- Could the authors test Self-Reward with an 8B model and more training data to assess whether it scales?