- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an online self-distillation method for computer-use agents. A GUI analyzer examines pre- and post-action observations to provide guidance and a step-level value judgment. The policy then scores the same sampled response with and without this guidance, using the token-level log-probability differences and value gating to construct self-distillation signals. These signals are combined with trajectory-level GRPO to optimize the policy.
Critical strengths
- The method uses step-level judgments to regulate self-distillation signals. A value gate checks whether guidance-induced probability shifts align with these judgments, without requiring an additional trainable gating network.
- The paper presents a reasonably comprehensive set of ablation studies, with results supporting the contribution of each core component to overall performance.
Critical weaknesses
- The method relies on sufficiently reliable guidance and step-level judgments from the GUI analyzer, which is fine-tuned on annotations produced by a stronger proprietary expert. However, agreement with the expert does not establish the correctness of those annotations. An independent assessment of annotation quality, together with an analysis of robustness to erroneous feedback, would strengthen the paper.
- Although the paper includes additional prompt-only and GRPO-with-PRM baselines, the comparisons could be broadened to include GiGPO and alternative policy optimization methods such as GSPO, Dr. GRPO, and DAPO. Comparisons under matched backbones, interaction budgets, and training resources would help establish whether the gains extend beyond the chosen GRPO baseline.
- Analyzer construction cost. Constructing the analyzer requires collecting 1,776 trajectories, obtaining expert annotations for 34,188 steps, and performing supervised fine-tuning. A more complete accounting of these upfront costs, together with a data-scaling study, would help assess the method’s practical efficiency and the amount of annotated data actually required.
- Comparison with direct expert feedback. The reported agreement with expert step-level judgments does not establish comparable guidance quality or downstream policy performance. A controlled comparison between using the proprietary expert directly and using the fine-tuned analyzer would clarify the performance–cost trade-off and provide stronger evidence for the effectiveness of distillation.
Questions for authors
- How reliable are the expert annotations, and how robust is the method to incorrect feedback?
- How does the method compare with GiGPO, GSPO, Dr. GRPO, and DAPO under matched settings?
- What is the total cost of constructing the analyzer, and can comparable performance be achieved with fewer annotated samples?
- How does the fine-tuned analyzer compare with direct expert feedback in downstream performance and cost?