acceptodds
Withdrawn from ICLR 2027

Adaptive-Group Process Rewards for GUI Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Training graphical user interface (GUI) agents with reinforcement learning suffers from sparse outcome rewards, leading to inefficient learning. Process rewards offer a promising direction for credit assignment to intermediate actions; however, accurately evaluating an action requires identifying evidence across long-horizon GUI trajectories, as its value may depend on information beyond its immediate state and observation. In this paper, we propose an adaptive-group process reward framework that dynamically partitions a GUI trajectory into semantically coherent subgoal groups, enabling process reward estimation at both the action and group levels. At the action level, a critic evaluates each action based on the visual changes and subsequent actions within the group for action correctness and contribution to the subgoal. At the group level, the framework aggregates action-level evaluations to assess subgoal completion and contribution to the overall task, and further refines these assessments by leveraging the context of all groups in the trajectory. The process reward of each action is obtained by combining its action-level evaluation with the refined group-level evaluation. We further augment GRPO's outcome advantage with the proposed process rewards to provide finer-grained guidance for GUI agent training. Experiments on process reward benchmarks and downstream RL training demonstrate the effectiveness of the proposed framework.

Primary area:
reinforcement learning
Keywords:
Adaptive-Group, Process Reward, GUI Agent

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4hd9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an adaptive-group process reward framework for GUI agent RL training. The framework dynamically partitions a trajectory into subgoal groups using task instructions and agent responses, then evaluates each action at two levels: action-level and group-level, with a final refinement stage that revisits group scores in full-trajectory context. The resulting process reward shapes GRPO's step-level advantage. Experiments on show improvements over outcome-only GRPO, with all five critic stages conducted by a frozen Qwen3.6-27B.

Critical strengths
  1. The adaptive grouping mechanism addresses a limitation of existing process reward methods. Fixed-window and milestone-based approaches either miss long-range evidence or fail to credit valid alternative paths, while dynamic grouping based on agent responses naturally aligns the evaluation scope with the executed trajectory's semantic structure.

  2. The two-level evaluation design is well-motivated. By combining action-level execution correctness with group-level subgoal contribution through a product aggregation, the framework ensures that both local correctness and global usefulness are jointly necessary.

  3. The paper validates the framework across three complementary dimensions: process-reward accuracy, step-level error detection, and downstream RL training.

Critical weaknesses
  1. All five critics are conducted in a sequence by the same frozen Qwen3.6-27B. If an error occurs at an early stage. For instance, actions from different subgoals are incorrectly assigned to the same group during the grouping phase. How does this propagate to subsequent stages? Is there any mechanism to detect or correct such errors? The experiments and ablations only report end-to-end overall accuracy and results with entire stages removed, without analyzing the robustness of the pipeline to intermediate-step errors.

  2. Action-level scoring evaluates each action using only subsequent actions within the same group as context. However, an in-group action's value may affect other group. For example, an action in group 1 enable a critical step in group 3. The current group-level refinement does consider cross-group dependencies by using later group results to reassess whether an earlier group was reused. If an action in group 1 has a cross-group impact on group 3, is there any mechanism to adjust that action's individual score?

  3. Why only use Qwen3.6-27B as the critic model? What is the effectiveness if choose larger or smaller size model as judger?

  4. The related work section discusses several GUI process reward methods, including GUI-Shepherd, GUI-PRA, ADMIRE, StainFlow, and GUIDE. However, the experimental evaluation includes only SeekJudge as a competing baseline, while the remaining approaches are all general model. This is insufficient to demonstrate the effectiveness of the proposed framework.

  5. In table 3, this paper uses Qwen3.6-27B as the critic, while several baselines CUAjudge, SeekJudge and OSThemis employ smaller critic models (Qwen3-VL-8B and SeekJudge-9B). Without same-size baselines, the method's effectiveness remains unclear. Beside, the precision metric is not better than CUAjudge that use a smaller size critic model.

  6. Please provide ablations experiments on OSworld benchmark.

Questions for authors

refer to weaknesses

Reviewer_aY64OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a process reward method for training GUI agents. It first divides a completed trajectory into groups of actions that appear to serve the same subgoal. A set of critics then scores each action, scores each group, and revises the group scores using the full trajectory. The final reward for an action is the product of its action score and group score. These rewards are added to the GRPO advantage during training. The method is evaluated on CUARewardBench, CUAStepBench, and three OSWorld applications.

Critical strengths
  • Grouping actions by the subgoal actually followed by the agent is a reasonable way to handle long GUI trajectories. The action-level and group-level scores capture different types of errors.

  • The paper evaluates both reward quality and downstream RL. It also compares the same Qwen3.6-27B critic under other prompting methods and provides useful ablations of grouping, scoring, refinement, and aggregation.

Critical weaknesses
  • The experimental comparison misses some closely related recent methods. In particular, ADMIRE and StainFlow also provide adaptive process rewards for GUI-agent RL, while GUIDE uses semantic trajectory segmentation for GUI evaluation. Direct comparisons with ADMIRE and StainFlow, together with a clearer discussion or segmentation comparison with GUIDE, would better establish the contribution of the proposed adaptive grouping.

  • The method uses several Qwen3.6-27B critic calls for each trajectory, while the baselines may use less computation. The paper reports throughput, but not the number of critic calls, tokens, or total compute. It is therefore difficult to tell whether the improvement comes from adaptive grouping or additional critic computation. Moreover, the 3.1-point gain over SeekJudge on OSWorld is reported without variance across runs. The training and test splits and the number of training trajectories are also unclear.

Questions for authors
  • How many critic calls and tokens does each method use per trajectory? Can the authors provide a compute-matched baseline with the same Qwen3.6-27B budget?
Reviewer_rZpeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes adaptive-group process rewards for GUI agents. It partitions executed trajectories into semantic subgoal groups, evaluates execution and contribution at both action and group levels, and refines group judgments using cross-group evidence. Action and group scores are multiplied to shape GRPO advantages. Experiments cover two reward benchmarks and downstream training on three OSWorld applications.

Critical strengths

Separating execution correctness from task contribution is a sensible choice for GUI evaluation. It provides a way to distinguish correctly executed but unhelpful actions from useful progress within unsuccessful trajectories. The prompts and scoring table make the intended procedure easy to inspect.

I also appreciate the evaluation of both reward accuracy and downstream training. The reported CUARewardBench gains are 10.5–14.0 percentage points over the same judge backbone under the official templates. On OSWorld, the three-run mean improves by 4.1 points over outcome-only GRPO and 3.1 points over SeekJudge. These results suggest practical value, although the evaluation details below need clarification.

Critical weaknesses

W1. The evaluation details limit confidence in the reported gains. Under a single pooled binary evaluation, the five CUARewardBench metrics for Ours match TP/TN/FP/FN counts of 158/116/41/22, totaling 337 samples. The same-backbone OpenCUA-Reflector row matches 346 samples, which is the benchmark’s reported size (CUARewardBench). A single evaluation over all 346 samples cannot produce 81.3% accuracy rounded to one decimal place: 281 correct predictions give 81.2%, while 282 give 81.5%. Different aggregation procedures or reporting errors could explain this, but the actual sample counts and evaluation procedure need to be clarified.

For OSWorld, the training/test task split, total training budget, checkpoint selection, and meaning of “3-run mean” are also unclear. These omissions make it difficult to assess the robustness of the 3.1-point gain over SeekJudge and whether the evaluation covers unseen tasks.

W2. The contribution beyond existing adaptive evaluation methods is not sufficiently isolated. GUIDE already partitions trajectories into contiguous subtask units using the textual action sequence without screenshots. StainFlow dynamically identifies task stages and constructs evidence windows from relevant steps. Both are cited, but the paper’s broad contrast between prior preset structures and its own adaptive grouping does not adequately reflect this overlap. Although the experiments include SeekJudge and other baselines, they do not directly establish the added value of the proposed scoring components relative to these closely related mechanisms.

The aggregation ablation also leaves a threshold issue unresolved. Section 4.1 reports a threshold of 0.4, while Table 6 uses an “equal sum” without specifying its normalization or separate threshold calibration. For unchanged action/group scores of 0.8 and 0.4, multiplication gives 0.32, whereas summation gives 1.20 and averaging gives 0.60. At a threshold of 0.4, either additive form produces a different classification. Threshold-sensitivity results and a controlled mechanism comparison would make the contribution more convincing.

Questions for authors

Q1: What are the actual sample counts and confusion matrices for Tables 1, 2, 5, and 6? How were metrics aggregated, and how were missing or invalid predictions handled?

Q2: Please specify the OSWorld task split, training budgets, and checkpoint-selection rules. Does “3-run mean” refer to independent training runs or repeated evaluations, and what are the individual results?

Q3: Could the authors provide threshold-sensitivity results and a controlled comparison isolating the added scoring components beyond adaptive trajectory segmentation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.