acceptodds
Under review as a conference paper at ICLR 2027

SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents

Abstract

GUI agents automate tasks on digital devices by grounding language instructions in visual interfaces. Existing work has improved GUI action prediction through group-relative reinforcement learning, which assigns credit by comparing rewards across sampled responses. However, with binary rewards, this comparison gives spatially different misses identical credit and yields no relative learning signal when every response fails. To address these limitations, we propose Spatial Credit Assignment (SCA), which uses click geometry to refine group-relative credit. Specifically, for a group with both successful and failed actions, SCA predicts each held-out click's reward from the other clicks and uses the prediction error to refine credit. If every sampled action fails, it instead ranks the clicks by distance to the annotated target; if every action succeeds, it keeps the standard group credit. The resulting credit weights the sampled responses in the policy update; spatial information is therefore used during training, while inference uses the policy alone. We then measure the error and directional alignment of the resulting policy updates relative to the exact return gradient in a controlled synthetic study. Across GUI grounding and offline action-prediction benchmarks, SCA improves performance across grounding domains and achieves the strongest results among reinforcement-fine-tuned models on most action-prediction metrics across the evaluated GUI suites, with consistent gains in our experiments.

Primary area:
foundation or frontier models, including LLMs
Keywords:
GUI, Agent, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Jux7OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Spatial Credit Assignment (SCA), a modification of group-relative reinforcement learning for GUI agents. The motivation is that binary success rewards discard useful spatial information where all-miss groups receive no relative reward signal at all. SCA-Residual uses leave-one-out spatial regression over sampled click coordinates to predict reward and constructs credit from the prediction residual, gated by estimated predictive quality. SCA-Prox assigns relative credit based on distance to the annotated target. The method is evaluated using a Qwen2.5-VL-3B backbone on GUI grounding and offline action-prediction benchmarks.

Critical strengths
  1. The paper identifies a concrete limitation of reward-only group-relative credit that is particularly relevant to GUI grounding. Binary reward treats spatially distinct failures identically, and all-miss groups provide no group-relative learning signal. I find the decomposition into mixed-hit and all-miss regimes conceptually clean. However, many papers tried to solve the same problem.
  2. The paper makes a meaningful effort to analyze the credit estimator rather than relying only on downstream benchmark accuracy.
Critical weaknesses
  1. The main downstream experiments do not isolate the source of the reported improvement. SCA combines two qualitatively different interventions: Residual and Prox. However, the paper does not report matched downstream ablations for Residual-only, Prox-only, or a simple target-distance/dense-reward baseline.

  2. The gains are relatively small. For example, the ScreenSpot-Pro average improves from 25.2 for GUI-R1 to 26.1 for SCA, and many individual grounding improvements are around 1 percentage point. Given the possible difference of the training data, this gain is not significant.

  3. The baselines are too old. Only Qwen2.5-VL is used as backbone model, and later contributions such as GUI-Actor and SE-GUI are not compared.

Questions for authors

Can the authors provide matched downstream ablations for (i) the base binary GRPO objective, (ii) SCA-Residual only, (iii) SCA-Prox only, (iv) simple target-distance/Gaussian reward shaping, and (v) full SCA, using the same initialization, data, evaluator, compute budget, and preferably matched seeds?

Reviewer_4eZsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Spatial Credit Assignment (SCA) to improve reinforcement learning for GUI agents. SCA-Residual uses leave-one-out regression to refine credit for mixed-success groups, while SCA-Prox ranks failed clicks by their distance to the target when all sampled actions fail. The approach is simple and promising, but the current experiments leave questions about baseline comparability and the contributions of the two components.

Critical strengths
  1. The paper addresses a concrete limitation of binary group-relative rewards: failed clicks at different locations receive the same credit, and all-failure groups provide no learning signal. The proposed solution is intuitive and easy to understand.
  2. The synthetic gradient analysis provides useful evidence for the method's design, while the benchmark results show consistent gains across several GUI settings.
Critical weaknesses
  1. SCA is compared mainly against results reported in prior work, without establishing matched training budgets and evaluation protocols. It is therefore unclear how much of the improvement comes from the proposed method itself.
  2. The main results evaluate SCA-Residual and SCA-Prox together, without sufficient ablations to establish their individual benefits. The evaluation also uses a single backbone and recorded GUI states, leaving its effectiveness in closed-loop interaction untested.
Questions for authors
  1. Were the baselines retrained under the same conditions as SCA? If not, can the authors provide a controlled comparison against standard GRPO using the same backbone, data, and training budget?
  2. What are the results of SCA-Residual and SCA-Prox individually? Does each component improve performance over standard GRPO?
  3. Does SCA improve task completion in closed-loop GUI environments, where actions affect subsequent observations?
Reviewer_9QxNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how click locations can provide more informative training feedback for GUI agents. Binary rewards treat failed clicks alike and provide no relative advantage when every sampled action fails. The proposed method, SCA, handles these cases differently. In groups containing both hits and misses, Residual fits a reference reward for each click using the other clicks, then blends normalized reward residuals with the original GRPO advantages through a gate. When all clicks miss, Prox assigns credit according to distance from the annotated target. The model architecture is unchanged at inference. The authors report improvements on GUI grounding and offline action-prediction benchmarks, alongside synthetic experiments examining the error, direction, and scale of policy updates.

Critical strengths
  • Prox addresses a clear problem: binary GRPO provides no relative advantage when all clicks fail, even though their distances from the target differ. Using available annotations to supply geometric feedback is well motivated and does not change the inference-time architecture. This is the more convincing part of the method.

  • The synthetic analysis examines update direction and scale alongside gradient error, helping distinguish useful corrections from merely shrinking the update. The authors also acknowledge that leave-one-out fitting does not make an action-dependent reference unbiased. This makes the mechanism and its limitations easier to assess.

Critical weaknesses
  • The benefit of Residual in mixed groups is not sufficiently established. It assigns credit relative to a predicted reference; the worked example gives a failed click higher final advantage than successful clicks. This does not itself prove an incorrect update, but the paper needs evidence that such behavior helps task success. Leave-one-out fitting does not ensure an unbiased update, and positive gradient alignment in limited synthetic settings does not establish its benefit in GUI training.

  • The evaluation does not isolate Residual from the more directly motivated Prox branch. Matched comparisons among GRPO, Residual-only, Prox-only, and the full method are missing. Since target geometry is available, a key baseline would keep successful clicks at the highest reward and assign lower, distance-based rewards to failed clicks. The dense-reward extension does not provide a sufficient controlled comparison, and published baselines use incompletely matched settings. The overall gains therefore do not establish that Residual adds enough value to justify its complexity.

Questions for authors
  • Under matched data, initialization, training budget, and evaluation, how do GRPO, Residual-only, Prox-only, and full SCA compare with a simple rule that gives successful clicks the highest reward and failed clicks lower rewards based on target distance? Do these controls show a consistent benefit from Residual beyond Prox and direct geometric supervision?

  • How often do failed clicks receive higher final advantages than successful clicks during GUI training, and what evidence connects this behavior to improved task success?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.