- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether GUI agents can recognize unsafe web environments and refuse risky interactions. The authors introduce GUI-WebMinefield, a benchmark of annotated webpage screenshots covering multiple risk categories. They propose Constraint-Conditioned Optimization, which augments ordinary GUI training examples with synthetic constraints that either permit execution or require refusal, without training on harmful webpages. They further introduce Operation Advantage Calibration, which separately normalizes rewards for format, action, and coordinate correctness and assigns the resulting advantages to the corresponding token spans. Experiments compare the combined method against a GUI-R1 baseline and a GRPO-based constraint-training variant, evaluating both safety and general GUI capabilities.
Critical strengths
- I think GUI-WebMinefield is a useful GUI safety benchmark of annotated risky webpages with task and action annotations. Unlike execution-focused benchmarks, it tests whether agents refuse risky interactions in harmful environments.
- An interesting approach to safety generalization. Constraint-conditioned training on ordinary GUI examples with synthetic constraints improves safety on harmful webpages without exposure to such environments during training.
Critical weaknesses
-
OAC’s advantage allocation may introduce insufficiently justified bias. Restricting rewards to selected token spans may omit cross component gradients arising from autoregressive dependencies, while independent normalization changes reward weights. The paper should clarify the resulting objective and when this trade-off is beneficial.
-
The scale experiments offer limited evidence of generalization. With 3,570 training examples (1,000 with synthetic constraints) and 1,600 benchmark screenshots across seven categories, the results do not establish generalization to broader environments, unseen constraints, or diverse interaction contexts.
Questions for authors
- Can the authors establish when the tradeoff introduced by OAC is beneficial? A theoretical characterization, diagnostic metric, or targeted ablation would help justify the design beyond empirical gains.
- The training data and synthetic constraints appear limited relative to the diversity of risk categories and interaction contexts in the benchmark. Can the authors provide data-scaling experiments to show how performance changes with more examples, constraint types, and environments?
3 comments
Thank you for the detailed response and the additional data-scaling experiments. I will maintain my weak acceptance recommendation.
Thank you for your helpful suggestions and comments. We are glad that our responses could address your concern. If you have any further questions, we are ready to provide clarification at any time.
We thank the reviewer for the constructive suggestions.
Cross-component Gradients. OAC changes advantage allocation and thus the resulting gradient. When reasoning is present, its tokens receive joint feedback from both components. In this example,
\qquad A_{\mathrm{think}}=\frac{r_{\mathrm{think}}-\mu_{\mathrm{think}}}{\sigma_{\mathrm{think}}+\epsilon}.$$ With or without reasoning, component updates remain coupled through the autoregressive context and shared parameters. For fixed rollout advantages and span masks, the gradient of the policy term in the objective below, evaluated at $\theta=\theta_{\mathrm{old}}$ where clipping is inactive and excluding KL, is $$g_{\mathrm{OAC}}=\left.\mathbb{E}_{\pi_{\mathrm{old}}}\left[\operatorname{Mean}_{t:m_t=1}A_t\nabla_\theta\log\pi_\theta(y_t\mid x,y_{\lt t})\right]\right|_{\theta=\theta_{\mathrm{old}}}.$$ OAC assigns advantages by output field, changing the aggregate-reward gradient while retaining autoregressive dependence. We provide further theoretical analysis in Response 1 to Reviewer kMZ7. **Normalization.** OAC computes action and argument advantages separately. Specifically, $A_a=(r_a-\mu_a)/(\sigma_a+\epsilon)$ and $A_p=(r_p-\mu_p)/(\sigma_p+\epsilon)$, with statistics computed within each rollout group. Each field receives feedback based on its own relative correctness and variability, while reasoning receives the normalized joint reward above. This changes the relative weighting of component feedback. These normalized advantages $A_t$ used in the gradient above: $$A_t=A_{\mathrm{fmt}} +\mathbf1_{t\in S_a}A_a +\mathbf1_{t\in S_p}A_p +\mathbf1_{t\in S_{\mathrm{think}}}A_{\mathrm{think}},$$ The full objective, maximized with PPO clipping and KL regularization, is $$\mathcal{J}_{\mathrm{OAC}}=\mathbb{E}_{\pi_{\mathrm{old}}}\left[\operatorname{Mean}_{t:m_t=1}\min\\!\left(\rho_t A_t,\\,\operatorname{clip}(\rho_t,1-\delta,1+\delta)A_t\right)\right]-\beta\mathcal{L}_{\mathrm{KL}},$$ where $\rho_t=\pi_\theta(y_t\mid x,y_{<t})/\pi_{\mathrm{old}}(y_t\mid x,y_{<t})$, $S_d$ denotes a component’s token span, and $m_t$ masks valid response tokens. The reasoning term is omitted when no reasoning span is used. > **Response to Weakness 2 & Question 2: Generalization and Data Scaling** **Generalization to Unseen Constraints and Environments.** Our existing experiments examine generalization across environments, constraint formulations, interaction contexts, and model backbones. - First, our training and evaluation data cover web, desktop, and mobile environments. These benchmarks span different operating systems and substantially different task objectives, from explicit action instructions to long-horizon planning. They also use different interaction schemes and action definitions. - Second, GUI-WebMinefield tests all models on real harmful webpages that are never used in training, directly assessing transfer to unseen harmful environments. - Third, we evaluate grounding, low-level execution, high-level execution, OSWorld long-horizon tasks, adversarial instructions, and benign boundary cases. **These evaluations test the method’s generalization across different unseen settings.** The results show overall improvements in safety and usability. - We also examine the effects of constraints. Table 20 (revised paper) shows that stricter constraints further improve safety, while Table 29 shows that greater diversity in training constraints improves safety alignment. **Test-time constraints differ from training constraints in content, style, and topic.** **Data Scaling.** We further examine how the scale of training data affects safety alignment and task performance by varying the number of training examples with synthetic constraints. | Constraint Examples | w/o Safety: SR | w/o Safety: RR | w/o Safety: ScreenSpot | w/ Safety: SR | w/ Safety: RR | w/ Safety: ScreenSpot | |---|---:|---:|---:|---:|---:|---:| | 250 | 79.94 | 0.00 | 86.24 | 66.69 | 20.38 | 81.68 | | 500 | 80.56 | 0.00 | 86.87 | 63.69 | 21.50 | 84.91 | | 1,000 | 84.94 | 0.00 | 87.8 | 64.44 | 24.44 | 85.8 | | 2,000 | 80.38 | 0.00 | 87.42 | 60.56 | 27.44 | 86.64 | We expanded the training data by adding 600 constraint-conflicting and 400 constraint-aligned examples while keeping the unconstrained examples unchanged. Due to time constraints, the new examples were not individually reviewed by humans. The results show that increasing training data can improve safety alignment and task capability. The model trained with 2,000 constraint-augmented examples refuses risky requests more often when safety constraints are provided. Further gains in usability are limited, as our method **focuses on safety-constrained decisions in existing GUI scenarios** without introducing additional GUI task types.