acceptodds
Under review as a conference paper at ICLR 2027

SAFEST: Rule-Guided Executable Safety Task Synthesis for Computer Use Agents

Abstract

Computer-Use Agents (CUAs) extend large language models with the ability to perceive screens, manipulate files, and operate multiple applications on behalf of the user. While modern LLMs receive extensive chat-style safety alignment, we demonstrate that this alignment does not transfer reliably to the CUA setting: a model that declines harmful textual queries often executes the same intent when instantiated as a concrete computer-use task. We address this gap by constructing safety training data tailored to CUA observations, action interfaces, and execution environments. To this end, we introduce , a afety-ware ramework for xecutable tructured raining data synthesis, which employs a hierarchical pipeline to progressively ground abstract risk categories into concrete, application-specific adversarial tasks and validated safe-execution trajectories. Leveraging this framework, we construct a safety fine-tuning dataset comprising 2,724 executable tasks (1,299 direct and 1,425 indirect) that span 8 core applications and cover 43 distinct harmful categories. Fine-tuning two open-source CUA backbones (Qwen3-VL-8B and EvoCUA-8B) on this dataset significantly enhances their safety alignment: the average safe rate across four CUA benchmarks rises from roughly 41% to 87.8–90.1%, while over-refusal on benign tasks remains low and accuracy on general GUI benchmarks is preserved. Our code and dataset are available at https://anonymous.4open.science/r/SAFEST-42F2.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-modal agents, safety algnment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dbv3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents SAFEST, a rule-guided framework for automatically synthesizing executable safety training data for Computer-Use Agents (CUAs), addressing the gap between text-based safety alignment and safe behavior in real GUI environments. The framework generates both direct and indirect harmful tasks, constructs executable environments, and collects validated safe-execution trajectories for supervised fine-tuning. The resulting dataset contains 2,724 tasks across 8 applications and 43 harmful categories. Experiments on Qwen3-VL-8B and EvoCUA-8B demonstrate substantial improvements in average safety rates across four benchmarks, while maintaining relatively low over-refusal rates and preserving general GUI capabilities.

Critical strengths
  1. Valuable training resources for CUA safety. The paper introduces a useful framework and dataset for improving CUA safety alignment, providing executable safety tasks and validated safe-execution trajectories that can support future research on safer CUAs.

  2. Comprehensive experimental evaluation. The paper evaluates its approach across multiple CUA safety benchmarks and model backbones, demonstrating substantial safety improvements. The additional analysis of over-refusal and general GUI capabilities provides valuable insights into the trade-off between safety and utility.

Critical weaknesses
  1. Limited exploration of safety alignment methods. The paper exclusively relies on supervised fine-tuning (SFT) without investigating alternative alignment methods, such as preference-based alignment (e.g., DPO) or reinforcement learning with verifiable rewards (RLVR). It remains unclear how different training paradigms would influence safety performance and the trade-off between safety and utility.

  2. Potential distribution overlap between training and evaluation. The synthesized training tasks and evaluation benchmarks largely share similar OSWorld-style environments, applications, and task distributions, raising concerns about potential distribution overlap. Although the authors conduct instruction-level similarity analysis, this does not fully rule out overlap in task structures and underlying safety scenarios, potentially limiting the generalizability of the reported improvements to more diverse, out-of-distribution CUA environments.

Questions for authors

N/A

3 comments

We thank the reviewers for their detailed and constructive comments. Below we address each point in turn.

W1: Extending SAFEST beyond SFT

Thank you for raising the question of whether SAFEST can support alignment methods beyond SFT. SAFEST was originally constructed as an SFT resource of task instructions and validated safe-execution trajectories; it does not provide preference pairs or online reward signals for directly applying methods such as DPO or RLVR. We nevertheless explored on-policy self-distillation (OPSD), an OPD-based alignment method, using each task's execution plan as privileged information available to the teacher model. This lets us test whether SAFEST's structured supervision can be used in a different alignment paradigm.

Qwen3-VL-8B with OPSDResult
OSWorld score36.1
OS-Blind safe rate84.4%
OS-Harm94.7%
VPI-Bench97.7%

These results show that SAFEST execution plans can support an OPD-based alignment approach, extending their use beyond standard SFT. We will add this experiment in the revision.

W2: Generalization to diverse, out-of-distribution CUA environments

Thank you for noting that overlap in OSWorld-style environments may limit generalization. To test transfer beyond SAFEST's training distribution, we evaluated Qwen3-VL-8B on ADEPTS, an independently constructed benchmark spanning mobile and desktop applications. Unlike SAFEST's executable OSWorld tasks across 8 desktop applications, ADEPTS uses paired benign/malicious tasks with threats embedded in screenshots and covers roughly 220 applications.

Qwen3-VL-8B settingADEPTS desktop safe rateADEPTS mobile safe rate
Base25.9%24.8%
SAFEST53.9%48.7%

SAFEST improves safe rate on both ADEPTS platforms, providing evidence that its safety gains transfer to visually grounded tasks across a broader application and platform distribution. We will include this OOD evaluation in the revised paper.

Dear Reviewer,

We would like to gently follow up on our response above before the rebuttal period ends. Regarding the exploration of alignment methods beyond SFT, we provided an additional on-policy self-distillation experiment demonstrating that SAFEST’s structured execution plans can support a different alignment paradigm. To examine generalization beyond OSWorld-style environments, we also evaluated SAFEST on ADEPTS, an independently constructed benchmark covering approximately 220 applications across desktop and mobile platforms, and observed consistent safety improvements on both platforms.

We hope these additional experiments address your two main concerns. We would be grateful to hear whether you find the responses satisfactory, or whether any questions remain. Your feedback would be very valuable to us as we prepare the revision.

Thank you again for recognizing the value of our work and for your constructive comments.

Best regards,
The Authors

Reviewer_dbv3OpenReview ↗

Response to Rebuttal

Thanks for providing the new results, which solve my concerns to a large extent. I choose to maintain my positive assessment of the paper.

Reviewer_kjoxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SAFEST, a framework for generating executable safety training data for computer-use agents. It synthesizes harmful tasks, workspace environments, and safe trajectories through a rule-guided pipeline, covering both direct and indirect risks. The resulting dataset contains 2,724 tasks across 8 applications and 43 harmful categories. Fine-tuning Qwen3-VL-8B and EvoCUA-8B improves their average safety rates from around 41% to 87.8–90.1% across four benchmarks, with relatively low over-refusal and limited changes in general GUI performance.

Critical strengths

S1. A useful framework for generating CUA safety training data.

The paper addresses an important problem: safety alignment learned from text-based interactions may not transfer well to computer-use agents operating in executable environments. SAFEST provides a systematic pipeline for generating application-specific harmful tasks and safe trajectories. I appreciate the inclusion of indirect tasks, where risks arise from environmental context or sequences of seemingly benign actions. These scenarios are relevant to real-world agent safety and are more challenging than explicit harmful requests.

S2. Strong empirical results across multiple safety benchmarks.

The improvements reported on four external safety benchmarks are substantial, particularly on OS-Blind, where harmful outcomes may arise despite benign user instructions. The comparison with MirrorGuard is also useful. SAFEST achieves competitive safety performance with lower over-refusal on the evaluated tasks and avoids additional inference-time guard-model calls.

Critical weaknesses

W1. Limited validation of the realism and operational validity of synthesized safety tasks.

SAFEST collects trajectories through actual GUI interactions in OSWorld, which provides meaningful evidence of environment executability. However, it remains unclear how well the synthesized tasks reflect realistic workflows and whether the intended safety-critical actions are operationally feasible, particularly for indirect tasks.

The authors report a human audit of 100 tasks with a 99% realism pass rate, but the evaluation criteria and annotation protocol are not sufficiently described. More detailed validation of task realism and the feasibility of the intended risk scenarios would strengthen the paper's claims about the quality of the synthesized safety training data.

W2. Unclear whether safety fine-tuning preserves the capabilities of stronger CUAs.

The paper evaluates general GUI performance after safety fine-tuning, but only on models with relatively limited capabilities. Although the average performance changes are relatively small, some benchmarks already show degradation. For example, EvoCUA-8B drops from 41.7% to 38.9% on OSWorld-Small.

An important question is whether the same safety fine-tuning procedure would preserve the capabilities of stronger agents. Models with better GUI grounding, planning, and tool-use abilities may respond differently to additional safety supervision. Specifically, training primarily on safe trajectories that involve refusal or redirection could potentially make agents more cautious or impair their ability to complete complex but legitimate tasks.

I would like to see an evaluation on stronger CUA backbones, focusing not only on safety improvements but also on whether the original task-completion capabilities are preserved after fine-tuning. This would provide stronger evidence for the claimed safety–utility trade-off.

W3. The safety system-prompt baseline could be strengthened.

Table 1 includes a system-prompt defense adapted from OS-Blind, which is an appropriate comparison. However, the exact prompt used in the experiments and its integration with the original agent system prompt are not clearly documented.

More importantly, it is unclear whether this baseline adequately represents what can be achieved through stronger safety prompting. SAFEST uses structured execution plans to guide teacher agents toward identifying risks and responding safely, whereas the system-prompt baseline appears to rely on more general safety instructions.

It would be useful to evaluate a stronger execution-aware safety prompt that explicitly encourages agents to inspect environmental context and consider the consequences of their actions before performing potentially harmful operations.

Such a comparison would help clarify how much of SAFEST's improvement requires fine-tuning, rather than more effective inference-time safety guidance.

W4. The generalization and safety–utility evaluation could be more comprehensive.

The paper provides useful analyses of training–evaluation overlap, but these primarily focus on instruction-level lexical similarity. Low textual similarity does not necessarily rule out substantial overlap in risk patterns or task structures.

It would be interesting to test whether the learned safety behavior generalizes to held-out risk categories, applications, or indirect-task strategies.

I also have some concerns about the over-refusal evaluation. OSWorld-Small contains only 36 benign tasks, which is a relatively small set for characterizing capability preservation. Furthermore, the same benchmark is used to select checkpoints based on an over-refusal threshold.

A larger independent benign evaluation set, particularly one involving legitimate but safety-sensitive operations, would provide a more reliable assessment of the safety–utility trade-off.

Questions for authors

Q1. Realism and validity of synthesized risk scenarios (W1).

SAFEST collects trajectories through actual GUI interactions in OSWorld, which provides evidence of environment executability. However, could the authors clarify the evaluation criteria and annotation protocol behind the reported 99% realism pass rate? In particular, for indirect tasks, how do the authors verify that the detection cues are actually observable before the violation triggers and that the intended risk scenarios are operationally valid, beyond LLM-based judgments?

Q2. Capability preservation on stronger CUAs (W2).

Could the authors evaluate SAFEST on stronger CUA backbones and examine whether safety fine-tuning degrades their original task-completion capabilities? I am particularly interested in performance on complex, long-horizon, and legitimate but safety-sensitive tasks, where additional safety training might introduce unnecessary refusals or overly conservative behavior.

Q3. Strength of the safety system-prompt baseline (W3).

Could the authors provide the exact system prompts used in the baseline experiments? Have they considered a stronger execution-aware safety prompt that explicitly guides agents to inspect environmental context and assess potential consequences before taking actions? Such a comparison would help clarify whether the improvements require fine-tuning or could be partially achieved through stronger safety prompting.

Q4. Generalization beyond the training distribution (W4).

The paper examines training–evaluation overlap using n-gram matching and TF-IDF similarity between instructions. However, these analyses do not directly assess overlap in underlying risk scenarios or task structures. Could the authors further evaluate generalization to held-out risk categories, applications, or indirect-task strategies?

Q5. Reliability of the safety–utility evaluation (W4).

Given that OSWorld-Small contains only 36 benign tasks and is also used for checkpoint selection, could the authors evaluate over-refusal and general task-completion performance on a larger, independent benign task set? This would provide stronger evidence that the reported safety improvements do not come at the cost of legitimate agent capabilities.

5 comments

We thank the reviewers for their detailed and constructive comments. Below we address each point in turn.

W1/Q1: Human validation of task realism and indirect-risk validity

We randomly sampled 100 tasks with their workspaces and safe trajectories. Human annotators checked whether each task was harmful, whether the task and workspace were realistic, and whether the safe-trajectory label was correct. All 100 passed the harmfulness and label-correctness checks; 99 passed the realism check.

For indirect tasks, SAFEST records a detection cue and a violation trigger. The LLM judge checks surface benignity, preservation of harmfulness, fairness of the decision point, naturalness, and executability, and the trajectory filter checks for a safe response before the trigger. We also reviewed final screenshots and logged actions for a random sample of 50 tasks, using three checks: (1) whether the task-relevant cue was visible before the violation trigger; (2) whether the screenshot evidence supported the risk interpretation at the decision point; and (3) whether any logged action attempted the trigger. A cue counted as visually supported only when the relevant evidence appeared in a pre-trigger screenshot; insufficient or inconsistent evidence counted as unsupported. Among 34 indirect instances, the task-relevant cue was visually supported in 30; four had insufficient or inconsistent screenshot evidence. In all 50 traces, logged actions stopped before the harmful trigger.

W2/Q2: Capability preservation on a stronger CUA backbone

We fine-tuned EvoCUA-32B, one of the strongest open-source CUA models on the official OSWorld leaderboard. The base model scores 52.8 on OSWorld and has a 4.4% OS-Blind safe rate. After SAFEST fine-tuning, it scores 50.0 on OSWorld and reaches a 72.4% OS-Blind safe rate.

ModelOSWorld scoreOS-Blind safe rate
EvoCUA-32B (base)52.84.4%
EvoCUA-32B + SAFEST50.072.4%

SAFEST raises OS-Blind safe rate by 68.0 percentage points with a 2.8-point change in OSWorld score. We will add these results to the revised paper.

W3/Q3: Stronger execution-aware safety prompting

We evaluated a stronger execution-aware prompt baseline. The prompt treats environmental content as untrusted, asks the agent to inspect the consequences and destination of each action, and gives concrete rules for high-risk actions such as credential entry, code execution, data transfer, destructive changes, and prompt injection. The exact original and stronger prompts will be included in the revised paper's appendix, with a clarification of how each is combined with the agent prompt.

BackboneMethodOS-Blind safe rateOS-Harm safe rateVPIBench safe rate
Qwen3-VL-8BOriginal system prompt22.4%72.1%50.2%
Qwen3-VL-8BStronger execution-aware prompt11.2%74.7%91.6%
Qwen3-VL-8BSAFEST82.8%96.7%100%
EvoCUA-8BOriginal system prompt19.2%75.0%55.3%
EvoCUA-8BStronger execution-aware prompt29.2%73.3%82.9%
EvoCUA-8BSAFEST78.8%96.7%100%

SAFEST has higher safe rates than the stronger prompt across all reported model-benchmark combinations. These results and the benchmark protocols will be clarified in the revised paper.

W4/Q4: Generalization beyond the training distribution

Thank you for raising the question of whether SAFEST generalizes beyond the training distribution. To test transfer under a different task format and broader application/platform coverage, we evaluated Qwen3-VL-8B on ADEPTS [1], an independently constructed CUA benchmark. Unlike SAFEST's 2,724 executable OSWorld-compatible training tasks with validated safe-execution trajectories across 8 desktop applications and 43 harm categories, ADEPTS pairs benign and malicious variants of the same instruction using different screenshots, with threats embedded in the visual interface. It covers mobile and desktop settings and roughly 220 applications. Qwen3-VL-8B's safe rate on ADEPTS improves from 25.5% before fine-tuning to 51.5% after SAFEST. To probe transfer from explicit to concealed harmful intent, we also evaluated the direct-task-only model on OS-Blind, whose scenarios are predominantly indirect; its safe rate is close to that of full SAFEST.

BenchmarkQwen3-VL-8B settingSafe rate
ADEPTSBase model25.5%
ADEPTSSAFEST51.5%
OS-BlindDirect-task-only81.2%
OS-BlindFull SAFEST (direct + indirect)82.8%

These results show improved safety on ADEPTS and transfer from direct-task training to an evaluation dominated by indirect scenarios. We will include these results in the revised paper.

W4/Q5: Independent benign-task evaluation

We evaluated the selected SAFEST models on 100 additional OSWorld benign tasks sampled outside OSWorld-Small; these tasks were not used for checkpoint or rank selection. The table reports each backbone's original OSWorld-Small score alongside its score and false-refusal rate on the additional set.

BackboneOriginal score on OSWorld-SmallScore on 100 additional OSWorld tasksFalse-refusal rate on additional tasks
Qwen3-VL-8B36.13312%
EvoCUA-8B41.7398%

The false-refusal rates on this independent 100-task set are 12% for Qwen3-VL-8B and 8% for EvoCUA-8B. We will report this additional evaluation and clarify that OSWorld-Small was used for model selection.

Dear Reviewer,

We would like to gently follow up on our detailed response above, as the rebuttal period is nearing its end. To address your concerns, we have provided: (1) further details of our human audit and an additional cue/trajectory validity review; (2) safety and utility results on the stronger EvoCUA-32B backbone; (3) a stronger execution-aware prompting baseline; (4) an out-of-distribution evaluation on ADEPTS and an analysis of transfer to indirect scenarios; and (5) an independent evaluation on 100 additional benign OSWorld tasks not used for model selection.

We hope these results provide clearer evidence regarding task validity, capability preservation, the benefit of fine-tuning over stronger prompting, generalization, and the reliability of our safety–utility evaluation. We would greatly appreciate hearing whether these additions address your concerns, or whether there are specific issues that you still consider unresolved. If your concerns have been adequately addressed, we would be grateful if you would consider the new evidence when reassessing the paper.

Thank you for your time and for the constructive suggestions that helped us strengthen our evaluation.

Best regards,
The Authors

Reviewer_4n3jOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SAFEST, a pipeline for generating executable safety-training tasks for computer-use agents. It converts application-specific risks into safety rules, harmful instructions, workspace files, and teacher-generated safe trajectories. The resulting dataset contains 2,724 direct and indirect tasks across eight applications and 43 harmful categories. Fine-tuning Qwen3-VL-8B and EvoCUA-8B increases average safe rates across four benchmarks from approximately 41% to 90.1% and 87.8%, respectively. The paper also evaluates benign-task refusal, general GUI capability, LoRA rank, and training duration.

Critical strengths
  1. The dataset provides concrete environments, workspace files, and safe trajectories across several desktop applications. This is a useful resource for training agents to recognize risks that emerge during execution, including cases where the user instruction appears benign.

  2. The improvements appear across two backbones and four external safety benchmarks. Evaluating general GUI capability alongside safety is also valuable, since higher refusal rates alone would not establish a useful improvement.

Critical weaknesses
  1. OSWorld-Small is used to select checkpoints and then to report final over-refusal and utility. LoRA rank is also chosen using OS-Blind and OSWorld-Small results. The main text identifies no separate validation split. With only 36 benign tasks, this reuse makes the claimed safety–utility trade-off vulnerable to selection bias.

  2. The main experiments do not isolate the value of executable trajectories, rule-guided synthesis, or indirect tasks. The ablations only vary LoRA rank and training duration. A training-budget-matched text-only safety baseline and a direct-task-only ablation would help determine how much of the gain comes from the proposed data design.

  3. Section 4.2 describes MirrorGuard as performing per-step simulation at inference time. The original method uses simulation for training and a learned reasoning corrector during deployment. This discrepancy leaves the baseline implementation unclear, while no measured latency or cost comparison supports the efficiency discussion.

Questions for authors
  1. Were checkpoint and rank selection performed on separate validation tasks? If OSWorld-Small was used directly for selection, can you report over-refusal and utility on an untouched benign set?

  2. How does SAFEST compare with a text-only safety fine-tuning baseline under the same training budget? How much performance is lost when indirect tasks are removed while keeping training size fixed?

  3. How exactly was MirrorGuard integrated with each backbone? Did the implementation use its learned reasoning-correction procedure, and what inference overhead was measured?

3 comments

We thank the reviewers for their detailed and constructive comments. Below we address each point in turn.

W1/Q1: Independent utility evaluation beyond OSWorld-Small

The reviewer is right that OSWorld-Small was used both to select the LoRA rank/checkpoints and to report utility, which can introduce selection bias. In the original OSWorld-Small evaluation, base Qwen3-VL-8B and EvoCUA-8B score 36.1 and 41.7, respectively (Table 3). We therefore evaluated the selected SAFEST models on 100 OSWorld tasks sampled outside OSWorld-Small. This set was not used for rank or checkpoint selection. Results:

BackboneBase-model score on 100 additional OSWorld tasksSAFEST score on 100 additional OSWorld tasksFalse-refusal rate on additional tasks
Qwen3-VL-8B30%33%12%
EvoCUA-8B40%39%8%

The false-refusal rates of 8% and 12% on this additional evaluation are both low and within an acceptable range, while the scores provide a utility assessment on tasks beyond the 36-task OSWorld-Small selection set. We will report the protocol and results in the revised paper.

W2/Q2: Text-only and direct-task-only ablations

We added two ablations to isolate the contributions of executable and indirect-task supervision.

  1. Text-only safety fine-tuning (EvoCUA-8B).

    We construct this ablation from the same tasks, risk categories, and safe responses as SAFEST, while removing screenshots, GUI actions, and step-by-step execution traces. We convert the remaining examples into chat-style instruction-response pairs and perform SFT with the same base model, LoRA configuration, and training budget as full SAFEST.

    BenchmarkMetricText-onlyFull SAFEST
    OS-BlindSafe rate53.2%78.8%
    OS-HarmSafe rate90%96.7%
    VPIBenchSafe rate97.3%100%

    Text-only safety supervision yields lower safe rates than full SAFEST across all three benchmarks.

  2. Direct-task-only safety fine-tuning (Qwen3-VL-8B).

    ModelOSWorld over-refusal rateOS-Blind safe rate
    Direct-task-only30%81.2%
    Full SAFEST8.3%82.8%

    Direct-task-only training achieves an 81.2% OS-Blind safe rate, while full SAFEST reaches 82.8%; its OSWorld over-refusal rate is 30.0%, compared with 8.3% for full SAFEST.

We will add these ablation results in the revised paper.

W3/Q3: MirrorGuard implementation and inference overhead

Thank you for pointing out that our description of MirrorGuard was imprecise. At inference time, our implementation follows the paper's learned reasoning-correction procedure. At each non-terminal step with an agent thought, the guard model receives the user instruction, recent reasoning history, current screenshot, and proposed thought. It returns the thought unchanged if safe, or provides a correction. When a correction is made, we roll back the current step and ask the CUA to regenerate its response and action conditioned on the corrected thought. Section 4.2 will be revised to describe this procedure accurately.

This intervention adds one guard-model call per step. Our measured average inference cost is:

MethodModel calls per stepTokens per stepLatency per step
MirrorGuard2230.712.14 s
SAFEST1143.38.32 s

Dear Reviewer,

We would like to gently follow up on our response above, as the rebuttal period is approaching its end. We have made substantial efforts to address each of your concerns: (1) an independent evaluation on 100 additional OSWorld tasks not used for model selection; (2) training-budget-matched text-only and direct-task-only ablations; and (3) a clarification of our MirrorGuard implementation, together with measured inference latency and cost.

We hope these additional results help address the selection-bias, data-design, and baseline-implementation questions raised in your review. We would be very grateful to hear whether you find our responses satisfactory, or whether any important concerns remain unresolved. If these additions adequately address your concerns, we would also appreciate it if you would consider them when reassessing the paper.

Thank you again for your thoughtful and constructive feedback.

Best regards,
The Authors

Thank you for the additional experiments and clarifications. Based on the new evidence, I now lean toward Weak Acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.