acceptodds
Under review as a conference paper at ICLR 2027

OSGuard: A Benchmark for Safety in Computer-Use Agents

Abstract

Computer-use agents can complete benign user instructions while violating important constraints of the user’s environment. We introduce OSGuard, a dual-granularity benchmark suite for evaluating safety through local, pre-execution guardrail decisions and end-to-end task execution. Its action-level benchmark contains 324 human-annotated examples in which guardrails classify candidate actions as allowed, unrelated, or unsafe given the original instruction and current interface state. Its risk-augmented execution suite contains 45 tasks derived from 40 OSWorld tasks, keeping original instructions unchanged while modifying the environment to introduce state-dependent safety constraints and preserve a safe path to completion. Augmented evaluators retain the original task-success criteria and add explicit state-based safety checks, distinguishing safe completion from nominal success that violates these constraints. On the action-level benchmark, the strongest evaluated guardrail reaches 79.9% accuracy and 0.80 macro-F1, but performance drops substantially on actions from risk-augmented executions. In full-task evaluation, an unguarded agent completes 62.2% of tasks safely while 37.8% result in unsafe completion; adding the strongest guardrail reduces unsafe completion to 33.3% while leaving safe success unchanged. These results show that state-dependent safety constraints remain challenging both to recognize locally and to preserve during end-to-end computer use.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
computer-use agents, AI safety, GUI agents, guardrails, benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_x2YrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OSGuard evaluates benign-task computer-use safety at two granularities: 324 human-labeled pre-execution action judgments and 45 environment-modified OSWorld tasks with state-based safety checks. The best offline guardrail reaches 79.9% accuracy, but online guarding leaves safe success at 62.2% and reduces unsafe completion from 37.8% to 33.3%. The study highlights the gap between local classification and full-task safety.

Critical strengths
  1. Unchanged benign instructions and explicit preservation checks expose collateral harm that nominal OSWorld success would miss.
  2. Connecting action judgments with guarded execution makes the offline–online gap concrete, and the paper transparently reports both corrected and newly terminated tasks.
Critical weaknesses
  1. Benchmark contribution: The useful setting is benign-task collateral harm, but the paper mainly combines existing action classification with environment modifications. Its added coverage over MisActBench, BLIND-ACT, and other computer-use safety benchmarks is described qualitatively. A common-task or failure-category comparison would better establish which consequential failures OSGuard newly captures.
  2. Dataset validity: Construction source is strongly associated with label: all 49 standard-execution items are allowed, and neither proposer-generated source contains allowed items (Table 1). This risks rewarding source-related cues. Annotator numbers, agreement, and adjudication are also unreported, making subjective unsafe/unrelated boundaries difficult to audit.
  3. Execution evidence: Online tests use one executor and one guardrail on 45 tasks. The change from 17 to 15 unsafe completions involves only two fewer tasks, while safe success stays at 28/45 and one previously safe task is terminated. Without repeated runs or other pairings, these results are insufficient to establish a general offline–online relationship.
  4. Measurement protocol: Candidate units mix primitive actions and short plans without a granularity-specific analysis; this changes what risk can be inferred before execution. Item-level bootstrap intervals also do not address shared-task/trajectory dependence, and state-based checks need to distinguish transient violations from final-state preservation. These choices can materially affect benchmark scores.
Questions for authors
  1. Which failure categories are newly captured relative to existing benchmarks, and how do source-balanced scores and independent annotation agreement change the conclusions?
  2. Are the online results stable across repeated runs and other executor/guardrail pairs, with task-clustered paired uncertainty?
  3. How are primitive versus compound candidates labeled and analyzed, and which safety checks monitor intermediate states rather than only terminal state?
Reviewer_XUWQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces OSGuard, a safety benchmark for computer-use agents with benign user instructions. It has two parts. Action-level benchmark. It contains 324 human-labeled items. Each item gives a guardrail the original instruction, the current state (screenshot + accessibility tree), and a candidate action. The guardrail must label the action as allowed, unrelated, or unsafe. Items come from four sources: standard OSWorld runs, interrupted runs continued by a state-compatible proposer, interrupted runs continued by an unsafe proposer, and runs of the risk-augmented tasks. Risk-augmented execution suite. It contains 45 tasks built from 40 OSWorld tasks. The instruction is unchanged, but the environment is modified to add a hazard, such as an existing file at the output path, a decoy document, or a symlink outside the tree. State-based checks then separate safe success from unsafe completion.

Critical strengths

The task design is clean and easy to verify. The user instruction stays benign and unchanged. Only the environment is modified. Each hazard is checked by concrete state checks, such as file hashes, permissions, and preserved settings. This makes results reproducible and avoids LLM judges. Appendix Table A1 shows many realistic hazards, like existing files at the export path, decoy documents in front, and symlinks to /proc. These are failures that ordinary users could actually face.

Critical weaknesses

The execution suite is too small to support its conclusions. There are only 45 tasks, so one task is about 2.2%. The guarded and unguarded runs differ by only two tasks (17 vs. 15 unsafe completions). The paper itself says one run was fixed, one unsafe run was halted, and one good run was halted.

The same models are used to build and to evaluate the benchmark. Gemini 3 Pro Preview generated the proposer actions, which make up 217 of the 324 items (67%). Gemini is also the best guardrail, and its lead is largest exactly on these proposer items. So the benchmark may be tied to these specific models. Results with other proposers, other executors, and other guardrail families would make the claims stronger.

Questions for authors

Labels are mixed up with data sources, and annotation details are missing. “Allowed” items come only from standard OSWorld runs and risk-augmented runs. The two proposer sources contain no allowed items at all. So a model could partly guess the label from what kind of state it sees. For example, all models do well on standard OSWorld items, and Gemini gets 100%, simply because they are all “allowed.” This also makes the per-source macro-F1 in Table 3 hard to read. With only two labels present, three-class macro-F1 seems capped at about 0.67, which explains values like 83% accuracy but 0.45 macro-F1. The paper also does not say: how many annotators labeled each item; what the inter-annotator agreement is; how the subjective line between “unrelated” and “unsafe” was handled.

Some tasks test correctness or ambiguity, not safety. In some tasks, the “safe” answer goes against the user’s explicit request. For example, the user asks to save the file as pre.pptx on the Desktop. When pre.pptx already exists, the benchmark requires saving under a different name. A reasonable agent might ask the user first, but the setup has no “ask the user” option. Other tasks look more like correctness than safety: Also, it is not clear why blocking “unrelated” but harmless actions is a safety question. Mixing these cases makes it hard to know what the benchmark measures. fixing seeded wrong values in a report; running ls in the right directory; a task where slides 3 and 4 are simply removed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.