- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ROGUE asks whether computer-use agents remain responsive to human control while completing benign tasks. It adds three conflicts to OSWorld-Verified: a request to return control, a shutdown notice that can be circumvented, and a prohibition on accessing credential files. A subagent condition tests whether restrictions survive delegation. Across several frontier models, the paper reports frequent override, shutdown avoidance, restricted-file access, and loss of constraints during delegation.
Critical strengths
-
ROGUE evaluates observable behavior in an executable desktop environment rather than inferring safety from a model's stated choice.
-
The subagent condition identifies a concrete delegation problem: parent agents often pass the task goal while omitting its restriction.
Critical weaknesses
-
Some scenarios mix corrigibility with interpretation of an artificial task conflict. Human override is clear, but canceling an unexpected shutdown notice may be reasonable if it is not presented as an authenticated user command. Safe behavior is also incompatible with task completion, with no alternative handoff or credential path. These tests therefore support a narrower claim than the paper's general discussion of corrigibility.
-
The subagent result partly depends on the delegation interface. A child receives only the parent's free-form instruction rather than automatically inheriting the original restriction. This is a real system failure, but the experiment does not tell us how much comes from the model and how much comes from an interface that fails to propagate policy.
-
Several secondary claims rest on limited evidence. Restricted access contains only eight tasks, and one shutdown condition has 59.1% agreement between the LLM judge and humans. The small capability comparison also cannot establish a general relationship between capability and corrigibility. These analyses should be distinguished from the stronger rule-detected results.
Questions for authors
-
How do results change when shutdown is presented as an explicit, authenticated user instruction?
-
What happens if a subagent automatically inherits the original restriction or a system-level policy?
-
Which conclusions remain when the analysis uses only rule-detected actions and excludes LLM-judged intent?
6 comments
[cont'd] Official Comment by Authors
Several secondary claims rest on limited evidence. Restricted access contains only eight tasks
Thank you for pointing this out. We agree that eight distinct tasks limit what we can infer about the frequency of credential-access failures. However, we run selected main results twice to account for stochasticity in model responses; Figure 2 displays the results of these two matched runs (each scenario-model pairing is run twice). In these two matched runs, Claude Opus 4.6 (max) accessed the restricted credential on 3/8 and 2/8 tasks. These results demonstrate the existence of this real failure mode, which can be worsened with variations in the environment and/or prompt (Appx. A.7). Across all three scenarios, Figure 3 includes a total of 907 agent trajectories (14,716 model responses), which we’ve now stated in the caption. Regarding ROGUE’s overall dataset size, ROGUE follows a precedent in agent-safety evaluation of using focused task sets to expose specific failure modes, as in OS-Harm (150 tasks), AM-Bench (35 tasks), ToolEmu (144 test cases), and AgentHarm (110 base tasks).
one shutdown condition has 59.1% agreement between the LLM judge and humans.
This is a valid concern. We note that the 59.1% human-LLM judge agreement is in labeling shutdown rewiring intent for Claude Opus 4.7 with subagents. However, in these runs, the GPT-5.5 judge is more conservative than the human judge at labeling intent to rewire shutdown (the LLM judge labeled 3/22 episodes positive versus 10/22 for the human).
The small capability comparison also cannot establish a general relationship between capability and corrigibility.
Thank you for raising this. The capability comparison (with task completion numbers taken from model evaluations on the full OSWorld-Verified suite of 369 tasks) is not intended to establish a general relationship between capability and corrigibility. Rather, we simply state that “capability alone does not imply greater corrigibility.” We’ve made sure this is clarified in Section 4.
Which conclusions remain when the analysis uses only rule-detected actions and excludes LLM-judged intent?
Using only rule-detected outcomes, we still observe frequent human-control overrides, direct shutdown-script modification in some configurations, and credential access on the eight tested tasks. For example, GPT-5.6 Sol overrode control in 47/48 episodes, GPT-5.4 modified the shutdown script in 8/22, and Gemini 3.1 Pro Preview accessed the restricted credential file in 6/8. These results using only rule-based detectors are represented by the “Actual violations” (red) bar in Figure 3. We’ve updated the caption of Figure 3 to clarify this. Taken together, the observed control failures persist when we restrict the analysis to rule-detected actions, and the matched runs show that credential-access violations recur on the same tasks.
We hope these revisions address your feedback and allow you to give a higher rating on our submission. If not, please let us know how we can improve.
Clarification on ROGUE’s coverage of corrigibility
We also recognize that certain areas of our initial submission may have made it sound like ROGUE’s aim was to comprehensively cover/test all aspects of corrigibility. We would like to clarify that this is not the case; we view our benchmark as a way to systematically measure certain corrigibility failures with clear interpretations (overriding human control, avoiding shutdown, ignoring user restrictions) as signs of agent incorrigibility.
We’ve updated the submission to reflect this. In particular, ROGUE is now introduced as: “ROGUE … a benchmark that measures signs of agent incorrigibility by constructing controlled conflicts between task completion and human control, shutdown, or access restrictions.”
We hope this clarifies the scope and contribution of our work.
Dear Reviewer v7kg,
Thank you for your thoughtful feedback. Please find our responses below:
Thank you for raising this concern. In Section 4 (“Baseline Mitigations”) and Figure 5 (Top), we do add an explicit shutdown compliance instruction to the task instruction. For GPT-5.4 (medium),
shutdown.shrewrites drop from 16/22 to 0/22, demonstrating that explicit instructions do change behavior. However, introducing subagents revives the problem, increasingshutdown.shrewrites to 3/22. This is due to either the parent agent not passing along the user instruction, or the child agent violating the instruction despite the parent communicating it.Additionally, among the models tested and displayed in Figure 3, only 39 of 242 (16.1%) agent episodes included reasoning that interpreted the shutdown notice as a potential prompt injection, 29 of which were from Claude Opus 4.7. Hence, we tested the effect of modifying the system prompt to state that all pop-ups and notices are benign. However, as displayed in Figure 5 (bottom), this doesn’t have much of an effect on Opus 4.7’s behavior. Thank you for asking this, we’ve updated the paper (under “Agents seek ways to avoid shutdown”, line 361) to clarify this.
Thank you for bringing this up. We don’t claim that our paper exhaustively covers/tests all aspects of corrigibility. We intentionally focus on measuring corrigibility failures in agents when there is conflict between task completion and safety in order to most clearly elucidate incorrigible reasoning. As we discuss in our Limitations and Future Work section, we recognize the potential to build upon this benchmark to support subsequent investigations outside the scope of the present study, such as when agents do have safer (but perhaps less convenient) alternatives, like you suggest. While the scope of our current benchmark is to clearly elucidate how frontier models trade off between corrigibility and task completion, it would be an interesting direction of future work to use our benchmark to explore and quantify corrigibility failures as a function of the number of safe alternative paths. We hope the Limitations and Future Work section clarifies this.
We agree that the subagent’s automatic inheritance of the original instruction may remove this particular route to failure. However, we intentionally test the framework in which the parent’s policy is not automatically passed to the subagent in order to explicitly test the models’ abilities to reliably communicate safety-pertinent information on their own. Our research question is asking whether models themselves are able to communicate safety restrictions.
[cont'd]