acceptodds
Under review as a conference paper at ICLR 2027

SIR: Self-improving Red-teaming for Computer Use Agents

Abstract

Computer-use agents (CUAs) are agents powered by vision-language models (VLMs) that perceive a screen and operate an operating system through mouse, keyboard, and terminal interactions to automate everyday digital tasks. Their exposure to untrusted content creates a risk of indirect prompt injection (IPI), where an adversary embeds instructions in content the agent reads to redirect it toward actions that violate the user's intent. Evaluations based on fixed, hand-written injections may underestimate the risk posed by adaptive adversaries. We present **SIR**, a black-box self-improving IPI framework that (i) composes task-specific injections from a small library of reusable red-teaming principles stated in plain language and (ii) uses an iterative feedback loop to analyze unsuccessful attack trajectories and distill new, named principles that are retained in a shared library and reused across tasks. We target operating-system-level compromise and evaluate outcomes through deterministic checks on filesystem, service, and permission state rather than an LLM judge. An attack counts as successful only when both the adversarial objective and the benign user task are completed in the same execution. We evaluate three frontier CUAs, allowing SIR up to 10 attack attempts per case. It achieves joint attack success rates of 24% on Claude Opus 4.8 and 28% on Gemini 3.5 Flash, compared with 4% and 0%, respectively, for the benchmark's fixed, hand-written injection. The red-teaming principles discovered against one victim also improve attacks against other victims, including a different model family, without additional feedback.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Indirect Prompt Injection, Computer-Use Agents, AI agent security, Red-Teaming attacks, Vision-Language Models, LLM safety

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Cc8UOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SIR, a black-box indirect prompt injection framework for computer-use agents. It builds injections by combining basic attack principles, and it learns new principles from failed runs and saves them in a library shared across tasks. The paper tests it on three target models and 50 RedTeamCUA cases, and gets the evaluation results.

Critical strengths

Attack results are checked against the real system state. The problem setting is close to real-world risk, and the evaluation is fairly solid. The target is real consequences at the operating-system level. It does not use an LLM judge, and it also requires the normal task to be completed. This is important for security evaluation.

The design of transfer experiment with the frozen principle library is meaningful.

Critical weaknesses

The most important claim ("principles learned from failures are effective and transferable") is weakly supported:

  1. The main result is tested in-sample. The principle library was discovered on these 50 evaluation cases, and then evaluated on the same 50 cases. Discovery and testing are not separated.
  2. The cross-model transfer is small: only 2 to 5 more cases succeeded (out of 50). The cross-task transfer is narrow: the held-out tasks are also software installation workflows, with the same injection location, the same benchmark, and the same attacker.

The main selling point, "learning from failures," is not tested on its own. The introduction present "learning from failures instead of successes" as the core novelty. But there is no experiment that compares "extracting principles from successes" with "extracting principles from failures." So we cannot tell whether the gain comes from the failure signal, or just from having a shared strategy memory plus multiple attempts. Given existing mechanisms such as EVA and CoP, the lack of these comparisons directly affects my judgment of the novelty and contribution.

Questions for authors
  1. Can you fully separate principle discovery from evaluation? For example, discover the library on a subset of cases, and evaluate on tasks that were never seen (including different types of benign tasks) and on different adversarial goals, with multiple repeated runs.
  2. Can you add a control experiment: the same framework, but the analyzer extracts principles only from successful trajectories (or from both successes and failures)? This would show whether the "failure" signal itself is important.
  3. In the cross-model transfer experiment, why were only P9 and P10 chosen?
  4. If StepJack were also given ten real executions (e.g., by sampling ten different attack chains), would SIR's advantage still hold?
7 comments
AuthorsOpenReview ↗

Responses to Reviewer Cc8U (Part 1)

1. The main result is tested in-sample. The principle library was discovered on these 50 evaluation cases, and then evaluated on the same 50 cases. Discovery and testing are not separated. & Can you fully separate principle discovery from evaluation? For example, discover the library on a subset of cases, and evaluate on tasks that were never seen (including different types of benign tasks) and on different adversarial goals, with multiple repeated runs.

A: Thank you for raising this concern. However, we would like to kindly raise your attention to Appendix A in our paper, which provides a separate held-out evaluation with discovery and testing explicitly separated.

Specifically, the library is discovered on 50 cases from the browser-extension and tmux-alias workflows (reddit_1 and reddit_3). It is subsequently evaluated on a disjoint set of 50 cases involving a package-installation workflow (reddit_2). The two sets share no task identifiers, and the held-out task–objective combinations were not executed during discovery.

Before held-out evaluation, we freeze the discovered library and disable the analyzer. No failure-driven discovery or library updates occur on the held-out cases. We compare two conditions using the same one-shot pipeline and task set: Base, which receives only the predefined principles, and Transfer, which additionally receives the frozen discovered library. Table S1 in Appendix A reports:

Victim modelOriginal: discovery-enabled ASR on discovery setBase: one-shot ASR on held-out setTransfer: one-shot ASR on held-out set
Claude Opus 4.654%28%48%
Claude Opus 4.824%12%20%
Gemini 3.5 Flash28%6%26%

Each set contains 50 cases. The analyzer is disabled in both held-out conditions. The Original column is included for reference only; it uses a different task set and discovery protocol. The controlled comparison is Base versus Transfer.

Adding the frozen library improves held-out ASR by 20, 8, and 20 percentage points, respectively. Because both conditions use the same one-shot budget without test-time discovery, these gains provide evidence that the learned guidance remains useful on cases outside the discovery set.

To address the multiple repeated runs, we report five evaluations of the Transfer condition on the same 50 held-out cases. Each run retains the one-shot budget per case, with the discovered principles frozen and the analyzer disabled throughout evaluation. The results are:

Victim modelHeld-out ASR: mean ± sample SD
Claude Opus 4.647.20% ± 2.28%
Claude Opus 4.819.60% ± 2.19%
Gemini 3.5 Flash26.40% ± 2.19%

Results summarize five evaluation runs. ASR is calculated separately for each run, without pooling successes across runs.

The mean ASRs remain close to the original held-out results of 48%, 20%, and 26%, respectively, with sample standard deviations of 2.19–2.28 percentage points. These repetitions provide additional evidence of the repeatability of the observed held-out transfer performance without test-time discovery.

Appendix A details the held-out protocol and results (Table S1, lines 714–776), and Appendix U.4 reports the five repeated evaluations with the library frozen and discovery disabled (Table S20, lines 2167–2184).

AuthorsOpenReview ↗

Responses to Reviewer Cc8U (Part 2)

2. The cross-model transfer is small: only 2 to 5 more cases succeeded (out of 50).

A: Thank you for pointing out that the original cross-model transfer experiment is thin on the number of cases. To examine whether the observed gains persist over a larger task set, we expanded this evaluation from 50 to 100 cases by sampling 50 additional tasks alongside the original 50.

We retained the original cross-model transfer protocol. P9 and P10 were discovered on Claude 4.6 and remained frozen when applied to Claude 4.8 and Gemini 3.5, without further principle discovery on either target model. Each principle was added separately to the base inventory, with the remaining experimental settings unchanged.

VictimBase+ P9+ P10
Claude 4.811%19%15%
Gemini 3.57%22%14%

On Claude 4.8, P9 and P10 produce gains of 8% and 4%. On Gemini 3.5, the gains is 15% and 7%.

The direction of improvement is positive in all four comparisons. The strongest result is P9 on Gemini 3.5, where ASR increases from 7% to 22%, providing evidence that a principle discovered on a Claude victim can remain useful on a different model family. The benefits nevertheless vary by principle and victim: P10’s improvement on Claude 4.8 remains modest at four percentage points. These expanded results strengthen the evidence for cross-model reuse of the two evaluated principles.

The expanded cross-model transfer evaluation on 100 cases is reported in Appendix U.4 (Table S21, lines 2186–2201).

3. software installation workflows, with the same injection location, and the same attacker.

A: Thank you for raising the concern about the evaluation scope. We have added experiments on an additional application and injection surface, as well as an ablation using a different attacker model.

To evaluate whether SIR’s learning procedure remains effective beyond the forum setting, we applied it to OwnCloud, where the agent encounters injected content in shared PDF documents. Discovery was performed within this environment, allowing the library to incorporate feedback from the new setting.

We evaluated 50 cases per victim model:

ModelBaselineStepJackMUZZLESIR
Claude 4.612%26%22%62%
Claude 4.88%14%10%28%
Gemini 3.58%16%14%36%

SIR achieves ASRs of 62%, 28%, and 36% on Claude 4.6, Claude 4.8, and Gemini 3.5, respectively, exceeding the strongest listed baseline by 36, 14, and 20 percentage points. These results support the applicability of SIR’s learning procedure to an additional application and injection surface. They evaluate adaptation through discovery in the new environment.

To address the concern about using the same attacker, we also replaced Grok‑4.3 with DeepSeek V3.2 as the attack-generation model in a separate experiment on the original forum benchmark:

VictimSIR with DeepSeek V3.2SIR with Grok‑4.3
Claude 4.656%54%
Claude 4.822%24%
Gemini 3.530%28%

The observed ASR changes by 2%, respectively, corresponding to one successful case per victim on the 50-case evaluation. Performance is therefore similar under the two tested attacker models, providing evidence that SIR’s effectiveness is not confined to Grok‑4.3.

The OwnCloud evaluation and attacker-model sensitivity experiment are presented in Appendices U.6 and U.7, respectively (Tables S23–S24, lines 2226–2270).

AuthorsOpenReview ↗

Responses to Reviewer Cc8U (Part 3)

4. Can you add a control experiment: the same framework, but the analyzer extracts principles only from successful trajectories (or from both successes and failures)? This would show whether the "failure" signal itself is important.

A: Thank you for suggesting this control. We conducted additional experiments comparing four configurations: base principles without discovery, principles extracted only from successful trajectories, principles extracted from both successful and failed trajectories, and SIR’s original failure-only discovery.

We follow the paper’s evaluation procedure and use the same definition of success across conditions: both the adversarial objective and the benign task must be completed in the same execution. The additional controls change which trajectories supply the analyzer’s principle-discovery process. Results over 50 cases per victim are:

Victim modelC0: Base, no discoveryC2: Success-only miningC3: Success + failure miningC1: Failure-only mining
Claude 4.634%42%50%54%
Claude 4.810%18%20%24%
Gemini 3.58%16%22%28%

The results provide three observations.

First, successful trajectories are also informative: success-only mining improves ASR over the base configuration by eight percentage points on each victim. Second, incorporating failures improves on success-only mining. Mixed mining raises ASR from 42% to 50% on Claude 4.6, 18% to 20% on Claude 4.8, and 16% to 22% on Gemini 3.5. This provides evidence that unsuccessful executions contribute useful information beyond the guidance extracted from successful examples. Third, failure-only mining achieves the highest observed ASR on all three victims. Compared with success-only mining, its gains are 12%, 6%, and 12%, respectively. Compared with mixed mining, its gains are smaller—4, 4, and 6 percentage points. These findings support our choice of failure-driven discovery under the evaluated protocol.

The success-only, mixed, and failure-only mining configurations and their results are described in Appendix U.1 (Table S16, lines 2066–2095).

5. In the cross-model transfer experiment, why were only P9 and P10 chosen?

A: Thank you for asking us to clarify the provenance of the transferred principles. In Section 4.4, we use Claude 4.6 as the source model and Claude 4.8 and Gemini 3.5 as the transfer targets. P9 (Error Corroboration) and P10 (Conditional Deferred Execution) were the only feedback-derived principles obtained in the Claude 4.6 discovery run used for this experiment. We therefore evaluated both principles from that run, adding each separately to the predefined base inventory.

The number of transferred principles reflects the output of this particular source discovery run; it is not a limit on the number of principles SIR can discover. Both principles were frozen after source-model discovery and applied to the target models without further principle discovery or modification.

SIR also learned additional principles in the discovery runs involving other victim models, as illustrated in Appendix Q. Those principles belong to the broader discovery results rather than the Claude 4.6 source set evaluated in Section 4.4. Including principles learned directly on the transfer targets would mix cross-model transfer with target-specific discovery.

Reviewer_KTghOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SIR (Self-Improving Red-Teaming), a black-box framework for generating indirect prompt injection attacks against computer-use agents. SIR consists of two main components: (1) compositional attack generation, which combines reusable attack principles to construct task-specific injections, and (2) failure-driven principle discovery, which analyzes unsuccessful attack trajectories and extracts new strategies into a shared principle library for future attacks. Experiments on 50 RedTeamCUA cases across three frontier CUAs demonstrate improvements over hand-crafted attacks and existing automated baselines, including MUZZLE and StepJack. The authors also investigate the transferability of discovered principles across different victim models and tasks.

Critical strengths
  1. The paper addresses security vulnerabilities in computer-use agents, particularly indirect prompt injection attacks that can lead to unauthorized operating-system-level actions. This is a timely and practically relevant research direction.

  2. Unlike approaches relying primarily on LLM-based judges, SIR uses deterministic system-state checks to evaluate attack outcomes. Requiring both adversarial success and benign task completion provides a more meaningful evaluation of task-preserving compromises.

  3. SIR consistently outperforms the evaluated baselines across three CUAs. The ablation experiments indicate that failure-driven feedback improves attack success rates beyond the initial compositional strategy library.

  4. The paper investigates whether discovered attack principles can generalize across different victim models and held-out tasks, providing useful preliminary evidence of reusable adversarial strategies.

Critical weaknesses
  1. Limited methodological novelty. The core framework follows a relatively standard generate–evaluate–reflect–update pipeline, combining existing ideas from compositional prompting, failure analysis, and memory-based self-improvement. Similar mechanisms have already been explored in prior work, including Reflexion, AutoDAN-Turbo, EVA, and MUZZLE.

  2. Insufficient analysis of the proposed principle discovery mechanism. The most important claimed contribution is the discovery of reusable attack principles from failed trajectories. However, the discovery process primarily relies on an LLM to analyze failures and generate natural-language strategies.

Questions for authors

see weakness

8 comments
AuthorsOpenReview ↗

Responses to Reviewer KTgh (Part 1)

1. Limited methodological novelty. The core framework follows a relatively standard generate–evaluate–reflect–update pipeline, combining existing ideas from compositional prompting, failure analysis, and memory-based self-improvement. Similar mechanisms have already been explored in prior work, including Reflexion, AutoDAN-Turbo, EVA, and MUZZLE.

A:Thank you for highlighting the relationship to these methods. We agree that the generate–evaluate–reflect–update loop is established, and we do not claim reflection, compositional prompting, or memory individually as new. Our contribution concerns what is learned from unsuccessful CUA executions, how that knowledge is represented and shared, and how it guides subsequent attacks under an execution-verified objective.

Specifically, SIR analyzes unsuccessful end-to-end web-to-OS executions and produces candidate principles containing a failure diagnosis, proposed mechanism, composition guidance, and example. These principles update a shared library during discovery and guide attack generation across tasks. The learning process uses a joint-success criterion: an execution must complete both the adversarial objective and the benign task. This criterion determines both which executions remain sources for discovery and when a task is retired.

The following comparison makes the relationship to prior work explicit:

MethodExperience used for learningKnowledge representationHow knowledge guides subsequent attemptsEvaluated objective
ReflexionThe acting agent’s trajectories and task feedbackVerbal self-reflections in episodic memoryConditions subsequent trials on previous reflectionsComplete the agent’s assigned task
AutoDAN-TurboPrompt–response records and improvements in jailbreak scoresNamed strategies with definitions and examples, indexed through response embeddings and score improvementsRetrieves, reuses, and combines strategies through a lifelong libraryElicit harmful textual responses
EVAAtomic GUI responses guide rejection/neglect-based mutations; successful mutation traces are distilledRules specifying scenario context, trigger mechanism, and a parameterized message templateDiscovers rules offline, then retrieves and instantiates them for zero-shot deploymentInduce a targeted action on an injected pop-up
MUZZLEWeb-agent execution traces and feedback on unsuccessful attacksRanked injection surfaces, candidate payloads, and failure diagnosesRevises payloads or switches injection surfaces within an investigation; a shared principle library is not describedComplete an adversarial objective in the web environment
SIREnd-to-end web-to-OS executions that do not satisfy the joint-success criterionCandidate principles specifying a name, failure diagnosis, proposed mechanism, composition guidance, and exampleUpdates a shared library across tasks; combines base principles during composition and incorporates discovered guidance during refinementComplete both the adversarial objective and the benign task in the same execution, verified through system-state checks

The closest connections are therefore acknowledged explicitly. AutoDAN-Turbo already demonstrates reusable strategy discovery and combination, but its learning signal comes from scored textual interactions. EVA already combines failure analysis with rule extraction; its reusable rules are distilled from successful mutation traces involving atomic GUI decisions. MUZZLE already uses failed executions to improve attacks, with reflection directing payload revision and injection-surface selection. SIR connects unsuccessful complete CUA executions to an evolving, shared principle representation that guides attacks across tasks. The proposed diagnoses remain hypotheses; their usefulness is assessed through subsequent execution rather than assumed from their wording.

Section 2 clarifies SIR’s relationship to prior adaptive attack methods and memory-based self-improvement (lines 162–179), and Appendix B provides the detailed comparison of learning signals, knowledge representations, reuse mechanisms, and evaluation objectives (Table S3, lines 810–856).

AuthorsOpenReview ↗

Responses to Reviewer KTgh (Part 2)

2. Insufficient analysis of the proposed principle discovery mechanism.

A: Thank you for raising this concern. We agree that an LLM-generated explanation alone is insufficient evidence of discovering a useful attack principle. In SIR, the analyzer proposes candidate guidance, while its practical value is assessed through subsequent victim executions, controlled ablations, and transfer experiments.

The discovery procedure has two stages. First, the analyzer examines a failed execution and proposes an explanation based on the recorded feedback. Second, it produces a candidate principle containing a name, failure diagnosis, proposed mechanism, composition guidance, and example. These candidates are incorporated into the shared library and guide subsequent injection generation. We regard their explanations as hypotheses rather than verified descriptions of the victim’s internal defenses.

The analyzer does not determine attack success. Success requires both the adversarial objective and the benign task to complete in the same execution, verified by deterministic checks on the resulting environment state, including filesystem changes and service states. These checks do not use an LLM judge.

To examine the discovery mechanism more directly, we conducted four additional ablations. All variants follow the paper’s evaluation procedure, using the same 50 cases, victim configurations, maximum of ten victim deployments per case, and deterministic joint-success criterion. The interventions are:

  • Task-local memory: Each task maintains an initially empty discovered-principle memory that persists across its attempts but is never shared with other tasks. The predefined base inventory remains available, and Top-K = 5 applies separately to each task’s discovered memory.
  • Raw failure summaries: We replace distilled principles with raw summaries of failed trajectories while retaining shared memory.
  • Success-mining: We retain the analyzer and principle-discovery procedure but extract principles only from jointly successful trajectories. These principles enter shared memory before the successful task is retired and can guide subsequent attempts on other tasks.
  • Fixed random-text library: We replace learned-principle guidance with five entries containing random word sequences in the same entry format. The library contains 2,275 characters, matching the average learned-library length across the three victims, and remains fixed across attempts. It is constructed without an analyzer or principle-generation LLM.
ConfigurationClaude 4.6Claude 4.8Gemini 3.5
Base principles, no discovery34%10%8%
Task-local memory44%18%20%
Raw failure summaries38%12%18%
Success-mining42%18%16%
Length-matched random strategies24%10%12%
Full SIR54%24%28%

These comparisons examine different aspects of the mechanism. The raw-summary comparison supports the usefulness of the structured principle representation over the evaluated summary format. The task-local comparison supports sharing discoveries across tasks. The success-mining comparison provides evidence that unsuccessful executions are a useful source of guidance, with failure-driven discovery achieving higher observed ASR on all three victims. The length-matched random-library control further examines the improvement comes from the learned content rather than simply supplying additional strategy text.

We also evaluate reuse separately from performance on the discovery tasks. In the held-out experiment (Appendix A in our paper), the library is frozen before evaluation, the analyzer is disabled, and both conditions use the same one-shot pipeline.

Victim modelOriginal: discovery-enabled ASR on discovery setBase: one-shot ASR on held-out setTransfer: one-shot ASR on held-out set
Claude Opus 4.654%28%48%
Claude Opus 4.824%12%20%
Gemini 3.5 Flash28%6%26%

Adding the discovered library improves ASR from 28% to 48% on Claude 4.6, 12% to 20% on Claude 4.8, and 6% to 26% on Gemini 3.5. Because no discovery occurs on the held-out tasks, these gains provide evidence that the stored guidance remains useful beyond the executions from which it was learned.

Furthermore, Table 2 examines individual discovered principles. P9 and P10 are learned on Claude 4.6 and added separately to the base inventory when attacking other victims, without further discovery:

VictimBase principles+ P9+ P10
Claude 4.810%16%14%
Gemini 3.58%18%12%

These interventions provide evidence of usefulness for the evaluated principles, beyond the aggregate performance of the complete framework.

The discovery controls are reported in Appendix U.1 (Table S16, lines 2056–2098), with evidence of principle reuse presented in Section 4.4 (lines 467–494) and Appendix A (Table S1, lines 714–776).

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer KTgh,

Thank you again for your thoughtful comments on the methodological contribution and the evidence for principle discovery. As the discussion period draws to a close, we would like to briefly follow up on the two main points in our response:

  • Relationship to prior work: Section 2 and the new comparison in Appendix B (Table S3) explicitly acknowledge the established reflection and memory-based learning framework. We clarify that our contribution concerns how unsuccessful end-to-end web-to-OS executions inform a shared principle library that guides attacks across tasks.
  • Evidence for useful principles: The additional controls examine task-local memory, raw failure summaries, successful trajectories, and random-text guidance (Appendix U.1, Table S16). The held-out and cross-model evaluations further examine whether the learned guidance remains useful with discovery disabled (Appendix A and Section 4.4).

We also clarify that the analyzer proposes candidate principles: their explanations remain hypotheses, and attack success is determined by deterministic execution checks.

We hope these additions make the contribution and its supporting evidence clearer. If either concern remains unresolved, we would appreciate the opportunity to discuss it further before the discussion closes. Thank you again for your careful review.

Reviewer_v9VLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

SIR attacks computer-use agents with adaptive indirect prompt injection in a controlled web and operating-system environment. It composes injections from natural-language attack principles and uses failed trajectories to propose new principles for a shared library, which later attempts and tasks reuse without updating any parameters. Outcomes are scored by deterministic system-state evaluators, and success requires the adversarial objective and the benign task to both complete within the same run. Frozen-library experiments probe reuse across models and tasks.

Critical strengths
  • Scoring real system effects, with adversarial and benign outcomes recorded separately, isolates the compromises that hide inside an apparently successful task and tells them apart from attacks that merely break the task.
  • Execution feedback is turned into reusable and readable principles. The feedback ablations and frozen-library transfer experiments together argue that the gain is more than the effect of sampling many independent attacks.
  • The authors report uncertainty, paired comparisons, and multiple-comparison corrections, plus baseline adaptations, execution settings, and costs. Learning during the main evaluation is also kept distinct from testing a frozen library on held-out tasks.
Critical weaknesses
  • The existing controls do not isolate the principle representation, failure-driven learning, and shared memory. The feedback ablation shows that feedback helps overall, but not that named principles beat raw failure summaries or generic reflection, or that shared memory beats task-local memory. Head-to-head comparison with the closely related adaptive methods discussed in the paper is also thin.
  • How much attacker access the feedback loop actually needs is still open. Discovery consumes actions, state information, and model narration, none of which is available to an attacker who can only submit content, and running different models does not substitute for removing feedback channels one at a time inside a fixed model. Frozen-library transfer lowers the feedback burden at the target but not the access and cost requirements of discovery.
  • Shared-library learning needs more evidence of stability. Do not know how sensitive it is to task order, independent discovery runs, update schedules, other applications, or other injection surfaces. A higher attack success rate also does not by itself confirm that the analyzer's account of the victim's internal defenses is accurate.
Questions for authors
  • Can you compare named principles against raw failure summaries, generic reflection, a length-matched random library, task-local memory, and principles derived from successful trajectories, matching both victim deployment and total compute?
  • How does performance change when screenshots, actions, or model narration are removed from discovery feedback while the victim model stays fixed? What feedback access and cost are needed to build a transferable library?
  • Are the results and the learned libraries stable across randomized task orders, independent discovery runs, and synchronous updates?
  • Can you evaluate transfer to different applications and injection surfaces, and separate whether a discovered principle is actually useful from whether its account of the victim's defenses is right?
7 comments
AuthorsOpenReview ↗

Responses to Reviewer v9VL (Part 1)

1. The existing controls do not isolate the principle representation, failure-driven learning, and shared memory. The feedback ablation shows that feedback helps overall, but not that named principles beat raw failure summaries or generic reflection, or that shared memory beats task-local memory. & Can you compare named principles against raw failure summaries, generic reflection, a length-matched random library, task-local memory, and principles derived from successful trajectories, matching both victim deployment and total compute?

A: Thank you for highlighting this concern. We have added four controls to examine the contributions of memory sharing, principle representation, the source of learning signals, and learned strategy content. Unless otherwise specified below, all controls follow the experimental procedure described in the paper, using the same 50 cases, victim configurations, maximum of 10 victim deployments per case, and deterministic joint-success criterion.

ConfigurationClaude 4.6Claude 4.8Gemini 3.5
Base principles, no discovery34%10%8%
Task-local memory44%18%20%
Raw failure summaries38%12%18%
Success-mining42%18%16%
Length-matched random strategies24%10%12%
Full SIR54%24%28%

First, task-local memory. We retain principle discovery but maintain a separate discovered-principle memory for each task. Each memory starts empty, persists across that task’s attempts, and is never shared with other tasks. The predefined base inventory remains available, and the same Top-K = 5 retention rule applies separately to each task’s discovered memory. This variant achieves ASRs of 44%, 18%, and 20% on Claude 4.6, Claude 4.8, and Gemini 3.5, respectively, compared with 54%, 24%, and 28% for full SIR. Sharing discoveries therefore yields observed gains of 10, 6, and 8 percentage points, supporting the benefit of making learned guidance available across tasks.

Second, raw failure summaries. We replace distilled principles with raw failure summaries while retaining shared memory. This variant achieves ASRs of 38%, 12%, and 18%, compared with 54%, 24%, and 28% for full SIR. The corresponding gains of 16, 12, and 10 percentage points support the usefulness of distilled principle guidance relative to the tested summary representation.

Third, success-mining. We retain the analyzer and principle-discovery procedure but supply only trajectories satisfying the joint-success criterion, excluding unsuccessful trajectories. Encoded and unreadable system-output content is removed through the same preprocessing procedure. The extracted principles are incorporated into shared memory before the successful task is retired, making them available to subsequent attempts on other tasks. We additionally inspect successful trajectories manually to verify the observed outcomes. Success-mining achieves ASRs of 42%, 18%, and 16%, compared with 54%, 24%, and 28% for failure-driven discovery. Both improve over the no-discovery baseline of 34%, 10%, and 8%. Thus, successful trajectories also provide useful experience, while failure-driven discovery achieves higher observed ASR under the evaluated protocol.

Fourth, a random-library control. We replace the learned-principle guidance with a fixed library of five entries, matching SIR’s maximum library capacity. The entries follow the same format as discovered principles but contain randomly assembled word sequences and randomly generated PascalCase names, without coherent strategy guidance. No analyzer or principle-generation LLM is used to construct this library. Its total length is 2,275 characters, matching the average learned-library length across the three victim models, and the same random library is held fixed across attempts. This condition achieves ASRs of 24%, 10%, and 12%, compared with 54%, 24%, and 28% for full SIR. The observed gains of 30, 14, and 16 percentage points support the usefulness of learned guidance relative to similarly formatted random text matched to the average library length.

We report the memory-sharing, representation, learning-signal, and random-library controls in Appendix U.1 (Table S16, lines 2056–2098).

AuthorsOpenReview ↗

Responses to Reviewer v9VL (Part 2)

2. Head-to-head comparison with the closely related adaptive methods discussed in the paper is also thin.

Regarding head-to-head comparisons, we further implemented Adaptive StepJack, which uses feedback from actual victim executions to optimize subsequent attacks. In this variant, we replace the offline-only refinement loop with an adaptive loop allowing up to ten real victim executions per case, including the initial deployment. After an unsuccessful execution, the observed victim feedback is supplied to the attack-generation process to revise the attack for the next attempt. Thus, the variant can use both additional victim executions and execution feedback.

We evaluate this adaptation on the same 50 tasks using the same deterministic joint-success criterion as SIR: both the adversarial objective and the benign task must be completed in the same execution.

VictimSIR: up to 10 executionsAdaptive StepJack: up to 10 executionsOriginal StepJack: 1 execution
Claude 4.654%22%10%
Claude 4.824%16%12%
Gemini 3.528%20%14%

Allowing repeated execution and feedback improves StepJack’s observed ASR by 12, 4, and 6 percentage points, respectively. Nevertheless, SIR retains higher observed ASR on all three victims, with margins of 32, 8, and 8 percentage points over Adaptive StepJack.

These results show that SIR’s observed advantage persists when StepJack is given the same maximum number of real victim executions.

The Adaptive StepJack protocol and comparison with SIR are reported in Appendix U.5 (Table S22, lines 2204–2221).

3. How much attacker access the feedback loop actually needs is still open. Discovery consumes actions, state information, and model narration, none of which is available to an attacker who can only submit content, and running different models does not substitute for removing feedback channels one at a time inside a fixed model. & How does performance change when screenshots, actions, or model narration are removed from discovery feedback while the victim model stays fixed?

A: Thank you for raising this valuable question. Our discovery setting is the harness-level black-box setting described in Section 3.2. We apply SIR within the RedTeamCUA sandbox [1], whose implementation records screenshots, executed actions, evaluation outcomes, and model-returned responses. SIR uses this execution feedback without access to the victim’s weights, gradients, token probabilities, system prompt, or hidden raw chain of thought.

Screenshots and actions are standard parts of the interaction interface for visual computer-use agents. For example, OSWorld [2] provides environment observations and executes agent-generated mouse and keyboard actions, while UI-TARS operates on screenshots and produces GUI actions. These interfaces explain the availability of trajectory feedback within our sandbox.

To directly examine dependence on model narration, we conducted a within-model ablation that withheld narration from SIR’s discovery feedback while retaining the other feedback channels and keeping the victim configuration and attack budget unchanged:

Victim modelWithout narrationWith narration
Claude 4.638%54%
Claude 4.810%24%
Gemini 3.512%28%

Removing narration lowers ASR for all three victims. This provides direct evidence that model-returned narration is useful for SIR’s discovery process, beyond the remaining execution feedback. SIR still achieves successful attacks without narration, although its effectiveness is reduced.

Section 3.2 in our paper specifies the feedback available during discovery (lines 209–215), and Appendix U.2 reports the within-model narration ablation (Table S17, lines 2101–2117).

[1] Redteamcua: Realistic adversarial testing of computer-use agents in hybrid web-os environments, ICLR Oral 2026

[2] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments, NeurIPS 2024

AuthorsOpenReview ↗

Responses to Reviewer v9VL (Part 3)

4. Frozen-library transfer lowers the feedback burden at the target but not the access and cost requirements of discovery. & What feedback access and cost are needed to build a transferable library?

A: Thank you for raising this interesting question. Our claim is that the resulting library can be reused on unseen tasks without repeating the discovery process.

Following Appendix A, we take the library learned after each discovery epoch, freeze it, and apply it to the same 50 held-out transfer tasks to measure ASR with the analyzer disabled and no further library updates. R0 uses only the base principles. The ASRs are evaluated independently at every two epochs and are not accumulated across epochs.

Completed discovery epochsClaude 4.6Claude 4.8Gemini 3.5
R0 — before discovery28%12%6%
R236%14%18%
R444%16%22%
R648%18%26%
R848%20%26%
R1048%20%26%

These results suggest that obtaining a useful transferable library does not always require exhausting the ten-epoch discovery budget. For Claude 4.6, the library learned after six epochs already achieves the final observed transfer ASR of 48%, with no further improvement through R10. Retrospectively, using this earlier library would have preserved the same observed transfer ASR while avoiding the discovery cost of the remaining five epochs. Claude 4.8 and Gemini 3.5 similarly reach their final observed ASRs at R8 and R6. This suggests an opportunity to reduce library-building costs by stopping discovery when additional updates provide little benefit.

The frozen-library evaluation at successive discovery checkpoints is reported in Appendix U.2 (Table S18, lines 2119–2139).

5. Shared-library learning needs more evidence of stability. Do not know how sensitive it is to task order, independent discovery runs, update schedules & Are the results and the learned libraries stable across randomized task orders, independent discovery runs, and synchronous updates?

A: Thank you for raising the question of shared-library stability. We have added experiments examining sensitivity to task order and independent discovery runs.

For task-order sensitivity, we evaluate five task orderings. Under each ordering, we restart from the base principle library and rerun the complete discovery procedure, allowing the shared library to evolve as tasks are processed. For independent discovery, we perform five runs from the same base library under a fixed task order, independently rebuilding the shared library in each run. Principles remain shared across tasks within each run; no discovered library is carried over between runs. Both experiments retain the original 50-task evaluation set, success criterion, and per-task deployment budget.

VictimOriginal ASRTask reorder: mean ± SDIndependent discovery: mean ± SD
Gemini 3.5 Flash28%28.00% ± 1.41%25.60% ± 2.61%
Claude 4.6 Opus54%53.60% ± 0.89%53.20% ± 4.38%
Claude 4.824%23.20% ± 1.10%22.80% ± 3.03%

Each result reports the mean and sample standard deviation across five runs.

The task-order experiments show relatively small observed variation, with standard deviations of 0.89–1.41 percentage points. Independent discovery produces greater variation, with standard deviations of 2.61–4.38 percentage points, but the mean ASRs remain close to the original results. Every individual run also exceeds the corresponding best-baseline ASR reported in the paper. These results support the consistency of the observed performance advantage across the tested orderings and discovery runs, while acknowledging that the precise outcome depends on the discovery trajectory.

Regarding “synchronous updates,” we would appreciate it if you could specify the update schedule and experimental comparison you would like us to evaluate. This would help us design an experiment that directly addresses your concern.

The task-order sensitivity and independent discovery experiments are described in Appendix U.3 (Table S19, lines 2143–2163).

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you again for your careful reading of SIR and for your constructive suggestions. As the discussion period draws to a close, we would like to follow up on our responses and briefly summarize the additions to the revised paper.

We have added controls examining principle representation, cross-task memory sharing, and the use of successful and unsuccessful trajectories for discovery (Appendix U.1). We also report an ablation of model-returned narration, evaluations of intermediate discovery checkpoints, and repeated experiments across task orderings and independent discovery runs (Appendices U.2–U.3).

To examine reuse, we added five evaluations of the frozen library on held-out tasks and expanded the cross-model transfer evaluation to 100 cases (Appendix U.4). Further experiments compare SIR with Adaptive StepJack under the same maximum victim-deployment budget, evaluate discovery in OwnCloud shared PDF documents, and test a different attacker model (Appendices U.5–U.7). Section 2 and Appendix B also clarify the relationship to prior methods and the scope of our contribution.

These additions are included in the revised paper, with the new text marked in blue. We hope they help clarify both the evidence for SIR and the limits of the conclusions supported by our experiments.

If any concerns remain after reading our responses, we would appreciate the opportunity to discuss them. We are happy to provide further details or focused analyses where feasible during the remaining discussion period.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.