- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SIR, a black-box indirect prompt injection framework for computer-use agents. It builds injections by combining basic attack principles, and it learns new principles from failed runs and saves them in a library shared across tasks. The paper tests it on three target models and 50 RedTeamCUA cases, and gets the evaluation results.
Critical strengths
Attack results are checked against the real system state. The problem setting is close to real-world risk, and the evaluation is fairly solid. The target is real consequences at the operating-system level. It does not use an LLM judge, and it also requires the normal task to be completed. This is important for security evaluation.
The design of transfer experiment with the frozen principle library is meaningful.
Critical weaknesses
The most important claim ("principles learned from failures are effective and transferable") is weakly supported:
- The main result is tested in-sample. The principle library was discovered on these 50 evaluation cases, and then evaluated on the same 50 cases. Discovery and testing are not separated.
- The cross-model transfer is small: only 2 to 5 more cases succeeded (out of 50). The cross-task transfer is narrow: the held-out tasks are also software installation workflows, with the same injection location, the same benchmark, and the same attacker.
The main selling point, "learning from failures," is not tested on its own. The introduction present "learning from failures instead of successes" as the core novelty. But there is no experiment that compares "extracting principles from successes" with "extracting principles from failures." So we cannot tell whether the gain comes from the failure signal, or just from having a shared strategy memory plus multiple attempts. Given existing mechanisms such as EVA and CoP, the lack of these comparisons directly affects my judgment of the novelty and contribution.
Questions for authors
- Can you fully separate principle discovery from evaluation? For example, discover the library on a subset of cases, and evaluate on tasks that were never seen (including different types of benign tasks) and on different adversarial goals, with multiple repeated runs.
- Can you add a control experiment: the same framework, but the analyzer extracts principles only from successful trajectories (or from both successes and failures)? This would show whether the "failure" signal itself is important.
- In the cross-model transfer experiment, why were only P9 and P10 chosen?
- If StepJack were also given ten real executions (e.g., by sampling ten different attack chains), would SIR's advantage still hold?
7 comments
Responses to Reviewer Cc8U (Part 2)
2. The cross-model transfer is small: only 2 to 5 more cases succeeded (out of 50).
A: Thank you for pointing out that the original cross-model transfer experiment is thin on the number of cases. To examine whether the observed gains persist over a larger task set, we expanded this evaluation from 50 to 100 cases by sampling 50 additional tasks alongside the original 50.
We retained the original cross-model transfer protocol. P9 and P10 were discovered on Claude 4.6 and remained frozen when applied to Claude 4.8 and Gemini 3.5, without further principle discovery on either target model. Each principle was added separately to the base inventory, with the remaining experimental settings unchanged.
| Victim | Base | + P9 | + P10 |
|---|---|---|---|
| Claude 4.8 | 11% | 19% | 15% |
| Gemini 3.5 | 7% | 22% | 14% |
On Claude 4.8, P9 and P10 produce gains of 8% and 4%. On Gemini 3.5, the gains is 15% and 7%.
The direction of improvement is positive in all four comparisons. The strongest result is P9 on Gemini 3.5, where ASR increases from 7% to 22%, providing evidence that a principle discovered on a Claude victim can remain useful on a different model family. The benefits nevertheless vary by principle and victim: P10’s improvement on Claude 4.8 remains modest at four percentage points. These expanded results strengthen the evidence for cross-model reuse of the two evaluated principles.
The expanded cross-model transfer evaluation on 100 cases is reported in Appendix U.4 (Table S21, lines 2186–2201).
3. software installation workflows, with the same injection location, and the same attacker.
A: Thank you for raising the concern about the evaluation scope. We have added experiments on an additional application and injection surface, as well as an ablation using a different attacker model.
To evaluate whether SIR’s learning procedure remains effective beyond the forum setting, we applied it to OwnCloud, where the agent encounters injected content in shared PDF documents. Discovery was performed within this environment, allowing the library to incorporate feedback from the new setting.
We evaluated 50 cases per victim model:
| Model | Baseline | StepJack | MUZZLE | SIR |
|---|---|---|---|---|
| Claude 4.6 | 12% | 26% | 22% | 62% |
| Claude 4.8 | 8% | 14% | 10% | 28% |
| Gemini 3.5 | 8% | 16% | 14% | 36% |
SIR achieves ASRs of 62%, 28%, and 36% on Claude 4.6, Claude 4.8, and Gemini 3.5, respectively, exceeding the strongest listed baseline by 36, 14, and 20 percentage points. These results support the applicability of SIR’s learning procedure to an additional application and injection surface. They evaluate adaptation through discovery in the new environment.
To address the concern about using the same attacker, we also replaced Grok‑4.3 with DeepSeek V3.2 as the attack-generation model in a separate experiment on the original forum benchmark:
| Victim | SIR with DeepSeek V3.2 | SIR with Grok‑4.3 |
|---|---|---|
| Claude 4.6 | 56% | 54% |
| Claude 4.8 | 22% | 24% |
| Gemini 3.5 | 30% | 28% |
The observed ASR changes by 2%, respectively, corresponding to one successful case per victim on the 50-case evaluation. Performance is therefore similar under the two tested attacker models, providing evidence that SIR’s effectiveness is not confined to Grok‑4.3.
The OwnCloud evaluation and attacker-model sensitivity experiment are presented in Appendices U.6 and U.7, respectively (Tables S23–S24, lines 2226–2270).
Responses to Reviewer Cc8U (Part 3)
4. Can you add a control experiment: the same framework, but the analyzer extracts principles only from successful trajectories (or from both successes and failures)? This would show whether the "failure" signal itself is important.
A: Thank you for suggesting this control. We conducted additional experiments comparing four configurations: base principles without discovery, principles extracted only from successful trajectories, principles extracted from both successful and failed trajectories, and SIR’s original failure-only discovery.
We follow the paper’s evaluation procedure and use the same definition of success across conditions: both the adversarial objective and the benign task must be completed in the same execution. The additional controls change which trajectories supply the analyzer’s principle-discovery process. Results over 50 cases per victim are:
| Victim model | C0: Base, no discovery | C2: Success-only mining | C3: Success + failure mining | C1: Failure-only mining |
|---|---|---|---|---|
| Claude 4.6 | 34% | 42% | 50% | 54% |
| Claude 4.8 | 10% | 18% | 20% | 24% |
| Gemini 3.5 | 8% | 16% | 22% | 28% |
The results provide three observations.
First, successful trajectories are also informative: success-only mining improves ASR over the base configuration by eight percentage points on each victim. Second, incorporating failures improves on success-only mining. Mixed mining raises ASR from 42% to 50% on Claude 4.6, 18% to 20% on Claude 4.8, and 16% to 22% on Gemini 3.5. This provides evidence that unsuccessful executions contribute useful information beyond the guidance extracted from successful examples. Third, failure-only mining achieves the highest observed ASR on all three victims. Compared with success-only mining, its gains are 12%, 6%, and 12%, respectively. Compared with mixed mining, its gains are smaller—4, 4, and 6 percentage points. These findings support our choice of failure-driven discovery under the evaluated protocol.
The success-only, mixed, and failure-only mining configurations and their results are described in Appendix U.1 (Table S16, lines 2066–2095).
5. In the cross-model transfer experiment, why were only P9 and P10 chosen?
A: Thank you for asking us to clarify the provenance of the transferred principles. In Section 4.4, we use Claude 4.6 as the source model and Claude 4.8 and Gemini 3.5 as the transfer targets. P9 (Error Corroboration) and P10 (Conditional Deferred Execution) were the only feedback-derived principles obtained in the Claude 4.6 discovery run used for this experiment. We therefore evaluated both principles from that run, adding each separately to the predefined base inventory.
The number of transferred principles reflects the output of this particular source discovery run; it is not a limit on the number of principles SIR can discover. Both principles were frozen after source-model discovery and applied to the target models without further principle discovery or modification.
SIR also learned additional principles in the discovery runs involving other victim models, as illustrated in Appendix Q. Those principles belong to the broader discovery results rather than the Claude 4.6 source set evaluated in Section 4.4. Including principles learned directly on the transfer targets would mix cross-model transfer with target-specific discovery.
Responses to Reviewer Cc8U (Part 1)
1. The main result is tested in-sample. The principle library was discovered on these 50 evaluation cases, and then evaluated on the same 50 cases. Discovery and testing are not separated. & Can you fully separate principle discovery from evaluation? For example, discover the library on a subset of cases, and evaluate on tasks that were never seen (including different types of benign tasks) and on different adversarial goals, with multiple repeated runs.
A: Thank you for raising this concern. However, we would like to kindly raise your attention to Appendix A in our paper, which provides a separate held-out evaluation with discovery and testing explicitly separated.
Specifically, the library is discovered on 50 cases from the browser-extension and tmux-alias workflows (
reddit_1andreddit_3). It is subsequently evaluated on a disjoint set of 50 cases involving a package-installation workflow (reddit_2). The two sets share no task identifiers, and the held-out task–objective combinations were not executed during discovery.Before held-out evaluation, we freeze the discovered library and disable the analyzer. No failure-driven discovery or library updates occur on the held-out cases. We compare two conditions using the same one-shot pipeline and task set: Base, which receives only the predefined principles, and Transfer, which additionally receives the frozen discovered library. Table S1 in Appendix A reports:
Each set contains 50 cases. The analyzer is disabled in both held-out conditions. The Original column is included for reference only; it uses a different task set and discovery protocol. The controlled comparison is Base versus Transfer.
Adding the frozen library improves held-out ASR by 20, 8, and 20 percentage points, respectively. Because both conditions use the same one-shot budget without test-time discovery, these gains provide evidence that the learned guidance remains useful on cases outside the discovery set.
To address the multiple repeated runs, we report five evaluations of the Transfer condition on the same 50 held-out cases. Each run retains the one-shot budget per case, with the discovered principles frozen and the analyzer disabled throughout evaluation. The results are:
Results summarize five evaluation runs. ASR is calculated separately for each run, without pooling successes across runs.
The mean ASRs remain close to the original held-out results of 48%, 20%, and 26%, respectively, with sample standard deviations of 2.19–2.28 percentage points. These repetitions provide additional evidence of the repeatability of the observed held-out transfer performance without test-time discovery.
Appendix A details the held-out protocol and results (Table S1, lines 714–776), and Appendix U.4 reports the five repeated evaluations with the library frozen and discovery disabled (Table S20, lines 2167–2184).