- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that the indirect prompt injection generated by existing methods are less effective when they are perceived as clearly harmful and unrelated to the user’s request. The paper empirically find that agent will recover from failures and dynamically adjust the action strategies, which attackers can attack by helpful recovery guidance. They thus proposed the helpinject that presents adversarial instructions as solutions to task-related problems inferred from external context. They further introduce the helpinject-opt to be the red-teaming method for black-box agents. The paper states that helpinject achives strong attack effectiveness and helpinject-opt improves the attack effectiveness.
Critical strengths
- The paper proposes their method from a motivated problem, where it investigates the vulnerability of the tool-using agents. The paper provides the explanation of how the malicious instructions may become more persuasive when presented as prerequisites for completing tasks.
- The empirical evaluation of the attacks and the justification of the failure of existing defenses are good.
Critical weaknesses
- In figure 1, it is not intuitive how the attacker would understand the tool space of the agent in order to inject the prompt. Also, it seems like if the tools are trustworthy (e.g., they would in most of the time) then the attack would not be easy.
- From the abstract, reader is expected to see your empirical investigation of how agent would survive from the recovery and how this motivates the papers helpinject. However, it is not clear how this is conducted. It would be nice if the author could add more qualitative case study and quantitative results.
- The assumption in the treat model maybe a bit strong. The paper states that the attacker can modify content C while do not know task T. However, T could be carried by content C. For example, For tasks of emailing, C would have stronger indicator of sending emails. Modifying C is essentially modifying task in this case, which is expected to results in low task success rate afterwards.
- The attack maybe sensitive to the surrogate model based on the actual model. It would be nice if the paper could help testing its sensitivity.
Questions for authors
- Can the author provide more examples of active failure recovery can be leveraged in practice? Where does the paper construct such datasets.
- Could the paper provide a clear picture of how helpinject is motivated in practice.
- Could the authors explain more about the threat model, e.g., how content can be modified when the attacker does not have access to the task.
- Could the paper provide the sensitivity analysis of the surrogate model of when it will be successful and when it will fail.