acceptodds
Withdrawn from ICLR 2027

HelpInject: Reframing Adversarial Goals as Helpful Solutions for Indirect Prompt Injection in LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model (LLM) agents have demonstrated outstanding capabilities in decision-making and interaction with external environments, yet remain highly vulnerable to indirect prompt injection (IPI) attacks, where attackers embed malicious instructions into external content to manipulate agent actions. IPI payloads generated by existing methods are often less effective when they are perceived as clearly harmful or unrelated to the user's request. We empirically investigate agents' behavior under task disruptions and find that they frequently attempt to recover from failures and dynamically adjust their action strategies based on external guidance. This behavior creates an opportunity for attackers to redirect agents through seemingly helpful recovery guidance. Building on this observation, we propose HelpInject, an IPI framework that presents adversarial instructions as solutions to task-related problems inferred from external context. We further introduce HelpInject-Opt, a training-free red-teaming method for black-box agents. It uses surrogate likelihood scores to guide iterative payload refinement. Evaluations on multiple benchmarks across five target LLMs demonstrate that HelpInject achieves stronger attack effectiveness than the baselines. Within limited query budgets, HelpInject-Opt further improves attack effectiveness. These results reveal that aligning adversarial instructions with task completion objectives can amplify the security risks posed by IPI attacks against LLM agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Indirect Prompt Injection, Agent Security, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MberOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that the indirect prompt injection generated by existing methods are less effective when they are perceived as clearly harmful and unrelated to the user’s request. The paper empirically find that agent will recover from failures and dynamically adjust the action strategies, which attackers can attack by helpful recovery guidance. They thus proposed the helpinject that presents adversarial instructions as solutions to task-related problems inferred from external context. They further introduce the helpinject-opt to be the red-teaming method for black-box agents. The paper states that helpinject achives strong attack effectiveness and helpinject-opt improves the attack effectiveness.

Critical strengths
  1. The paper proposes their method from a motivated problem, where it investigates the vulnerability of the tool-using agents. The paper provides the explanation of how the malicious instructions may become more persuasive when presented as prerequisites for completing tasks.
  2. The empirical evaluation of the attacks and the justification of the failure of existing defenses are good.
Critical weaknesses
  1. In figure 1, it is not intuitive how the attacker would understand the tool space of the agent in order to inject the prompt. Also, it seems like if the tools are trustworthy (e.g., they would in most of the time) then the attack would not be easy.
  2. From the abstract, reader is expected to see your empirical investigation of how agent would survive from the recovery and how this motivates the papers helpinject. However, it is not clear how this is conducted. It would be nice if the author could add more qualitative case study and quantitative results.
  3. The assumption in the treat model maybe a bit strong. The paper states that the attacker can modify content C while do not know task T. However, T could be carried by content C. For example, For tasks of emailing, C would have stronger indicator of sending emails. Modifying C is essentially modifying task in this case, which is expected to results in low task success rate afterwards.
  4. The attack maybe sensitive to the surrogate model based on the actual model. It would be nice if the paper could help testing its sensitivity.
Questions for authors
  1. Can the author provide more examples of active failure recovery can be leveraged in practice? Where does the paper construct such datasets.
  2. Could the paper provide a clear picture of how helpinject is motivated in practice.
  3. Could the authors explain more about the threat model, e.g., how content can be modified when the attacker does not have access to the task.
  4. Could the paper provide the sensitivity analysis of the surrogate model of when it will be successful and when it will fail.
Reviewer_ZyhTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HelpInject, an indirect prompt injection (IPI) framework that reframes adversarial objectives as necessary actions for resolving task-related problems, thereby inducing LLM agents to perform unauthorized actions. The authors further introduce HelpInject-Opt, which uses surrogate-based conditional log-likelihood scoring to optimize attack payloads in black-box settings.

Critical strengths
  1. The paper connects agents' task-recovery behavior with indirect prompt injection and introduces a problem–solution framing strategy. This provides a plausible behavioral explanation for why task-related malicious instructions may be more effective than conventional instruction overrides.

  2. The experiments cover two benchmarks and five target models, including defense evaluations, ablation studies, random-seed analysis, and sensitivity to model configurations. These evaluations provide supporting evidence for the proposed approach.

  3. Table 3 shows that removing problem descriptions or surrogate scoring generally reduces attack success rates. Appendix A.4 further provides comparisons under matched victim-agent query budgets, helping to assess the contribution of the optimization procedure.

Critical weaknesses
  1. The paper investigates agent behavior using simulated tool failures and benign recovery suggestions. However, the actual attacks involve fabricated failure descriptions embedded in ordinary external content and malicious recovery actions. This difference makes it difficult to determine whether the observed attack effectiveness is primarily attributable to the proposed recovery mechanism.

  2. The surrogate evaluation prompt in Appendix B.3 does not include the actual user task and instead uses a generic task description. Moreover, the adversarial objective already appears in the injected content. Consequently, increases in conditional likelihood may partially reflect textual overlap rather than a genuine increase in the probability of executing the targeted tool action.

  3. AutoInject and HelpInject-Opt use different initial attack formulations, making it difficult to isolate the contribution of the optimization strategy. Although Appendix A.4 includes matched-query-budget comparisons against repeated HelpInject generation, these experiments are limited to subsets evaluated with Qwen-3.

  4. The improvements are not consistent across all settings. For example, on InjecAgent with GLM-4.7, HelpInject-Opt achieves 50.1% ASR, below ChatInject's 54.2%. Furthermore, Table 5 reports an average utility under attack of only 40.6% for HelpInject, suggesting substantial disruption to legitimate tasks. The main ASR comparisons also lack corresponding uncertainty estimates.

Questions for authors
  1. Could the authors distinguish the effects of genuine tool failures from fabricated failure messages embedded in external content through additional controlled experiments? This would help clarify whether task-recovery behavior is the primary mechanism underlying the attacks.

  2. How can the authors establish that the surrogate likelihood score predicts actual malicious tool execution rather than merely reflecting the presence or phrasing of the adversarial objective, particularly when the surrogate does not observe the real user task?

  3. Could the authors compare optimization methods using the same initial attack formulation and computational budget, and extend the matched-query-budget evaluations in Appendix A.4 to additional target models?

  4. Could the authors report confidence intervals for the main ASR results and provide a more detailed analysis of the relationship between attack success and legitimate task completion? This would help assess the robustness and practical implications of the reported improvements.

Reviewer_dxFcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HelpInject, an indirect prompt injection framework that reframes adversarial instructions as necessary solutions to task-related problems, motivated by observations of LLM agents' recovery behavior following execution failures. The paper then introduces HelpInject-Opt, a training-free black-box optimization approach that iteratively refines attack payloads using log-likelihood scores. Experiments across AgentDojo and InjecAgent demonstrate the improvement on attack success rates over the related baselines.

Critical strengths
  • The proposed method, HelpInject, achieves substantial improvements over the evaluated baselines, particularly on models that exhibit strong resistance to conventional prompt injection attacks. The paper also evaluates the proposed approach on various benchmarks across diverse models.
  • The proposed method does not require access to the models' parameters and gradients.
Critical weaknesses
  • I think the novelty of the paper is somewhat limited and the comparison with one of the important related works seems missing. The main idea of HelpInject, which presents adversarial instructions as necessary steps toward legitimate task completion, quite overlaps with previous persuasion-based injection strategies. In particular, [1] ChatInject (although the paper cited the paper) already explores persuasion-driven multi-turn dialogue generation, but the comparison appears to use only its basic chat-template variant rather than its more relevant persuasive multi-turn variants. Furthermore, the closely related concurrent work [2] LogicInception similarly exploits fabricated task dependencies to induce unauthorized actions, while introducing an explicit instruction-free, discourse-level mechanism. LogicInception is neither cited nor discussed in the paper. These similarities raise concerns about whether HelpInject introduces a genuinely new attack mechanism or primarily represents an incremental refinement of existing contextual persuasion strategies.

  • Also, the problem–solution payload construction largely relies on an auxiliary LLM to generate plausible error descriptions and a fixed template that presents the adversarial goal as a required remedy. Notably, the generic-problem variant achieves performance close to the full method in several AgentDojo domains, suggesting that much of the effectiveness may originate from the fixed prerequisite framing rather than context-specific problem inference.

  • Furthermore, HelpInject-Opt largely follows a conventional LLM-guided iterative prompt search procedure, combining candidate rewriting, top-k selection, and surrogate likelihood scoring. The comparison with AutoInject seems not well controlled. AutoInject optimizes short suffixes using a smaller policy model, whereas HelpInject-Opt freely rewrites entire payloads using a substantially larger instruction-tuned model. Other relevant automated red-teaming methods can be added to the experimental comparisons to improve the paper. Consequently, the results do not clearly establish the superiority of the proposed optimization strategy over existing black-box search methods.

  • The paper investigates recovery behavior by replacing tool outputs with explicit execution failure messages. However, in the actual HelpInject threat model, tool executions succeed normally, while attacker-controlled external content merely claims that a problem has occurred. Here, responding to an execution failure is not equivalent to trusting an unverified failure claim inside retrieved content. Consequently, the experiments do not sufficiently establish whether HelpInject specifically exploits recovery behavior or simply benefits from persuasive prerequisite framing.

[1] ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents, ICLR 2026 [2] LogicInception: Discourse-Level Indirect Prompt Injection via Causal Chain Construction. (Anonymous ACL submission, ARR May, publically available)

Questions for authors
  • How does HelpInject fundamentally differ from the persuasion-driven multi-turn variants of ChatInject and the fabricated causal-dependency approach of LogicInception? Could you provide comparisons against persuasive multi-turn ChatInject and context-aware false-error baselines under matched generation settings?

  • The motivating experiment simulates actual tool execution failures, whereas HelpInject inserts fabricated failure claims into otherwise successful tool responses. Could you experimentally distinguish recovery-specific behavior from general prerequisite framing or persuasive instruction following?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.