acceptodds
Under review as a conference paper at ICLR 2027

Asking for Trouble: The Help Channel as an Injection Surface

Abstract

Agents are increasingly trained to stop and ask users for help when a task is underspecified or ambiguous. Safety classifiers and other checks screen what a user sends to an agent, but the responder's reply to a question the agent asked is usually assumed to be a trusted channel into the agent's environment and context. We study an attacker who controls this help channel, on a text-to-SQL benchmark whose tasks are deliberately underspecified. Across eight models, a steering cue appended to a reply consistently lets the attacker choose which question the agent asks next. On four models the cue raises payload delivery from 67% to 89% of trials and raises the attack success rate (ASR) on one model, though grouped by task neither increase separates from chance. When the responder answers only one question after the cue, the increase in ASR separates from chance on three of seven models for off-task attacks over two seeds and on one model for on-task attacks. Trained injection guards have difficulty discriminating between poisoned and clean replies. The best LLM-based detector we evaluate flags 40% of injected attacks at a false-positive rate of about two percent. We test a defensive prompt that tells the agent to treat each reply as an unverified claim, check it against the database and state a conflict instead of acting on it. The prompt lowers ASR from 36% to 15%, with a change in clean task pass that does not separate from run-to-run variation. An adaptive attacker with ten attempts per trial raises the defended ASR from 0% after one attacker-written attempt to 11% over three models. However, that increase comes entirely from one model in a text-only tool mode, so whether the defense holds under adaptation is open. Because poisoned replies are often indistinguishable from genuine helpful replies, we suggest treating solicited replies as claims to verify against the task before acting on them, and limiting how much help an untrusted party may supply in high-risk applications.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent security, prompt injection, help-seeking agents, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_PjHhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes using question responses as an attack surface for prompt injection against LLM agents. The proposed method appends a ''cue'' to the end responses given to the LLM agent (e.g. from a RAG database or another agent) which push them towards asking specific questions about ambiguous tasks. The targeted answer then contains a prompt injection payload which attempts to push the agent towards deleting sensitive data etc.

The method is compared against direct prompting and approaches without a cue. Defenses and adaptive attack approaches are also studied.

Critical strengths
  • Targeting the help channel appears to be well motivated.

  • The paper tests a number of reasonable defense approaches and is honest with the findings and limitations.

Critical weaknesses

Threat Model

The threat model isn't well motivated and seems somewhat arbitrary. If I am understanding correctly, it appears that the attacker can only alter the answers to a few (or even just one) question that the agent may ask. The attack then pushes the agent towards asking this particular (set of) question(s) which contains a payload. Why wouldn't the attacker just have full control over all question answers? Why would they take this approach if they can already append further instructions to the end of the original response?

Writing

The paper writing is very hard to follow and doesn't have a linear or logical flow. The abstract and intro are primarily at fault here, as the authors quickly jump towards reporting results and findings before the threat model or attack method are even motivated or explained. 3/4 of the abstract reports findings, referring to ''attack success rates'' and ''payloads'' before these terms are even defined within the context of the paper. Attack ''payloads'' are mentioned multiple times throughout the text, but we never get a solid example until page 5. The paper's writing needs to be completely restructured.

Results

Figure 3 reports that models are more likely to ask specific questions when prompted to do so. This isn't particularly surprising, but still a good result to report. Figure 4 then shows that using the cue method works better than not using it, but this is just a direct consequence of the agent being more likely to ask the payload containing question, so it's unclear if the cue itself adds anything or if it just helps in guaranteeing that the agent actually reads the payload. This table also measures defense performance against the ''no cue'' baseline which seems weird, shouldn't the proposed attack (with the cue) be evaluated here?

Method

Overall the proposed method is just another form of prompt injection but with a slightly different delivery method, it is unclear what the contributions are if any. The core mechanism, exploiting the ''help channel'' has already been explored by other works (as explained in the "Agents that ask, and attacks on their questions" section). The only distinction from these works is that the authors "ask what the attacker gains by choosing the question". The paper does not motivate why this is an interesting question to ask, nor do the results show that it leads to any unique or interesting outcomes.

Questions for authors
  • Is the cue mechanism (changing what question the agent asks) the core contribution of this work?

  • Why (in practice) would an attacker use this approach rather than just direct prompt injection?

1 comment

We thank the reviewer for the detailed review, which led to the largest changes in the revision, especially to the threat model and the writing.

Q1: "Is the cue mechanism (changing what question the agent asks) the core contribution of this work?" Yes, and we now state it more narrowly. The contribution is that a reply can steer which question the agent asks next, on all 11 models we test on SQL. As a result, the payload attached to that question reaches the agent more often (75% to 96%, Table 2), although App. A.3 shows that the rise is not statistically clear once test cases are grouped by task. We no longer claim that steering raises attack success over direct injection (see the next answer).

Q2: "Why (in practice) would an attacker use this approach rather than just direct prompt injection?" We agree that an attacker who controls every reply gains little from the cue in our tasks. A payload written into the first reply succeeds about as often as the cued attack (49% against 50% pooled, Table 2), and we narrowed the abstract and introduction accordingly. In our SQL tasks every ambiguity concerns the same query, so a payload on the first reply is never far from the task. Steering may matter more where the agent's open questions are unrelated to each other, as in long-horizon workflows, which we name in the introduction and conclusion as future work.

Threat model: "Why wouldn't the attacker just have full control over all question answers?" The reviewer's reading was reasonable, since the submission said the attacker controls "some" replies while our experiments route every question to the attacker's source. Section 3 now gives the attacker the answer source for every question. It also explains how an attacker gains that control without breaking into the agent, for example by posting in a shared chat channel or by controlling a subagent that writes the reply.

Results: "... unclear if the cue itself adds anything or if it just helps in guaranteeing that the agent actually reads the payload." We agree with this reading. The cue's measured effect is on delivery, and once a payload is delivered we do not detect a rise in compliance (Figure 3(b)). On the three models tested, the cue still steers without its instruction clause and a placebo sentence does not steer (App. C.4), which suggests that steering is more than instruction-following.

Results: "... shouldn't the proposed attack (with the cue) be evaluated here?" We agree that this was a gap, and we now test the defensive prompt against the cued attack (Table 2(b)). The prompt lowers attack success under the cue from 51% to 20% on DeepSeek-V3.2 and from 47% to 22% on Kimi K2.5, but barely on Qwen3-235B (44% to 38%).

Method: "The core mechanism, exploiting the 'help channel' has already been explored by other works ..." We agree that ASPI is the closest work. It shows that the clarification channel is an attack surface, and our Table 1 reproduces that finding, which we cite as theirs. ASPI's agent gets one round of clarification, while ours asks several questions in turn, so an earlier reply can choose the next one. The revised Related work states which results replicate and which are new.

Writing: "The abstract and intro are primarily at fault here ..." We agree that our exposition was dense, and we restructured the abstract and introduction as suggested. The abstract now sets up the help channel and the attacker before any result, and defines the steering cue and the payload at first use. The introduction gives a concrete payload, App. D gives a full trace, and Section 4 is about a third shorter.

Reviewer_12HmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at how an attacker can sneak instructions into an AI agent’s clarification replies, nudging it toward certain follow-up questions and sometimes getting it to run unwanted SQL commands. The clearest finding is that a small added sentence can reliably shift what the agent asks next, though the evidence that this makes attacks more effective than simply inserting the malicious instruction directly is fairly limited.

Critical strengths

The strongest part is the controlled experiments: the authors show that one added sentence can shift the agent’s next question across eight models. The paper also goes beyond measuring question order to test actual harmful actions and defenses, and it’s fairly upfront about where the results are inconclusive.

Critical weaknesses

some concerns about the work:

  1. Since the attacker already controls or modifies the replies, why not just control all outputs? Also, it doesn’t seem like steering questions clearly outperform just inserting the malicious instruction directly. I’m not sure “the attacker only appends one sentence to a reply” is a particularly interesting threat model here.

  2. Also, the attack entrance seems to be just a prompt injection case where attackers can modify the tool output of ask_human. It feels like it’s basically prompt injection in tool output in general, so it doesn’t seem to add much beyond that.

  3. The evaluation is also limited to one SQL benchmark with simulated responders. That doesn’t have to be on this benchmark, right? It’s basically prompt injection in tool outputs. Also, the headline attack and defense results rely on small task sets and aren’t particularly interesting. I’m not sure what the main takeaways from these results are.

Questions for authors

See above for the main concerns.

1 comment

We thank the reviewer for the review. Their first point changed how we frame the paper, and we address it first.

W1: "... why not just control all outputs? Also, it doesn't seem like steering questions clearly outperform just inserting the malicious instruction directly." We agree with both points. The revised threat model gives the attacker the answer source for every question (Section 3), and the earlier wording that suggested only "some" replies was a mistake. Under that threat model, we do not establish that steering beats direct injection, because a payload written into the first reply succeeds about as often as the cued attack (49% against 50% pooled, Table 2), with the direction differing by model. We therefore narrowed the claim in the abstract and introduction and withdrew the one-answer-budget result that the submission used to motivate the cue. We now claim that a reply can steer which question the agent asks next on all 11 models we test, so that the payload attached to that question reaches the agent more often (75% to 96%, Table 2). This rise is not statistically clear when test cases are grouped by task (App. A.3).

W2: "... seems to be just a prompt injection case where attackers can modify the tool output of ask_human." We agree that in our harness the reply is a tool return (Section 3). The agent still treats it differently from ordinary tool output, since the same payload is followed more often as a help reply on all four models we test (Table 1), although the two conditions also differ in when and how often the payload arrives. The cue also steers without its instruction clause, while a placebo sentence does not (App. C.4), which suggests the effect is more than ordinary instruction-following. Screening the reply's text does not reliably catch the attack either. The trained injection guards we test flag poisoned and clean replies at about the same rate (Tables 5 and 6). A screen for destructive statements catches every payload that gives the statement in SQL, but it misses nearly all payloads that describe the deletion in prose and every redefinition (Section 5.2).

W3: "... limited to one SQL benchmark with simulated responders. [...] small task sets [...] I'm not sure what the main takeaways from these results are." We agree that the simulated responder is a real limitation. We repeat the steering and harm experiments on the benchmark's code-repair track, where the cue raises delivery on GPT-5.4 from 17 of 25 attacks to all 25 (Table 8), and we test the defensive prompt on eight further models (App. C.6). Table 15 (App. C.9) lists the models, tasks, attacks and repetitions of every experiment, so readers can judge each result's size. The introduction now states three takeaways as contributions. A reply can steer which question the agent asks next, filtering the reply's text does not reliably catch the attack, and defenses for this channel must be tested against rewritten payloads, since a payload rewritten as the task owner's standing requirement is executed more often under the defensive prompt (Table 3). We believe a study with human responders on a second benchmark, and long-horizon workflows such as LHAW, are the most important future directions.

Reviewer_DnDHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the influence of untrusted text. They focus on settings where A.I. agents ask users for clarification, framing this interaction pattern as a steering opportunity for attackers.

The authors claim this situation is difficult to mitigate with LLM-judges or classifiers. As an alternative, they propose a prompt-based method which warns the A.I. to treat user/tool (ask_human) text as untrusted, and directs the agent to conduct additional verification against a SQL database.

The primary contributions are deeper exploration of role-based attacks (exploits that take advantage of how LLMs assign authority to different channels), a way to consistently steer clarification requests, and a proposed mitigation strategy.

Critical strengths

Realism/Simplicity

  • The authors employ a relatively simple and straightforward setup
  • The task setting is pretty realistic, albeit a text-to-sql agent generally wouldn’t (or at least shouldn’t) have these kinds of (destructive) database privileges
Critical weaknesses

Framing

  • This paper narrowly focuses on a specific LLM-user interaction (agents asking for clarification). It’s unclear what distinguishes this from general interactions A.I. agents have with users. The literature on positional attacks, generally involves text that is injected within designated channels (offset using tags like <user></user>). This “clarification” scenario is generally not allocated its own channel. Perhaps the authors could clarify how “follow-ups” are different from other user, or tool-based attacks in general.

Scope

  • Requires SQL verification
  • Very similar to Scale AI’s ASPI (Sehwag et. al, 2026)

Presentation

  • Section 4 is quite sprawling and can be condensed
  • Section 5: condense the paragraphs about the steering cue’s effects into ~1-2 paragraphs, and add graphics to illustrate the effect (perhaps something like fig. 1)
  • Visualizations, in general, would be helpful – it’s especially hard for me to envision the steering cue, “payload” construction, and “influenced responses.” One idea could be to diagram out a single attack in its entirety.
Questions for authors
  • What best distinguishes your work from prior work (e.g. Scale's paper on "Seeking Ambiguity Clarification")
  • Is there a reason we should focus so specifically on "clarification requests," compared to general role-based attacks (I think I'm still uncertain about whether the effects we're seeing are specific to clarification or just the influence of having a conversation)
  • Overall, how should I think more generally about what "clarification" represents?
1 comment

We thank the reviewer for the close reading and for the concrete suggestions on framing and presentation, which we adopted. The changes are marked in blue in the revised PDF.

Q: "... how should I think more generally about what 'clarification' represents?" We see clarification as the agent asking what the task owner meant, for example what an ambiguous term in the request means. This differs from looking up a fact, because the agent can check a fact against the data, while the data alone cannot tell it what the owner meant. Unless the owner confirms the reply, the agent has to take it on trust. Section 3 now defines the help channel in these terms.

W1: "... how 'follow-ups' are different from other user, or tool-based attacks ..." Q: "... just the influence of having a conversation." We agree that in our harness the reply is a return value of the ask_human tool (Section 3). Having a conversation is common to both conditions we compare, since in both the agent asks its questions and gets the same correct replies. Even so, the same payload is followed more often as a help reply than as ordinary tool output on all four models we test, for example 26% against 15% on DeepSeek-V3.2 and 55% against 39% on GPT-5.6 Terra (Table 1). Because the payload arrives once in the help condition and on every search result in the tool condition, this comparison does not isolate the channel itself (App. A.3), and a design that does is future work.

W2: "Very similar to [...] ASPI ..." Q: "What best distinguishes your work from prior work ..." We agree that the two works are close, and the revised Related work states which of our findings replicate ASPI and which are new. ASPI shows that the clarification channel is an attack surface, and our Table 1 reproduces that finding on our benchmark, which we cite as theirs. ASPI's tasks leave one piece of information missing, with one round of clarification. Each of our tasks leaves three to five terms unclear, so the agent asks several questions in turn and an earlier reply can choose which comes next. ASPI names this multi-turn setting as a limitation of its design. We also test LLM-based detectors and a defensive prompt, including against an adaptive attacker.

W3: "Requires SQL verification." We agree that our defensive prompt checks replies against the database, and we no longer present it as general. App. C.7 states that the combined prompt was tested only on SQL. In an exploratory follow-up on 35 further SQL tasks, it lowers how often the agent adopts a cued redefinition by about 22 points without lowering clean task success, although we do not detect a drop in how often the result is corrupted (App. C.8). The abstract now ends on the broader point instead, which is to test any defense for this channel against rewritten payloads.

"... a text-to-sql agent generally wouldn't (or at least shouldn't) have these kinds of (destructive) database privileges." We absolutely agree. Where the task needs no writes, least privilege stops destructive payloads from executing, and we added this to the Discussion. Under read-only access the agent still issues a destructive statement in 41% of test cases, and none executes (Table 11). What remains is the payload that redefines a term, which runs as an ordinary read-only query.

W4: "Section 4 is quite sprawling [...] diagram out a single attack in its entirety." Yes, and thank you for the suggestion. Figure 1 now walks through one attack on an example task, from the cue that changes the agent's next question to the agent running the deletion, and App. D gives a recorded trace of a full attack. Section 4 is about a third shorter, with details moved to App. A.3.

Comments on the paper

1 comment

We thank the reviewers for their detailed reviews, and the changes are marked in blue in the revised PDF.

The reviewers' main concern is whether steering an agent's clarifying questions also raises attack success. On our SQL tasks, we do not find that the cue raises attack success over direct injection. Because every ambiguity concerns the same query, an attacker who controls every reply can put the payload on the first reply. However, we expect steering to matter more in 2 settings that we have not tested. First, in long-horizon workflows, the agent's early questions can be unrelated to the attacker's goal. Second, in multi-agent settings, the attacker may control only some of the answers. We also withdrew the one-answer-budget result that motivated the cue in the submission.

Instead, the revision states 3 findings. First, a reply can change which question the agent asks next. A cue appended to a correct reply steers the next question on each of the 11 models we test on SQL (Section 5.1). In our main harm experiment on 3 models, the cue raises payload delivery from 75% to 96%, although the rise is not statistically significant when we group test cases by task (App. A.3). As a result, the party that answers a question can shape what the agent asks for next. To address the concern that our results cover only SQL, we repeat these experiments on the benchmark's code-repair track. On this track, the cue raises delivery from 68% to 100% of attacks on GPT-5.4 and from 60% to 96% on GPT-5.6 Terra (Table 8).

Second, the defensive prompt lowers attack success, but its protection depends on the payload's wording and it costs clean task success. We now test the prompt against the cued attack, where it lowers attack success (Table 2). On 8 additional models, observed attack success falls with the prompt on the 4 where the attack runs often enough to test, while clean task success falls from 68% to 48%, pooled over the 8 models (App. C.6). Moreover, payloads rewritten as the task owner's standing requirement get past the prompt more often (Table 3). We therefore recommend testing defenses for this channel against rewritten payloads.

Third, screening the reply's text does not reliably catch the payload. The trained injection guards we test flag poisoned and clean replies at similar rates (Tables 5 and 6). A screen for destructive statements catches each of the 15 payloads that contain an explicit destructive statement, without flagging any of the 15 clean replies. However, on 30 other destructive payloads, the screen misses the 28 that describe the deletion in prose, and it misses every redefinition (Section 5.2). In contrast, read-only database access stops destructive payloads from running, and we added this to the Discussion.

In addition, ASPI shows that clarification replies are an attack surface, and we build on it by studying steering across several clarification turns (Section 2). In Table 1, the same payload is followed more often as a help reply than as tool output. Because the payload also arrives at different times and rates in the two conditions, this comparison does not isolate the channel. In Section 3, the attacker controls the answer source for every question.

Finally, we rewrote the abstract and introduction, shortened Section 4, and list the size of every experiment in Table 15. We agree that one benchmark with simulated responders is a real limitation. We believe the 2 untested settings above and a study with human responders are the most important future directions.

We again thank the reviewers for their detailed feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.