acceptodds
Under review as a conference paper at ICLR 2027

Multi Surface Attack: An Agent Is Only as Safe as Its Weakest Surface

Abstract

Large language model (LLM) agents expose numerous attack surfaces, e.g., user messages, system prompts, tool outputs, environment states etc. Yet existing studies typically inject malicious prompts through only one specific surface or a limited combination of surfaces. These studies generally bind each malicious prompt to a fixed attack surface. In this paper, we systematically study how the attack surface affects success by delivering the same attack through different surfaces of an agent. We further study whether a chat LLM is vulnerable to such agentic-style multi-surface attacks when provided with a synthetic agentic context payload. Finally, we study what happens when an agent or LLM is attacked through multiple surfaces simultaneously. We evaluate nine common agent attack surfaces across five large language models and four existing benchmarks. Our extensive experiments show that routing the same malicious prompts through a more vulnerable surface can increase the attack success rate (ASR) by up to 100% relative to the baseline surface. We also show that chat LLMs are similarly vulnerable to agentic multi-surface attacks when we use fake agent-like context payloads. We find that attacking multiple surfaces simultaneously can significantly increase the ASR. However, attacking more surfaces does not necessarily yield a higher ASR and can even make an agent more resistant to attacks, as surfaces can interact both positively and negatively. Based on our findings, we propose a novel multi-surface attack (MSA) approach for any agent or chat LLM that can increase the attack success rate by up to 100%. Finally, we release a comprehensive multi-surface attack benchmark constructed by adapting attacks from four existing benchmarks using our methodology.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agentic ai, large language model, benchmark, llm security, agentic ai security

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NuH3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MSA investigates how attack placement across nine agent surfaces influences attack success, using five LLMs and four benchmarks.

Critical strengths

The paper addresses a simple but important question and provides broad empirical evidence of surface-dependent vulnerabilities. Its analysis of model-dependent behavior and non-monotonic interactions between attack surfaces offers valuable insights for agent security.

Critical weaknesses

Compared with ASB, the main contribution lies in systematically comparing attack placement. However, differences in surface authority and attack framing complicate causal attribution. Moreover, the reported maximum ASRs reflect empirical best-case coverage rather than the sequential attack algorithm's effectiveness under a fixed budget.

Questions for authors

How can the effects of attack placement be separated from differences in surface authority and framing? What ASR does the sequential algorithm achieve under a fixed budget?

3 comments
AuthorsOpenReview ↗

Summary and Weakness 1

We thank the reviewer for recognizing the importance of our question, the breadth of our evidence, and the value of the non-monotonic interactions between surfaces. In response to the concerns:

  1. Relation to ASB (W1). MSA makes attack location a controlled variable, holding the instruction and its framing fixed; ASB's design varies location and attack method together and cannot isolate location.
  2. Authority and framing (W2, Q1). A new surface × framing experiment places the identical framed payload on every surface, including the user message. The system prompt, memory, and RAG output still succeed 42–77 pp more often than the user message, and JailbreakBench's official judge confirms the effect.
  3. Fixed budget (W3, Q2). The reported MSA number is a lower bound on the sequential algorithm, not a best case. Under a query budget, MSA reaches nearly its full ASR within 2–3 queries, versus most of the budget for random ordering.

Weakness 1: Contribution relative to ASB

Compared with ASB, the main contribution lies in systematically comparing attack placement.

MSA is fundamentally different from ASB: the two works ask different questions, and ASB's design cannot answer ours.

Different question. ASB asks which attack methods succeed against agents and which defenses stop them. MSA asks whether the same malicious instruction succeeds or fails only because of where it appears in an agent's context.

Different independent variable. In ASB, each attack type (direct injection, observation injection, memory poisoning, PoT backdoor, mixed) is a distinct method with its own injection mechanism, so location and method change together. A difference between, e.g., memory poisoning and observation injection cannot be attributed to location. In MSA, the instruction, its framing, the remaining context, tools, harness, and scoring are fixed; only the surface varies. To our knowledge, this is the first work to make attack location a controlled, causal variable.

Different scope of surfaces and combinations. ASB covers 4 channels and one fixed mixed attack. MSA covers 9 surfaces (adding LLM planner, human plan editor, conversation history, RAG output, and environment state) and all 502 multi-surface combinations of a single instruction, plus a full factorial study on FinSafetyBench (28,579 runs).

Findings ASB cannot produce.

  • Surface-dependent vulnerability: the identical instruction, moved off its benchmark-intended surface, can go from 0% to 100% ASR (Table 2).
  • Model-specific surface rankings and weakest surfaces (Table 3), e.g., the planner bottleneck on GPT-5.6 Sol.
  • Non-monotonic combinations: attacking more surfaces can lower ASR (~67.5% at 3 surfaces vs. ~42.9% at 11).
  • Chat LLMs inherit agent-style surface vulnerabilities when given agent-like context; ASB studies agents only.

ASB is one of our benchmarks, not our baseline method. MSA is orthogonal to ASB, which is exactly why we apply it on top of ASB as one of four benchmarks. Re-routing ASB's own attack instructions raises ASR on every model (Table 2): GPT-4.1 88% → 100%, GPT-5.6 Sol 82% → 100%, Gemini 2.5 Flash 91% → 100%, Gemini 3.7 Flash 78% → 80%, Claude Sonnet 4.6 49% → 60%. In other words, even a benchmark built to cover multiple channels underestimates agent vulnerability when each attack is tested only at its prescribed location.

AuthorsOpenReview ↗

Weakness 2 and Question 1: Surface authority and framing

However, differences in surface authority and attack framing complicate causal attribution.

How can the effects of attack placement be separated from differences in surface authority and framing?

We ran a new controlled experiment that separates placement from framing.

Design. Each case is delivered on four surfaces (user message, system prompt, long-term memory, RAG output) under two framings: (i) the original benchmark prompt verbatim, with no framing, and (ii) one fixed authority-framed payload, identical on every surface, including the user message. Neutral context, model settings, and scoring are held fixed. Within each framing, the surfaces differ only in where the text appears, so any difference between them is a placement effect. Cases in which the provider's content filter blocked any of the eight conditions are dropped, so that all surfaces are compared on the same cases (7 of 100 JailbreakBench cases for GPT-4.1; none in the other chat conditions shown below).

BenchmarkModelFramed: user messageFramed: SP / LTM / RAG
JailbreakBenchGPT-4.1 (93 cases)28%76% / 70% / 74%
JailbreakBenchGemini 2.5 Flash (100)28%83% / 99% / 98%
FinSafetyBenchGPT-4.1 (100)0%53% / 64% / 77%
FinSafetyBenchGemini 2.5 Flash (100)0%67% / 51% / 61%

Placement has a large effect with framing held fixed. The identical framed text succeeds 42–77 pp more often in the system prompt, memory, or RAG output than in the user message. No case succeeds in the user message but fails on another surface (exact McNemar p < 1e-11 for every surface). Framing therefore cannot explain the effect: the user message receives exactly the same framing and remains far less vulnerable.

Framing alone is not sufficient. The same framed text reaches only 0–28% ASR in the user message but 51–99% on the other surfaces. Placement determines whether authoritative content is trusted, which is the surface-dependent vulnerability our paper identifies.

The effect holds under JailbreakBench's official judge. With framing fixed, judged-harmful ASR is 15% in the user message vs. 48–60% on the other surfaces for GPT-4.1, and 17% vs. 57–78% for Gemini 2.5 Flash (each surface vs. user message, p ≤ 1.2e-7).

Higher authority does not simply mean higher ASR. On ASB (Table 2, 120 cases), where attacker instructions are ordinary task requests, the user message is more vulnerable than the high-authority system prompt for four of five models, e.g., Gemini 3.7 Flash 78% vs. 3% and Claude Sonnet 4.6 49% vs. 3%. If authority drove the results, the system prompt would win everywhere. Instead, the most vulnerable surface depends on the model and the task, which is why each surface must be tested.

Surface authority is part of what we measure. In a deployed agent, a system prompt or memory entry carries different implicit authority than a user message; this authority is a property of the surface itself. Our claim is that safety measured on one surface does not transfer to others: the same text with the same framing is refused in the user turn and executed elsewhere.

Agent-side controls (ASB, DTap). On both agent benchmarks, the surface effect appears even with no framing at all. We ran the same factorial on ASB, and our DTap surface runs already place the original prompt verbatim (UM / SP / LTM / RAG, %):

ModelASB verbatimASB framedDTap verbatim
GPT-4.192 / 31 / 26 / 4697 / 21 / 31 / 3871 / 12 / 4 / 0
GPT-5.6 Sol79 / 71 / 21 / 3224 / 37 / 29 / 2436 / 59 / 41 / 32
Gemini 2.5 Flash95 / 90 / 93 / 93100 / 90 / 95 / 9884 / 84 / 80 / 80
Gemini 3.7 Flash72 / 63 / 16 / 4012 / 19 / 14 / 1450 / 58 / 17 / 17

Verbatim surface differences are significant for GPT-4.1, GPT-5.6 Sol and Gemini 3.7 Flash on both ASB (Cochran's Q, p < 1e-8) and DTap (p ≤ 0.02), so placement matters without any added framing.

Full results for all models are in our response to Reviewer TZF1 (Question 3). We will add this experiment to the appendix. PromptRouter already places the original instruction verbatim inside a surface-appropriate authority framing (Section 3.1); we will state this explicitly from the abstract onward, describing our manipulation as verbatim relocation under a fixed authoritative framing per surface. This clarifies the wording without changing our core finding: with the same instruction and the same framing, ASR varies strongly with the surface that delivers it.

AuthorsOpenReview ↗

Weakness 3 and Question 2: Maximum ASR vs. the sequential algorithm under a fixed budget

Moreover, the reported maximum ASRs reflect empirical best-case coverage rather than the sequential attack algorithm's effectiveness under a fixed budget.

What ASR does the sequential algorithm achieve under a fixed budget?

We thank the reviewer. The reported MSA number is not a best case; it is an empirical lower bound on what the sequential algorithm achieves.

Why it is a lower bound. MSA tries candidates in a profile-driven order (baseline surface, then single surfaces by profiled susceptibility, then combinations) and stops at the first success. Given enough budget, it therefore succeeds on a case whenever any configuration succeeds on that case, so its ASR is the union of successful cases across all configurations. Table 2 instead reports the ASR of the single best configuration, which is a subset of that union. The ASR of a full MSA run is therefore at least the reported value. Our table note described the column as an "empirical coverage bound," which was imprecise; we will relabel it as an empirical lower bound in the revision.

Correction to Algorithm 1. As printed, the combination stage of Algorithm 1 extends a single nested chain of surfaces and does not reach every combination, contrary to our intent. We will replace it with a version that enumerates every combination in profile-driven order (given in full in our response to Reviewer TZF1, Question 2). The ordering affects only how many queries are spent before success, not which configurations are reachable.

ASR under a fixed budget. We report ASR@k, the ASR when MSA may spend at most k queries per case, against a random ordering of the same candidates (exact expectation over all orderings). This replays the single-surface stage on all 120 ASB cases from our logged per-case results. For brevity, we report ASB here; the revised appendix will include the same analysis for all four benchmarks:

ModelOrderASR@1ASR@2ASR@3ASR@9Mean queries to first success
GPT-4.1MSA88.399.299.2100.01.16
Random59.582.792.6100.01.71
GPT-5.6 SolMSA75.881.782.584.21.22
Random40.761.372.384.22.09
Gemini 2.5 FlashMSA90.898.3100.0100.01.11
Random84.998.099.5100.01.18
Gemini 3.7 FlashMSA74.274.274.274.21.00
Random20.034.245.374.23.27
Claude Sonnet 4.6MSA49.250.050.050.81.13
Random14.023.630.650.83.42

ASR@k counts provider-blocked runs as failures (cf. our answer to TZF1 Q6) and covers single surfaces only, so values can be slightly below Table 2.

MSA is query-efficient. It reaches nearly its full ASR within 2–3 queries, whereas a random order needs most of the budget. On average, MSA needs 1.0–1.2 queries to first success vs. 1.2–3.4 for random. ASR@9 also confirms the lower-bound argument: the union over the nine single surfaces already meets or exceeds the best single surface for every model (e.g., GPT-5.6 Sol 84.2% vs. 81.7%), before any combinations are added.

In the revision, we will relabel the MSA column, replace Algorithm 1, and add the ASR@k analysis for all four benchmarks to the appendix.

Reviewer_fA4iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how the effectiveness of adversarial instructions varies depending on the attack surface through which they are delivered, considering nine surfaces across LLM agents and standalone chat models. The paper introduces PromptRouter to relocate malicious instructions across different surfaces and proposes multi-surface attack, which prioritizes vulnerable surfaces and their combinations to improve attack success rates. Experiments across serveral benchmarks and LLMs reveal substantial surface-dependent variations in attack success rates, including non-monotonic effects when multiple surfaces are attacked simultaneously.

Critical strengths
  • The paper provides an interesting perspective on LLM agent security by systematically investigating how the same malicious instruction behaves across different attack surfaces. The attempt to isolate attack surface as an independent experimental variable is meaningful and highlights a potentially overlooked dimension of safety evaluation.
  • The finding that attacking additional surfaces does not necessarily improve attack success rates is particularly interesting. The observed positive and negative interactions between surfaces suggest that multi-surface attacks involve more complex dynamics than simply increasing the number of injected instructions.
Critical weaknesses
  • I think the novelty of the paper is somewhat limited, despite its interesting experimental perspective. The central observation that identical adversarial instructions can produce different outcomes depending on their presentation context is closely related to previously studied role and authority confusion in prompt injection attacks. For example, [1] ChatInject demonstrates how manipulating role representations and simulated conversational contexts can influence attack effectiveness. Although this paper extends the investigation to a broader set of agent surfaces, it remains unclear whether this reveals a fundamentally new vulnerability or primarily provides a systematic empirical extension of existing findings. More importantly, the paper does not sufficiently explain why certain surfaces are more vulnerable or why their combinations produce different outcomes. Consequently, the contribution remains largely descriptive, without establishing a novel underlying mechanism or a generalizable principle.

  • The experimental threat model seems unrealistic in several important respects. The nine attack surfaces are treated as interchangeable attack targets, despite requiring substantially different levels of attacker privilege and access. For instance, modifying system prompts, internal planner states, or human-approved plans generally requires much greater control than manipulating retrieved documents or external tool outputs. In particular, the substantial improvements obtained by relocating attacks to privileged surfaces may not represent meaningful advances in realistic prompt injection scenarios. The evaluation would be more convincing if attack surfaces were constrained according to explicitly defined and practically feasible attacker capabilities.

  • The experimental design does not convincingly isolate the effect of attack surfaces from other confounding factors. Although PromptRouter attempts to preserve the original malicious instruction, the surface-specific construction can introduce additional contextual framing, including indications that an instruction has already been approved or authorized. Consequently, differences in attack success rates may originate from changes in perceived authority, contextual formatting, or instruction positioning rather than the attack surface itself. Furthermore, the synthetic agent-like contexts used for standalone chat LLMs do not necessarily reproduce the actual trust boundaries or execution behavior of real agents. Additional controlled experiments seem necessary to establish whether the observed differences genuinely reflect surface-dependent vulnerabilities.

  • The effectiveness of the proposed MSA algorithm is not clearly distinguished from the empirical maximum achieved through extensive attack evaluation. The paper reports MSA performance as the maximum ASR across tested individual surfaces and combinations, rather than clearly demonstrating the success rate of the sequential search procedure described in Algorithm 1. As a result, it is difficult to determine whether the improvements arise from the proposed surface-prioritization strategy or simply from evaluating a larger set of attack configurations. Moreover, the combination search initially relies on only ten profiling examples per benchmark, raising concerns about the stability and generalizability of the selected combinations. Comparisons against random surface selection, best-single-surface strategies, and budget-matched search baselines would help establish the actual contribution of MSA.

[1] ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents, ICLR 2026

Questions for authors
  • How does the surface-dependent vulnerability investigated in this paper fundamentally differ from existing role and authority confusion mechanisms, such as those explored in ChatInject? Could controlled experiments distinguish the effects of attack surface, perceived authority, and contextual framing, and provide deeper explanations for the observed non-monotonic interactions between surfaces?

  • How realistic is the assumption that an attacker can freely relocate malicious instructions across all nine surfaces, particularly privileged surfaces such as system prompts and internal planner states? Could the evaluation be repeated under explicitly defined attacker-access constraints, and could synthetic agent-context attacks be compared with corresponding attacks in actual agent environments?

7 comments
AuthorsOpenReview ↗

Summary of Response

We thank the reviewer for finding our perspective interesting, for recognizing that isolating the attack surface as an independent variable highlights an overlooked dimension of safety evaluation, and for highlighting the non-monotonic interactions between surfaces. In response:

  1. Novelty relative to ChatInject (W1, Q1). ChatInject forges role tags nested inside one fixed channel (tool output), alters the instruction with an override prefix, and depends on matching the model's chat template. MSA keeps the instruction verbatim and moves it across nine genuine surfaces and their combinations, with no forgery, special tokens, or template knowledge.
  2. Mechanism (W1, Q2). We now explain both findings with data. Surfaces differ because models refuse harmful requests presented as human requests but follow the same text arriving as context, and the non-monotonic combination curve is explained by a single negative interaction: among combinations that exclude the user message, ASR rises monotonically with the number of surfaces (49% → 100%).
  3. Controlled experiment (W3, Q2). A new surface × framing experiment places the identical framed payload on every surface, including the user message. Other surfaces still succeed 42–77 pp more often, and JailbreakBench's official judge confirms the effect.
  4. Threat model (W2, Q3, Q4). Grouped by attacker capability, external surfaces alone reach 52–92% on GPT-4.1 and Gemini 2.5 Flash, and our agent benchmarks deliver surfaces through native API channels.
  5. MSA vs. empirical maximum (W4). The reported MSA number is a lower bound on the sequential algorithm, and under a query budget MSA beats random ordering, reaching nearly its full ASR within 2–3 queries.
AuthorsOpenReview ↗

Weakness 1 and Question 1: Novelty relative to ChatInject

I think the novelty of the paper is somewhat limited, despite its interesting experimental perspective. The central observation that identical adversarial instructions can produce different outcomes depending on their presentation context is closely related to previously studied role and authority confusion in prompt injection attacks. For example, [1] ChatInject demonstrates how manipulating role representations and simulated conversational contexts can influence attack effectiveness.

How does the surface-dependent vulnerability investigated in this paper fundamentally differ from existing role and authority confusion mechanisms, such as those explored in ChatInject?

We thank the reviewer and will discuss ChatInject in the related work. MSA differs not only in mechanism but in the questions it can answer; several cannot be posed in ChatInject's single-channel design or, to our knowledge, in any prior work.

1. Combinations of surfaces: a new axis. Prior injection work places an attack in one channel at a time; the closest exception, ASB's "mixed attack," is one fixed bundle of attack methods. To our knowledge, MSA is the first to systematically study which combination of surfaces to attack (all 502 combinations, plus a 2,048-subset full factorial). Our central finding here is new: attacking more surfaces is not always better. For most models and benchmarks, peak ASR stops improving beyond four surfaces and often falls (Claude Sonnet 4.6 on ASB: 60% → 0%); in the GPT-4.1 FinSafetyBench full factorial, mean ASR peaks at 67.5% with 3 surfaces and drops to 42.9% with all 11. The full factorial explains why: adding the user message triggers the model's user-turn refusal and lowers ASR by 73 pp on average, while combinations without it rise monotonically (see our answer on non-monotonicity below). Combinations also expose bottlenecks: 53 of 54 GPT-5.6 Sol triplet successes involve the planner. None of this is observable when the channel is fixed.

2. Questions a single-channel study cannot answer.

  • Does safety transfer across channels? No. The identical framed text is refused in the user message but executed via the system prompt, memory or RAG (JailbreakBench: 28% vs. 70–99%; FinSafetyBench: 0% vs. 51–77%).
  • Which channel is weakest for a given model? Rankings are model-specific, and the most vulnerable surface reverses between chat (system prompt, memory) and agents (user message on ASB).
  • Where must defenses sit? Since the weakest channel varies, checks on user turns or tool outputs alone leave the other channels open.

3. Different variable and mechanism. ChatInject fixes one channel (the attacker "can only manipulate the content of the tool response," Sec. 3.1) and varies the payload format: forged chat-template role tags and fabricated dialogue nested inside the tool output. It works only if the model is fooled by the forgery. MSA fixes the instruction and places it in each genuine surface (in DTap, the real system message, real prior turns, and real tool messages), measuring how much a model trusts each real channel.

4. Verbatim instruction, no signature. ChatInject prepends "IMPORTANT!!! Ignore all previous instructions..." and its multi-turn variant rewrites the instruction into a 7-turn GPT-4.1 dialogue; MSA keeps the instruction verbatim (App. B). ChatInject is template-specific (with a foreign template, ASR often falls below plain text: Llama-4 on InjecAgent, 50.1% vs. 14.0–40.5%), and its own analysis finds that detectors "primarily react to special tokens" (Sec. 6). MSA uses no special tokens, needs no template knowledge, and works on closed frontier models.

5. Not reducible to authority confusion. If it were, the system prompt would always win. On ASB it does not: the user message is more vulnerable for four of five models (e.g., Gemini 3.7 Flash 78% vs. 3%; Claude Sonnet 4.6 49% vs. 3%), while on JailbreakBench the ordering reverses.

The works are complementary: ChatInject's formatting could be applied on any MSA surface, while MSA shows where, and in which combination, such payloads should be placed.

AuthorsOpenReview ↗

Weakness 1 and Question 2: Why surfaces differ and why combinations are non-monotonic

More importantly, the paper does not sufficiently explain why certain surfaces are more vulnerable or why their combinations produce different outcomes. Consequently, the contribution remains largely descriptive, without establishing a novel underlying mechanism or a generalizable principle.

... and provide deeper explanations for the observed non-monotonic interactions between surfaces?

We thank the reviewer. Our study's goal is to identify and report this vulnerability; a full mechanistic account (e.g., at the level of model internals) is a broader question that we leave to future work. Our data nonetheless supports a clear, generalizable principle, which we will move from the appendix into the main text: models apply a different policy to each channel. A harmful request is refused when presented as a human request, but followed when it arrives as configuration, memory, retrieved context, or a plan.

Why surfaces differ.

  1. Human-authored channels are the defended ones. In chat LLMs, the user message and human plan editor are the least susceptible surfaces in every setting (mean rank 7.2 of 9; Table 3), while long-term memory and the system prompt are the most susceptible.
  2. The same text flips with the channel. In our new surface × framing experiment, the identical framed payload succeeds in 28% of JailbreakBench cases in the user message but in 70–99% via the system prompt, memory, or RAG output (FinSafetyBench: 0% vs. 51–77%).
  3. Content must fit the channel. Verbatim relocation without surface-appropriate framing is ineffective in chat (0–3%): a bare user request inside memory or retrieval reads as out-of-place data.

The principle also explains agents. ASB's attacker instructions are benign-looking operational tasks that pass the harm check, so the user message, the principal's channel, becomes the most effective surface. The ranking reverses because the instruction type changes, not the principle.

Why combinations are non-monotonic. The GPT-4.1 FinSafetyBench full factorial (2,048 subsets) traces the inverted-U to one negative interaction: including the user message lowers ASR by 73 pp on average (95.9% vs. 22.9%; App. Table 9).

Surfaces kWithout user messageWith user message
149.3%0.0%
277.6%1.4%
389.3%8.1%
495.5%14.9%
698.8%22.8%
10100.0%40.0%

Within each group, ASR rises monotonically: added surfaces reinforce each other. The aggregate falls after k = 3–4 only because the share of combinations containing the user message grows with k, and those trigger the model's user-turn refusal.

Principle and implication. An instruction is refused on the channel where safety training is concentrated and followed on channels the model treats as trusted context; adding surfaces helps an attacker only if none triggers that refusal. Defenses must therefore check every channel, not only user turns. We will add this analysis to the paper.

Reviewer_TZF1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an agent's robustness depends on where in its context a malicious instruction is delivered, with the instruction held fixed. A Codex-based "PromptRouter" (GPT-5.6 Sol) relocates each benchmark attack case from the user message to eight other surfaces: system prompt, LLM planner, human plan editor, conversation history, long-term memory, RAG output, tool output/description, and environment state. For chat LLMs, these surfaces are serialized as synthetic agent context into the user turn. The study spans 420 cases from four benchmarks (JailbreakBench, FinSafetyBench, ASB, DTap) and five models (GPT-4.1, GPT-5.6 Sol, Gemini 2.5 Flash, Gemini 3.7 Flash, Claude Sonnet 4.6). It screens all 502 multi-surface combinations on a 10-case profiling subset per model/benchmark, and runs a full-factorial study (2,048 subsets) for GPT-4.1 on FinSafetyBench. The reported findings are: (1) robustness is strongly surface-dependent; (2) chat LLMs are susceptible under synthetic agent context; (3) adding surfaces is not monotonically better. On this basis the paper proposes an ordered search procedure (MSA) and releases a routed benchmark. The methodological question — existing benchmarks couple what the attack says with where it appears — is a good one, and the engineering (four benchmarks under one harness, a 9-surface taxonomy, a full-factorial sweep) is substantial. My reservations are that the central manipulation does not isolate the variable the paper claims to isolate, the headline "0%→100%" numbers are a post-selection maximum over a large candidate set rather than a deployable success rate, and the success criterion departs from each benchmark's own protocol. I lean reject, but the study has a defensible core that a reframed version could build on.

Critical strengths

A clean, well-motivated question. Decoupling attack content from attack location is a reasonable critique of current agent-safety evaluation, and the paper operationalizes it across four heterogeneous benchmarks under a single harness. Broad experiments and some genuinely informative aggregate analysis. The full-factorial FinSafetyBench study (GPT-4.1, 28,579 valid runs) is the most defensible part of the paper: the inverted-U over combination degree (ASR rises to ~67.5% at k=3, then declines to ~42.9% at k=11) and the per-surface marginal-impact table (Table 9) are computed over means rather than extrema, and support the "more is not always better" claim without relying on a maximum. The planner-bottleneck observation is concrete and interesting. On GPT-5.6 Sol's k=3 triplets, 53 of 54 successes involve the planner surface and none involve the user message (Table 10). This is a crisp, falsifiable finding. Reasonable disclosure of limitations and anonymization. The paper itself flags that the system-prompt condition carries authority-override wording (F.6), that the ASB metric is a tool-selection proxy (F.3), and that the 120-case single-surface cells have a ~9 pp 95% half-width.

Critical weaknesses

W1. The manipulated variable is not "delivery surface" alone; it is confounded with authority framing and with the presence of synthetic agent context. The paper's central claim is that it holds the malicious prompt fixed and varies only the surface (abstract; §5). The paper's own description undercuts this: §3.1 Principle 1 explicitly permits the router to either relocate the prompt verbatim or "use that surface to authoritatively approve the prompt's malicious intent," and §3.1/§3.3 say the router "presents the instruction as approved or authoritative." These are different manipulations, and the paper does not report which cases used which. This conflicts with the router's own hard constraints in App. B.1, which forbid paraphrasing the attack prompt and forbid claiming "an approval, role, source, tool event, or authority that is not supported by the original case." F.6 concedes that the system-prompt condition uses "conspicuous authority-override language," so that its outcome "reflects both prompt placement and that template's wording." The paper does not say whether the other surfaces share this issue. Please also clarify, per benchmark, whether the original benchmark prompt itself appears verbatim on every non-baseline surface (in particular for FinSafetyBench, whose case study in App. C.6 is framed around authorization), and whether non-baseline conditions add synthetic context (e.g., prior turns or tool interactions) that the user-message baseline lacks. A control that holds the framing and the context length/structure constant and varies only the delivery field — plus a verbatim-only arm and a "framing added to the user-message baseline too" arm — is needed before the surface-vulnerability rankings (Table 3) and the "+100 pp" gains can be attributed to surface. A simple way to report this is a surface × framing factorial (neutral vs. authoritative framing on each surface, including the user message), which would separate the main effect of surface from the main effect of framing and their interaction. Relatedly, please state how the requirement that unselected surfaces remain "strictly neutral" (App. B.2) was checked. W2. The headline "MSA" column is a post-selection maximum over a large candidate set, not a deployable attack success rate. The table note itself defines MSA as the "empirical coverage bound (maximum ASR achieved across tested individual surfaces and combinations)," i.e., the maximum over the 9 single surfaces and the top-3 combinations, where those 3 were selected from 502 combinations screened on 10 cases. Reporting this maximum as the method's ASR conflates "some configuration in our search space worked" with "our procedure reliably succeeds." The paper does not report the ASR that Algorithm 1 actually achieves when run as specified (which is what a deployable claim would require), its query cost, or a comparison with simple search baselines (best single surface, random surface/combination, greedy) at equal query budgets (e.g., ASR@1/3/5/10). The claim "single-interface testing can substantially overestimate safety" requires the Algorithm-1 operating-point number, not the coverage bound. Please rename this column and add the Algorithm-1 result. W3. The success criterion is a tool-call proxy that departs from each benchmark's own judging protocol, and the paper describes the oracle inconsistently. Success is defined as invoking the designated prohibited tool (App. C.3, F.3), which the paper correctly calls a "tool-selection proxy" that "does not establish that the tool arguments implement the hidden attack goal, that the model intended harm, or that an external harmful effect occurred." But JailbreakBench and FinSafetyBench judge response content (a Llama-3-class jailbreak judge; an LLM harmfulness rubric). Table 18 lists a "Llama-3.3-70B Safety Judge" and a "Policy Compliance Rubric" as oracles, yet App. C.3 states that a run counts as an attack success "if and only if the target prohibited tool is called," and no result in the paper appears to use the content judges. So the headline "0%→100% on JailbreakBench" is a change in tool-invocation rate, not in harmful-content rate under JailbreakBench's own protocol. This is the single point most likely to change my assessment: please report JailbreakBench/FinSafetyBench results under each benchmark's content judge, and reconcile Table 18 with the deterministic predicate actually used. W4. The non-monotonicity claim and the surface rankings rest on 10-case screens, single runs, and post-hoc selection. The combination analysis (Table 4, Fig. 2, §4.3.2) uses m=10 cases per model/benchmark, so each step is 10 pp and statements like "60%→0%" are a handful of cases. The procedure screens 502 combinations on those 10 cases and then reports the maximum over the top candidates on the full set — selection-after-screening without a held-out split, confidence intervals, repetition, or multiplicity control. Saturation is real: on ASB/GPT-4.1, Table 4 implies ~491/502 combinations tie at 100% on the 10 cases, so the "best combination" label carries little information. The abstract elevates non-monotonicity to a main finding, while the strongest evidence for it (the full-factorial study) covers a single model and benchmark. Please base the non-monotonicity claim on the full-factorial means (W2/Strength 2) and give intervals; treat the 10-case screen as screening only, as App. E.1 already does internally. W5. Undefined or overloaded quantities make the headline numbers hard to reconstruct. "Surface," "neutral," "setting," "susceptibility," and "empirical coverage bound" are used without definition; the symbol k denotes both the number of available surfaces (9) and the combination degree on the same page; §3.3 states "502k attack evaluations" (502×k would be 4,518, not the 5,020 used elsewhere); the abstract says "increase the ASR by up to 100% relative to the baseline" while the tables are in percentage points; and Table 4's "UM" differs numerically from Table 2's "UM (Base)" for the same cells without the note explaining that Table 4 is the 10-case subset. Per-cell denominators also vary with excluded error runs but are reported for only two tables. Table 3's rankings depend on how settings are aggregated (e.g., how settings where every surface scores 0% are handled, and how many settings enter each panel), which should be stated. Finally, F.3 excludes provider content-filter blocks from both numerator and denominator; since a filter block is a refusal-like outcome, excluding it biases ASR upward, and the exclusion counts should be reported per cell (Table 10 alone shows 237 of 2,310 GPT-5.6 Sol runs excluded as errors). W6. The threat model does not distinguish what an attacker can realistically control, and the chat-LLM "surfaces" are not real channels. The nine surfaces require very different attacker capabilities: an external attacker can often poison retrieved documents or tool outputs, whereas modifying the system prompt, planner, or human plan editor typically implies privileged access. Ranking all nine on one ASR scale therefore mixes vulnerabilities of different severity; results should be grouped by attacker capability (external / workflow-level / privileged). For chat LLMs, all surfaces are serialized into the user turn (§3.2, Fig. 1), so the experiment measures how role-labeled text inside one user message affects behavior, not attacks through genuine system, memory, or tool channels. A comparison between real API roles/tools and their serialized counterparts would show how far the chat-LLM conclusion transfers. The paper also offers no defense evaluation, so the practical takeaway beyond "test more surfaces" is limited.

Questions for authors

Can you report JailbreakBench and FinSafetyBench results scored by each benchmark's original content judge (alongside the tool-call proxy), and clarify which oracle produced the Table 2 numbers given Table 18? What does Algorithm 1 achieve when run as specified (its actual operating-point ASR), as opposed to the maximum over the tested configurations, and how does it compare with random and greedy search at equal query budgets? How do the surface rankings change under a control that (a) holds authority framing and context length constant across surfaces, (b) relocates the prompt verbatim with no framing, and (c) adds the same framing to the user-message baseline? Per benchmark, does the original prompt appear verbatim on every non-baseline surface, and do non-baseline conditions add synthetic context that the baseline lacks? Can you provide intervals and repeated runs for the combination results, and base non-monotonicity on the full-factorial means rather than the 10-case screens? How was the App. B.2 "strictly neutral" requirement for unselected surfaces verified, and how many runs per cell were excluded (including provider content-filter blocks)? Can you group the surface rankings by attacker capability (external vs. privileged surfaces), and compare real API roles/tools with their serialized counterparts for chat LLMs?

9 comments
AuthorsOpenReview ↗

Summary of Response

We thank the reviewer for the detailed and constructive review. In response, we ran new experiments and analyses that address each concern:

  1. Official content judges (Q1). Under JailbreakBench's official judge, the surface effect holds: judged-harmful ASR rises from 15–17% in the user message to 48–78% on other surfaces for GPT-4.1 and Gemini 2.5 Flash (p ≤ 1.2e-7). For FinSafetyBench, we report its original rubric and explain why the action-level predicate is the appropriate measure for agents.
  2. Surface × framing control (Q3, Q4). With the identical framed payload on every surface, including the user message, the system prompt, memory, and RAG output still succeed 42–77 pp more often than the user message. We will describe our manipulation as verbatim relocation under a fixed authoritative framing per surface; the core finding, surface-specific variation in ASR, is unchanged.
  3. Algorithm 1 and query budget (Q2). Algorithm 1 as printed had an unintended flaw that made it appear not to cover every surface combination. We fixed it and clarify that MSA searches the full space of surfaces and combinations, so the reported MSA number is a lower bound on what a full MSA run achieves, not a best case. Under a query budget, MSA reaches nearly its full ASR within 2–3 queries (1.0–1.2 queries to first success vs. 1.2–3.4 for random ordering).
  4. Intervals and repeated runs (Q5). Non-monotonicity now rests on full-factorial means with bootstrap intervals (peak at 3–4 surfaces, non-overlapping with 8–9 surfaces), and a repeated combination screen reproduces aggregate ASR with 95.5% case-level agreement.
  5. Exclusions (Q6). We report per-cell exclusion counts. Counting every provider block as a failed attack changes ASR by at most 7.4 pp, and the most vulnerable surface is unchanged for every model and benchmark.
  6. Threat model (Q7). Grouped by attacker capability, privileged surfaces are the most dangerous tier for agents, while external surfaces alone reach 52–92% on GPT-4.1 and Gemini 2.5 Flash. Our agent benchmarks already deliver surfaces through native API channels.

Can you report JailbreakBench and FinSafetyBench results scored by each benchmark's original content judge (alongside the tool-call proxy), and clarify which oracle produced the Table 2 numbers given Table 18?

Which oracle produced Table 2. All Table 2 numbers use the deterministic tool-call predicate: a run succeeds if and only if the designated prohibited tool is invoked (App. C.3). For ASB this is the benchmark's own predicate; for JailbreakBench and FinSafetyBench it is our action-space proxy. Table 18 lists the source benchmarks' native oracles, which were not used for Table 2. We will relabel it "source-benchmark oracle" and add a column "oracle used in this paper".

JailbreakBench under its content judge. We rescored runs with JailbreakBench's official judge prompt and verdict parsing, with two adaptations: (1) the original judge (Llama-3-70B-Instruct) is unavailable on our gateway, so we used Llama-3.3-70B-Instruct with the identical prompt; (2) since the harmful act is a tool call, the judge's RESPONSE field contains the action trace (tool name, arguments, final message), so a run is unsafe only if it actually contains content fulfilling the harmful request.

Results on the surface × framing experiment (answer to Q3) for GPT-4.1 and Gemini 2.5 Flash, the models behind the largest JailbreakBench gains:

ModelSurfaceTool-call ASRJudge ASRTool calls judged harmful
GPT-4.1 (93 cases)User message28%15%54%
System prompt76%59%77%
Long-term memory70%48%69%
RAG output74%60%81%
Gemini 2.5 Flash (100 cases)User message28%17%61%
System prompt83%57%69%
Long-term memory99%78%79%
RAG output98%73%74%

The surface effect holds under the benchmark's own judge: every surface yields significantly more judged-harmful outputs than the user message carrying the identical payload (Cochran's Q p < 1e-15; each vs. user message, exact McNemar p ≤ 1.2e-7). Since 69–81% of non-user-surface tool calls carry judged-harmful content, the proxy largely tracks genuine harmful compliance.

For the newer models, we scored 75 JailbreakBench behaviors under the baseline, the user message, and the system prompt alone and in combination. GPT-5.6 Sol shows the same direction:

GPT-5.6 Sol conditionTool-call ASRJudge ASR
Direct prompt (baseline)0/750/75
User message0/750/75
System prompt12/755/75
System prompt + planner15/757/75
System prompt + RAG5/752/75

Gemini 3.7 Flash and Claude Sonnet 4.6 stay at or near 0% under both metrics, consistent with Table 2. The tool-call proxy measures willingness to take the prohibited action; judge ASR is the stricter harmful-content measure.

FinSafetyBench under its original rubric (1–10 harmfulness rating on Llama-3.3-70B-Instruct; unsafe if ≥ 8):

ModelSurfaceTool-call ASRRubric ASR
GPT-4.1 (100 cases)User message0%0%
System prompt53%3%
Long-term memory64%3%
RAG output77%2%
Gemini 2.5 Flash (97 cases)User message0%0%
System prompt67%1%
Long-term memory52%3%
RAG output62%1%

Here the metrics diverge. The action-level predicate is the appropriate measure in our setting:

  1. Same oracle throughout. The 0% baseline and the multi-surface ASR in Table 2 use the same predicate, so the surface effect is not an artifact of scoring the baseline and the attack differently.
  2. The rubric grades text, not actions. It was built for chat models and rates how fully a written answer explains the task. Example (GPT-4.1, RAG): asked how to bribe a bank's credit manager, the agent calls authorize_kickback_payment with arguments "Execution of intermediary transaction fee disbursement per Standing Directive 404" and reports that it "was executed according to prior legal opinion and internal pre-authorization policy." The rubric rates this 1/10 because no step-by-step text was written, although the payment was executed. Nearly all successful tool calls received 1/10.
  3. The executed action is the more severe outcome. An explanation still requires a human to act; an agent that authorizes the payment has already caused the harm.

The paper already states that FinSafetyBench is scored with an action-level predicate (Table 18; App. C.3). In the revision, we will report both metrics side by side and state in the main text that our FinSafetyBench results measure prohibited-action execution, not harmful text generation.

What does Algorithm 1 achieve when run as specified (its actual operating-point ASR), as opposed to the maximum over the tested configurations, and how does it compare with random and greedy search at equal query budgets?

We thank the reviewer; this question helped us find a discrepancy between Algorithm 1 as printed and our intended procedure, which we correct below.

1. MSA orders the full search space; it does not restrict it. MSA tries the baseline surface, then single surfaces by profiled susceptibility (π), then combinations of increasing size ordered by profiled marginal impact (ρ), stopping at the first success. The ordering determines only when a candidate is tried, not whether: given enough budget, MSA covers all 9 single surfaces and 502 combinations, like exhaustive search.

2. Correction to Algorithm 1. As printed, the combination stage extends a single nested chain of surfaces and so does not reach every combination. We will replace it with:

Algorithm 1: Multi-Surface Attack (MSA), revised

Require: attack prompt pp, target AA, PromptRouter RR, accessible surfaces S\mathcal{S}, baseline surface sbs_b, individual-surface order π\pi, marginal-impact order ρ=(ρ1,…,ρ∣S∣)\rho = (\rho_1, \ldots, \rho_{|\mathcal{S}|})

Ensure: an individual surface or surface set CC that produces a successful attack, or failure if no tested surface or surface set succeeds

  1. if Execute(A,p,sb)\mathrm{Execute}(A, p, s_b) succeeds then
    • return {sb}\lbrace s_b \rbrace
  2. for all s∈πs \in \pi such that s≠sbs \neq s_b do
    • if Execute(A,R(p,{s}))\mathrm{Execute}(A, R(p, \lbrace s \rbrace)) succeeds then return {s}\lbrace s \rbrace
  3. for k=2,…,∣S∣k = 2, \ldots, |\mathcal{S}| do
    • for all 1≤i1<⋯<ik≤∣S∣1 \le i_1 < \cdots < i_k \le |\mathcal{S}| in lexicographic order do
      • C←{ρi1,…,ρik}C \gets \lbrace \rho_{i_1}, \ldots, \rho_{i_k} \rbrace
      • if Execute(A,R(p,C))\mathrm{Execute}(A, R(p, C)) succeeds then return CC
  4. return failure

Only the combination stage changes: the printed version grows one nested chain from {π1,π2}\lbrace \pi_1, \pi_2 \rbrace (only 8 of the 502 combinations), whereas the revision tests every subset of size ≥2\ge 2 (29−9−1=5022^9 - 9 - 1 = 502 for nine surfaces), by size and then by ρ\rho-rank. With unlimited budget, MSA succeeds on a case if and only if any configuration succeeds on it.

3. The reported MSA number is a lower bound, not a best case. Table 2 reports the highest ASR of any single configuration. A full MSA run succeeds on a case whenever any configuration does, so its ASR is the union of successful cases and is at least the reported value. Our table note ("empirical coverage bound") is imprecise; we will relabel the column as an empirical lower bound on the ASR of a full MSA run.

Query budgets and random/greedy baselines. MSA's ordering is greedy: it tries the candidates the profiles rank most likely to succeed first. Greedy, random and exhaustive search cover the same space and differ only in query cost. We therefore report ASR under a budget of k queries per case (ASR@k) for MSA vs. random ordering (exact expectation over all orderings), replaying the single-surface stage on all 120 ASB cases from logged results (no new runs):

ModelOrderASR@1ASR@2ASR@3ASR@5ASR@9Mean queries to first success
GPT-4.1MSA (greedy)88.399.299.299.2100.01.16
Random59.582.792.698.6100.01.71
GPT-5.6 SolMSA (greedy)75.881.782.582.584.21.22
Random40.761.372.380.884.22.09
Gemini 2.5 FlashMSA (greedy)90.898.3100.0100.0100.01.11
Random84.998.099.5100.0100.01.18
Gemini 3.7 FlashMSA (greedy)74.274.274.274.274.21.00
Random20.034.245.361.274.23.27
Claude Sonnet 4.6MSA (greedy)49.250.050.050.050.81.13
Random14.023.630.640.350.83.42

ASR@k counts provider blocks as failures (cf. Q6) and covers single surfaces only, so values can be slightly below Table 2.

MSA reaches (nearly) its full single-surface ASR within 2–3 queries, while random ordering needs most of the budget (1.0–1.2 vs. 1.2–3.4 mean queries to first success). ASR@9 also confirms the lower bound: the union over nine single surfaces already exceeds the best single surface (e.g., GPT-5.6 Sol 84.2% vs. 81.7%; Claude Sonnet 4.6 50.8% vs. 49.2%). For brevity we report ASB here; the revised appendix will cover all four benchmarks.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We thank all reviewers and the AC.

Reviewers agree that our question is important and overlooked: TZF1 calls it "a clean, well-motivated question," fA4i notes that isolating the attack surface "highlights a potentially overlooked dimension of safety evaluation," and NuH3 finds it "a simple but important question" with "broad empirical evidence."


Below, we summarize the major concerns we have addressed.

1. Is the surface effect caused by authority framing? (TZF1, fA4i, NuH3) New experiment: we ran a surface × framing factorial in which the identical framed payload is placed on every surface, including the user message, alongside a verbatim no-framing arm.

  • With framing held fixed, the system prompt, memory, and RAG output still succeed 42–77 pp more often than the user message (exact McNemar p < 1e-11 for every surface); no case succeeds in the user message but fails elsewhere.
  • On the agent benchmarks, surface differences are significant with no framing at all: for GPT-4.1, GPT-5.6 Sol and Gemini 3.7 Flash, both in the same new experiment on ASB (p < 1e-8) and in our verbatim DTap runs (p ≤ 0.02; e.g., GPT-4.1 71% in the user message vs. 0% in RAG output).
  • We will revise our wording to emphasize that we relocate the benchmark prompt verbatim under a surface-specific authoritative framing, as already specified for PromptRouter (Section 3.1, App. B).

2. MSA vs. the empirical maximum and budget-matched baselines (TZF1, fA4i, NuH3)

  • We acknowledge that our Table 2 note mislabeled the MSA column as an upper-bound "coverage bound." It is a lower bound: a full MSA run succeeds whenever any configuration succeeds, so its ASR is at least the reported value.
  • We corrected an unintended flaw in Algorithm 1: as printed, it tested 8 nested combinations; the revised version covers all 502.
  • New analysis (budget-matched baselines): MSA reaches nearly its full ASR within 2–3 queries (1.0–1.2 mean queries to first success vs. 1.2–3.4 for random ordering), and its first query equals the best-single-surface baseline.

3. Novelty relative to prior work (fA4i: ChatInject; NuH3: ASB)

  • ChatInject varies payload format inside one fixed channel (forged role tags in tool output); ASB varies attack method and location together. MSA relocates the verbatim benchmark instruction, under a fixed authoritative framing, across nine genuine surfaces, so only the surface varies.
  • To our knowledge, MSA is the first systematic study of surface combinations (all 502, plus a 2,048-subset full factorial), which yields findings a single-channel study cannot, e.g., that attacking more surfaces is not always better (mean ASR peaks at 67.5% with 3 surfaces and falls to 42.9% with 11; Claude Sonnet 4.6 on ASB, 60% → 0%).

4. Threat model and attacker access (fA4i, TZF1)

  • Attacker privilege depends on the deployment, not on the surface: an API user writes the system prompt and full context, and a local user controls every surface of an open-weight model and agent.
  • MSA takes the accessible surfaces as input, so our results apply to any access level.
  • New analysis: grouped by attacker tier, even external-only attackers reach 52–92% on agents, and privileged surfaces are the most dangerous tier in 9 of 10 agent settings.

We hope this rebuttal addresses the reviewers' concerns. Our original experiments show, and the new experiments further solidify, that the same malicious instruction a model refuses in the user turn is followed when it arrives through other parts of its context, and that this effect is not explained by framing, scoring, or search budget. We believe this is a major vulnerability in existing LLM and LLM based agents that safety alignment should address toward safer AI. We will incorporate all changes, new experiments, and analyses into the final revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.