acceptodds
Under review as a conference paper at ICLR 2027

Protocol Rendering Shapes Source Boundaries in LLM Agents

Abstract

LLM agents combine task instructions and external content in structured inputs. We study how Protocol Rendering, defined by wrapper syntax, field order, key style, and nesting, affects attack susceptibility when the receiver model, task, and attacker payload are held fixed. To analyze and investigate this effect, we introduce protocol-conditioned source separability (PCSS), which measures how distinctly a model represents identical neutral text placed in trusted versus external fields. We perform controlled experiments across five models and three attack settings and conclude three important findings : (1) Rendering alone changes attack outcomes. Switching from XML to nested JSON produces a 31% relative increase in attack success on Qwen3-14B memory-attack tasks, yet no wrapper family is consistently safest. (2) Weaker source separation is associated with higher memory-attack risk across renderings in three models spanning two model families. (3) Shifting external-message representations toward a trusted-source direction learned from neutral prompts without attack labels increases attack success by 9.2% on held-out Qwen3-14B memory-attack tasks, demonstrating a causal influence of source representations on attack susceptibility. These findings establish Protocol Rendering as a security-relevant design choice and support source separation as a contributor to rendering-dependent attack risk. https://anonymous.4open.science/r/ProtocolRenderingSafety-2848Code is available online.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Prompt Injection, Agent Safety, Information Provenance, Representation Intervention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uz8UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether the input format of LLM agents can affect security. The authors show that simple formatting choices such as XML vs. JSON or changes in field order and nesting can change attack success. To understand why, the paper examines how clearly the model distinguishes trusted instructions from external content. The authors then directly modify the model’s internal representation of external content to make it look more like trusted content. This makes the model more likely to follow the attacker, showing that source separation itself contributes to the security differences. The paper's main conclusion is that input formatting is an important part of an LLM agent’s security design.

Critical strengths
  • The paper identifies a practical security issue. The main finding is interesting, that is, changing the format of the same content can change attack success, even when other factors are fixed. This shows that formatting should be considered when designing and evaluating LLM agents.

  • The paper investigates the underlying reason why it happens. The authors show that attacks tend to be more successful when the model less clearly distinguishes trusted instructions from external content. They further intervene on the model’s internal representation and show that making external content appear more like trusted content increases attack success.

Critical weaknesses
  • The main explanation still needs more evidence. The paper shows two things: first, renderings with weaker source separation tend to have higher attack risk; second, directly changing a model’s internal representation of external content can increase attack success. But these two results do not complete the causal chain. The intervention does not change the rendering itself, and the paper does not show that a rendering becomes riskier because it weakens source separation. In other words, source separation may matter for attack behavior, while still not being the main reason different renderings behave differently. This is one of my major concerns because explaining the rendering effect is the key scientific contribution of the paper.

  • The strongest evidence comes from an artificial representation edit, but its relation to naturally occurring rendering differences is unclear. The causal experiment manually pushes hidden states in a chosen direction at a chosen layer and strength. The results show that the model is sensitive to this direction, but it does not show that natural modifications (e.g., changing XML to JSON) change the model in the same way or by a comparable amount. From my perspective, the experiment is convincing as evidence that source information can affect behavior, but it becomes much weaker as an explanation of the original rendering phenomenon.

Questions for authors
  • What evidence directly shows that the rendering effect is mediated by source separation?

  • What evidence supports the insight that naturally occurring rendering changes alter the model through the same source-related direction?

4 comments
AuthorsOpenReview ↗

Response to Reviewer uz8U

We thank the reviewer for the clear statement of the causal question. Revisions are highlighted in yellow, and the new analyses are collected in Appendix D.

W1 / Q1. Evidence that the rendering effect is mediated by source separation.

The evidence links source separation to rendering-dependent risk at two levels. Across 23 renderings (Section 3), lower final-token PCSS accompanies higher memory-attack risk in Qwen3-14B (ρ = −0.633), Qwen3-8B (ρ = −0.521), and Mistral-7B (ρ = −0.715). With the rendering fixed, moving external-message states toward the trusted-source direction raises attack success by 4.0 pp [1.33, 7.33] (Section 4, Appendix B.6).

The revision also tests whether natural rendering changes act through the edited direction. Switching from XML to nested JSON moves external-message states along it, and aligning this shift leaves the attacker-margin gap essentially unchanged (Section 4.4). The revision therefore presents PCSS as an internal indicator of rendering-dependent risk (Section 6.1), and the Abstract and Introduction are revised accordingly.

W2 / Q2. Do natural rendering changes alter the model through the same direction, and by a comparable amount?

Switching from XML to nested JSON moves external-message states along the same direction as the edit, by a smaller amount. The projection moves toward the trusted side by +0.135 [0.103, 0.169], in the same direction for 45 of the 50 held-out MA tasks (Section 4.4). On 256 fresh, position-matched item pairs, the frozen probe that defines the edit also detects the rendering change (d′ difference −1.52 [−1.58, −1.45], Section 3.5). The natural shift is two orders of magnitude smaller than the primary edit (α = 16), and aligning it changes the attacker-margin gap of 0.668 by 0.003 [−0.013, 0.019] (Appendix D.3). The edit therefore tests whether the direction can change adoption, and Section 4.4 describes it in these terms.

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer,

Thank you for your clear account of the causal chain and for finding the representation edit convincing evidence that source information can affect behavior. Section 4.4 projects the change from XML to nested JSON onto the edited direction, and Appendix D.3 aligns this shift across the two renderings. Section 3.5 applies the probe that defines the edit to fresh, position-matched item pairs. Section 6.1 relates source separation to the rendering effect.

If these analyses address your concerns, we would appreciate a re-evaluation of the submission. Thank you very much.

Reviewer_uz8UOpenReview ↗

Thanks for the response

Thank you for the response. It has largely addressed my second concern but the first one remains. In Section 4, the authors show that switching from XML to JSON changes the internal representations, but when they align these changes between the two renderings, the difference in attacker margin changes very little (0.003 vs. 0.668). This seems to imply that changes in source separation does not reflect why the two renderings lead to different attack outcomes. It is good that the authors tone down the claims, but I still feel that the main question of the explanation remains.

AuthorsOpenReview ↗

Response to follow-up

Thank you for noting that the second concern is largely addressed.

At layer 16 of Qwen3-14B, the attacker-margin gap between XML and nested JSON is carried by the structural tokens that delimit the sources. The alignment in Section 4.4 moves one direction at the message tokens. Appendix D.5 copies the full layer-16 states instead. We use 46 position-matched XML and nested-JSON attack inputs whose message and question tokens are identical, and the margin gap on them is 0.363 [0.186, 0.544]. Copying the XML states into nested JSON at the structural tokens (field boundaries, markers, the warning, and padding) recovers the gap, with a recovered fraction of R = 1.25 [0.95, 1.73]. Copying them at the message tokens gives R = −0.17 [−0.35, −0.06].

Reviewer_xJKUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether the format used to serialize instructions and external content—“Protocol Rendering”—changes prompt-injection susceptibility when the model, task, and payload are held fixed. In experiments across five open-weight models and three attack settings, the authors find that rendering changes attack success, with no wrapper family consistently safest; for example, on Qwen3-14B memory attacks, attack success rises from 52% under XML to 68% under nested JSON. They introduce protocol-conditioned source separability (PCSS), a probe-based measure of how differently a model represents identical text in trusted and external fields, and report that weaker final-token separation correlates with higher memory-attack risk across three receivers. Finally, they edit Qwen3-14B hidden states along a source direction learned from neutral prompts; trusted-ward edits increase attack success on held-out memory-attack tasks by 4 percentage points. The evidence supports rendering as a security-relevant variable and suggests a role for source representations, though it does not establish that source separation explains the rendering effects generally.

Critical strengths

1-The authors keep the receiver, task, and attack payload fixed while changing the rendering, and also evaluate matched clean tasks. The XML-versus-nested-JSON result for Qwen3-14B is particularly compelling because attack success rises by 16 percentage points while clean accuracy does not fall. This makes the central finding more than a comparison of different attacks or models.

2-PCSS is evaluated on held-out tasks, with additional family-centering and positional controls; the reported negative association between final-token separability and attack risk also appears in Qwen3-8B and Mistral-7B. The intervention further tests whether a learned source-related direction can affect behavior, rather than relying on probe decodability alone. These analyses add useful evidence beyond the behavioral sweep.

Critical weaknesses

1-The intervention shows that a particular hidden-state direction can increase attacker adoption in Qwen3-14B memory-attack tasks, but it does not show that this direction mediates the effect of changing renderings. The trusted-ward edit is tested on lower-risk formats, while the reverse edit on higher-risk formats is not reliable; the random-direction comparison is also suggestive rather than decisive (the reported finite-control statistic is 0.0606). Most importantly, the PCSS configuration set does not include nested JSON, even though XML versus nested JSON is a headline behavioral comparison. Thus the representational analysis does not directly explain that prominent result.

2-All attack content is placed in one external-message field within a single user message, and the experiments use greedy decoding and a limited set of benchmark tasks. The PCSS association is limited to memory attacks and three receivers, while the intervention is limited to Qwen3-14B memory attacks. These choices support a controlled proof of concept, but leave uncertain how well the conclusions transfer to native tool-result or retrieval channels, multi-turn agents, or more varied attack and task distributions.

Questions for authors

1-The abstract reports a 9.2% increase in attack success from the intervention, whereas the main results report a 4.0-percentage-point increase at the primary strength, with a 95% CI of [1.33, 7.33]. What precisely does the 9.2% figure measure, and how does it relate to the 4.0-point result?

2-The PCSS candidate set does not include nested JSON. Do the authors have source-separability measurements for the exact XML and nested-JSON configurations used in the headline comparison? If not, should the claim that PCSS helps explain rendering-dependent risk be understood as a broader association, rather than an explanation of that specific comparison?

3-Given that the trusted-ward and external-ward interventions are applied to different risk groups and the reverse-direction result is not reliable, what causal conclusion do the authors believe is supported—specifically, does the intervention show only that a source-related direction can increase susceptibility, or that source separation contributes to the effect of rendering changes?

2 comments
AuthorsOpenReview ↗

Response to Reviewer xJKU

We thank the reviewer for the thorough summary and the specific questions. Revisions are highlighted in yellow, and the new analyses are collected in Appendix D.

W1 / Q3. What causal conclusion does the intervention support?

Moving external-message states toward the learned trusted-source direction increases adoption of the attacker answer, and this holds in both rendering groups. The revision applies both edit directions to the same rendering groups (Section 4.3):

EditLower-risk renderingsHigher-risk renderings
Trusted-ward+4.00 pp+4.67 pp
External-ward−0.67 pp−0.67 pp

The external-ward intervals include zero. The asymmetry therefore reflects the edit direction rather than the rendering group.

Besides the random-direction comparison, a matched location control tests specificity. Applied to question tokens, the same edit yields an ASR change 5.33 pp [2.00, 8.67] smaller than the payload edit (Section 4.3, Appendix B.7).

Switching from XML to nested JSON moves external-message states along this direction by a smaller amount than the edit, and aligning this shift leaves the attacker-margin gap essentially unchanged (Section 4.4). The revision therefore presents the intervention as evidence that the source direction can change adoption, and PCSS as an internal indicator of rendering-dependent risk (Section 6.1).

W2. Scope of fields, decoding, receivers, and settings.

This paper evaluates single-turn inputs in which all fields share one user message, under greedy decoding, as stated in Appendix A.7. As described in Appendix A.2, the PI / MA / TA taxonomy follows G-Safeguard (Wang et al., 2025), and the tasks are drawn from MMLU (Hendrycks et al., ICLR 2021), G-Safeguard, and NetSafe (Yu et al., ACL Findings 2025). The PCSS association covers three receivers, and the revision adds the position-matched confirmation on fresh items in Qwen3-14B and Mistral-7B (Section 3.5). Native tool-result and retrieval channels and multi-turn agents are listed as future work (Section 6.2).

Q1. The 9.2% figure in the Abstract.

The 9.2% is the relative increase that corresponds to the 4.0-percentage-point result. Attack success rises from 43.3% to 47.3%, an absolute increase of 4.0 pp. The revised Abstract reports percentage points with baselines, and the XML versus nested-JSON result is reported the same way (16 pp, 52% to 68%).

W1 (PCSS configuration set) / Q2. PCSS for the exact XML and nested-JSON configurations.

The revision measures PCSS on the exact XML and nested-JSON configurations of Section 2, and nested JSON shows weaker separation. On the 50 held-out PCSS tasks, final-token d′ is 4.21 under XML and 3.56 under nested JSON (difference −0.65 [−1.31, −0.02], Section 3.5). On 256 fresh, position-matched item pairs, nested JSON also shows lower final-token d′ than XML in Qwen3-14B (−0.287 [−0.486, −0.074]) and Mistral-7B (−0.474 [−0.651, −0.318]). The result across the 23 renderings is an association, and Section 3.5 adds this measurement for the exact pair.

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer,

Thank you for highlighting the controlled comparison between XML and nested JSON. The revision now measures PCSS on these exact configurations and on fresh, position-matched item pairs (Section 3.5). It applies both edit directions to the same rendering groups (Section 4.3) and projects the change from XML to nested JSON onto the edited direction (Section 4.4). The Abstract now reports results as percentage points with baselines.

If our response resolves your concerns, we would be grateful if you would consider raising your score. Thank you very much.

Reviewer_Q6M3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examined how the protocol of rendering could impact the vulnerability of LLM agents to adversarial external content and provided an explanation on how the process of rendering can impact the susceptibility of attack by impacting the source distinction internally.

Critical strengths

The experiments hold task content and attack payloads fixed while varying their presentation, helping isolate rendering as a consequential experimental variable.

Critical weaknesses

Please check Questions For Authors

Questions for authors

Final token representation is the one that has the best control association, while the manipulation affects the representation of the external message through a separate probe and layer. The key issue here is whether changes in rendering affect the representation being manipulated and whether the latter accounts for the difference in behavior.

Positional confounding is high in terms of the representation used in the manipulation. Therefore, it makes less likely the inference that the message-local representation direction, which is used in the manipulation, constitutes a reliable mechanism. Controlling for position and controlling for surrounding context would improve interpretation of results.

As described in Appendix C.1, the warning refers to the peer message “below,” although it may have been presented in another position because of the field order. Consequently, the comparisons are subject to differing degrees of positional accuracy in the ability of the warning to identify the relevant message. Position neutrality and explicit reference to the external message field would constitute better control here.

The payload templates require the participant to write an argument or passage that makes the “wrong” answer correct. It is conceivable that the participant could accept the message but simultaneously know that the message was from an external source. In that case, there would not be evidence of confusion between task instructions and external information.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Q6M3

We thank the reviewer for the detailed technical questions. Revisions are highlighted in yellow, and the new analyses are collected in Appendix D.

Q1. Does rendering affect the manipulated representation, and does it account for the behavioral difference?

Rendering changes the message-span representation used in the intervention. On the 50 held-out MA tasks, switching from XML to nested JSON moves the external-message projection on the edited direction toward the trusted side by +0.135 [0.103, 0.169], in the same direction for 45 of the 50 tasks (Section 4.4).

This natural shift is two orders of magnitude smaller than the primary edit. Aligning it across the two renderings changes the attacker-margin gap of 0.668 by 0.003 [−0.013, 0.019], and the generated answers do not change (Appendix D.3). The revision therefore presents PCSS as an internal indicator of rendering-dependent risk (Section 6.1).

The two representations serve different purposes. The edit acts on the external message's own token states. The final-token state aggregates all fields, so an edit there would not be specific to the external message.

Q2. Positional confounding of the message-local representation, and control for position and surrounding context.

Under exact position matching, the message-span probe used in the intervention still detects the rendering change. The revision adds a confirmation on 256 fresh item pairs whose design was fixed before extraction (Section 3.5, Appendix D.1). XML, flat JSON, and nested JSON share leaf keys, leaf values, and leaf order. Whitespace padding in field gaps matches the token positions of both records, the final input token, and the input length. Under this design, the frozen message-span probe detects the rendering change (d′ difference −1.52 [−1.58, −1.45]). A common final-token probe fit on separate items gives the same direction in both receivers:

ReceiverNested JSON − XML, final-token d′ [95% CI]
Qwen3-14B−0.287 [−0.486, −0.074]
Mistral-7B−0.474 [−0.651, −0.318]

Five of the six final-token rendering contrasts remain significant after Holm correction.

Q3. Position-neutral warning.

The revision adds a position-neutral warning that names the external-message field and contains no spatial term, and re-evaluates the field-order comparisons that move the message relative to the warning (Appendix D.2). In all eight renderings of Section 2, the warning precedes the external message, so the original wording is accurate for those comparisons.

Q4. Adopting external content while recognizing its source.

The paper's claim concerns how distinctly the two sources are encoded and how much weight external content receives (Section 6.1). It does not require the model to mistake the external message for an instruction. The instruction to write an argument (Appendix C.2) is given to the payload-generation model, and the receiver sees only the resulting message. Each payload is generated once and reused verbatim under every rendering (Section 2.1), so the persuasive content is identical across conditions. PCSS is measured on neutral messages without attack content or labels (Section 3.1).

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer,

Thank you for your detailed technical questions on our representation analyses. The revised PDF adds one item for each question:

  • a confirmation on 256 fresh, position-matched item pairs, read with the probe used in the intervention (Section 3.5)
  • the projection of the XML-to-nested-JSON change onto the edited direction (Section 4.4)
  • a position-neutral warning that names the external-message field (Appendix D.2)
  • the interpretation of the results as source encoding and the weight given to external content (Section 6.1)

If these additions resolve your concerns, we would appreciate it if you could re-evaluate the submission. Thank you very much.

My remaining concern is how much the measured shift in the message representation explains the behavioral difference between renderings. I agree with the revised interpretation of PCSS as a risk indicator. But the experiment leaves open why the same content produces different answers across renderings. I will keep my score.

AuthorsOpenReview ↗

Response to the remaining concern

Thank you for the assessment of PCSS as a risk indicator.

The message-token shift explains little of the attacker-margin gap. At layer 16 of Qwen3-14B, the gap is carried by the structural tokens. Aligning that shift changes the gap by 0.003 of 0.668 (Section 4.4). Appendix D.5 then copies full layer-16 states from XML into nested JSON on 46 position-matched inputs with identical message and question tokens, where the gap is 0.363 [0.186, 0.544]. Copying at the structural tokens (field boundaries, markers, the warning, and padding) recovers the gap (R = 1.25 [0.95, 1.73], the recovered fraction). Copying at the message tokens does not (R = −0.17 [−0.35, −0.06]).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.