acceptodds
Under review as a conference paper at ICLR 2027

Recovering and Exploiting Latent Presentation Preferences in Large Language Models

Abstract

LLM-based search, question-answering, and recommendation systems must select among multiple, potentially conflicting external sources. This creates an attack surface: a small amount of attacker-controlled content can disproportionately steer model outputs through changes in presentation, even when the underlying claim is fixed. Existing content-optimization attacks rely mainly on end-to-end outcomes that conflate attention allocation, internal evaluation, and source competition, providing coarse guidance for optimization. We introduce a preference-guided adversarial content optimization framework that recovers model-specific presentation preferences from claim-level Information Attention and sparse Acceptance Vectors and uses these preferences to guide downstream attacks while preserving the core claim and linguistic naturalness. Profiling 120 presentation forms in 11 categories across seven models reveals structured, model-specific preferences. Preference scores correlate strongly with downstream attack success (–). Causal interventions further demonstrate that attention and acceptance components influence evidence adoption. In retrieved-evidence poisoning, our attacks outperform the strongest baselines by 8.67–31.87 percentage points with one controlled source competing against nine sources supporting the correct answer. They also achieve the best target ranks in generative recommendation manipulation, with mean-rank gains of 0.27–2.32 positions. The attacks transfer to black-box models, exceeding the strongest baselines by 23.06–41.03 percentage points in poisoning success rate. These results establish latent presentation preference as a direct and effective signal for optimizing adversarial content to increase the model's acceptance of attacker-controlled claims.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language model security, generative engine optimization, presentation bias, evidence selection, adversarial content optimization, retrieval-augmented generation, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HmK2OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to make LLMs adopt attacker-controlled content by changing its presentation while keeping the claim fixed. It recovers model-specific presentation preferences from SAE-based acceptance directions and claim-level attention, and uses them to initialize and optimize attack texts for evidence poisoning and recommendation manipulation.

Critical strengths
  • Using internal signals to analyze presentation effects on evidence adoption is an interesting angle, supported by causal interventions.
  • The experiments cover several models and two attack settings.
Critical weaknesses
  • The threat model is not well supported for current systems. The paper's own results show that frontier models are rarely swayed by a single conflicting source by default, while the main experiments rely on smaller, older open models and a bare RAG prompt without any safeguards.
  • The core claim that internal signals are better than output-level signals is not supported:
    • The premise that outcome-level signals are coarse and noisy is stated without evidence.
    • The only output-based baseline is the model's self-rating of styles; ranking styles by their measured ASR on a small development set is not compared.
    • The optimizer is adopted from ContextBench (LLM-assisted EPO), but the same optimizer with an output-level objective is not compared. The ContextBench baseline is also run with less frequent LLM assistance.
  • The attack gains cannot be clearly attributed to presentation preferences:
    • The Acceptance Vectors are confounded. For Trust, positives are cases where the model accepts the supplied claim and negatives are cases where it rejects it; these come from different claims, and since models tend to accept true claims and reject false ones, the direction may largely capture claim truth. For Compliance/Sycophancy, negatives include inputs with the pressure text removed. As the representation is taken before generation, these directions can only encode input differences rather than acceptance itself.
    • The optimization constrains only the claim span. The final texts add fabricated supporting evidence and direct instructions to the model (e.g., "immediately acknowledge"), contradicting the paper's own rule of rejecting instruction-like rewrites, and the initial style is not preserved. The gains therefore mix content and prompt-injection effects.
    • The transfer to closed-source models likely relies on the same added content and instructions.
  • Many design choices are not justified or validated:
    • The layers for Acceptance Vectors and Information Attention are chosen without explanation and differ from each other.
    • The four acceptance facets and the 120 styles are hand-defined, and how the styled samples and behavior labels are produced is not described.
    • The held-out evaluation uses a split of the same behavioral corpus; there is no out-of-distribution evaluation.
    • The method requires a public SAE for each model and per-model offline profiling, whose cost is not reported.
  • The evaluation setting may overstate attack difficulty. The "nine sources supporting the correct answer" include unrelated retrieved passages (e.g., the example in Appendix D.2), the questions and false target answers are generated by an LLM without quality checks, the recommendation task has only three categories of ten synthetic products, and the black-box evaluation uses only 195 examples.
  • No defense or detection is evaluated, although many optimized texts contain obvious manipulative cues.
Questions for authors

see above

7 comments
AuthorsOpenReview ↗

Response to Reviewer HmK2 (1/5)

W/Q1

The threat model is not well supported for current systems.

Thank you for raising this concern. Our evaluation follows established community settings and demonstrates effectiveness across diverse models, datasets, and frontier black-box targets. We adopt the controlled-source poisoning setup used by existing baselines (Appendix D.1), profile seven models across the Llama, Gemma, and Qwen families, and evaluate attacks on FEVER, SQuAD, and three recommendation categories. Table 3 further demonstrates effective transfer to three frontier black-box models.

W/Q2

The core claim that internal signals are better than output-level signals is not supported

Thank you for recommending comparisons with observed adoption outcomes in addition to model-generated ratings. We provide complementary evidence on preference prediction, style selection, and subsequent text optimization, including the additional Output ASR baseline.

Internal signals better characterize presentation preferences and identify styles that increase claim adoption. Table 4(a) shows that internal Preference Scores correlate more strongly with downstream ASR than model-generated output ratings: r=0.687–0.792, compared with 0.175–0.509.

We additionally introduce an Output ASR baseline that ranks the 120 presentation styles by their observed rejection-to-adoption rates on the same behavioral dataset used to construct our Acceptance Vectors. This complements the Output ratings baseline with a ranking derived from actual model behavior.

Using exactly the downstream evaluation data of Table 18, we compare the effectiveness of styles selected by these three rankings. For each ranking, we divide the 120 styles into consecutive groups of 20. Each value below is the group’s mean ASR divided by the model’s mean ASR across all 120 styles.

ModelMethod1–2021–4041–6061–8081–100101–120
Gemma-2-9B-ITOurs (Internal)1.3581.1291.0390.9910.8360.648
Output ratings1.2071.0541.0151.0090.9170.798
Output ASR1.0900.9891.0011.0341.0190.867
Llama-3.1-8B-InstructOurs (Internal)1.4211.0710.9690.9650.9420.632
Output ratings1.1330.9611.0741.0630.8960.873
Output ASR1.1520.9181.0030.9511.0840.894
Qwen3-8BOurs (Internal)1.3281.1370.9790.9960.9250.636
Output ratings1.1611.1251.0471.0110.8950.762
Output ASR1.1011.0880.9960.8611.0360.918

Internal-guided optimization achieves higher ASR and larger improvements over template initialization under the same optimization budget. We compare the effectiveness of attack-text optimization guided by internal versus output-based objectives on Qwen3-8B using FEVER. Each of the three methods first selects its top-40 presentation styles using its own signal—internal Preference Scores, output ratings, or observed ASR—and uses these styles to initialize attack texts.

Our method then runs EPO with an optimization objective combining the internal Preference Score and a PPL regularizer. For both output-based methods, we replace the internal Preference Score with the model’s output-rating score while retaining the PPL regularizer, and replace gradient-based candidate proposals with prompt-based proposals.

We evaluate all three methods under the same optimization budget (iterations, candidates per iteration and assistant-LLM call frequency), both with and without LLM assistance.

Optimization settingOutput-ratings rankingOutput-ASR rankingOurs (Internal)
Target + Assist53.77%54.19%65.33%
Target only51.89%53.13%62.73%
Style templates only44.51%42.63%48.00%

Under the same optimization budget, our method achieves higher ASR and larger gains over template initialization, with and without LLM assistance. These results support the advantage of our proposed internal objective over output-based objectives for guiding attack-text optimization.

AuthorsOpenReview ↗

Response to Reviewer HmK2 (2/5)

W/Q 3.1

The Acceptance Vectors are confounded.

Thank you for raising concerns about the construction of the Acceptance Vectors and the interpretation of representations extracted before generation.

Trust is supervised by claim adoption, with factual correctness balanced across positive and negative samples. Both the acceptance and rejection groups contain factually correct and incorrect claims in matched proportions. Positive and negative labels reflect whether the model adopts the supplied claim. This construction targets acceptance versus rejection while controlling for factual correctness.

Compliance and Sycophancy characterize acceptance under pressure, rather than general claim acceptance. Positive examples are inputs in which the model adopts the target stance under explicit compliance pressure or sycophancy-related user cues. The negative pool combines two complementary controls:

  1. Pressure-present non-adoption examples: the pressure cues remain, but the model does not adopt the target stance.
  2. Pressure-removed non-adoption examples: the pressure cues are removed while retaining the same claim and question, and the model does not adopt the target stance. The two types of negatives provide complementary contrasts. Pressure-present negatives contrast resisting pressure with yielding to it, while pressure-removed negatives contrast maintaining the original stance without pressure with adopting the target stance under pressure. We combine these negatives using a 2:1 sampling design to balance the two contrasts. Together, they support extracting representations associated with acceptance under compliance or sycophancy pressure.

Pre-generation representations can encode behaviorally relevant states and causally influence subsequent responses. Extracting representations before generation does not restrict them to surface-level input differences. Representation Engineering explicitly uses representations immediately preceding a model’s prediction to read out high-level concepts [1]. More directly, refusal directions extracted from post-instruction tokens, before answer generation, can induce or suppress subsequent refusal through activation interventions [2].

Our experiments provide corresponding evidence for claim adoption: with the input text held fixed, interventions on selected acceptance features and claim-directed attention shift the token-level decision margin in the expected directions (Figure 4; Appendix D.5). Forward and reverse interventions, together with random controls, support the causal relevance of these internal states to subsequent decisions.

[1] Zou A, Phan L, Chen S, et al. Representation engineering: A top-down approach to ai transparency[J]. arXiv preprint arXiv:2310.01405, 2023.

[2] Arditi A, Obeso O, Syed A, et al. Refusal in language models is mediated by a single direction[J]. Advances in Neural Information Processing Systems, 2024, 37: 136037-136083.

AuthorsOpenReview ↗

Response to Reviewer HmK2 (3/5)

W/Q3.2 - 3.3

The optimization constrains only the claim span. The transfer to closed-source models likely relies on the same added content and instructions.

Thank you for raising the concern that additional supporting content and direct instructions could contribute to the reported gains. We address this by distinguishing the experiments validating presentation preferences from subsequent attack optimization, clarifying the shared baseline settings, and quantifying target-adoption directives in the optimized texts.

Our experiments support the proposed mechanism linking presentation changes to model acceptance : Our framework modifies text in two stages:

  1. Preference-guided template rewriting. We instantiate high-preference presentation templates by inserting the target claim into the designated placeholder. The template and its generic surrounding text are predefined independently of individual claims. This stage uses rule-based substitution without generating claim-specific supporting content.
  2. LLM-assisted gradient optimization. We further optimize the template-instantiated text while preserving the target claim verbatim. This stage can introduce claim-specific background, explanations, or supporting assertions, such as the scarcity statement in M2.

The preference-validation experiments precede LLM-assisted optimization. Figures 3–4 and Table 18 all use first-stage, template-instantiated texts. Figure 3 establishes score–ASR correlations of r=0.687–0.792r=0.687–0.792, and Table 18 confirms the behavioral effectiveness of preference rankings. Figure 4 establishes that interventions on selected acceptance features and claim-directed attention causally affect the token-level decision margin under matched template-instantiated inputs. These results demonstrate that presentation preferences improve claim adoption, without confounding effects from claim-specific supporting evidence.

Our content-editing scope and assistant-LLM budget follow established baseline settings. Existing GEO and poisoning methods permit additional supporting content. AutoGEO [1]’s rules encourage source citations, statistics, examples, and explanatory context, while PoisonedRAG [2] constructs misleading passages to induce attacker-chosen answers. These precedents support allowing supporting-content additions during attack optimization.

In our evaluation, methods share the same controlled source and competing context. Table 13 details their resource budgets, our method and ContextBench both use three assistant-LLM rewrite calls per attack text. Within this shared content-editing setting, our two-stage framework uses internal signals to profile, select, and optimize model-preferred presentations, achieving 8.67–31.87 percentage-point ASR improvements over the strongest baseline across the evaluated poisoning settings (Table 1).

Our method does not rely on command-based templates and outperforms the explicit-command baseline. Coercive Command is included among the 120 presentation strategies as a diagnostic dimension for studying how models respond to instructional content. It is excluded from template selection for our main attacks. On FEVER, the selected high-preference templates achieve higher ASR (without gradient optimization) than Coercive Command:

ModelSelected templateTemplate ASRCoercive Command ASR
Llama-3.1-8B-InstructAdministrative Document Style46.33%33.33%
Qwen3-8BEmphatic Markup59.00%42.67%
Reviewer_8jbsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether internal model signals can recover preferences over how a fixed claim is presented and guide adversarial content optimization. It constructs four sparse acceptance-related directions from behavioral supervision, combines presentation induced changes along these directions with claim-level attention, and profiles 120 presentation styles across seven models. High scoring styles initialize an LLM-assisted evolutionary search that optimizes the internal objective while retaining the target claim. Evaluation includes retrieved-evidence poisoning, recommendation manipulation, and transfer to black-box models. The main supporting evidence comprises correlations between upstream preference scores and downstream style-only attack success, sparse-versus-dense and optimization ablations, and matched activation/attention interventions on 96 FEVER examples. The reported attack improvements are substantial, although the poisoning results are conditional on the controlled source entering the context.

Critical strengths

1.The internal scores predict style-only attack success reasonably well across three models. This is an interesting and useful finding.

2.The intervention study is fairly convincing, with forward/reverse interventions and random controls. The appendix also explains its limitations clearly.

Critical weaknesses

1.What is the benefit of new objective? I think paper should compare it with a direct output based objective using the same optimizer and budget.

2.I think keeping the original claim unchanged does not mean only the presentation changes. Some examples in paper add instructions or new information, such as “Stocks are dwindling.” These additions could explain part of the gains.

3.Some pressure-removed inputs are labeled negative by construction. It is unclear whether the model actually rejects the target stance in these cases.

4.the submission is more of an integration of existing technologies; it does not propose any new concepts or uncover new mechanisms, and its level of innovation is limited.

5.The denominators or averaging procedures behind some ASR values are unclear.

Questions for authors

Please refer to weakness

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8jbs (1/4)

W/Q1

What is the benefit of new objective? I think paper should compare it with a direct output based objective using the same optimizer and budget.

Thank you for suggesting a direct comparison with an output-based objective under the same optimization budget. We examine the benefit of internal signals in both selecting effective presentation styles and optimizing attack texts.

Internal signals better characterize presentation preferences and identify styles that increase claim adoption. Table 4(a) shows that internal Preference Scores correlate more strongly with downstream ASR than model-generated output ratings: r=0.687–0.792, compared with 0.175–0.509.

Using exactly the downstream evaluation data of Table 18, we compare the effectiveness of styles selected by these two rankings. For each ranking, we divide the 120 styles into consecutive groups of 20. Each value below is the group’s mean ASR divided by the model’s mean ASR across all 120 styles.

ModelMethod1–2021–4041–6061–8081–100101–120
Gemma-2-9B-ITOurs (Internal)1.3581.1291.0390.9910.8360.648
Output ratings1.2071.0541.0151.0090.9170.798
Llama-3.1-8B-InstructOurs (Internal)1.4211.0710.9690.9650.9420.632
Output ratings1.1330.9611.0741.0630.8960.873
Qwen3-8BOurs (Internal)1.3281.1370.9790.9960.9250.636
Output ratings1.1611.1251.0471.0110.8950.762

Internal-guided optimization achieves higher ASR and larger improvements over template initialization under the same optimization budget. We compare the effectiveness of attack-text optimization guided by internal versus output-based objectives on Qwen3-8B using FEVER. Each method first selects its top-40 presentation styles using its own signal—internal Preference Scores or output ratings—and uses these styles to initialize attack texts.

Our method then runs EPO with an optimization objective combining the internal Preference Score and a PPL regularizer. For the output-based method, we replace the internal Preference Score with the model’s output-rating score while retaining the PPL regularizer, and replace gradient-based candidate proposals with prompt-based proposals.

We evaluate both methods under the same optimization budget (iterations, candidates per iteration and assistant-LLM call frequency), both with and without LLM assistance.

Optimization settingOutput-ratings rankingOurs (Internal)
Target + Assist53.77%65.33%
Target only51.89%62.73%
Style templates only44.51%48.00%

Under the same optimization budget, our method achieves higher ASR and larger gains over template initialization, with and without LLM assistance. These results support the advantage of our proposed internal objective over the output-based objective for guiding attack-text optimization.

AuthorsOpenReview ↗

Response to Reviewer 8jbs (2/4)

W/Q2

I think keeping the original claim unchanged does not mean only the presentation changes. Some examples in paper add instructions or new information, such as “Stocks are dwindling.” These additions could explain part of the gains.

Thank you for pointing out that preserving the target claim does not itself exclude additional information or instructions. We clarify when such additions arise in our framework, which experiments validate presentation preferences, and how frequently target-adoption directives occur in the optimized texts.

Our experiments support the proposed mechanism linking presentation changes to model acceptance : Our framework modifies text in two stages:

  1. Preference-guided template rewriting. We instantiate high-preference presentation templates by inserting the target claim into the designated placeholder. The template and its generic surrounding text are predefined independently of individual claims. This stage uses rule-based substitution without generating claim-specific supporting content.
  2. LLM-assisted gradient optimization. We further optimize the template-instantiated text while preserving the target claim verbatim. This stage can introduce claim-specific background, explanations, or supporting assertions, such as the scarcity statement in M2.

The preference-validation experiments precede LLM-assisted optimization. Figures 3–4 and Table 18 all use first-stage, template-instantiated texts. Figure 3 establishes score–ASR correlations of r=0.687–0.792r=0.687–0.792, and Table 18 confirms the behavioral effectiveness of preference rankings. Figure 4 establishes that interventions on selected acceptance features and claim-directed attention causally affect the token-level decision margin under matched template-instantiated inputs. These results demonstrate that presentation preferences improve claim adoption, without confounding effects from claim-specific supporting evidence.

Our content-editing scope and assistant-LLM budget follow established baseline settings. Existing GEO and poisoning methods permit additional supporting content. AutoGEO [1]’s rules encourage source citations, statistics, examples, and explanatory context, while PoisonedRAG [2] constructs misleading passages to induce attacker-chosen answers. These precedents support allowing supporting-content additions during attack optimization.

In our evaluation, methods share the same controlled source and competing context. Table 13 details their resource budgets, our method and ContextBench both use three assistant-LLM rewrite calls per attack text. Within this shared content-editing setting, our two-stage framework uses internal signals to profile, select, and optimize model-preferred presentations, achieving 8.67–31.87 percentage-point ASR improvements over the strongest baseline across the evaluated poisoning settings (Table 1).

Our method does not rely on command-based templates and outperforms the explicit-command baseline. Coercive Command is included among the 120 presentation strategies as a diagnostic dimension for studying how models respond to instructional content. It is excluded from template selection for our main attacks. On FEVER, the selected high-preference templates achieve higher ASR (without gradient optimization) than Coercive Command:

ModelSelected templateTemplate ASRCoercive Command ASR
Llama-3.1-8B-InstructAdministrative Document Style46.33%33.33%
Qwen3-8BEmphatic Markup59.00%42.67%
AuthorsOpenReview ↗

Response to Reviewer 8jbs (3/4)

The "immediately acknowledge" directive in R4 arose in a specific Time Urgency example during LLM-assisted gradient optimization. Our evaluations indicate that such directives are uncommon. For recommendation manipulation, our method achieves the best mean rank across all six settings, alongside the lowest Bad Word Ratio (BWR) in most settings, including ties. (BWR measures the occurrence of recommendation-related command words and phrases.)

For retrieved-evidence poisoning, we additionally used DeepSeek-V4 to audit all 1,500 optimized texts per model–dataset setting. Informed by BIPIA [3] and InjecGuard [4], our rubric checks whether the passage contains a behavioral command directed at the assistant that prescribes adoption of the target conclusion.

ModelFEVER: detected target-adoption directivesSQuAD: detected target-adoption directives
Llama-3.1-8B-Instruct1/1,500 (0.07%)3/1,500 (0.20%)
Qwen3-8B5/1,500 (0.33%)3/1,500 (0.20%)

Even if every text identified as containing a target-adoption directive were counted as an attack failure, ASR would decrease by at most 0.33 percentage points. The substantial gains would remain, leaving our core conclusion unchanged: internal preference signals effectively guide content optimization and improve attack success.

[1] Wu Y, Zhong S, Kim Y, et al. What generative search engines like and how to optimize web content cooperatively[C]//International Conference on Learning Representations. 2026.

[2] Zou W, Geng R, Wang B, et al. {PoisonedRAG}: Knowledge corruption attacks to {Retrieval-Augmented} generation of large language models[C]//34th USENIX Security Symposium (USENIX Security 25). 2025: 3827-3844.

[3] Yi J, Xie Y, Zhu B, et al. Benchmarking and defending against indirect prompt injection attacks on large language models[C]//Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 2025: 1809-1820.

[4] Li H, Liu X. Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models[J]. arXiv preprint arXiv:2410.22770, 2024.

Reviewer_co54OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how presentation affects LLM acceptance of a claim and whether internal signals can predict and exploit this sensitivity. It constructs four Acceptance Vectors in sparse-autoencoder feature space and combines them with a claim-directed Information Attention measure. These signals profile 120 presentation styles across seven models and guide gradient-based search with auxiliary LLM rewrites. Experiments evaluate conflicting-source question answering, recommendation manipulation, and transfer to black-box models. A separate intervention study tests the causal influence of selected internal states and claim-directed attention on a token-level decision margin.

Critical strengths
  1. The connection between internal signals and downstream behavior is interesting and well motivated. The preference scores predict style-only attack success (Figure 3), and using them for optimization produces substantial gains across the poisoning and recommendation tasks. This gives the approach both interpretive and practical value.

  2. I found the intervention study in Appendix D.5 useful. The forward and reverse interventions, different strengths, and random controls provide evidence that the selected internal states can influence the model's decision. This adds substance to the proposed connection between the readout and model behavior.

Critical weaknesses
  1. My main concern is what counts as a change in presentation. Appendix B excludes added instructions, yet R4 in Appendix E asks the model to "immediately acknowledge" the claim, and M2 adds "Stocks are dwindling." These changes introduce an instruction or new information that could itself affect the answer. This makes it difficult to tell how much of the improvement comes from presentation preferences. A clearer definition, supported by an evaluation of rewrites that satisfy it, would strengthen the paper's central interpretation.

  2. The specific contribution of the internal objective is less clear than the overall attack improvement. The approach combines model-specific profiling, iterative search, and LLM rewriting. Table 5 helps explain the components, but does not compare internal guidance with an output-based objective under the same search procedure. Such a comparison would help establish why the proposed score is especially useful for optimization.

Questions for authors
  1. Do the presentation constraints in Appendix B also apply to the optimized attacks? Have you checked how often the rewrites add instructions or new factual assertions, and whether the gains remain on examples without these additions?

  2. Do you have a comparison using the same search procedure and a similar budget, with an output-based objective in place of the internal score?

3 comments
AuthorsOpenReview ↗

Response to Reviewer co54 (1/3)

W/Q1

Do the presentation constraints in Appendix B also apply to the optimized attacks? Have you checked how often the rewrites add instructions or new factual assertions, and whether the gains remain on examples without these additions?

Thank you for highlighting the ambiguity in our definition of presentation and the potential effects of added information or instructions. We clarify the definition, distinguish template-based profiling from subsequent attack optimization, and provide additional evidence below.

For preference profiling, we study how content transformations affect claim adoption in generative engines. Each transformation retains a designated target claim verbatim while modifying its surrounding text. These transformations cover claim-independent patterns of linguistic expression, discourse organization, rhetorical framing, and source attribution, as well as the addition of generic supporting statements or contextual information. The patterns and generic additions are predefined independently of individual claims and instantiated by inserting the target claim into a designated placeholder.

Our experiments support the proposed mechanism linking presentation changes to model acceptance : Our framework modifies text in two stages:

  1. Preference-guided template rewriting. We instantiate high-preference presentation templates by inserting the target claim into the designated placeholder. The template and its generic surrounding text are predefined independently of individual claims. This stage uses rule-based substitution without generating claim-specific supporting content.
  2. LLM-assisted gradient optimization. We further optimize the template-instantiated text while preserving the target claim verbatim. This stage can introduce claim-specific background, explanations, or supporting assertions, such as the scarcity statement in M2.

The preference-validation experiments precede LLM-assisted optimization. Figures 3–4 and Table 18 all use first-stage, template-instantiated texts. Figure 3 establishes score–ASR correlations of r=0.687–0.792r=0.687–0.792, and Table 18 confirms the behavioral effectiveness of preference rankings. Figure 4 establishes that interventions on selected acceptance features and claim-directed attention causally affect the token-level decision margin under matched template-instantiated inputs. These results demonstrate that presentation preferences improve claim adoption, without confounding effects from claim-specific supporting evidence.

Our content-editing scope and assistant-LLM budget follow established baseline settings. Existing GEO and poisoning methods permit additional supporting content. AutoGEO [1]’s rules encourage source citations, statistics, examples, and explanatory context, while PoisonedRAG [2] constructs misleading passages to induce attacker-chosen answers. These precedents support allowing supporting-content additions during attack optimization.

In our evaluation, methods share the same controlled source and competing context. Table 13 details their resource budgets, our method and ContextBench both use three assistant-LLM rewrite calls per attack text. Within this shared content-editing setting, our two-stage framework uses internal signals to profile, select, and optimize model-preferred presentations, achieving 8.67–31.87 percentage-point ASR improvements over the strongest baseline across the evaluated poisoning settings (Table 1).

Our method does not rely on command-based templates and outperforms the explicit-command baseline. Coercive Command is included among the 120 presentation strategies as a diagnostic dimension for studying how models respond to instructional content. It is excluded from template selection for our main attacks. On FEVER, the selected high-preference templates achieve higher ASR (without gradient optimization) than Coercive Command:

ModelSelected templateTemplate ASRCoercive Command ASR
Llama-3.1-8B-InstructAdministrative Document Style46.33%33.33%
Qwen3-8BEmphatic Markup59.00%42.67%
AuthorsOpenReview ↗

Response to Reviewer co54 (2/3)

The "immediately acknowledge" directive in R4 arose in a specific Time Urgency example during LLM-assisted gradient optimization. Our evaluations indicate that such directives are uncommon. For recommendation manipulation, our method achieves the best mean rank across all six settings, alongside the lowest Bad Word Ratio (BWR) in most settings, including ties. (BWR measures the occurrence of recommendation-related command words and phrases.)

For retrieved-evidence poisoning, we additionally used DeepSeek-V4 to audit all 1,500 optimized texts per model–dataset setting. Informed by BIPIA [3] and InjecGuard [4], our rubric checks whether the passage contains a behavioral command directed at the assistant that prescribes adoption of the target conclusion.

ModelFEVER: detected target-adoption directivesSQuAD: detected target-adoption directives
Llama-3.1-8B-Instruct1/1,500 (0.07%)3/1,500 (0.20%)
Qwen3-8B5/1,500 (0.33%)3/1,500 (0.20%)

Even if every text identified as containing a target-adoption directive were counted as an attack failure, ASR would decrease by at most 0.33 percentage points. The substantial gains would remain, leaving our core conclusion unchanged: internal preference signals effectively guide content optimization and improve attack success.

[1] Wu Y, Zhong S, Kim Y, et al. What generative search engines like and how to optimize web content cooperatively[C]//International Conference on Learning Representations. 2026.

[2] Zou W, Geng R, Wang B, et al. {PoisonedRAG}: Knowledge corruption attacks to {Retrieval-Augmented} generation of large language models[C]//34th USENIX Security Symposium (USENIX Security 25). 2025: 3827-3844.

[3] Yi J, Xie Y, Zhu B, et al. Benchmarking and defending against indirect prompt injection attacks on large language models[C]//Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 2025: 1809-1820.

[4] Li H, Liu X. Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models[J]. arXiv preprint arXiv:2410.22770, 2024.

AuthorsOpenReview ↗

Response to Reviewer co54 (3/3)

W/Q2

Do you have a comparison using the same search procedure and a similar budget, with an output-based objective in place of the internal score?

Thank you for asking us to clarify the contribution of the internal objective beyond the overall attack gains. We provide additional comparisons with output-based signals to assess preference prediction, style selection, and subsequent text optimization.

Internal signals better characterize presentation preferences and identify styles that increase claim adoption. Table 4(a) shows that internal Preference Scores correlate more strongly with downstream ASR than model-generated output ratings: r=0.687–0.792, compared with 0.175–0.509.

Using exactly the downstream evaluation data of Table 18, we compare the effectiveness of styles selected by these two rankings. For each ranking, we divide the 120 styles into consecutive groups of 20. Each value below is the group’s mean ASR divided by the model’s mean ASR across all 120 styles.

ModelMethod1–2021–4041–6061–8081–100101–120
Gemma-2-9B-ITOurs (Internal)1.3581.1291.0390.9910.8360.648
Output ratings1.2071.0541.0151.0090.9170.798
Llama-3.1-8B-InstructOurs (Internal)1.4211.0710.9690.9650.9420.632
Output ratings1.1330.9611.0741.0630.8960.873
Qwen3-8BOurs (Internal)1.3281.1370.9790.9960.9250.636
Output ratings1.1611.1251.0471.0110.8950.762

Internal-guided optimization achieves higher ASR and larger improvements over template initialization under the same optimization budget. We compare the effectiveness of attack-text optimization guided by internal versus output-based objectives on Qwen3-8B using FEVER. Each method first selects its top-40 presentation styles using its own signal—internal Preference Scores or output ratings—and uses these styles to initialize attack texts.

Our method then runs EPO with an optimization objective combining the internal Preference Score and a PPL regularizer. For the output-based method, we replace the internal Preference Score with the model’s output-rating score while retaining the PPL regularizer, and replace gradient-based candidate proposals with prompt-based proposals.

We evaluate both methods under the same optimization budget (iterations, candidates per iteration and assistant-LLM call frequency), both with and without LLM assistance.

Optimization settingOutput-ratings rankingOurs (Internal)
Target + Assist53.77%65.33%
Target only51.89%62.73%
Style templates only44.51%48.00%

Under the same optimization budget, our method achieves higher ASR and larger gains over template initialization, with and without LLM assistance. These results support the advantage of our proposed internal objective over the output-based objective for guiding attack-text optimization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.