acceptodds
Under review as a conference paper at ICLR 2027

Meta-SecAlign: A Utility-Preserving Training Recipe for Prompt Injection Defense

Abstract

Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to AI agents. By fine-tuning on simulated prompt injections, SecAlign, a leading open defense, reports LLMs with good test-time robustness and negligible benign utility drop. By scaling up training and evaluations, however, we find that SecAlign actually suffers from significant utility degradation, especially in agentic tasks where the threat of prompt injection is prominent. Motivated by this, we propose Meta-SecAlign for utility-preserving defense by (1) randomized injection position during training to avoid shortcut learning and (2) self-generated responses as high-quality indistribution training labels. Across general knowledge, instruction following, and agentic workflows (on tool-calling and web-navigation), Meta-SecAlign maintains almost all the undefended LLM’s utility while achieving better overall security than SecAlign against various static and GCG adaptive attacks. Experiments use Llama-3.1-8B, Llama-3.3-70B, Llama-4-Scout, Qwen3-4B, and Qwen3.6-27B on 6 prompt injection benchmarks including AgentDojo, InjecAgent, WASP, SEP, etc.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
prompt injection, large language models, LLM security, preference optimization, AI agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1rHzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper revisits SecAlign, a DPO-based defense for indirect prompt injection, and shows that at 70B scale it loses most of its agent utility (AgentDojo drops from 59.8% undefended to 6.2%; WASP from 62.2% to 45.9%). The authors trace this to two problems in SecAlign’s preference data. First, injections are always placed at the end of the data, so the model learns to ignore whatever instruction comes last. Second, the response labels come from an old text-davinci-003 dataset that is off-distribution for current models. Meta-SecAlign fixes both: it places injections at the start or end of the data at random, and it generates both preferred and dispreferred responses from the base model itself. On Llama-3.3-70B this keeps utility within 2.7 percentage points of the undefended model on eight benchmarks, brings ASR to single digits on six injection benchmarks, and holds up better than SecAlign under a GCG adaptive attack. The recipe is also applied to Llama-3.1-8B, Llama-4-Scout, Qwen3-4B, and Qwen3.6-27B, and the paper shows that LoRA α can be adjusted at test time to move along the utility–security trade-off.

Critical strengths
  1. Showing that SecAlign breaks down at 70B scale (AgentDojo utility 6.2%) is a useful finding for the community. The paper identifies the failure mechanism—the model outputs nothing when the user prompt is the last instruction in the context (Figure 2)—and the fix is simple.

  2. The experiments are broad: five model families up to 109B, eight utility benchmarks, six security benchmarks, and a GCG adaptive attack. The test-time LoRA-α interpolation (Figure 3) is a practical way to adjust the utility–security trade-off without retraining.

Critical weaknesses
  1. The paper presents two separate takeaways (+69.0 pp from position randomization, +10.9 pp from self-generated labels), but combining Tables 1 and 4 shows that neither technique works on its own. Position randomization with davinci labels gives 15.5% AgentDojo utility, self-generated labels with end-only injection also give 15.5%, and only the combination reaches 84.5%. Presenting them as independent contributions in the introduction is not supported by the data, and it undercuts the mechanism in Section 3.1: if the end-position shortcut were the whole story, randomizing position should help regardless of the label source, and it does not. This matters because the paper’s methodological narrative rests on this attribution.

  2. The headline result—Llama-3.3-70B utility rising from 59.8% to 84.5% after defense—should not be read as utility preservation. Footnote 1 links the increase to two non-security failure patterns in the undefended model (no tool call emitted, tool-call loops) and says the cause is unclear. The likely reading is that fine-tuning repaired the base model’s tool calling rather than preserved it, and every other family shows a small drop instead (e.g., Qwen3-4B from 43.3% to 42.3%). The clean evidence for the paper’s claim is those small drops, and they should be the headline, not the 70B jump.

Questions for authors
  1. Does position randomization alone reduce the empty-output rate in Figure 2 at all when davinci labels are kept?

  2. What fraction of undefended Llama-3.3-70B failures are empty tool calls or loops, and does a control SFT on clean Alpaca data (no DPO, no injections) also bring utility to about 84%?

  3. Can you report SecAlign baseline numbers for the four other model families in Table 6?

  4. Can you run at least one adaptive attack against the defended models on an agentic benchmark (AgentDojo or InjecAgent)?

Reviewer_djNPOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Meta-SecAlign, a utility-preserving training recipe for defending large language models against indirect prompt injection attacks. The authors identify substantial utility degradation in the existing SecAlign framework, particularly in agentic tasks such as tool calling and web navigation.

To address this issue, Meta-SecAlign introduces two modifications to preference-based defensive fine-tuning: (1) randomizing the positions of simulated prompt injections during training to mitigate positional shortcut learning, and (2) using self-generated responses from the original model as training labels to reduce distributional mismatch. The approach retains the original DPO training objective.

Experiments across multiple model families and sizes, including Llama and Qwen models, show that Meta-SecAlign substantially improves the utility-security trade-off compared with SecAlign. The evaluation covers general knowledge, instruction following, agentic workflows, and both static and adaptive prompt injection attacks.

Critical strengths
  1. Important and practically relevant problem. The paper addresses the utility degradation introduced by defensive fine-tuning, which is particularly important for deploying secure LLM-based agents. The observation that existing defenses can severely impair agentic capabilities provides a meaningful motivation.

  2. Simple and effective training improvements. The proposed techniques are straightforward to implement and do not require architectural changes or additional inference-time computation. Despite their simplicity, they produce substantial improvements in the utility-security trade-off.

  3. Comprehensive experimental evaluation. The paper evaluates multiple model families and scales across a diverse collection of benchmarks covering general knowledge, instruction following, tool calling, and web navigation. The inclusion of adaptive GCG attacks further strengthens the evaluation.

  4. Useful ablation studies. The paper investigates both injection position randomization and response-label selection, providing empirical evidence that these design choices contribute to preserving model utility while maintaining robustness against prompt injection.

Critical weaknesses
  1. Insufficient analysis of the shortcut-learning hypothesis. The authors attribute SecAlign's utility degradation to positional shortcut learning, where the model learns to ignore instructions appearing near the end of the input. However, the evidence is primarily based on qualitative examples and downstream performance improvements. More systematic analysis of model behavior across different instruction positions and message roles would be valuable to validate this explanation.

  2. Limited evaluation against strong adaptive attacks. Although the paper includes GCG-based adaptive attacks, Meta-SecAlign still exhibits substantial attack success rates under this setting (e.g., 47.1% on Llama-3.3-70B). Evaluating a broader range of adaptive, agent-specific attacks would provide stronger evidence of the defense's robustness in realistic adversarial environments.

Questions for authors

see weakness

Reviewer_ZsFNOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies system-level security risks in Claw-like AI agents, where persistent access to tools, credentials, and external services enables attacks spanning multiple runtime components. The authors analyze these agents through a computer-systems lens, identifying weaknesses in isolation, privilege management, persistent-state protection, and data–instruction separation. They introduce SafeClawArena, a benchmark of 406 adversarial tasks covering skill supply-chain integrity, persistent-state exploitation, cross-boundary data flow, and indirect prompt injection, evaluated in containerized agent platforms using canary-based taint tracking. Experiments across three platforms and five frontier models reveal substantial vulnerabilities, including malicious plugins that succeed regardless of the underlying model, while comparisons of hardened platforms show that security improvements can depend on restricting functionality and vary across model backbones.

Critical strengths
  1. The paper makes a practical contribution, particularly from an engineering perspective, and provides valuable guidance for balancing utility and security in agent alignment.
  2. The proposed approaches are intuitively effective and reasonable.
  3. Extensive experiments demonstrate the effectiveness of the proposed approaches in improving security while preserving utility.
Critical weaknesses
  1. The randomized injection position strategy only considers the beginning and end of the data. Why not use a fully randomized strategy that inserts injected instructions at arbitrary positions within the data? Is there any advantage to restricting insertion to the beginning or end?
  2. Although the paper has practical value for agent safety alignment, its primary techniques—randomized injection positions and in-domain response learning—offer limited academic novelty.
  3. It is surprising that Meta-SecAlign significantly improves utility on AgentDojo for Llama-3.3-70B but not for the other models (Qwen3-4B, Qwen3.6-27B, and Llama-4-Scout). This raises concerns about whether the effectiveness of randomized injection positions and in-domain response learning demonstrated in Table 1 is specific to Llama-3.3-70B. Although Meta-SecAlign improves security on the other models, these improvements may stem from the underlying SecAlign mechanism rather than the two proposed techniques. Analyzing the individual effectiveness of these two components on the other three models would further demonstrate their generality.
  4. Meta-SecAlign could be regarded as a foundation model and should therefore transfer easily to other agent harnesses, such as OpenClaw. The evaluation focuses on earlier security benchmarks for tool-using agents. I am curious about how it performs on more recent harness-based agent benchmarks, such as DTap [1] and SafeClawArena [2].

References:

  • [1] DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents
  • [2] Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens
Questions for authors

Please refer to weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.