acceptodds
Under review as a conference paper at ICLR 2027

AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks

Abstract

Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one. Existing detection defenses typically classify any input containing an instruction as malicious, leading to misclassification of benign inputs containing instructions that align with the intended task. In this work, we account for the instruction hierarchy and distinguish among three categories: inputs with misaligned instructions, inputs with aligned instructions, and non-instruction inputs. We introduce AlignSentinel, an alignment-aware detection method that flags an input only when the instruction it carries overrides the higher-priority instruction rather than whenever it carries an instruction at all, using a three-class classifier over features derived from the LLM's attention maps. To support evaluation, we construct the first systematic benchmark containing inputs from all three categories. Experiments on both our benchmark and existing ones—where inputs with aligned instructions are largely absent—show that AlignSentinel accurately detects inputs with misaligned instructions and substantially outperforms baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Prompt Injection, Detection Defense, LLM Agent Security

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qbRqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AlignSentinel to detect an input only when the instruction it carries overrides the higher-priority instruction. Compared with existing works, AlignSentinel distinguishes misaligned instructions, aligned instructions, and non-instruction inputs and flags only the first category. AlignSentinel includes an open-weight detection model on attention features between the two input regions. The paper conducts evaluations on an LLM-generated benchmark with 8 domains for direct and indirect injection. The results show that AlignSentinel achieves almost zero false-positive and false-negative rates and has powerful generalization on IHEval and OpenPromptInjection.

Critical strengths
  1. The problem formulation addresses an evaluation gap between benign instructions and malicious injections. Testing whether a detector distinguishes them from conflicting instructions is informative and effective.

  2. Comprehensive evaluations are conducted on self-curated benchmarks and existing benchmarks.

  3. The proposed AlignSentinel is simple yet effective.

Critical weaknesses
  1. I can hardly agree with the paper's arguments on "Existing detection defenses typically classify any input containing an instruction as malicious" and "existing prompt injection benchmarks do not account for instruction hierarchy." Nearly all prompt injection papers consider instruction hierarchy for their benchmark curation. I cannot see any difference between your curated benchmark and existing benchmarks.

  2. The proposed benchmark lacks construction details and its synthetic construction may be hacked via training. The agent and domain splits share a similar generator and construction recipe. And the generation process deliberately creates aligned, opposite, and extended constraints. I cannot see anything new or novel in the construction.

  3. Weak baselines are compared. The performance of frontier LLMs is missing.

  4. Overall, the paper lacks novelty and comprehensive baseline comparisons. Attention-based detection itself is already established by AttnTracker and the benchmark has nothing new.

Questions for authors

Please refer to my listed weaknesses, especially Weakness 1.

Reviewer_j2GxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AlignSentinel, an alignment-aware three-class detector for prompt-injection attacks. Different from prior binary detectors that flag any instruction-containing input as malicious, this work distinguishes misaligned-instruction (malicious override), aligned-instruction (benign task-consistent guidance), and non-instruction inputs. It extracts detection features from cross-token attention interactions between suspicious input and higher-priority governing instructions, and provides two model variants: Avg-first (early average pooling) and Enc-first (token-wise encoding before pooling). It builds a new multi-domain benchmark covering direct / indirect prompt injection with all three sample categories. Evaluations on their own benchmark, IHEval and OpenPromptInjection show strong detection performance, and Enc-first generally yields better robustness.

Critical strengths
  1. It brings a valuable new problem formulation: explicitly modeling instruction-hierarchy semantics, which addresses the high false-positive defect of existing detectors that misclassify benign aligned instructions as attacks. This points out an important blind spot in current prompt-injection defense research.
  2. The method leverages internal attention interaction signals rather than plain text semantics and two architectural variants are well designed and compared. Enc-first preserves fine-grained token-pair signals and achieves better cross-domain generalization.
  3. A dedicated benchmark is constructed, covering 8 domains, direct & indirect injection, and contains the previously-missing aligned-instruction samples, which is a useful community resource.
  4. Comprehensive evaluations: cross-model ablation, cross-benchmark transfer test, adaptive adversarial attack test, controlled comparison against multiple strong published baselines.
Critical weaknesses
  1. The approach assumes full white-box access to attention maps of the detection LLM, yet this prerequisite is not clearly stated in the paper’s threat model. Mainstream commercial black-box LLM APIs do not expose attention weights, so the method cannot work for black-box production environments.
  2. The method extracts fine-grained pairwise token attention features, which introduces non-negligible memory and computation overhead. The paper does not systematically analyze the inference latency and memory cost brought by attention extraction, especially when the input prompt sequence is long.
  3. The paper does not fully study the impact of classification threshold selection. Since the core motivation of AlignSentinel is reducing false positives on benign aligned instructions, more detailed threshold ablation is needed to illustrate the precision-recall trade-off across different operating points.
Questions for authors
  1. The whole benchmark is LLM-generated synthetic data. Could the authors quantify the performance drop when evaluating on real‑world human-written prompt-injection / benign aligned-instruction samples, instead of only AI-generated synthetic instances? It is unclear whether the attention-based pattern learned from synthetic data transfers well to human‑written inputs.
  2. Could the authors analyze the computational and memory overhead of attention feature extraction across different input lengths, and discuss whether this overhead limits practical deployment?
  3. Could the authors provide full precision-recall curves and threshold ablation results, to demonstrate how the detector balances false positive and false negative rates across different threshold choices?
Reviewer_CEN3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors present AlignSentinel, a prompt injection defense which trains a classifier to distinguish attention pattern signatures characteristic of data, malicious injected instructions, and aligned instructions; the approach shows strong results in the author's evaluations.

Critical strengths

Detection of prompt injection via its effect on the internal activations of the targeted model is a promising direction; as these must be influenced by a successful attack, the approach has the potential to correctly distinguish instructions from data. The experimental results and adaptive attack evaluation indicate that AlignSentinel may have realized some of this potential.

Critical weaknesses

While the adaptive attack results are promising, and the adaptive attack design is appropriate, the lack of any supplementary materials with which to evaluate the defense and experimental setup make it difficult to treat the findings as fully reliable. Prominent prompt injection defenses (for instance, DataSentinel [1]) have claimed near-perfect detection accuracy, similar to that reported by AlignSentinel, but have been later been found to be fundamentally flawed [2, 3].

[1] Liu et al. DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks. S&P 2025.

[2] Choudhary et al. How Not to Detect Prompt Injections with an LLM. AISec 2025.

[3] Nasr et al. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections. USENIX Security 2026.

Questions for authors

The following additional adaptive attack evaluations would strengthen my confidence in the robustness of AlignSentinel:

Are the adaptive attack results consistent across additional optimization-based attacks such as AutoDAN [4]?

Are the adaptive attack results consistent with higher step counts, and an increased λ\lambda on the detection-evasion term?

Is AlignSentinel robust to embedding-space attacks with similar objectives? While these are stronger than a realistic adversary, such an evaluation would substantially strengthen the evidence that the adaptive attack results are not due to the difficulty of discrete optimization in token space.

[4] Liu et al. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. ICLR 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.