- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes an architectural prompt injection defense. It assumes labels for the untrusted (data) part of the input. It proposes to learn a re-embedding (rotation) matrix for the data tokens. Unlike prior work, only the re-embedding matrix is learned, making the method parameter efficient. The method is conceptually similar to some ideas from prior work (ISE, ASIDE and SecAlign), but combines them in a novel and interesting way, bringing in an improvement in security at a lower training cost. The fact that learning the re-embedding alone can work (with the model frozen) is the central thing I learned from this paper. Which is a somewhat useful, if minor, practical contribution. However, the baseline comparisons, analysis and ablations do not provide much additional clarity or justification of the design choices.
Critical strengths
The paper studies an important problem. It proposes an interesting method for defending against prompt injections. The proposed method is very parameter-efficient and shows promising results on security and utility, as measured by standard (if somewhat outdated) benchmarks.
Critical weaknesses
Baselines and comparisons
W1
The training data described in 3.2.2 and the CPO objective defined there very closely resemble the training data used to train SecAlign. It is one of the best known methods in the field. The closest method to your work, ASIDE, to which you compare, is an architecture based defense, it does not use adversarial (with instructions injected) training data. Your method does use adversarial data, so it is both an architecture-based and data-based defense. I believe it should be compared to SecAlign because of this. Arguably, your method uses fewer trainable parameters than a SecAlign LoRA would, so this probably motivates a comparison of similar resources. In any case, this comparison is necessary.
W2
Comparison to ASIDE is in your favor, both in terms of trainable parameters and security/utility. But again, you use adversarial data. As a potential ablation to better understand how different parts of your method contribute - I suggest training your method on non-adversarial data (Alpaca is a good option).
Evaluation
W3
The field of adversarial defense of LLMs is starting to move on from static attacks, since their results strongly underestimate what an adaptive attacker can do (https://arxiv.org/abs/2510.09023). Security against an adaptive attack would be a much more practically useful and informative evaluation than SEP or StruQ. One option would be to try RL-Hammer (https://arxiv.org/abs/2510.04885) although there are many options.
W4
I understand the potential resource constraints, but single-turn, pure-chat-based security benchmarks like SEP and StruQ are starting to lose relevance since the shift of the field towards agentic use-cases. Prompt injection is most relevant for agents, rather than just chatbots, since they have affordances and can cause problems in the system. Thus, evaluation in agentic benchmarks would make this paper much stronger. AgentDojo or InjecAgent could be an option. Importantly, retaining utility on SEP and HotpotQA hides utility loss on more complex behaviors, for example tool calling.
I would be open to raising my score if the concerns are adressed.
Questions for authors
Q1
Why did you decide to frame the method as two separate components? It seems that you apply a learned rotation matrix of the structure defined in 3.2.1 using the training data and signal defined in 3.2.2. Those are not separate components, right? Both this description and the picture in Figure 2 hinder understanding of what happens, as it looks like a two-stage pipeline, which it is not.