- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Self-Intervention, a prompt-injection defense for large reasoning models (LRMs). Instead of directly classifying external content as malicious, the method continuously monitors goal drift, i.e., whether the model’s reasoning or actions begin to deviate from the original trusted user request. The authors attach lightweight monitor heads to the frozen reasoning model and use hidden states to detect token-level drift during generation. Once persistent drift is detected, the method rolls back the recent reasoning tokens, inserts a textual reminder, and resumes generation, allowing multiple interventions when necessary. The approach requires neither updates to the base model nor an additional LLM during inference. Experiments on InjectAgent and AgentDojo, including strong adaptive attacks such as MAPO, PAIR, and PISmith, show that Self-Intervention substantially improves robustness while largely preserving benign task performance. Overall, the paper reframes prompt-injection defense from input-level attack detection to reasoning-level goal-drift monitoring and online intervention.
Critical strengths
- Self-Intervention adds lightweight monitor heads to the frozen reasoning model to detect goal drift during generation. Once drift is detected, it rolls back recent reasoning tokens, inserts a reminder, and resumes generation. The method does not require updating the base model or invoking an additional LLM during inference, making it relatively lightweight and practical.
- The paper evaluates the defense against several optimization-based adaptive attacks, including MAPO, PAIR, and the RL-based PISmith. Self-Intervention consistently achieves substantially lower attack success rates than the baselines, demonstrating strong robustness beyond fixed or static prompt-injection attacks.
Critical weaknesses
- The experiments are mainly conducted on Qwen3.6-27B, leaving it unclear whether the proposed monitor generalizes across different model families and scales. The effectiveness of Self-Intervention may depend on model-specific hidden representations.
- Although the additional parameters are relatively small, the method continuously evaluates eight monitor heads at every generated token and may require rollback and regeneration after an intervention. The paper does not provide a systematic analysis of latency, throughput, or computational overhead in practical deployment.
- Self-Intervention relies on the final-layer hidden states of the underlying reasoning model. Therefore, the method is not directly applicable to black-box or API-only models where internal representations are inaccessible, which limits its deployment scope.
- The paper reports cases where the reasoning appears to return to the original task, but the final tool call still follows the injected instruction. This suggests that safe-looking reasoning does not necessarily ensure safe downstream actions, particularly for high-risk agent operations.
- The monitor is trained by pairing two normal requests and treating pursuit of the second request as goal drift when it is hidden from the monitor. While elegant, it remains unclear how well this synthetic formulation captures subtle and complex goal manipulation in real-world prompt-injection attacks. The use of an external LLM judge for labeling may also introduce supervision noise.
Questions for authors
Please see weakness
3 comments
response to rebuttal
The rebuttal partially solved my concern. Especially on weakness 1,2.
Thank you for your follow-up! We’re glad that our rebuttal has helped address some of your concerns. Please let us know if there are any remaining concerns we could further clarify.
We would like to thank you for your detailed comments and feedback. We address the specified weaknesses below.
W1. Generalization across models
Our method generalizes to other model families; we report below the effectiveness of our defense with Muse Glimmer 30B and Gemma 4 26B-A4B. ASR@1 and ASR@5 evaluate MAPO attacks on InjecAgent (Table 1) and the last three columns evaluate on AgentDojo (Table 2).
W2. Runtime overhead
On AgentDojo benign tasks, the full eight-head defense adds 9.06% end-to-end wall-clock overhead, including monitoring, hint prefill, stopping, and regeneration, and generates only 2.71% more tokens. We attribute this efficiency to infrequent intervention on benign tasks and reuse of the retained KV cache when generation resumes. Specifically, only 3/360 turns (across 3 out of the 97 tasks) triggered intervention; this is less than 1% of total turns. One turn ended in blocking (0.28% of all turns).
W3. Hidden-state access
Our defense is best characterized as a model-based method. Thus, we envision that our method will be most relevant for anyone hosting models with access to their hidden states. It might be possible to apply a text-based monitor to protect models that expose reasoning traces but where the internal representation is not accessible; we consider this out-of-scope and do not evaluate it in our work.
W4. Reasoning and action mismatch
We agree; we will mention this limitation in our revised manuscript.
W5. Training data and attack coverage
We appreciate the concern. We believe that our scheme’s success against three different adaptive optimization-based attacks, along with the fact that it substantially outperforms all prior methods (Table 1), is strong evidence for our method’s effectiveness. But we appreciate that it is hard to be sure we have covered all possible attack strategies.