acceptodds
Under review as a conference paper at ICLR 2027

Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience

Abstract

Large language model (LLM) agents are increasingly deployed in tool-augmented environments, but their reliance on external inputs makes them highly vulnerable to prompt injection attacks that can hijack task objectives. Existing defenses largely rely on static expert demonstrations or preference optimization, limiting their ability to generalize to diverse and adaptive attack patterns. In this work, we propose Self-Reflection Fine-Tuning (SRFT), a training framework that enables agents to improve robustness by learning from their own failure experiences under adversarial conditions. Instead of passively imitating expert behaviors, SRFT exposes the agent to compromised trajectories constructed via injected attacks, and leverages an expert model to generate structured self-reflection reasoning that contrasts unsafe and optimal actions. This reflective supervision teaches the agent to identify malicious instructions, reason about their consequences, and maintain alignment with the original user intent. We instantiate this framework in SR-Agent, built on Llama-3.1-8B-Instruct and Qwen3-8B, and evaluate it on both static and adaptive prompt injection benchmarks. Experimental results show that SRFT substantially reduces attack success rates while preserving task performance, and demonstrates strong generalization under adaptive attacks. These findings suggest that learning from failure via self-reflection is a promising direction for building robust and secure LLM agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent security, indirect prompt injection attack, defense

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Qhw2OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Self-Reflection Fine-Tuning (SRFT) to improve tool-using agents’ robustness against prompt injection. The framework constructs attacked trajectories, samples candidate actions from the agent, and uses an expert model to generate rationales contrasting these actions with the desired actions. The agent is then supervised fine-tuned to produce the rationales and correct actions. Experiments evaluate security and utility on AgentDojo and robustness against target-specific RL-Hammer attacks on InjecAgent.

The problem is important and the results are promising. However, I find the methodological distinction from rationale-augmented supervised fine-tuning insufficiently established, and the experimental coverage too limited to support the paper’s broad claims about generalization and adaptive robustness.

Critical strengths
  1. The paper addresses an important and practically relevant problem: defending tool-using agents against prompt injection while preserving legitimate task performance.

  2. The paper is generally clear and easy to follow. The stages of data construction, reflection generation, and fine-tuning are presented coherently.

  3. The evaluation includes useful training-time and inference-time ablations, as well as an analysis of inference token usage. In particular, the comparison with training on the same attacked trajectories without reflection provides evidence that the reflection supervision contributes under the evaluated attack setting.

Critical weaknesses
  1. The methodological contribution beyond supervised fine-tuning needs clearer justification. Section 4.5 uses a standard supervised objective, and the reflections are generated by an external expert model. The main contribution therefore appears to lie in constructing failure-conditioned training targets rather than introducing a distinct learning algorithm. Although the ablations support the usefulness of these targets, they do not fully establish the necessity of agent-specific failure experience compared with expert-generated safety rationales that do not depend on the agent’s sampled actions. The paper’s broad contrast between SFT and SRFT consequently seems overstated.

  2. The adaptive attack evaluation is too narrow. RL-Hammer is the only adaptive attack framework evaluated, while AgentDojo uses only the default “important instructions” attack. Additional attacks that appear stronger in the literature, such as AutoInject (Chen et al., 2026), adaptive attack (Andriuschenko et al., 2024), and the evolutionary search attacks (Nasr et al., 2025), would provide stronger evidence of robustness against strong baselines. Robustness against one attacker-training procedure is insufficient to establish broad robustness against adaptive adversaries.

  3. The evidence for generalization remains limited. The paper evaluates Llama-3.1-8B and Qwen3 at 4B and 8B, which provides some cross-family evidence. Nevertheless, the strongest adaptive comparison with an existing defense is concentrated on Llama-3.1-8B, while the Qwen3 adaptive experiments compare primarily against the undefended model. Broader matched comparisons across backbones and attack methods are needed to substantiate the claimed generality.

  4. Some conclusions about practicality and utility are stronger than the results support. On Qwen3, both benign utility and utility under attack decrease after SRFT, despite the security improvements. The trade-off calculation is also problematic. It is computed as utility under attack + 100 - ASR, but it ignores benign utility, which people are generally more concerned about. For the same reason, a higher number in the trade-off calculation does not mean this method is more convincing. I personally find this metric confusing and would recommend removing it in future supervisions.

Questions for authors
  1. What is the intended methodological distinction between SRFT and supervised fine-tuning on failure-conditioned, expert-generated rationales? Would expert-generated rationales without access to the agent’s sampled candidate actions achieve comparable results?

  2. How does SRFT perform against additional target-adaptive attacks, particularly AutoInject and evolutionary search, that are designed for prompt injection? Do the relative improvements persist under comparable attack budgets?

  3. Table 1 currently only has comparisons with Llama-3.1-8B. Can the authors provide performance results for additional models to show the method's applicability beyond Llama-3.1-8 B?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer Qhw2 (Part 1/2)

We thank the reviewer for the patient and professional review, as well as the valuable suggestions for improvement. The changes are marked in blue in our revision. Our responses to the reviewer's questions are below:

W1 : The contribution beyond SFT needs clearer justification. SRFT uses a standard supervised objective with expert reflections, and the ablations do not fully show that agent-specific failure experience is needed compared with expert rationales that do not depend on sampled actions.

A1: We agree that SRFT uses a standard supervised loss, and our contribution is how the training targets are built. Each reflection is built on candidate actions sampled under attack, and the expert compares them with the optimal action and explains why some tool calls fail the user's goal and what will happen after them. So besides imitating the right action, the agent also learns to check its next action against the user's goal, and it does this by itself at inference time.

To figure out whether this action analysis really matters compared with expert safety rationales, we conduct an ablation on Qwen3-8B under RL-Hammer. The without failure-action analysis variant still has expert-written rationales that find the injection and explain its consequences, and it only removes the comparison between the sampled actions and the optimal action. All other settings are kept the same, eg., the attacked trajectories and the expert actions. The results are shown below:

Training variantFinal RL-Hammer ASR (%) ↓
Full SRFT17.0
Without failure-action analysis65.0
Without reflection, thinking enabled60.0
Without reflection, thinking disabled60.0

We observe that ASR increases from 17.0% to 65.0% after removing the action analysis, which is close to training without reflection (60.0%). It shows that expert rationales without the action analysis can not reproduce the robustness of SRFT, and reflecting on concrete actions is really important. In the revised version, we will make it clear that SRFT differs from SFT in how the training targets are built.


W2 : The adaptive evaluation is too narrow. RL-Hammer is the only adaptive attack, AgentDojo only uses the default attack, and stronger attacks such as AutoInject, Andriushchenko et al. and evolutionary search are missing.

A2: Thanks for the suggestion. To further validate the robustness, we add one more AgentDojo template and one different adaptive attack.

First, we add the tool_knowledge template, so each of the 949 task–injection pairs in AgentDojo is tested under both important_instructions and tool_knowledge. A pair counts as compromised if either template succeeds, and UA is averaged across templates. The results are shown below:

Llama-3.1-8BBenign Utility ↑UA ↑ASR ↓RL-Hammer ASR ↓
Base27.8423.8712.9698.5
SRFT37.1129.291.4817.5

We observe that SRFT reduces AgentDojo ASR from 12.96% to 1.48% and improves UA from 23.87% to 29.29%, so the gains still hold beyond the single template.

Second, we adapt PAIR, an iterative attack-refinement method, to indirect prompt injection on Qwen3-8B. A Llama-3.1-8B-Instruct attacker, without fine-tuning, writes the injected tool content and keeps revising it based on the target responses and attack feedback. We use 100 InjecAgent cases with 30 parallel streams × 3 rounds, so there are up to 90 target queries per case, and both targets use temperature 0.6 with the same configuration as our Qwen RL-Hammer evaluation. A case counts as compromised if any query leads to an attacker-directed tool call. The results are shown below:

Query budgetQwen3-8B ASR (%) ↓SR-Agent-Qwen3-8B ASR (%) ↓
30 queries (first round)27.010.0
Up to 90 queries66.023.0

We observe that SRFT reduces ASR from 27.0% to 10.0% with 30 queries, and from 66.0% to 23.0% with the full 90 queries. It shows that SRFT also resists iterative attack refinement, which works in a different way from RL-Hammer. These results further demonstrate that the robustness of SRFT holds across various attacks, and against both RL-trained and iterative-refinement attackers.

AuthorsOpenReview ↗

Responses to Reviewer Qhw2 (Part 2/2)

W3 : Evidence for generalization is limited. The strongest comparison with an existing defense is only on Llama-3.1-8B, and the Qwen3 adaptive experiments only compare with the undefended model.

A3: Thanks for the suggestion. We evaluate SRFT on Llama-3.1-8B, Qwen3-4B and Qwen3-8B, which covers two model families and two scales, and SRFT consistently improves security under both static and adaptive attacks across them.

First, we compare SRFT and Meta-SecAlign on the same Llama-3.1-8B backbone, and AgentDojo follows the important_instructions protocol. The results are shown below:

DefenseBenign Utility (%) ↑UA (%) ↑AgentDojo ASR (%) ↓RL-Hammer ASR (%) ↓
Meta-SecAlign23.7119.070.9569.5
SRFT37.1129.081.2617.5

We observe that SRFT has higher benign utility and UA with comparably low AgentDojo ASR, and much lower RL-Hammer ASR (17.5% vs. 69.5%).

Second, on Qwen3-4B, SRFT reduces AgentDojo ASR from 10.85% to 0.84%, and we further compare it with Meta-SecAlign on the same backbone. The results are shown below:

DefenseBenign Utility (%) ↑ASR (%) ↓
Meta-SecAlign42.300.70
SRFT50.520.84

We observe that SRFT has 8.22 points higher benign utility with comparably low ASR. On Qwen3-8B, SRFT also reduces RL-Hammer ASR from 59.5% to 17.0%, which further supports its effectiveness across model families.


W4 : Claims about practicality are stronger than the results. Utility drops on Qwen3, and the trade-off metric ignores benign utility.

A4: Thanks for pointing this out. We agree that benign utility should be considered, so we replace the trade-off metric with Benign Utility − ASR, and still report Benign Utility, UA and ASR separately. The revised results are shown below:

ModelBenign Utility ↑UA ↑ASR ↓Benign − ASR ↑
Qwen3-4B57.7350.5810.8546.88
Meta-SecAlign-4B42.30—0.7041.60
SRFT-4B50.5249.210.8449.68
Qwen3-8B60.8250.4716.9743.85
SRFT-8B51.5546.681.0550.50

Actually, the utility drop is a common problem for security alignment. On Qwen3-4B, Meta-SecAlign drops benign utility by 15.43 points, while SRFT only drops it by 7.21 points with comparably low ASR. Under the revised metric, SRFT improves the score from 46.88 to 49.68 on Qwen3-4B and from 43.85 to 50.50 on Qwen3-8B, which is also higher than 41.60 of Meta-SecAlign-4B. Moreover, on Llama-3.1-8B, SRFT improves benign utility from 27.84% to 37.11% and UA from 23.08% to 29.08%, while Meta-SecAlign lowers benign utility to 23.71%, so the utility drop does not happen on every backbone. These results further demonstrate that SRFT gets a better trade-off between utility and security than the state-of-the-art defense.


Q1 : What is the methodological distinction between SRFT and SFT on failure-conditioned expert rationales? Would expert rationales without the sampled actions work as well?

A5: As discussed in A1, the difference is that the reflections in SRFT are built on sampled candidate actions and their consequences. The without failure-action analysis variant keeps the expert rationales that find the injection and explain its risks, and only removes the analysis of candidate actions, and its RL-Hammer ASR goes from 17.0% to 65.0%. So expert rationales without the analysis of sampled actions can not reach comparable results, which shows the extra value of reflecting on candidate actions.


Q2 : How does SRFT perform against AutoInject and evolutionary search under comparable attack budgets?

A6: As discussed in A2, we evaluate PAIR adapted to indirect prompt injection with the same query budget for both targets. SRFT reduces ASR from 27.0% to 10.0% with 30 queries and from 66.0% to 23.0% with 90 queries, so the improvement still holds under a different attack framework.


Q3 : Can the authors provide results on models beyond Llama-3.1-8B?

A7: As discussed in A3, we also evaluate SRFT on Qwen3-4B and Qwen3-8B. On AgentDojo, SRFT reduces ASR from 10.85% to 0.84% on Qwen3-4B and from 16.97% to 1.05% on Qwen3-8B, and under RL-Hammer it reduces Qwen3-8B ASR from 59.5% to 17.0%. These results show that SRFT also works well beyond Llama-3.1-8B, across different model families and scales, and we will add them to Table 1 in the revised version.


Thanks again for the valuable comments. We hope our responses address your concerns, and if so, we would appreciate it if you could consider raising your rating. We are also happy to discuss any further questions.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer Qhw2,

Thank you again for your valuable feedback. We have added three experiments to the revised paper, with changes marked in blue:

  • Clean-SFT control. Compared with clean SFT using the same trajectories and training recipe, SRFT achieves higher benign utility (37.11% vs. 28.87%) and lower RL-Hammer ASR (17.5% vs. 74.0%) (Section 5.5.1). This indicates that generic agentic fine-tuning alone does not explain SRFT’s gains.
  • Additional AgentDojo attack template. Across two attack templates, SRFT reduces ASR from 12.96% to 1.48% while improving utility under attack from 23.87% to 29.29% (Appendix C.2).
  • Additional adaptive attack. Under PAIR on Qwen3-8B, SRFT reduces cumulative ASR from 66.0% to 23.0% with up to 90 queries per case (Appendix C.5). This shows that the robustness gains extend beyond RL-Hammer to iterative attack refinement.

As the discussion period is ending soon, we would appreciate your feedback on our responses. If they have addressed your concerns, we would be grateful if you would consider raising your score.

Best regards,
The authors

Reviewer_ML5GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Self-Reflection Fine-Tuning (SRFT), which enables an agent to learn from "failure experience" to defend against prompt injection. The pipeline is: inject adversarial instructions into clean expert trajectories, let the agent roll out to expose failures, have an expert LLM generate structured self-reflection reasoning (contrasting unsafe and optimal actions and analyzing consequences), and supervise fine-tuning of the agent with these reflections. It is instantiated as SR-Agent (Llama-3.1-8B-Instruct and Qwen3-8B/4B). On AgentDojo it reduces ASR from 7.59% to 1.26% (Llama) and from 16.97% to 1.05% (Qwen3-8B), while maintaining or slightly improving general capability; under the RL-Hammer adaptive attack, ASR stays below 20%, whereas Meta-SecAlign exceeds 60%. Ablations show that removing "failure experience analysis" raises ASR to 65%, removing reflective reasoning raises it to 60%, and disabling thinking at test time raises ASR from 1.05% to 14.33%.

Critical strengths
  1. Robustness under adaptive attack is hard evidence: under a strong adaptive attack like RL-Hammer, which is specifically designed against Meta-SecAlign, SR-Agent consistently keeps ASR < 20% while Meta-SecAlign rises above 60%, and this holds across both Llama and Qwen3 backbones. This demonstrates generalization better than ASR numbers on static benchmarks and directly supports the core claim that "learning from failure experience > pattern-matching suppression."
  2. The ablation design is convincing: removing "failure experience analysis" at train time raises ASR from 17% to 65%, and disabling thinking at test time raises ASR from 1.05% to 14.33%—the two ablations separately isolate the roles of the training signal and explicit reasoning at inference, showing that robustness comes not from memorizing safe responses but from the reflective process itself.
Critical weaknesses
  1. The "self-reflection" framing overstates the mechanism, and baselines are thin:Reflections are generated by an external expert (Claude Sonnet 4.6), not by the agent itself; the agent only supplies failure actions and is SFT-trained to imitate expert-written reflections. This is failure-conditioned expert distillation rather than genuine self-reflection, so the novelty over SecAlign/Meta-SecAlign is largely the supervision form, not a different learning paradigm. The main table also compares only against Meta-SecAlign, omitting StruQ, SecAlign, and CaMeL.
  2. Unexplained utility gains and missing limitations:Benign Utility rises substantially (e.g., Banking 25→43.75, All 27.84→37.11), which a defense should not typically produce and which is left unexplained—possibly agentic SFT rather than the security training, but untested. Travel's Utility under Attack (7.14) is also below the undefended model (12.86), masked by the aggregate Trade-off metric. There is no dedicated Limitations section, and the 38 synthesized injection tasks leave a clear diversity ceiling unaddressed.
Questions for authors

Q1. Benign Utility rises substantially on several suites (e.g., Banking 25→43.75). Please decompose this gain into (a) task-capability gains from the SRFT training data itself acting as agentic SFT, versus (b) evaluation-protocol/sampling differences. If it is mainly (a), does this imply that part of the "security improvement" comes from a stronger task baseline rather than from defense per se?

Q2. The "self-reflection" is generated by an expert LLM rather than by the agent itself. How would results change if reflections were generated by the agent itself (same model)? This bears on the method's naming and its essence.

Q3. The training data contains only 38 malicious injection tasks. Can you provide a scaling curve of injection-task diversity/count versus ASR under adaptive attack, to assess whether 38 is already saturated?

Q4. On the Travel suite, Utility under Attack (7.14) is lower than the undefended baseline (12.86)—why does defense instead harm task completion under attack on this suite? Does this indicate that SRFT over-suppresses under certain task structures?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer ML5G (Part 1/3)

We thank the reviewer for the patient and professional review, as well as the valuable suggestions for improvement. The changes are marked in blue in our revision. Our responses to the reviewer's questions are below:

W1 : Reflections are written by an external expert, so SRFT is failure-conditioned expert distillation and the novelty over SecAlign/Meta-SecAlign is mainly the supervision form. The main table also only compares with Meta-SecAlign, missing StruQ, SecAlign and CaMeL.

A1: We agree that the reflections are written by an expert during training. The "self" in SRFT means the agent reflects on its own next action at inference time. To learn this, each training reflection is built on candidate actions sampled from a reference agent under attack, and it explains why these actions fail the user's goal and what will happen if they are executed. SecAlign and Meta-SecAlign optimize preferences between legitimate and injection-following responses, and normal reasoning finetuning teaches the model why the correct output is correct. SRFT further teaches the model what a wrong action looks like from the agent side and why it goes against the user. So at inference time, the agent checks its own next action against the user's goal before acting, and it does not need the expert or the injection label anymore. This check does not rely on recognizing a known injection pattern, so it still works when an adaptive attacker like RL-Hammer keeps finding new injections.

In fact, this reflection is learned in training and really used at inference time. When we turn off thinking for SR-Agent-Qwen3-8B, AgentDojo ASR goes from 1.05% to 14.33%. Adding the same reflection instruction to the prompt of the untrained model only reduces ASR from 16.97% to 14.75%.

The ablation below also supports this. Actually, the without failure-action analysis variant is very close to the expert distillation the reviewer describes. It keeps the expert-written analysis that finds the injection and its consequences, and only removes the comparison with the sampled actions, and all other settings are kept the same. The results on Qwen3-8B under RL-Hammer are shown below:

Training variantFinal RL-Hammer ASR (%) ↓
Full SRFT17.0
Without failure-action analysis65.0
Without reflection, thinking enabled60.0
Without reflection, thinking disabled60.0

We observe that ASR goes from 17.0% to 65.0% after removing the action analysis, which is close to having no reflection at all (60.0%). It shows that the expert's safety reasoning alone does not help much, and the robustness mainly comes from reflecting on candidate actions, which is exactly what makes SRFT different from existing reasoning-based defenses.

Moreover, SRFT does not need an expert to write the reflections. We let Qwen3-8B write the reflections by itself from the trajectory context, and the expert actions and training recipe are kept the same. The results on AgentDojo under the original single-template protocol are shown below:

Qwen3-8B variantBenign Utility ↑UA ↑ASR ↓
Undefended base60.8250.4716.97
SRFT: expert-generated reflections51.5546.681.05
SRFT: target-model-generated reflections53.6146.582.32

We observe that ASR still drops from 16.97% to 2.32% with comparable utility, and expert reflections only give a bit stronger security (1.05%). It shows that the reflections written by the target model itself also work well.

About the baselines, we mainly compare with Meta-SecAlign-8B, since it is a state-of-the-art model-level defense on the same Llama-3.1-8B backbone. It is also an improved version of SecAlign, so it can represent the StruQ/SecAlign line. Under the original AgentDojo protocol, SRFT gets 37.11% benign utility and 29.08% UA, compared with 23.71% and 19.07% for Meta-SecAlign, and the ASR is comparably low, 1.26% vs. 0.95%. Under RL-Hammer, SRFT has 17.5% ASR while Meta-SecAlign has 69.5%.

We further evaluate the released SecAlign checkpoint (Llama-3-8B-Instruct) on AgentDojo, and adapt the agent context to its instruction/data format. SecAlign only gets 7.22% benign utility and 8.22% UA. Its ASR is 0%, but 99.5% of the attacked trajectories do not have any tool call, so its security comes with very limited agent functionality. CaMeL is a system-level defense based on execution and information-flow control, so it can be used together with model-level training like SRFT. These comparisons further demonstrate that SRFT gives competitive security and better keeps the agent functionality, especially against adaptive attacks. We will add all these discussions in the revised version.

AuthorsOpenReview ↗

Responses to Reviewer ML5G (Part 2/3)

W2 : Benign utility rises a lot, which may come from agentic SFT. Travel UA drops below the base, there is no Limitations section, and the 38 injection tasks limit diversity.

A2: We agree that training on expert trajectories alone can improve tool use. Moreover, the reflections in SRFT also analyze safe but suboptimal actions, which gives extra supervision for task execution. To figure out how much of the gain comes from generic agentic SFT, we train a clean-SFT baseline on the same 3,698 expert trajectories with the same recipe, and remove all injections and reflections. We evaluate on AgentDojo with both important_instructions and tool_knowledge, where a task–injection pair counts as compromised if either template succeeds and UA is averaged across templates, and we also run RL-Hammer. The results are shown below:

Llama-3.1-8BBenign Utility ↑UA ↑ASR ↓RL-Hammer ASR ↓
Base27.8423.8712.9698.5
Clean SFT28.8725.4510.4374.0
SRFT37.1129.291.4817.5

We observe that clean SFT only improves benign utility from 27.84% to 28.87%, while SRFT reaches 37.11%. Clean SFT also keeps ASR high, 10.43% on AgentDojo and 74.0% under RL-Hammer, compared with 1.48% and 17.5% for SRFT. It shows that generic agentic SFT can not explain the gains of SRFT, and we think part of the extra gain comes from the action-level reflection.

On Travel, benign completion improves from 2/20 to 5/20, but UA drops from 18/140 to 10/140. To check whether SRFT rejects too many legitimate instructions, we look at the failed trajectories across all AgentDojo suites under important_instructions, and only 2/100 of the cases where SRFT loses to the base end with a refusal or an injection warning. So the drop is more about task execution than widespread over-suppression.

About the diversity, the training data is actually much more diverse than the 38 malicious tasks. Besides them, we include 285 benign but off-task requests, which teach the agent that instructions in tool outputs can conflict with the user's goal even when they are not clearly malicious. Combined with 686 trigger templates, they give 3,455 distinct injection strings, and the insertion position in the tool observation also varies. Moreover, we train on Toucan and test on AgentDojo and InjecAgent/RL-Hammer, which have different environments and tool APIs, and no AgentDojo or InjecAgent attack goal shares a six-word span with any training injection. So the results are not from memorizing the training prompts. We will add all these discussions and a Limitations section in the revised version.

AuthorsOpenReview ↗

Responses to Reviewer ML5G (Part 3/3)

Q1 : Please decompose the benign utility gain into agentic SFT effects and evaluation differences. Does part of the security gain come from a stronger task baseline?

A3: As shown by the clean-SFT baseline in A2, all models are evaluated under the same settings, so the gains are not from evaluation differences. Generic agentic SFT only gives a small utility gain from 27.84% to 28.87%, while SRFT reaches 37.11%. The security gain also can not be explained by a stronger task baseline alone, since clean SFT still has 10.43% AgentDojo ASR and 74.0% RL-Hammer ASR, compared with 1.48% and 17.5% for SRFT. These results further demonstrate the extra contribution of the reflection beyond generic agentic SFT.


Q2 : How would the results change if the reflections were generated by the agent itself?

A4: As shown in A1, when Qwen3-8B writes the reflections by itself with everything else kept the same, AgentDojo ASR still drops from 16.97% to 2.32%, compared with 1.05% with expert reflections, and utility stays comparable. So SRFT does not need an expert to write the reflections, and at inference time the agent always reflects by itself.


Q3 : Is 38 injection tasks already saturated? Can you provide a scaling curve of task diversity versus adaptive ASR?

A5: As discussed in A2, our training data includes 38 malicious tasks, 285 benign but off-task requests, and 3,455 distinct injection strings from 686 trigger templates, and our cross-environment and adaptive results show that SRFT generalizes beyond the training distribution.


Q4 : Why is Travel UA lower than the undefended baseline? Does SRFT over-suppress under some task structures?

A6: As discussed in A2, Travel benign completion improves from 2/20 to 5/20, while UA drops from 18/140 to 10/140. Only 2/100 of the cases where SRFT loses to the base involve a refusal or an injection warning, so the drop is more about task execution than over-suppression.


Thanks again for the valuable comments. We hope our responses address your concerns, and if so, we would appreciate it if you could consider raising your rating. We are also happy to discuss any further questions.

Reviewer_UquZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SRFT trains agents on teacher-generated explanations comparing correct actions with alternatives sampled from a reference agent. The authors insert prompt injections into expert tool-use trajectories, sample candidate next actions, and generate analyses of the user goal, injection, and candidate consequences. The explanations and correct actions supervise LoRA fine-tuning. Evaluations cover AgentDojo's default important_instructions attack on Llama-3.1-8B and Qwen3 models, defense-specific RL-Hammer attackers on InjecAgent, general-capability tests, and AgentDyn. ASR falls in static and adaptive tests. On AgentDojo, task completion improves for Llama and decreases for Qwen.

Critical strengths
  1. The method goes beyond teaching the agent to identify an injection or imitate a safe action. It compares candidate actions with the expert action and explains how they could derail the user's task.
  2. The action-analysis ablation and evaluation against defense-specific adaptive attackers provide useful evidence for the method's effectiveness.
Critical weaknesses
  1. The value of failure experience needs clearer evidence. ReasAlign and RETA already use reasoning-based supervision to defend against injections. SRFT adds comparisons between sampled and expert actions, but its ablation removes the entire action-analysis component, so it does not show whether reference-agent failures are more useful than teacher-generated wrong actions. The paper should isolate this difference and clarify that the method learns from offline, teacher-analyzed action samples rather than verified execution failures.
  2. Task utility is not consistently preserved. On Qwen3-8B, benign utility drops from 60.82% to 51.55%, and utility under attack also decreases; Qwen3-4B shows a similar trade-off. On AgentDyn, SR-Agent-Llama completes only 10% of benign tasks. The authors should explain these failures and distinguish limited task capability from rejection of legitimate instructions.
  3. Adaptive evaluation remains limited. The adaptive results average only two attacker runs and show considerable variation. They also report ASR on 100 InjecAgent cases without legitimate-task completion, while AgentDojo uses only one static attack. More runs and task-completion results under adaptive attacks would strengthen the claim of robust and useful defense.
Questions for authors
  1. Which reference policy generated the candidate actions? Does using its failures outperform generic or teacher-generated alternatives when the action-analysis structure and supervision length are comparable?
  2. Beyond the brief discussion of ReasAlign, what distinguishes SRFT from existing reasoning-supervised defenses? A controlled comparison with the closest feasible recipe would help establish the added value.
  3. Can the authors report additional attacker runs and legitimate-task completion under adaptive attack, especially for the ablation models?
  4. What causes the Qwen utility losses? On AgentDyn, how often do failures involve rejecting legitimate third-party instructions rather than inability to complete the task?
5 comments
AuthorsOpenReview ↗

Responses to Reviewer UquZ (Part 1/4)

We thank the reviewer for the patient and professional review, as well as the valuable suggestions for improvement. The changes are marked in blue in our revision. Our responses to the reviewer's questions are below:

W1 : ReasAlign and RETA already use reasoning-based supervision, and the ablation does not show whether reference-agent failures are more useful than teacher-generated wrong actions. The paper should also make clear that the actions are offline samples analyzed by the teacher, not verified execution failures.

A1: Thanks for the valuable question. We agree that SRFT uses the same supervised loss as ReasAlign and RETA, and the main difference is what the reasoning talks about. In SRFT, each reflection is built on the actions sampled from a reference agent under attack, eg., following the injection or making a safe but suboptimal tool call, and the expert explains why each of them fails the user's goal and what will happen if it is executed. So besides knowing why the injection is malicious, the agent also learns to check its own next action against what the user really wants. We would like to clarify that these sampled actions are not executed in the environment. The expert analyzes their consequences from the context, so we can get these failure cases without really running harmful tool calls.

To figure out how much this action analysis helps, we conduct an ablation on Qwen3-8B under RL-Hammer. The without failure-action analysis variant still has the expert-written analysis that finds the injection and explains its risks, and it only removes the comparison between the sampled actions and the optimal action. All other settings are kept the same, eg., the attacked trajectories and the expert actions. The results are shown below:

Qwen3-8B training variantFinal RL-Hammer ASR (%) ↓
Full SRFT17.0
Without failure-action analysis65.0
Without reflection, thinking enabled60.0
Without reflection, thinking disabled60.0

We observe that removing the action analysis increases ASR from 17.0% to 65.0%, which is close to training without reflection (60.0%). So only analyzing the injection can not bring the adaptive robustness, and comparing the candidate actions is the key part of SRFT. These results further demonstrate the advantage of SRFT over existing reasoning-based defenses, and show that grounding the reflection in concrete actions is really needed for agent security.

Actually, compared with teacher-generated wrong actions, the sampled actions have two advantages. First, they are sampled from a policy model under the attacked context, and we do not ask it to make mistakes, so they are the failures that really happen in this context, eg., partly following the injection or calling a related but wrong tool. Wrong actions written by a teacher on purpose tend to be obvious, like directly calling the attacker's tool, and the model can easily tell them apart from the correct action without really learning why they fail. Second, the sampled actions naturally include safe but suboptimal tool calls, which a teacher asked to write wrong actions usually would not produce, and reflecting on them also gives supervision for task execution.

AuthorsOpenReview ↗

Responses to Reviewer UquZ (Part 2/4)

W2 : Utility is not consistently preserved. Benign utility drops on Qwen3, and SR-Agent-Llama only completes 10% of benign tasks on AgentDyn.

A2: Actually, the utility drop is a common problem for prompt injection defenses, since the model learns to be more careful with the content in tool outputs. For example, Meta-SecAlign also loses a lot of utility. We run Meta-SecAlign on the same Qwen3-4B backbone, and the results on AgentDojo are shown below:

Qwen3-4BBenign Utility (%) ↑ASR (%) ↓
Undefended57.7310.85
Meta-SecAlign42.300.70
SRFT50.520.84

We observe that Meta-SecAlign drops benign utility by 15.43 points, while SRFT only drops it by 7.21 points, and both keep ASR below 1%. On Llama-3.1-8B, Meta-SecAlign also lowers benign utility from 27.84% to 23.71%, while SRFT improves it to 37.11%.

To look at utility and security together, we further compute Benign Utility − ASR and UA − ASR, and the results are shown below:

ModelBenign Utility ↑UA ↑ASR ↓Benign − ASR ↑UA − ASR ↑
Qwen3-4B57.7350.5810.8546.8839.73
Meta-SecAlign-4B42.30—0.7041.60—
SRFT-4B50.5249.210.8449.6848.37
Qwen3-8B60.8250.4716.9743.8533.50
SRFT-8B51.5546.681.0550.5045.63

We observe that SRFT improves both metrics on the two models. On Qwen3-4B, Benign − ASR goes from 46.88 to 49.68, which is also higher than 41.60 of Meta-SecAlign-4B, and on Qwen3-8B it goes from 43.85 to 50.50. It shows that SRFT gets a better trade-off between utility and security than the state-of-the-art defense, so it is more practical for agent security.

For AgentDyn, the undefended Llama-3.1-8B only completes 7/60 benign tasks and SRFT completes 6/60, so the gap is only one task. It shows that the low completion mostly comes from the limited ability of the base model, and it is not evidence that SRFT systematically rejects legitimate instructions.

AuthorsOpenReview ↗

Responses to Reviewer UquZ (Part 3/4)

W3 : Adaptive evaluation is limited. RL-Hammer averages only two attacker runs and reports no task completion, and AgentDojo uses only one static attack.

A3: Thanks for the suggestion. We would like to clarify that RL-Hammer is not evaluated on only 100 fixed attacks. Each attacker is trained for 20 epochs and every checkpoint is tested on all 100 InjecAgent cases, so one attacker run has about 2,000 attack evaluations generated during training.

Actually, InjecAgent itself only checks whether the agent calls the attacker's tool, and it does not define when the user task is completed, so we can not directly get task completion on it. So to evaluate legitimate-task completion under RL-Hammer, we split the responses at the final checkpoint by whether they follow the attack and whether they are valid outputs. The results are shown below:

Llama-3.1-8B targetASR (%) ↓Non-attack outputs (%)Invalid outputs (%) ↓
Base98.51.00.5
SRFT17.580.02.5

We observe that SRFT reduces ASR from 98.5% to 17.5%, and 80.0% of its outputs are valid non-attack outputs with only 2.5% invalid. It shows that SRFT still works normally in most cases under adaptive attack, and the lower ASR is not from broken outputs. Moreover, AgentDojo can directly measure task completion, and on Llama-3.1-8B SRFT improves Benign Utility from 27.84% to 37.11% and UA from 23.08% to 29.08%.

We further add the tool_knowledge template to AgentDojo, so each of the 949 task–injection pairs is tested under both important_instructions and tool_knowledge. A pair counts as compromised if either attack succeeds, and UA is averaged across the two templates. The results are shown below:

Llama-3.1-8BBenign Utility (%) ↑UA (%) ↑ASR (%) ↓
Base27.8423.8712.96
SRFT37.1129.291.48

We observe that SRFT reduces ASR from 12.96% to 1.48%, and improves UA from 23.87% to 29.29%. It shows that SRFT keeps both better security and stronger task completion under the two templates.

We also adapt PAIR to indirect prompt injection on Qwen3-8B as another adaptive attack. A Llama-3.1-8B-Instruct attacker, without fine-tuning, keeps revising the injected tool content based on the target responses and attack feedback. We use 100 InjecAgent cases with 30 parallel streams × 3 rounds, so there are up to 90 target queries per case, and both targets use temperature 0.6 with the same configuration as our Qwen RL-Hammer evaluation. A case counts as compromised if any query leads to an attacker-directed tool call. The results are shown below:

Query budgetQwen3-8B ASR (%) ↓SR-Agent-Qwen3-8B ASR (%) ↓
30 queries (first round)27.010.0
Up to 90 queries66.023.0

We observe that with the full 90-query budget, SRFT reduces ASR from 66.0% to 23.0%. These results further demonstrate that the robustness of SRFT holds under different static templates and different adaptive attacks, and it also keeps better legitimate-task completion on AgentDojo.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.