- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AgentFalcon for step-level safety alignment of tool-using LLM agents. Observing that agents can often recognize unsafe behaviors when evaluating their own trajectories despite failing during execution, the authors score each prefix using the agent's own safe and unsafe token log-probabilities. They linearly residualize the unsafe increment against the safe increment to filter out benign task-sensitive steps, trigger safety reflection at the first outlier sentence, and iteratively construct local preference pairs for DPO training. Experiments on four backbones and four safety benchmarks show consistent ASR reductions while maintaining general agentic performance.
Critical strengths
- Comparing the same backbone as a task executor versus a trajectory evaluator nicely demonstrates the gap between latent risk awareness and action execution in multi-turn agents.
- The local preference pair construction handles both agent outputs and external tool/environment messages cleanly, and achieves strong empirical results across four backbones on both safety and general agentic benchmarks.
Critical weaknesses
- In Sec. 3.3, and are raw token log-probabilities without renormalization over
\{safe, unsafe\}. Both can increase simultaneously simply because adding increases the total probability mass assigned to valid label tokens, without needing the "Dual Evidence Sentence" hypothesis. Taking binary softmax or log-odds increments (, i.e., fixing ) removes shared probability-mass shifts directly. Meanwhile, Theorem 1 and Appendix A spend two pages deriving standard univariate OLS (), yet the paper never reports the fitted values or compares against / normalized probabilities. - The motivation of avoiding external models and reducing cost does not fully hold up. First, Appendix C.2 still relies on a frontier external model (
GPT-5.6 Sol) to filter all 9,302 candidate pairs. Second, running prefix forward passes using the target agent itself plus iterative regeneration retries is computationally heavy. In practice, training a lightweight dedicated safety classifier (e.g., 1B-4B) for step-level localization would be much cheaper, reusable across backbones, and directly calibrated to separate benign sensitive actions from real attacks. - The ablations in Fig. 4 do not cleanly isolate the localization score's contribution.
External Localizationperforms almost identically to AgentFalcon (9.3% vs. 8.7% ASR on AgentLAB; 2.2% vs. 2.0% on ASB), suggesting that the gains mainly come from the step-level reflective DPO framework rather than the self-residualization score. Also, the SFT ablation trains on full trajectories rather than local segments, mixing supervision scope with training objective.
Questions for authors
- What are the fitted values across the four backbones, and how does compare against fixing (log-odds) or binary softmax normalization over
\{safe, unsafe\}in both localization F1 (Sec. 3.4) and downstream ASR? - What is the total compute cost (tokens/wall-clock time) of prefix scoring and iterative regeneration, and how does the method perform without
GPT-5.6 Solpair filtering (p_filter)? - How does localized SFT (or full-trajectory DPO) perform in Fig. 4 when holding supervision scope or training objective fixed?
10 comments
Response to Reviewer fkGp (2/7)
W1-2 & Q1. Theorem 1 and Appendix A derive standard univariate OLS for , yet the paper never reports the fitted values. What are the fitted values across the four backbones?
R1-2: We apologize for omitting the fitted values in our manuscript. Below, we report the OLS-fitted residualization coefficients and localization thresholds across all four primary backbones:
Table R2. Calibration parameters across backbones.
| Backbone | ||
|---|---|---|
| Qwen3.5-4B | 0.791 | 2.477 |
| Qwen3.5-9B | 0.753 | 2.546 |
| Qwen3.6-35B-A3B | 0.788 | 2.429 |
| Ministral-3-14B-Instruct-2512 | 0.726 | 2.684 |
Across all four backbones, the fitted values consistently fall between 0.726 and 0.791 rather than approaching 1, which aligns with R1-1 where fixing degrades localization precision and downstream safety. We report these parameter values in Appendix D.2 of our revised manuscript.
Response to Reviewer fkGp (3/7)
W2-1 & Q2. Appendix C.2 still relies on a frontier external model, GPT-5.6 Sol, to filter all 9,302 candidate pairs. How does the method perform without GPT-5.6 Sol pair filtering?
R2-1: To verify whether AgentFalcon relies on GPT-5.6 Sol for pair filtering, we start from the same 9,302 candidate pairs on Qwen3.5-9B and compare GPT-5.6 Sol filtering with self-filtering by Qwen3.5-9B itself, open-weight filtering with Qwen3.5-4B and Qwen3.6-35B-A3B, and no filtering:
Table R3. Downstream performance by filtering strategy.
| Filter | Retained Pairs | AgentDojo ASR ↓ | AgentDojo TCR ↑ | ASB ASR ↓ | ASB TCR ↑ | AgentLAB ASR ↓ |
|---|---|---|---|---|---|---|
| GPT-5.6 Sol | 8,344 | 15.8 | 78.5 | 2.0 | 91.3 | 8.7 |
| Qwen3.5-9B self-filter | 8,133 | 15.4 | 77.2 | 2.3 | 90.3 | 9.0 |
| Qwen3.5-4B | 8,241 | 16.2 | 77.1 | 2.4 | 90.8 | 8.6 |
| Qwen3.6-35B-A3B | 8,527 | 15.9 | 77.4 | 1.9 | 91.0 | 8.8 |
| No filtering | 9,302 | 18.3 | 78.9 | 5.5 | 91.5 | 13.8 |
The results show that AgentFalcon does not rely on GPT-5.6 Sol, as self-filtering by the acting model itself and open-weight filtering achieve comparable downstream safety and utility without any external model. Even without filtering, AgentFalcon retains strong performance, while pair filtering further removes unsuccessful reflections. We include the downstream results for these filtering strategies in Section 5.5 of our revised manuscript.
Response to Reviewer fkGp (1/7)
We sincerely thank the reviewer for the insightful and constructive review. We particularly appreciate your recognition of our risk-awareness analysis and empirical evaluation. Below, we address each point under
WeaknessesandQuestions, and we sincerely hope that our clarifications and additional analyses resolve your concerns. Unless otherwise stated, all experiments use Qwen3.5-9B as the agent backbone.W1-1 & Q1. In Section 3.3, Si and Ri are not renormalized over the safe and unsafe labels, whereas log-odds increments with γ=1 or binary softmax remove shared probability-mass shifts directly. How does γ∗ compare against fixing γ=1 or binary softmax in localization F1 and downstream ASR?
R1-1: To verify whether retaining probability-mass information via fitted residualization improves over direct label normalization, we compare our score against fixing γ=1 and binary softmax. While normalizing over the
safeandunsafelabels removes shared shifts, it also discards total probability-mass changes that help distinguish benign sensitive steps from genuine risk emergence. Keeping all subsequent stages fixed, we evaluate our fitted residualization score ΔRi−γ∗ΔSi, log-odds increments ΔRi−ΔSi with γ=1, and binary-softmax probability differences:Table R1a. Localization performance by scoring method.
Table R1b. Downstream performance by scoring method.
The results show that retaining probability-mass information via fitted residualization achieves substantially higher localization precision and stronger downstream safety than direct label normalization. Full normalization triggers more false alarms on benign steps, introducing noisy interventions that weaken alignment. We add this comparison to Appendix D.2 of our revised manuscript.