acceptodds
Under review as a conference paper at ICLR 2027

Learning to Align Where Risk Emerges for LLM Agents

Abstract

LLM agents solve long-horizon tasks through multi-turn interaction and tool use, but also introduce safety risks such as privacy leakage and unauthorized tool execution. Runtime guardrails can mitigate these risks, yet if bypassed, the underlying model may still lack a reliable basis for safe decisions, motivating intrinsic safety alignment. However, trajectory-level alignment provides sparse supervision, obscuring where to intervene for safe execution. Step-level alignment offers denser supervision, but localizing steps that steer trajectories toward unsafe behavior typically relies on stronger external models, which may be unavailable for frontier agents and incur substantial API costs and privacy risks. Surprisingly, We find that the latent risk awareness of the acting LLM Agent can instead support reliable risk localization. We therefore propose AgentFalcon, which quantifies each sentence's information gain toward an unsafe judgment elicited from the acting LLM Agent and residualizes it against the corresponding safety information gain to identify sentences that drive unsafe behavior. It then intervenes at the earliest risk position, elicits safety reflection, and pairs the regenerated safe continuation with the original continuation for preference optimization and repeat this process to improve risk coverage and preference-data efficiency. Experiments across four LLM backbones show that AgentFalcon reduces unsafe behavior while preserving benign-task performance. On Qwen3.5-9B, it reduces the attack success rate on AgentLAB from 53.3% to 8.7% and attains a 4.1% harmful-task score and 96.8% refusal rate on AgentHarm. It also improves DeepPlanning accuracy from 15.4% to 16.6% and LiveClawBench from 25.2% to 28.6% which indicate stronger capabilities in long-horizon task planning and cross-service task execution.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, safety alignment, risk localization, preference optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fkGpOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AgentFalcon for step-level safety alignment of tool-using LLM agents. Observing that agents can often recognize unsafe behaviors when evaluating their own trajectories despite failing during execution, the authors score each prefix using the agent's own safe and unsafe token log-probabilities. They linearly residualize the unsafe increment against the safe increment to filter out benign task-sensitive steps, trigger safety reflection at the first outlier sentence, and iteratively construct local preference pairs for DPO training. Experiments on four backbones and four safety benchmarks show consistent ASR reductions while maintaining general agentic performance.

Critical strengths
  • Comparing the same backbone as a task executor versus a trajectory evaluator nicely demonstrates the gap between latent risk awareness and action execution in multi-turn agents.
  • The local preference pair construction handles both agent outputs and external tool/environment messages cleanly, and achieves strong empirical results across four backbones on both safety and general agentic benchmarks.
Critical weaknesses
  1. In Sec. 3.3, SiS_i and RiR_i are raw token log-probabilities without renormalization over \{safe, unsafe\}. Both can increase simultaneously simply because adding ziz_i increases the total probability mass assigned to valid label tokens, without needing the "Dual Evidence Sentence" hypothesis. Taking binary softmax or log-odds increments (ΔRi−ΔSi\Delta R_i - \Delta S_i, i.e., fixing γ=1\gamma = 1) removes shared probability-mass shifts directly. Meanwhile, Theorem 1 and Appendix A spend two pages deriving standard univariate OLS (γ∗={Cov}(ΔR,ΔS)/{Var}(ΔS)\gamma^* = \text\{Cov\}(\Delta R, \Delta S)/\text\{Var\}(\Delta S)), yet the paper never reports the fitted γ∗\gamma^* values or compares against γ=1\gamma = 1 / normalized probabilities.
  2. The motivation of avoiding external models and reducing cost does not fully hold up. First, Appendix C.2 still relies on a frontier external model (GPT-5.6 Sol) to filter all 9,302 candidate pairs. Second, running O(n)O(n) prefix forward passes using the target agent itself plus iterative regeneration retries is computationally heavy. In practice, training a lightweight dedicated safety classifier (e.g., 1B-4B) for step-level localization would be much cheaper, reusable across backbones, and directly calibrated to separate benign sensitive actions from real attacks.
  3. The ablations in Fig. 4 do not cleanly isolate the localization score's contribution. External Localization performs almost identically to AgentFalcon (9.3% vs. 8.7% ASR on AgentLAB; 2.2% vs. 2.0% on ASB), suggesting that the gains mainly come from the step-level reflective DPO framework rather than the self-residualization score. Also, the SFT ablation trains on full trajectories rather than local segments, mixing supervision scope with training objective.
Questions for authors
  1. What are the fitted γ∗\gamma* values across the four backbones, and how does γ∗\gamma^* compare against fixing γ=1\gamma = 1 (log-odds) or binary softmax normalization over \{safe, unsafe\} in both localization F1 (Sec. 3.4) and downstream ASR?
  2. What is the total compute cost (tokens/wall-clock time) of prefix scoring and iterative regeneration, and how does the method perform without GPT-5.6 Sol pair filtering (p_filter)?
  3. How does localized SFT (or full-trajectory DPO) perform in Fig. 4 when holding supervision scope or training objective fixed?
10 comments
AuthorsOpenReview ↗

Response to Reviewer fkGp (1/7)

We sincerely thank the reviewer for the insightful and constructive review. We particularly appreciate your recognition of our risk-awareness analysis and empirical evaluation. Below, we address each point under Weaknesses and Questions, and we sincerely hope that our clarifications and additional analyses resolve your concerns. Unless otherwise stated, all experiments use Qwen3.5-9B as the agent backbone.

W1-1 & Q1. In Section 3.3, SiS_i and RiR_i are not renormalized over the safe and unsafe labels, whereas log-odds increments with γ=1\gamma = 1 or binary softmax remove shared probability-mass shifts directly. How does γ∗\gamma^{\ast} compare against fixing γ=1\gamma = 1 or binary softmax in localization F1 and downstream ASR?

R1-1: To verify whether retaining probability-mass information via fitted residualization improves over direct label normalization, we compare our score against fixing γ=1\gamma = 1 and binary softmax. While normalizing over the safe and unsafe labels removes shared shifts, it also discards total probability-mass changes that help distinguish benign sensitive steps from genuine risk emergence. Keeping all subsequent stages fixed, we evaluate our fitted residualization score ΔRi−γ∗ΔSi\Delta R_i - \gamma^{\ast}\Delta S_i, log-odds increments ΔRi−ΔSi\Delta R_i - \Delta S_i with γ=1\gamma = 1, and binary-softmax probability differences:

Table R1a. Localization performance by scoring method.

Scoring MethodPrecision ↑Recall ↑F1 ↑
ΔRi−γ∗ΔSi\Delta R_i - \gamma^{\ast}\Delta S_i63.297.176.6
ΔRi−ΔSi\Delta R_i - \Delta S_i52.598.068.3
Binary softmax54.697.770.0

Table R1b. Downstream performance by scoring method.

Scoring MethodAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
ΔRi−γ∗ΔSi\Delta R_i - \gamma^{\ast}\Delta S_i15.878.52.091.38.7
ΔRi−ΔSi\Delta R_i - \Delta S_i23.676.46.289.811.6
Binary softmax23.176.86.189.411.3

The results show that retaining probability-mass information via fitted residualization achieves substantially higher localization precision and stronger downstream safety than direct label normalization. Full normalization triggers more false alarms on benign steps, introducing noisy interventions that weaken alignment. We add this comparison to Appendix D.2 of our revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fkGp (2/7)

W1-2 & Q1. Theorem 1 and Appendix A derive standard univariate OLS for γ∗\gamma^{\ast}, yet the paper never reports the fitted values. What are the fitted γ∗\gamma^{\ast} values across the four backbones?

R1-2: We apologize for omitting the fitted γ∗\gamma^{\ast} values in our manuscript. Below, we report the OLS-fitted residualization coefficients γ∗\gamma^{\ast} and localization thresholds μ\mu across all four primary backbones:

Table R2. Calibration parameters across backbones.

Backboneγ∗\gamma^{\ast}μ\mu
Qwen3.5-4B0.7912.477
Qwen3.5-9B0.7532.546
Qwen3.6-35B-A3B0.7882.429
Ministral-3-14B-Instruct-25120.7262.684

Across all four backbones, the fitted γ∗\gamma^{\ast} values consistently fall between 0.726 and 0.791 rather than approaching 1, which aligns with R1-1 where fixing γ=1\gamma = 1 degrades localization precision and downstream safety. We report these parameter values in Appendix D.2 of our revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fkGp (3/7)

W2-1 & Q2. Appendix C.2 still relies on a frontier external model, GPT-5.6 Sol, to filter all 9,302 candidate pairs. How does the method perform without GPT-5.6 Sol pair filtering?

R2-1: To verify whether AgentFalcon relies on GPT-5.6 Sol for pair filtering, we start from the same 9,302 candidate pairs on Qwen3.5-9B and compare GPT-5.6 Sol filtering with self-filtering by Qwen3.5-9B itself, open-weight filtering with Qwen3.5-4B and Qwen3.6-35B-A3B, and no filtering:

Table R3. Downstream performance by filtering strategy.

FilterRetained PairsAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
GPT-5.6 Sol8,34415.878.52.091.38.7
Qwen3.5-9B self-filter8,13315.477.22.390.39.0
Qwen3.5-4B8,24116.277.12.490.88.6
Qwen3.6-35B-A3B8,52715.977.41.991.08.8
No filtering9,30218.378.95.591.513.8

The results show that AgentFalcon does not rely on GPT-5.6 Sol, as self-filtering by the acting model itself and open-weight filtering achieve comparable downstream safety and utility without any external model. Even without filtering, AgentFalcon retains strong performance, while pair filtering further removes unsuccessful reflections. We include the downstream results for these filtering strategies in Section 5.5 of our revised manuscript.

Reviewer_ipgbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AgentFalcon for safety alignment of tool-using LLM agents. Motivated by the observation that agents can recognize unsafe behavior when reviewing their own trajectories despite exhibiting such behavior during execution, the method scores sentence-level changes in the model’s log-probabilities of safe and unsafe judgments. It residualizes the unsafe signal against the safe signal, selects the earliest position exceeding an outlier threshold, and elicits reflection and a revised continuation. Repeated interventions produce localized preference pairs for DPO-based training, with external-model quality filtering. Experiments across four backbones and four safety benchmarks report substantial safety improvements, while evaluations on three general agent benchmarks suggest preserved utility on Qwen3.5-9B.

Critical strengths
  1. Substantial safety improvements across multiple backbones and threat settings. Table 1 reports reductions in AgentLAB attack success rates from 57.2, 53.3, 46.0, and 57.2 percent to 9.8, 8.7, 6.9, and 9.9 percent, respectively. The consistency and magnitude of these improvements make the complete pipeline practically relevant and provide meaningful evidence in favor of the work.

  2. Safety is evaluated alongside task utility. On Qwen3.5-9B, AgentDojo attack success decreases from 73.6 to 15.8 percent while benign task completion increases from 76.5 to 78.5 percent. Table 2 also reports no degradation in the main metrics across three general agent benchmarks. Although the smaller gains require uncertainty estimates, these evaluations help establish that the safety improvements are not explained solely by indiscriminate refusal.

Critical weaknesses
  1. The central localization mechanism is not compared with the most direct baseline suggested by the paper’s own premise. If the acting model can recognize problems in its trajectories, a natural alternative is to ask that same model to inspect a trajectory, identify the earliest point requiring correction, and then use exactly the same reflection, localized preference construction, filtering, and training procedure. This comparison is missing. Removing residualization only compares against raw unsafe-label increments; external localization changes the localizer to a stronger model. Neither establishes the advantage of the proposed probability-based mechanism over direct self-localization. Consequently, it remains unclear whether sentence-level label scoring, residualization, and thresholding improve localization accuracy or downstream safety at a matched budget. No complete runtime or data-construction cost comparison establishes an efficiency advantage either. This is consequential because the missing comparison concerns the incremental value of the paper’s central algorithmic component, rather than an additional peripheral benchmark.

  2. The existing ablations do not sufficiently isolate the benefits attributed to localized supervision and iterative intervention. The SFT variant changes both the learning objective and the supervision scope, replacing localized preference learning with full-trajectory SFT. Its weaker performance therefore does not isolate the claimed dilution of localized corrections. Likewise, the single-round comparison does not report whether the number of retained preference pairs, training tokens, or generation budget is matched. Appendix C.2 reports approximately 1.9 candidate pairs per relevant trajectory for the iterative pipeline, leaving additional data quantity as a plausible contributor to its advantage. Without these controls, the results establish the effectiveness of the combined recipe more clearly than the specific mechanisms used to explain its gains.

Questions for authors
  1. Have you evaluated direct self-localization, where the same acting model identifies the earliest risk sentence through prompting, while all subsequent components and budgets are held fixed? If so, how does it compare in localization precision/recall and downstream safety and utility? A focused comparison would materially clarify the contribution of residualized scoring.

  2. How many retained preference pairs and training tokens are used in the single-round and iterative variants? Are their generation budgets matched? Is there a local-SFT comparison using the same preferred segments, which would separate supervision scope from the optimization objective?

  3. What are the measured costs of prefix scoring, regeneration, and external preference filtering? Appendix C.2 uses GPT-5.6 Sol to filter candidate pairs; is there evidence showing how much of the final improvement persists without this external safety supervision? These details would clarify the practical benefit and scope of relying on the acting model’s own risk perception.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ipgb (1/5)

We sincerely thank the reviewer for the insightful and constructive review. We particularly appreciate your recognition of our safety improvements across multiple backbones and threat settings, as well as our evaluation of task utility alongside safety. Below, we address your concerns under Weaknesses and Questions. Unless otherwise stated, all experiments use Qwen3.5-9B as the agent backbone. ASR denotes attack success rate, and TCR denotes benign-task completion rate. All performance metrics in the tables are percentages.

W1-1 & Q1. The localization mechanism is not compared with Direct Localization, where the same acting model identifies the earliest risk sentence through prompting while all subsequent components are held fixed. How does it compare in localization precision and recall as well as downstream safety and utility?

R1-1: Thank you for this suggestion. To assess the contribution of AgentFalcon's probability-based residualized scoring, we construct a Direct Localization baseline by prompting Qwen3.5-9B to identify the earliest risk position in its own trajectories, while keeping subsequent reflection, localized preference construction, filtering, and training procedures unchanged, and compare localization and downstream performance:

Table R1a. Localization performance by localizer.

LocalizerPrecision ↑Recall ↑F1 ↑
AgentFalcon63.297.176.6
Direct Localization50.188.664.0
GPT-5.6 Sol localization65.092.276.2

Table R1b. Downstream performance by localizer.

MethodAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
AgentFalcon15.878.52.091.38.7
Direct Localization24.472.78.184.217.7
GPT-5.6 Sol localization16.973.62.289.99.3

The results show that AgentFalcon substantially improves localization precision and recall over Direct Localization and achieves localization F1 comparable to GPT-5.6 Sol. More accurate localization also enables AgentFalcon to achieve lower downstream ASR and higher benign-task TCR than Direct Localization. We include the downstream results of this comparison in Section 5.4 of our revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ipgb (2/5)

W1-2 & Q3-1. It remains unclear whether residualized scoring improves localization or safety at a matched budget, as no complete runtime or data-construction cost comparison is reported. What are the measured costs of prefix scoring, regeneration, and external preference filtering?

R1-2: To evaluate AgentFalcon's data-construction efficiency, we compare it with Direct Localization (DL), which prompts the same Qwen3.5-9B model to identify the earliest risk position while keeping subsequent procedures unchanged. Both methods run on eight RTX 4090 GPUs, with localization batch size 6 per GPU and vLLM regeneration concurrency 256. We report localization and regeneration times, as well as external filtering costs.

Table R2a. Localization and regeneration costs.

StageMethodInstancesMean Input TokensMean Output TokensMean Time (s)Stage Time (s) ↓
LocalizationAgentFalcon16,9525,001--0.631240
LocalizationDL18,5224,756<100.603268
RegenerationAgentFalcon9,3023,4541,1670.806334
RegenerationDL10,8723,2881,2490.819378

Table R2b. Combined runtime and filtering costs.

MethodLocalization + Regeneration Time (s) ↓Candidates Submitted for FilteringMean Filter Input TokensFilter API Cost (USD) ↓
AgentFalcon5749,3025,357209.66
DL64610,8725,524256.45

AgentFalcon uses parallel prefix scoring with a prefix-restricted attention mask to compute safe and unsafe probabilities across all sentence boundaries in a single forward pass. As a result, AgentFalcon completes localization and regeneration in under 10 minutes, with lower combined wall-clock time and filtering API cost than DL. We report the localization and regeneration costs in Section 5.5 and describe parallel prefix scoring in Appendix D.1 of our revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ipgb (3/5)

W2-1 & Q2. In the ablations, the SFT variant replaces localized preference learning with full-trajectory SFT, changing both learning objective and supervision scope. Is there a local-SFT comparison using the same preferred segments?

R2-1: Thank you for this suggestion. To disentangle the effects of supervision scope and training objective, we evaluate two controlled variants that change one factor at a time: Localized SFT on the exact same preferred local segments used by Localized DPO, and Global DPO on full-trajectory preference pairs:

Table R3. Supervision scope and training objective ablations.

MethodAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
Localized DPO15.878.52.091.38.7
Localized SFT14.665.62.282.78.3
Global DPO20.478.17.690.613.2

The results show that localized supervision and contrastive DPO optimization jointly provide a better balance between safety and benign-task utility. Localized SFT slightly lowers ASR on AgentDojo and AgentLAB but substantially reduces benign-task TCR, consistent with overly conservative behavior. Global DPO preserves similar TCR but increases ASR across all three benchmarks, supporting the benefit of concentrating preference supervision at risk positions. We add these ablations to Section 5.4 of our revised manuscript.

Reviewer_jWGnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies safety alignment for tool-using LLM agents, focusing on the problem of identifying where risk first emerges within a long execution trajectory. The key observation is that an agent may execute unsafe actions while still being able to recognize the resulting trajectory as unsafe when asked to evaluate it afterward. Building on this gap between risk perception and action selection, the paper proposes AgentFalcon, which uses the acting model’s own safety judgments to localize risk-bearing sentences. Specifically, it measures how each sentence changes the model’s evidence for “unsafe” and “safe,” then residualizes unsafe information gain against safe information gain to reduce false positives from sensitive but legitimate operations. Experiments cover four model backbones and several safety benchmarks and reported results are strong, for example, on Qwen3.5-9B, AgentFalcon reduces AgentLAB attack success rate, while improving or preserving benign performance.

Critical strengths

The strongest aspect of the paper is the formulation of self-aware risk localization. Rather than using an expensive external model to inspect every step, AgentFalcon extracts a training signal from the acting model’s own latent safety awareness. Another major strength is the breadth and consistency of the empirical evaluation. The method is tested across four backbones of different sizes and architectures, multiple safety benchmarks, and several benign agent-capability benchmarks. The improvements are large on the main safety evaluations, while benign task performance is generally preserved and sometimes improved. The component-level ablations are also informative, removing residualization, reflection, iterative intervention, or localized preference optimization consistently worsens performance. This makes the empirical case substantially stronger than an evaluation limited to a single model or attack setting.

Critical weaknesses

The main weakness is that risk localization depends on the target model already recognizing unsafe behavior. If the model does not understand that a trajectory is risky, AgentFalcon may fail to identify and correct the relevant step. This could be particularly problematic for weaker models, subtle safety failures, or unfamiliar attack strategies. The paper would benefit from analyzing how localization and downstream safety performance change as the model’s initial risk-awareness accuracy decreases. Another weakness is that AgentFalcon estimates a single shared residualization coefficient and localization threshold by pooling sentence-level scores across the training tasks, and then keeps both fixed during iterative intervention. This assumes that a common linear correction and threshold are suitable across different tasks, tools, attack types, and trajectory stages. The paper does not analyze the sensitivity of localization to these global parameters or compare them with task, category, or stage-specific alternatives.

Questions for authors

Since AgentFalcon depends on the target model’s own safety and risk signals, how does its localization accuracy change for backbones with weaker initial risk awareness?

AgentFalcon estimates one shared residualization coefficient and localization threshold from pooled training-task scores and keeps them fixed throughout subsequent intervention. How sensitive are the results to these two parameters?

4 comments
AuthorsOpenReview ↗

Response to Reviewer jWGn (1/2)

We sincerely thank the reviewer for the insightful and constructive review. We particularly appreciate your recognition of our risk-awareness analysis and empirical evaluation. Below, we address each point under Weaknesses and Questions, and we sincerely hope that our clarifications and additional analyses resolve your concerns. Unless otherwise stated, all experiments use Qwen3.5-9B as the agent backbone.

W1 & Q1: Risk localization depends on the target model already recognizing unsafe behavior. How do localization accuracy and downstream safety performance change as the backbone's initial risk awareness decreases?

To investigate how AgentFalcon's localization accuracy, downstream safety, and general-task performance vary with the accuracy of the model's initial risk perception, we include Qwen3-8B, a model with weaker initial risk perception, alongside the four backbones and compare their localization accuracy and downstream safety performance:

Table R1a. Risk localization across backbones.

ModelPrecision ↑Recall ↑F1 ↑
Qwen3.5-4B60.897.975.0
Qwen3.5-9B63.297.176.6
Qwen3.6-35B-A3B61.495.374.7
Ministral-3-14B-Instruct-251256.496.171.0
Qwen3-8B55.192.169.0

Table R1b. Downstream performance across backbones, AF denotes alignment with AgentFalcon and Base denotes the original backbone.

ModelMethodAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
Qwen3.5-4BAF16.179.20.386.89.8
Qwen3.5-9BAF15.878.52.091.38.7
Qwen3.6-35B-A3BAF15.880.80.288.96.9
Ministral-3-14B-Instruct-2512AF18.772.11.562.79.9
Qwen3-8BBase74.864.713.164.761.2
Qwen3-8BAF28.164.96.267.318.7

The results show that, even for Qwen3-8B, which has weaker initial risk perception and lower localization accuracy, AgentFalcon substantially improves downstream safety and general-task performance. However, we also observe that the magnitude of these improvements on Qwen3-8B is slightly smaller than on the other models with stronger initial risk perception, indicating a clear positive correlation between initial risk perception and downstream safety performance. We report this cross-backbone comparison in Appendix D.3 of our revised manuscript.

W2-1 & Q2: AgentFalcon estimates a single shared residualization coefficient and localization threshold from pooled training-task scores and keeps them fixed during iterative intervention. How sensitive are localization and downstream results to these two global parameters?

To assess sensitivity to the residualization coefficient and localization threshold, we perturb each parameter separately and compare downstream safety performance:

AuthorsOpenReview ↗

Response to Reviewer jWGn (2/2)

Table R2. Default calibration parameters. kappa = 1.5 for every backbone.

Backboneγ\gammaμ\mu
Qwen3.5-4B0.7912.477
Qwen3.5-9B0.7532.546
Qwen3.6-35B-A3B0.7882.429
Ministral-3-14B-Instruct-25120.7262.684

Table R3. Complete sensitivity results for gamma, with kappa = 1.5 and mu recalculated. Performance metrics are percentages.

γ\gammaμ\muPrecision ↑Recall ↑F1 ↑AgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
0.904 (+20%)2.48955.497.670.722.677.35.689.612.8
0.866 (+15%)2.52557.097.471.920.877.64.190.49.3
0.828 (+10%)2.56759.497.373.819.278.33.090.89.0
0.790 (+5%)2.53261.597.175.316.178.72.191.18.7
0.753 (ours)2.54663.297.176.615.878.52.091.38.7
0.715 (-5%)2.56063.396.076.315.278.32.290.88.8
0.678 (-10%)2.62060.895.774.416.477.43.190.59.1
0.640 (-15%)2.64457.594.371.419.476.94.690.09.7
0.602 (-20%)2.68454.492.968.621.977.15.789.611.2

Table R4. Complete threshold-sensitivity results, with gamma = 0.753 fixed. Performance metrics are percentages.

κ\kappaμ\muPrecision ↑Recall ↑F1 ↑AgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
1.302.29558.597.873.218.177.24.090.49.5
1.352.35859.797.774.116.777.93.490.79.1
1.402.42161.297.475.215.978.32.891.28.9
1.452.48362.197.275.815.678.62.190.98.7
1.50 (ours)2.54663.297.176.615.878.52.091.38.7
1.552.60963.795.776.515.578.62.191.08.8
1.602.67164.293.676.216.478.22.291.48.8
1.652.73464.491.875.717.877.53.590.69.5
1.702.79764.589.274.919.877.14.189.69.5

The results show that AgentFalcon maintains comparable downstream safety and general-task performance when the residualization coefficient gamma is perturbed by approximately ±5% and the filtering threshold kappa ranges from 1.45 to 1.55. Although performance declines substantially as the perturbations increase, the residualization coefficient and filtering threshold exhibit considerable stability overall. In addition, estimating parameters by attack type or stage does not yield consistent gains over global estimation. Considering the trade-off between efficiency and performance, we therefore adopt globally estimated parameters. We describe the parameter sensitivity analysis and alternative estimation strategies in detail in Appendix D.2 of our revised manuscript.

W2-2: Using a fixed global linear correction and threshold across different tasks, attack types, and trajectory stages is not compared against category- or stage-specific alternatives.

To investigate the effect of different parameter estimation strategies, we add two variants: By attack type estimates a separate residualization coefficient and filtering threshold for each of the six task categories; By stage estimates these parameters from the trajectories undergoing regeneration at the current stage and applies them to localization in the next stage.

Table R5. Global, attack-type-specific, and stage-wise calibration.

MethodAgentDojo ASR ↓AgentDojo TCR ↑ASB ASR ↓ASB TCR ↑AgentLAB ASR ↓
Global (ours)15.878.52.091.38.7
By attack type15.179.32.488.48.9
By stage15.677.62.390.79.2

As shown in Table R5, neither variant yields consistent gains over global estimation. We therefore adopt globally estimated parameters to balance efficiency and performance.

AuthorsOpenReview ↗

Follow-up on Additional Experiments and Clarifications

Dear Reviewer,

We would like to follow up on our submitted response. We have provided additional experiments and further explanations to address your questions and concerns, and have incorporated these additions and clarifications into the revised manuscript. The experimental results are consistent with and further support our conclusions.

We hope you will have a chance to review our response and share your feedback on the questions we raised. Your feedback would help us better understand whether our clarifications and additional experiments have adequately addressed your concerns.

Thank you for your time and consideration.

I thank the authors for the detailed response and additional experiments. The weaker backbone analysis, parameter sensitivity studies, and comparison with attack type and stage specific calibration directly address my main concerns. While the behavior under extremely weak risk awareness remains an open question, the rebuttal substantially strengthens the paper. I will retain my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Review and Rebuttal Summary

We sincerely thank all reviewers for their time and valuable feedback. We are encouraged that all four reviewers rated the paper’s presentation and organization as good or excellent. Most reviewers emphasized the importance of analyzing the data construction costs of AgentFalcon’s self-reflective localization approach, reporting parameter values and examining their estimation strategies and sensitivity, and conducting controlled ablations to disentangle the training objective from the supervision scope. We also appreciate the reviewers’ positive comments on the paper’s clarity and structure.

In addition to our responses to the reviewers, we revised the manuscript during the rebuttal period with the following changes, marked in blue:

  1. We added Localized SFT and Global DPO ablations to disentangle the effects of the training objective and supervision scope. We also added comparisons with Direct Localization, which directly prompts the acting model to identify risk positions. (Section 5.4)

  2. We provided implementation details of AgentFalcon’s parallel prefix scoring and compared its data construction costs with Direct Localization across three stages: identifying the earliest critical risk position, regeneration, and filtering. (Section 5.4 and Appendix D.1)

  3. We conducted detailed sensitivity analyses of the residualization coefficient and filtering threshold, together with additional experiments comparing attack-type-specific and stage-specific parameter estimation strategies. (Appendix D.2)

  4. We strengthened the discussion of how AgentFalcon’s downstream performance varies with the model’s initial risk awareness and added experiments on Qwen3-8B. (Appendix D.3)

  5. We added experiments on risk localization using a lightweight model. We also compared downstream performance under preference filtering with open-source models, no filtering, and filtering with an external model. (Section 5.4 and Appendix D.4)

We look forward to further discussions with the reviewers!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.