acceptodds
Under review as a conference paper at ICLR 2027

Steer Before You Act: Robust Thought-Guided Steering for Agent Behavioral Safety

Abstract

Large language model (LLM) agents perform long-horizon, complex tasks by reasoning iteratively and interacting with environments through external tool calls. Even when responding to legitimate user requests, agents may compromise behavioral safety through unauthorized or inappropriate tool calls that expose private information, disrupt services, or cause financial loss. Existing defenses often assess generated actions using auxiliary guard models, adding inference costs from verification and, when needed, regeneration. We introduce an activation-steering framework that steers the agent's internal thought representations before tool-call generation, without retraining the underlying language model or relying on auxiliary guard models. To improve steering reliability across tool-use scenarios, we propose Safety Preference Stability, which favors steering directions with consistent safety utility under direction and construction-data perturbations. We evaluate our method on Qwen3 models (4B–14B) in ToolEmu, an environment for evaluating agents on high-risk tool-use tasks. Our framework achieves up to and the unprotected agent's safety rate in in-distribution and evolving scenarios, respectively, while providing – speedups per invocation over external defenses. Moreover, our framework improves legitimate-goal completion by up to 30.4% over the strongest external guard baseline.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Safety, Representation Engineering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_omEhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies behavioral safety of tool-using agents under legitimate user requests. A linear probe predicts decision risk from hidden states before the next Thought is generated. A steering direction is constructed from differences between Thought representations of risky proposals and teacher-generated risk-aware proposals under matched contexts. During deployment, steering is applied to the subsequent Thought–Action generation when the predicted risk meets or exceeds a threshold, while the acting model remains frozen. The main additional design is Safety Preference Stability (SPS), which selects a fixed intervention layer and direction using worst-case utility over a finite set of local direction perturbations, lower-tail utility under case-level bootstrap resampling, and utility dispersion. Experiments evaluate three Qwen3 model sizes on in-distribution ToolEmu cases and newly constructed evolving scenarios, reporting safety, helpfulness, legitimate-goal completion, and per-invocation overhead.

Critical strengths

The paper addresses a practically important problem: harmful tool use while carrying out legitimate requests. Its emphasis on preserving safe task progress is valuable, and evaluating legitimate-goal completion against fixed, method-independent references helps reveal the utility costs of excessive refusal. The deployment mechanism is lightweight, requiring a risk probe and a fixed steering intervention without additional calls to a large guard model. The controlled fixed-context measurements support lower per-invocation overhead than the evaluated external defenses. The case-level bootstrap design also preserves the grouping of dependent decisions from the same case during resampling.

Critical weaknesses
  1. The incremental contribution of SPS is not sufficiently isolated by the experiments. The mean activation-difference construction and additive intervention follow CAA (Rimsky et al., 2024). Conditional activation steering has been studied in CAST (Lee et al., 2025), and AgentLens (Luo et al., 2026) investigates risk detection and representation-level mitigation for multi-turn agents. I therefore view the main methodological contribution as the risk-aware paired construction and SPS-based selection criterion. SPS is reasonably motivated, but the evidence for its benefit over simpler selection criteria remains limited. Appendix C reports component ablations on Qwen3-8B, with partial Spearman correlations of 0.704 for the full score and 0.219, 0.139, and 0.602 after removing individual components. These results support the usefulness of the components for candidate ranking. However, the measured outcome is reference-tool alignment, which the paper explicitly distinguishes from argument-level safety and task completion. The paper does not clearly report a matched comparison between SPS and selection by nominal utility U(κ)U(\kappa) alone, using identical paired data, gating, intervention strength, and candidate layers. Nor does it report the SPS component ablations on final safety and completion metrics. Although Table 1 provides end-to-end results in evolving scenarios, these results do not isolate the contribution of SPS from the other framework components. Consequently, the practical benefit of the proposed stability criterion over simpler alternatives remains insufficiently established.

  2. The reported percentages require clarification of the evaluation and aggregation protocol. Appendix B.1 states that the in-distribution test set contains 108 cases, while Appendix B.6 defines safety and helpfulness as binary pass rates over trajectories with valid scores. However, the Qwen3-4B safety rates in Table 1—52.89% for GuardAgent, 39.67% for CAA, and 61.98% for Thought-Aligner—cannot be obtained by rounding an integer pass count divided by an integer denominator of at most 108 to two decimal places.

Questions for authors
  1. Please clarify how the percentages in Table 1 were computed, including the number of rollouts per case, any averaging across runs, and the pass counts, valid trajectory counts, and excluded evaluations for each method and metric. In particular, how do the 108 in-distribution cases yield the Qwen3-4B safety rates of 52.89%, 39.67%, and 61.98%?

  2. With paired data, gating, intervention strength, and candidate layers held fixed, how does full SPS compare with selection by nominal utility U(κ)U(\kappa) alone on final safety and legitimate-goal completion, particularly in evolving scenarios? Please provide these outcomes for the component ablations, together with uncertainty estimates. To assess the specific contribution of Thought-based construction, is there a controlled comparison with directions extracted from Action or the full proposal?

3 comments
AuthorsOpenReview ↗

Response to Reviewer omEh (Part I)

We thank the reviewer for their careful audit of our evaluation. In the response below, we provide detailed counts for Table 1 (Part I) and compare Safety Preference Stability (SPS) with nominal utility, component ablations, and alternative extraction spans (Part II).

W2 & Q1(a). Rollouts, valid scores, and excluded evaluations

Please clarify how the percentages in Table 1 were computed, including the number of rollouts per case, any averaging across runs, and the pass counts, valid trajectory counts, and excluded evaluations for each method and metric.

For each metric (safety or helpfulness), the valid-score pass rate is the fraction of scored trajectories with a ToolEmu score of 2 or 3 (Appendix B.6). A malformed response, a timeout or an unknown judgment can leave a metric unscored. For example, GuardAgent passes safety on 57/106 scored tasks (53.77%), or 57/108 designated tasks (52.78%) when missing scores count as failures.

Each case contributes one trajectory per method. For Qwen3-4B, the table gives safety and helpfulness over valid scores and over all 108 in-distribution (ID) tasks (missing counts as failure). Legitimate-goal completion (LGC) applies to 99 tasks with at least one legitimate goal. We average each task’s completed-goal fraction over scored tasks, then over all 99 with missing scores set to zero.

MethodSafety pass/fail/missingSafety % (valid scores)Safety % (all 108 tasks)Helpfulness pass/fail/missingHelpfulness % (valid scores)Helpfulness % (all 108 tasks)LGC scored/missing (of 99 eligible)Mean LGC % (scored tasks)Mean LGC % (all 99 eligible)
Base Agent29 / 79 / 026.85 (29/108)26.8520 / 88 / 018.52 (20/108)18.5299 / 041.7541.75
GuardAgent57 / 49 / 253.77 (57/106)52.7828 / 78 / 226.42 (28/106)25.9397 / 247.7746.80
ShieldAgent54 / 51 / 351.43 (54/105)50.0030 / 75 / 328.57 (30/105)27.7899 / 05.055.05
Thought-Aligner67 / 40 / 162.62 (67/107)62.0437 / 70 / 134.58 (37/107)34.2698 / 157.8257.24
Contrastive Activation Addition (CAA)43 / 65 / 039.81 (43/108)39.8130 / 78 / 027.78 (30/108)27.7899 / 0——
k-nearest neighbors (KNN), hard45 / 63 / 041.67 (45/108)41.6744 / 64 / 040.74 (44/108)40.7499 / 0——
KNN (Soft)47 / 61 / 043.52 (47/108)43.5240 / 68 / 037.04 (40/108)37.0499 / 051.6851.68
SPS (Ours)75 / 33 / 069.44 (75/108)69.4434 / 74 / 031.48 (34/108)31.4899 / 064.3164.31

With missing scores counted as zero, SPS leads Thought-Aligner by 7.41 safety points on the same 108 ID tasks (75 vs. 67 passes) and 7.07 LGC points on the same 99 eligible tasks (64.31% vs. 57.24%).

For SPS at 8B and 14B, the ID safety counts are 67/108 and 78/108, and helpfulness counts are 63/108 and 71/108, respectively. The complete method-by-metric tally here covers 4B. We will add the per-metric counts and denominator conventions to Appendix B.6, define a safety pass in the Table 1 caption, and report a follow-up comparison averaging repeated rollouts within each case.

Q1(b). Why the three published Qwen3-4B rates used 121 cases

In particular, how do the 108 in-distribution cases yield the Qwen3-4B safety rates of 52.89%, 39.67%, and 61.98%?

We designated 108 of the 144 ToolEmu cases for ID evaluation and 36 for offline development. We also ran 13 offline calibration cases to check that we could reliably extract safety and helpfulness scores from the evaluator outputs. The combined log contained 121 records. On the 108 designated test cases, the safety pass rates among valid scores were 57/106 (53.77%) for GuardAgent and 67/107 (62.62%) for Thought-Aligner.

The published 4B safety rates used 64/121 for GuardAgent, 48/121 for CAA and 75/121 for Thought-Aligner. For CAA, the corrected safety rate is 43/108 (39.81%). Qwen3-8B SPS helpfulness changes from 71/121 (58.68%) to 63/108 (58.33%).

We apologize for these reporting errors. We will correct the affected Table 1 entries and their reported gains and document the test-ID filter in Appendix B.1.

AuthorsOpenReview ↗

Response to Reviewer omEh (Part II)

W1 & Q2(a). Full SPS versus nominal utility

With paired data, gating, intervention strength, and candidate layers held fixed, how does full SPS compare with selection by nominal utility (U(\kappa)) alone on final safety and legitimate-goal completion, particularly in evolving scenarios?

To isolate the selector, we held the paired proposals, candidate layers, probe, gate (γ=0.50\gamma=0.50), steering strength (α=0.8\alpha=0.8) and decoding fixed on Qwen3-8B. CAA is an ungated baseline. Safety covers 108 ID and 72 Evolving tasks. LGC covers the 99 and 66 tasks with legitimate goals.

SelectorID safety / LGC (%)Evolving safety / LGC (%)Evolving vs. nominal (safety / LGC, points)
CAA (un-gated baseline)60.19 / —33.33 / 49.24-20.83 / -10.61
Nominal utility only56.48 / 69.0254.17 / 59.850.00 / 0.00
Full SPS (R_dir + R_tail + R_stab)62.04 / 73.9169.44 / 68.18+15.28 / +8.33

Compared with nominal utility, full SPS gains 5.56 safety points and 4.88 LGC points in ID, and 15.28 and 8.33 points in Evolving.

On Evolving, nominal vs. full SPS safety / LGC (%) was 50.00 / 53.03 vs. 63.89 / 56.82 on Qwen3-4B, and 59.72 / 52.27 vs. 75.00 / 59.09 on Qwen3-14B. The safety gains were 13.89 and 15.28 points, respectively.

We will add the matched nominal-utility comparison to Section 3.3 and report its ID and Evolving safety and LGC results in Appendix C.

Q2(b). Contribution of each SPS component

Please provide these outcomes for the component ablations, together with uncertainty estimates.

Qwen3-8B selectorID safety / LGC (%)Evolving safety / LGC (%)Evolving difference vs. full SPS (points)Safety standard error (SE), ID / Evolving (points)
Full SPS62.04 / 73.9169.44 / 68.180.00 / 0.004.67 / 5.43
Without directional term58.33 / 70.8859.72 / 62.88−9.72 / −5.304.74 / 5.78
Without lower-tail term57.41 / 69.7058.33 / 61.36−11.11 / −6.824.76 / 5.81
Without dispersion term61.11 / 72.7366.67 / 66.67−2.78 / −1.524.69 / 5.56

Removing the lower-tail term produces the largest observed Evolving safety drop (eight fewer passes), followed by the directional term (seven) and dispersion (two). The last column gives binomial standard errors for individual safety rates, 100p(1−p)/n100\sqrt{p(1-p)/n}, treating tasks as independent draws. Paired uncertainty for the differences and for LGC requires the per-task outcomes.

In Appendix C, we will add a rollout table alongside Table 4 showing how omitting each SPS term changes safety and LGC on both test sets.

Q2(c). Direction extraction span

To assess the specific contribution of Thought-based construction, is there a controlled comparison with directions extracted from Action or the full proposal?

On Qwen3-8B we varied only the offline span in Eq. 5. The online intervention in Eq. 8 still covered the subsequent Thought and Action tokens whenever the gate opened (gt=1g_t=1).

Offline extraction spanSelected layerID safety / LGC (%)Evolving safety / LGC (%)Evolving difference vs. Thought (points)
Thought only (Ours)2862.04 / 73.9169.44 / 68.18Baseline
Action only (tool name + arguments in JavaScript Object Notation (JSON))2652.78 / —44.44 / 52.27-25.00 / -15.91
Full proposal (Thought + Action)2858.33 / 69.7061.11 / 62.88-8.33 / -5.30

On unseen toolkits, Action alone scores 25.00 safety points below Thought alone. Pooling both spans narrows the gap to 8.33 points. We observed malformed arguments with Action-only directions. A risky action can call for the same precaution across toolkits even when names and argument formats change. A direction estimated from Thought may capture that shared reason, whereas including Action tokens can make the direction more sensitive to toolkit-specific syntax.

We will distinguish the extraction and intervention spans in Section 2.3 and add the three-span comparison to Appendix C, including the selected layer and rollout outcomes for each span.

Dear Reviewer,

Thank you again for your thoughtful comments. In the revised manuscript, we expanded the reporting of evaluation counts and missing-score handling in Appendix C (Tables 6–7), including pass/fail/missing counts for Qwen3-4B and SPS pass counts for Qwen3-8B and 14B. We also added matched end-to-end comparisons between SPS and nominal-utility selection, with component ablations, in Section 3.4 (Table 2) for Qwen3-8B and Appendix E.1 (Table 13) for Qwen3-4B and 14B. Finally, Appendix D.3 (Table 10) compares Thought-only, Action-only, and Thought+Action extraction with the online intervention span held fixed. We would appreciate your feedback on whether these additions address your main concerns, and we would be happy to clarify any remaining questions.

Best regards, The authors

Reviewer_sNU4OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a thought-guided agent safety steering framework for LLM Agent behavior safety. By applying guidance vectors to the internal thinking representation of the model before tool calls are generated, reasoning is directed to risk-aware action proposals, thereby avoiding unauthorized or harmful tool calls. The author further proposes the SPS criterion to robustly select the guidance layer and direction through the utility stability under direction perturbation and structural data perturbation. The experiment was conducted on Qwen3 (4B–14B) and ToolEmu environments, demonstrating the security improvement and reasoning efficiency advantages within distribution and evolution scenarios.

Critical strengths
  1. This approach intervenes via activation-based guidance before the "Thought" process is generated. It eliminates the need for auxiliary models during online inference, incurring an overhead of only 12.4%–16.4%—offering a distinct advantage in balancing efficiency and safety.
  2. The study formalizes Preference Stability into a quantifiable utility metric, evaluating the robustness of candidate directions through two complementary dimensions: Directional Safety Stability and Construction Safety Stability. This selection criterion goes beyond simple average safety gains by focusing on failure risks in individual cases and under distribution shifts—a well-motivated design.
  3. Beyond evaluating standard in-distribution scenarios, the authors construct Evolving Scenarios, incorporating unseen toolkits and novel risk settings to test the method’s cross-scenario transferability. The selective abstention cases (e.g., in smart home control) intuitively demonstrate the model’s ability to reject hazardous operations while still executing safe tasks, thereby avoiding the blanket, global rejections characteristic of approaches like ShieldAgent.
Critical weaknesses
  1. All experiments were conducted exclusively within ToolEmu. Since environmental feedback is simulated by a language model, the potential risks and consequences do not manifest in reality. The absence of validation using actual tool APIs casts doubt on the practical significance of the reported safety metrics.
  2. The study is limited to a single model family, lacking validation for cross-architecture generalization. All experiments were performed solely using the Qwen3 series, excluding other architectures such as LLaMA and Gemma. The effectiveness of activation steering relies heavily on the linear separability of internal model representations—a property that may vary significantly across different architectures.
  3. Ablation studies were restricted to the Qwen3-8B model, and only the Spearman correlation coefficient with "reference-tool alignment" was reported. The analysis lacks: (a) ablation studies for the 4B and 14B variants; (b) an assessment of each component's contribution to the final safety and helpfulness metrics; and (c) comparisons against simple baselines (e.g., selecting the direction with the highest average utility).
  4. There is a lack of ablation or sensitivity analysis regarding the choice of the teacher model. While DeepSeek-V4-Flash is used as the teacher model for counterfactual generation, the paper fails to analyze: (a) how the capabilities of the teacher model influence the quality of the steering directions; (b) whether the results would change if a different teacher model were used; and (c) how the quality of the risk-aware proposals generated by the teacher model is ensured.
  5. The study lacks experimental comparisons with recent methods such as AgentLens. Although the paper identifies AgentLens in the Related Work section as directly relevant to agentic safety steering, it does not include it as a baseline in the experiments. Given that both approaches focus on activation steering in agentic contexts, the absence of a direct comparison makes it difficult to evaluate the relative advantages of the proposed method.
Questions for authors
  1. Is it possible to reproduce the main results on at least one non-Qwen architecture to verify the cross-architecture generalizability of the method?
  2. How do the quality of the guidance direction and the final safety rate change when the teacher model is replaced with models of different capability levels (e.g., smaller or larger models)?
4 comments
AuthorsOpenReview ↗

Response to Reviewer sNU4 (Part I)

We thank the reviewer for their thoughtful assessment. In the response below, we report cross-family and teacher results (Part I), multi-scale ablations and the AgentLens comparison (Part II), and executable-tool results (Part III).

Q1 & W2. Transfer to Llama and Gemma

Is it possible to reproduce the main results on at least one non-Qwen architecture to verify the cross-architecture generalizability of the method?

We used the same 36 offline ToolEmu cases, three folds, α=0.8\alpha=0.8, and γ=0.50\gamma=0.50 on Llama-3.1-8B-Instruct. Safety Preference Stability (SPS) selected layer 19 of 32. The area under the receiver operating characteristic curve (AUC) was 0.746 for the last prefix token, 0.582 for the prefix mean and 0.854 for the Thought mean. Only the prefix is available before the next Thought. The tables report in-distribution (ID) and Evolving results, including legitimate-goal completion (LGC).

Llama-3.1-8B-InstructID safety / helpfulness / LGC (%)Evolving safety / helpfulness / LGC (%)Evolving difference vs. nominal: safety / LGC (points)
Base Agent42.59 / 40.74 / —27.78 / 13.89 / 50.76-23.61 / -7.58
Contrastive Activation Addition (CAA) (2024)51.85 / 44.44 / 56.0631.94 / 15.28 / 49.24-19.44 / -9.09
GuardAgent (2025)57.41 / 48.15 / —50.00 / 31.94 / 48.48-1.39 / -9.85
Nominal utility only58.33 / 51.85 / —51.39 / 30.56 / 58.330.00 / 0.00
Full SPS (Ours)65.74 / 59.26 / 71.2165.28 / 38.89 / 65.91+13.89 / +7.58
Gemma-2-9B-ItID safety / helpfulness / LGC (%)Evolving safety / helpfulness / LGC (%)
Base Agent50.93 / 44.44 / 53.0331.94 / 16.67 / 46.97
CAA (2024)58.33 / 31.48 / 46.4634.72 / 12.50 / 43.94
Nominal utility only62.96 / 51.85 / 59.6058.33 / 34.72 / 53.03
Full SPS (layer 27)68.52 / 59.26 / 66.6770.83 / 43.06 / 59.85

A dash denotes an LGC mean incompatible with the scoring grid. On Llama, SPS raises safety over the base actor by 23.15 points in ID and 37.50 in Evolving. LGC applies to tasks with at least one legitimate goal (99 of 108 ID tasks and 66 of 72 Evolving tasks). Compared with nominal utility on Evolving, SPS gains 13.89 safety points and 7.58 LGC points on Llama, and 12.50 and 6.82 points on Gemma.

We will add the Llama probe results to Section 3.2 and report the Llama and Gemma rollout comparisons in Section 3.3.

Q2 & W4(a–b). Teacher capability, direction quality, and rollout safety

How do the quality of the guidance direction and the final safety rate change when the teacher is replaced with models of different capability levels (e.g., smaller or larger models)?

We varied only the teacher, keeping the Qwen3-8B actor, prefixes, pair count and SPS procedure fixed.

TeacherDirection cosine vs. V4-ProActor ID safety / LGC (%)Actor Evolving safety / LGC (%)
Qwen3-4B (smaller)0.74254.63 / 63.1351.39 / 55.30
Qwen3-8B (self-teacher)0.83958.33 / 67.6859.72 / 61.36
DeepSeek-V4-Flash (default)0.95862.04 / 73.9169.44 / 68.18
DeepSeek-V4-Pro (frontier)1.00063.89 / 75.25Unreported

Direction quality. V4-Flash is closest to the V4-Pro direction (cosine 0.958). We will report direction similarities in Appendix B.5.

Rollout safety and completion. Evolving safety reaches 51.39% (4B teacher) and 59.72% (self-teacher), versus 30.56% for Base and 33.33% for CAA. Relative to V4-Flash, these teachers refuse unnecessarily more often (19.8% and 13.6% vs. 4.9%), and Evolving LGC falls by 12.88 and 6.82 points. On ID, V4-Pro exceeds V4-Flash by 1.85 safety points and 1.35 LGC points. We will add the teacher comparison to Appendix B.5 and summarize the rollout results in Section 3.4.

W4(c). Quality checks for teacher proposals

(c) how the quality of the risk-aware proposals generated by the teacher model is ensured.

We first filtered for a valid single-step ReAct response: a Thought followed by an Action with a legal tool name and arguments in valid JavaScript Object Notation (JSON), or a Final Answer. A blind audit of retained pairs at 81 risky steps then assessed whether the proposal resolved the risk, preserved safe progress, or refused unnecessarily when a safe read-only action or clarification was available.

TeacherRisk resolved (%)Safe progress (%)Unnecessary refusal (%)
Qwen3-4B (smaller)67.961.719.8
Qwen3-8B (self-teacher)77.872.813.6
DeepSeek-V4-Flash (default)92.687.74.9
DeepSeek-V4-Pro (frontier)97.593.81.2

We will document the schema filter and the three blind-audit criteria in Appendix B.5, alongside each teacher’s audit results.

AuthorsOpenReview ↗

Response to Reviewer sNU4 (Part II)

W3. Component ablations across model sizes

Ablation studies were restricted to the Qwen3-8B model, and only the Spearman correlation coefficient with "reference-tool alignment" was reported. The analysis lacks: (a) ablation studies for the 4B and 14B variants; (b) an assessment of each component's contribution to the final safety and helpfulness metrics; and (c) comparisons against simple baselines (e.g., selecting the direction with the highest average utility).

(a) Actor sizes. We therefore compared full SPS, each single-term ablation, and the selector that maximizes nominal utility on Qwen3-4B, 8B, and 14B. Each cell gives safety / helpfulness / LGC (%) for ID first and Evolving second. Safety and helpfulness cover 108 ID and 72 Evolving tasks. LGC applies to the 99 ID and 66 Evolving tasks with at least one legitimate goal. We will expand Appendix C to cover all three model sizes.

SelectorQwen3-4B: ID; EvolvingQwen3-8B: ID; EvolvingQwen3-14B: ID; Evolving
Nominal utility only62.96 / 27.78 / 59.93 ; 50.00 / 31.94 / 53.0356.48 / 53.70 / 69.02 ; 54.17 / 27.78 / 59.8565.74 / 59.26 / 71.21 ; 59.72 / 44.44 / 52.27
SPS w/o R_dir64.81 / 29.63 / — ; 55.56 / 36.11 / 54.5558.33 / 55.56 / 70.88 ; 59.72 / 31.94 / 62.8868.52 / 61.11 / — ; 66.67 / 50.00 / 55.30
SPS w/o R_tail63.89 / 28.70 / 60.61 ; 54.17 / 34.72 / 53.7957.41 / 54.63 / 69.70 ; 58.33 / 30.56 / 61.3667.59 / 60.19 / 72.73 ; 65.28 / 48.61 / 54.55
SPS w/o R_stab68.52 / 30.56 / 63.64 ; 61.11 / 40.28 / 56.0661.11 / 57.41 / 72.73 ; 66.67 / 34.72 / 66.6771.30 / 63.89 / — ; 72.22 / 54.17 / 57.58
Full SPS (Ours)69.44 / 31.48 / 64.31 ; 63.89 / 41.67 / 56.8262.04 / 58.33 / 73.91 ; 69.44 / 36.11 / 68.1872.22 / 65.74 / 75.93 ; 75.00 / 55.56 / 59.09

(b) Component contributions. At all three sizes, full SPS has the highest safety and helpfulness on both test sets and the highest Evolving LGC among these selectors. The largest Evolving safety drop comes from removing the lower-tail term, followed by removing the directional term. For example, on 14B these ablations lower safety from 75.00% to 65.28% and 66.67%, respectively; removing dispersion lowers it to 72.22%. We will add these per-component rollout metrics alongside the correlations in Table 4.

(c) Highest-average-utility baseline. Compared with nominal utility, full SPS gains 13.89–15.28 safety points on Evolving. We will include the nominal-utility baseline in the same Appendix C table for each model and test set.

W5. Comparison with AgentLens

Given that both approaches focus on activation steering in agentic contexts, the absence of a direct comparison makes it difficult to evaluate the relative advantages of the proposed method.

Agents are deployed across domains with different tools, permissions, and consequences for the same kind of action. In Figure 3(b), standard CAA improved the safety rate in 10 of 14 scenario categories, including Financial (about +21 percentage points), Security (+13.5), and Healthcare (+13.5). The changes in the other four categories were approximately −0.5 percentage points in Vehicle, −3 in Data/search, −3.3 in Crypto, and −15 in Cloud.

AgentLens detects risky states and steers within a low-dimensional subspace. SPS selects a direction for its stability by evaluating how much utility it retains when the direction is perturbed or the offline cases used to estimate it are resampled. This targets the variation across settings in Figure 3(b).

We adapted AgentLens (Luo et al., 2026) to ToolEmu and compared both methods on Qwen3-8B using the same 36 offline cases, paired proposals, and rollout budget.

Test setAgentLens: safety / helpfulness / LGC (%)Full SPS: safety / helpfulness / LGC (%)SPS difference: safety / LGC (points)
ID (108 cases; 99 LGC-eligible)58.33 / 52.78 / 68.1862.04 / 58.33 / 73.91+3.70 / +5.72
Evolving (72 cases; 66 LGC-eligible)51.39 / 26.39 / 59.0969.44 / 36.11 / 68.18+18.06 / +9.09

The safety gap grows from four additional passing cases in ID (67 vs. 63) to thirteen in Evolving (50 vs. 37). Mean invocation latency was 5.634 s for SPS and 5.912 s for AgentLens. With the steering mechanism held fixed on the same actor, nominal utility gives 54.17% Evolving safety and 59.85% LGC, compared with 69.44% and 68.18% for full SPS.

We will connect the sign changes in Figure 3(b) to the SPS selection criterion in Section 3.2, describe the AgentLens adaptation in Section 3.1, and report its rollout results in Table 1 and latency in Section 3.4.

AuthorsOpenReview ↗

Response to Reviewer sNU4 (Part III)

W1. Evaluation with executable tools

All experiments were conducted exclusively within ToolEmu. Since environmental feedback is simulated by a language model, the potential risks and consequences do not manifest in reality. The absence of validation using actual tool APIs casts doubt on the practical significance of the reported safety metrics.

We agree that ToolEmu’s simulated observations leave open whether an agent’s actions change the underlying state. We therefore evaluated Qwen3-8B on Agent-SafetyBench (Zhang et al., 2024), using the steering direction selected offline on ToolEmu without retraining. In this evaluation, tool calls execute against Python environment objects whose state persists across turns. The environments cover filesystem, database and financial, email, calendar, and Internet of Things (IoT) and operating-system tasks.

We selected 120 multi-turn tasks with benign user intent. Each offered both an opportunity to take an unsafe action and a legitimate goal the agent could fulfill. We counted any attempted unauthorized or unsafe call as a safety failure, including attempts blocked by parameter or permission checks. We verified legitimate-goal completion (LGC) directly against the Python environment state after execution.

MethodSafety % (pass / total)State-verified LGC (%)Mean invocation latency (s)
Base Agent29.17 (35/120)53.334.785
GuardAgent (2025)57.50 (69/120)49.1711.820
AgentLens (2026)59.17 (71/120)60.835.812
Nominal utility only60.83 (73/120)62.505.540
Full SPS (Ours)75.00 (90/120)70.835.546

Full SPS passed 90 of 120 tasks on safety, compared with 35 for the base actor (+45.83 percentage points). Against nominal utility, SPS gained 14.17 points in safety and 8.33 in state-verified LGC. Against AgentLens, it gained 15.83 points in safety and 10.00 in LGC, with lower mean invocation latency (5.546 vs. 5.812 s). These gains extend to executable Python tools with deterministic state transitions. Their scope is the selected benchmark environments.

We will report the Agent-SafetyBench comparison in Section 3.4 and document the task selection criteria, treatment of blocked calls, and post-execution state checks in Appendix B.7.

Dear Reviewer,

Thank you again for your thoughtful comments. In the revised manuscript, we extended the evaluation to Llama-3.1-8B and Gemma-2-9B (Section 3.5, Table 3) and added 120 executable Agent-SafetyBench tasks with state-verified completion (Appendix D.5, Table 12). We added AgentLens to the Qwen3-8B main comparison (Table 1) and report the matched ToolEmu results in Appendix D.4 (Table 11). We also added a four-teacher comparison and proposal-quality audit in Appendix D.2 (Tables 8–9), alongside end-to-end SPS component ablations in Section 3.4 (Table 2) and Appendix E.1 (Table 13). We would appreciate your feedback on whether these additions address your main concerns, and we would be happy to clarify any remaining questions.

Best regards, The authors

Reviewer_PVTEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies inference-time safety for tool-using agents. A linear probe predicts risky decisions from the context before the next thought is generated. When the probe triggers, the agent is steered using a direction constructed from matched risky and teacher-generated safer thoughts. Safety Preference Stability (SPS) selects the intervention layer and direction using perturbations and resampling of the construction data. Experiments on three Qwen3 models use ToolEmu and newly constructed scenarios, reporting safety and task-completion gains with lower invocation latency than external guard models.

Critical strengths

The division between offline construction and online intervention is appealing. Using the same decision context for the risky and safer proposals makes the contrast reasonably well controlled, while deployment needs only the probe and a fixed steering vector. Table 2’s invocation-level latency measurements support the practical attraction of intervening before a tool call is generated.

The paper evaluates useful task progress alongside safety. That is consequential here: a defense that stops most actions can otherwise look successful. The evolving-scenario results are promising, particularly for Qwen3-8B, where Table 1 reports safety increasing from 30.56% to 69.44% and legitimate-goal completion from 50.00% to 68.18%. The paper also reports cases where completion falls, which helps qualify the task-preservation claim.

Critical weaknesses
  • The main results cannot yet be reconciled with the stated evaluation population. Appendix B.1 specifies 108 in-distribution test cases, and B.6 defines safety as a binary proportion over trajectories with valid scores. However, Table 1 entries such as 52.89% and 61.98% for Qwen3-4B cannot be obtained by rounding a single proportion with at most 108 observations. Repeated runs or another aggregation rule could explain this, but neither is documented.
  • The treatment of missing scores adds another uncertainty: these are excluded separately for each metric, without the corresponding counts. Before relying on the comparisons, I would need the actual evaluated populations and aggregation procedure.
  • SPS is the distinctive methodological contribution, but its effect is not directly isolated. The natural control would use the same counterfactual data, probe, candidate layers, and steering strength, and select the candidate using nominal utility alone. Table 4 instead ablates correlations with reference-tool alignment. As the paper explicitly notes, that proxy does not evaluate argument-level safety or task completion. The current evidence therefore leaves unclear whether robustness-based selection improves actual rollouts beyond the other design choices.
Questions for authors
  • What are the evaluated-case counts, valid-score counts, and number of runs per case behind Table 1? Please explain the percentage aggregation and confirm that methods use the same held-out cases. Do the comparisons hold with a common evaluation population and consistent treatment of missing scores?
  • With all other components fixed, how does SPS compare with selecting the candidate by nominal utility alone? Rollout safety and completion on both scenario sets would directly address the contribution of the proposed selection rule.
  • How are cases allocated among probe training, direction construction, utility evaluation, safe-behavior evaluation, and the folds in Figure 4? How is the probe threshold calibrated? Appendix B.1 gives the broad train/test split but not these allocations.
3 comments
AuthorsOpenReview ↗

Response to Reviewer PVTE (Part I)

We thank the reviewer for their careful reading. In the response below, we give the Table 1 counts and missing-score checks (Part I), then compare Safety Preference Stability (SPS) with nominal utility and explain the offline split and probe calibration (Part II).

W1, W2 & Q1(a). Case counts, valid scores, and runs per case

What are the evaluated-case counts, valid-score counts, and number of runs per case behind Table 1?

For each metric (safety or helpfulness), the valid-score pass rate is the fraction of scored trajectories with a ToolEmu score of 2 or 3 (Appendix B.6). A malformed response, a timeout or an unknown judgment can leave a metric unscored. For example, GuardAgent passes safety on 57/106 scored tasks (53.77%), or 57/108 designated tasks (52.78%) when missing scores count as failures.

We designated 108 of the 144 ToolEmu cases for in-distribution (ID) evaluation and 36 for offline development. We also ran 13 offline calibration cases to check that we could reliably extract safety and helpfulness scores from the evaluator outputs. The combined log contained 121 records. On the 108 designated test cases, the safety pass rates among valid scores were 57/106 (53.77%) for GuardAgent and 67/107 (62.62%) for Thought-Aligner.

The published 4B safety rates used 64/121 for GuardAgent, 48/121 for Contrastive Activation Addition (CAA) and 75/121 for Thought-Aligner. For CAA, the corrected safety rate is 43/108 (39.81%). Qwen3-8B SPS helpfulness changes from 71/121 (58.68%) to 63/108 (58.33%).

We will correct the affected entries and gains in Table 1 and report a follow-up comparison averaging repeated rollouts within each case.

Q1(b). Aggregation and held-out case IDs

Please explain the percentage aggregation and confirm that methods use the same held-out cases.

Each case contributes one trajectory per method. For Qwen3-4B, the table gives safety and helpfulness over valid scores and over all 108 ID tasks (missing counts as failure). Legitimate-goal completion (LGC) applies to 99 tasks with at least one legitimate goal. We average each task’s completed-goal fraction over scored tasks, then over all 99 with missing scores set to zero.

MethodSafety pass/fail/missingSafety % (valid scores)Safety % (all 108 tasks)Helpfulness pass/fail/missingHelpfulness % (valid scores)Helpfulness % (all 108 tasks)LGC scored/missing (of 99 eligible)Mean LGC % (scored tasks)Mean LGC % (all 99 eligible)
Base Agent29 / 79 / 026.85 (29/108)26.8520 / 88 / 018.52 (20/108)18.5299 / 041.7541.75
GuardAgent57 / 49 / 253.77 (57/106)52.7828 / 78 / 226.42 (28/106)25.9397 / 247.7746.80
ShieldAgent54 / 51 / 351.43 (54/105)50.0030 / 75 / 328.57 (30/105)27.7899 / 05.055.05
Thought-Aligner67 / 40 / 162.62 (67/107)62.0437 / 70 / 134.58 (37/107)34.2698 / 157.8257.24
CAA43 / 65 / 039.81 (43/108)39.8130 / 78 / 027.78 (30/108)27.7899 / 0——
k-nearest neighbors (KNN), hard45 / 63 / 041.67 (45/108)41.6744 / 64 / 040.74 (44/108)40.7499 / 0——
KNN (Soft)47 / 61 / 043.52 (47/108)43.5240 / 68 / 037.04 (40/108)37.0499 / 051.6851.68
SPS (Ours)75 / 33 / 069.44 (75/108)69.4434 / 74 / 031.48 (34/108)31.4899 / 064.3164.31

For SPS at 8B and 14B, the ID safety counts are 67/108 and 78/108, and helpfulness counts are 63/108 and 71/108, respectively. The complete method-by-metric tally here covers 4B. We will identify the test sets in the Table 1 caption, specify LGC averaging in Appendix B.2 and include the per-method counts in Appendix B.6.

Q1(c). Comparison on a common population

Do the comparisons hold with a common evaluation population and consistent treatment of missing scores?

With missing scores counted as zero, SPS leads Thought-Aligner by 7.41 safety points on the same 108 ID tasks (75 vs. 67 passes) and 7.07 LGC points on the same 99 eligible tasks (64.31% vs. 57.24%).

We apologize for these reporting errors. We will include these common-denominator comparisons in Section 3.3 and Appendix B.6.

AuthorsOpenReview ↗

Response to Reviewer PVTE (Part II)

W3 & Q2. Full SPS versus nominal utility

With all other components fixed, how does SPS compare with selecting the candidate by nominal utility alone? Rollout safety and completion on both scenario sets would directly address the contribution of the proposed selection rule.

We agree that the relevant control is a selector with access to the same data and steering mechanism. Both selectors use the same paired counterfactual proposals, candidate layers, frozen probe and gate (γ=0.50\gamma=0.50), steering strength (α=0.8\alpha=0.8), and decoding settings. The nominal selector maximizes U(κ)\mathcal{U}(\kappa); full SPS uses the stability score. Safety covers every task. LGC averages completion only over tasks with at least one legitimate goal.

ModelTest setAll tasksTasks eligible for LGCNominal safety / LGC (%)SPS safety / LGC (%)SPS minus nominal (safety / LGC, points)
Qwen3-4BID1089962.96 / 59.9369.44 / 64.31+6.48 / +4.38
Qwen3-4BEvolving726650.00 / 53.0363.89 / 56.82+13.89 / +3.79
Qwen3-8BID1089956.48 / 69.0262.04 / 73.91+5.56 / +4.88
Qwen3-8BEvolving726654.17 / 59.8569.44 / 68.18+15.28 / +8.33
Qwen3-14BID1089965.74 / 71.2172.22 / 75.93+6.48 / +4.71
Qwen3-14BEvolving726659.72 / 52.2775.00 / 59.09+15.28 / +6.82

The safety gain is larger on Evolving (13.89–15.28 points across the three model sizes) than on ID (5.56–6.48 points). LGC also increases in all six comparisons.

We will describe the matched control in Section 3.3 and add the six model-by-scenario comparisons to Appendix C.

Q3(a). Offline case allocation and the folds in Figure 4

How are cases allocated among probe training, direction construction, utility evaluation, safe-behavior evaluation, and the folds in Figure 4?

Neither the 108 ID nor the 72 Evolving test cases entered any fitting or selection step. We grouped the 36 offline cases into three folds of 12, keeping every step from a case together. Each fold contains 54 decision steps (27 risky and 27 safe); the total is 162.

For probe training, each fold used 24 cases for fitting and 12 for held-out predictions. We calibrated and froze the gate as described below before building any directions.

Build and score directions by fold. For each candidate layer, we used the 54 risky steps in the two build folds to estimate its direction, four perturbations along principal component analysis (PCA) axes, and 100 case-bootstrap vectors. On the remaining fold (27 risky and 27 safe steps), we evaluated the preference score on risky steps and the safe-behavior cost on safe steps and computed the SPS score as the sum of the directional, lower-tail, and dispersion terms. Figure 4 shows these fold-by-layer scores; its partial Spearman correlation controls for fold and layer.

Select one layer and refit its direction. We averaged each layer's validation score across the three folds and chose the layer with the highest mean. The zero-based indices were 20, 28, and 22 for Qwen3-4B, 8B, and 14B, respectively. At that single layer we recomputed the deployed direction with Eq. 6 on all 81 risky steps from the 36 offline cases, then froze the layer and direction for online evaluation.

We will add the case and step counts for each offline stage to Appendix B.1 and explain in the Figure 4 caption how fold scores determine the deployed layer.

Q3(b). Probe threshold calibration

How is the probe threshold calibrated?

Fit and calibrate the gate. For each fold, we trained a linear probe on the other 24 cases (54 risky and 54 safe steps) and scored the held-out 12. We pooled the 162 out-of-fold predictions and searched thresholds from 0.30 to 0.70 in steps of 0.05, using the true positive rate (TPR) and false positive rate (FPR) to maximize Youden's J=TPR−FPRJ=\mathrm{TPR}-\mathrm{FPR} subject to a safe-step false positive rate of at most 0.15. The selected threshold was 0.50 (Qwen3-8B: 85.2% true positives, 13.6% false positives). We refit the probe on all 36 offline cases and froze both the probe and threshold before constructing directions. Steering strength was also fixed before candidate selection.

We will specify the calibration objective, threshold grid, and false-positive constraint in Appendix B.1 and state before Eq. 7 when the refitted probe and threshold are frozen.

Dear Reviewer,

Thank you again for your thoughtful comments. In the revised manuscript, we expanded the reporting of evaluation counts and missing-score handling in Appendix C (Tables 6–7) and documented the case-level offline splits, out-of-fold probe calibration, and separation of test cases from direction selection in Appendix B.1. We also added a matched comparison between SPS and nominal-utility selection under the protocol described in Appendix D.1. The Qwen3-8B results are in Section 3.4 (Table 2), and the Qwen3-4B and 14B results are in Appendix E.1 (Table 13). We would appreciate your feedback on whether these revisions address your main concerns, and we would be happy to clarify any remaining questions.

Best regards, The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.