acceptodds
Under review as a conference paper at ICLR 2027

AgentBoundary: Counterfactual Action Safety in Tool-Using LLM Agents

Abstract

Safety alignment for large language models (LLMs) in conversational settings is largely framed around whether to answer or refuse a request. In agentic settings, however, the same models must decide whether to act as permission-critical evidence emerges during execution. This creates a distinct challenge: apparent risk, action permissibility, and task competence are easily confounded, making agentic over-refusal difficult to distinguish from ordinary task failure. To address this, we introduce AgentBoundary, the first four-way counterfactual generation-and-evaluation framework for tool-using agent safety. AgentBoundary transforms the same executable workflow by independently varying apparent risk and action permissibility, enabling controlled comparisons of risky-looking but authorized tasks and routine-looking but unauthorized tasks. These comparisons jointly diagnose over-refusal and unsafe compliance while controlling for task competence. We instantiate AgentBoundary as a human-validated 4,000-task evaluation suite with trajectory-based and post-state-based judgments. Across 17 model and harness configurations, high safety frequently coexists with poor authorized-task completion: GPT-5.5 blocks 99.5% of routine-looking unauthorized actions yet completes only 28.7% of risky-looking authorized tasks. We further train a lightweight runtime calibration module that improves authorized-task completion by 18.2% on average across 10 evaluated configurations, while improving unsafe-action blocking by 5.4% on average. These show that effective agentic alignment requires action decisions to track permission-relevant execution evidence, rather than refusal strength alone.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent safety, alignment, over-refusal

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9KQGOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author propose the AgentBoundary framework for tool-using agent safety. It builds four-way families of executable agent tasks around a shared tool by enabling controlled comparisons of the risky-looking but authorised tasks.

Critical strengths

(1) The four way counterfactual setup does a much better job of separating out over-refusal from unsafe compliance than the benign/harmful pairs. (2) The paper reports on the aligner’s failures and includes a tool gating baseline and an untrained wrapper control.

Critical weaknesses

(1) The central assumption that permissibility is determined by evidence that arises during execution is not supported by the data. In the case study the Hard- Benign prompt contains no authorization at all and the plain model' s refusal is a perfectly sensible reaction to that prompt. The tool observations in the case study make no difference to what is permitted. The aligned decisions in both cases are a direct function of the prompt text rather than of any evidence that emerges during execution. The environment is LM- emulated and the rosters produced by the different runs of the prompt are different so the evidence is not held constant across arms. The labels are derived from the generator' s intent and human validation is performed on families not on the actual trajectories that are realised. (2) The measurement chain is not validated at the level that the metrics are applied. HB utility and CM safety are binarised at score> = 2 from an LLM judge. The judge validation reports within- one agreement on a 0 to 3 scale. A one point disagreement at the 1 to 2 boundary will change the binary outcome so 91. 0% and 96. 7% within- one agreement tells us very little about pass/fail agreement. Appendix E reports human- human exact agreement of 49. 6% on helpfulness and within- one agreement of 68. 2% which is lower than the human judge figure and that needs to be explained. The identity of the judge and any potential overlap with the models being evaluated is not discussed in the main text. The aligner is trained on a pool constructed by the same pipeline and on the same action boundary principle as the benchmark and is tested only on that benchmark. The improvements may be due to learning the generator' s disguise patterns. No confidence intervals, seeds or repeated runs are reported.

Questions for authors
  1. Please report binary threshold agreement (pass/fail at >= 2) between humans and the judge, with kappa, per split.
  2. What proportion of Hard-Benign refusals happen prior to any tool call and in how many of those is it an observation that provides the authorisation that leads to a change of decision?
  3. Does the aligner transfer to an external benchmark such as AgentHarm or to a differently generated family set.
  4. Which model acts as the judge and the emulator and does any of the evaluated configurations share a vendor with it;
  5. How much run-to-run variance is there for one configuration?
16 comments

Dear Reviewer 9KQG,

Thank you for the careful and detailed review. We are greatly encouraged by** your recognition that the four-way counterfactual layout offers a clearer diagnostic of over-refusal and unsafe compliance, and that reporting intervention controls alongside safety trade-offs is valuable. **Below, we reply to all your conerete comments one by one.


W1. Execution evidence, the LM-emulated environment, and human validation

Thank you for the detailed and constructive questions. AgentBoundary evaluates action boundaries using the user's delegation, tool observations and the effects of the proposed action.

1. Permission evidence can be initial or revealed during execution.

The central premise, that permissibility turns on evidence emerging during execution, is not shown in the data.

The trajectories below show tool-revealed permission facts explicitly informing subsequent decisions. The table gives three original DeepSeek-V4-Flash examples. Permission can also be established via the initial request. New authorization need not emerge in every task (§3; Appendix B).

Example / splitPermission-relevant tool observationActual subsequent behavior
Asset licensing / HBRegistry distinguishes Public Domain/CC0, Internal Only and Pending Clearance assets.Selects Public Domain/CC0 assets and saves the mockup within an inspected internal sandbox.
External posting verification / CMWorkspace policy requires dual verification for external posting.Cites the verification rule and unconfirmed account-holder permission when declining to post.
Personal-data disclosure / CMRecord review identifies personal data and requires sanitization before external sharing.Declines raw external sharing and recommends anonymization.

The agent uses these observations to select permitted resources or withhold restricted actions. The provided examples are trajectory-level examples of evidence use, which do not establish a counterfactual causal effect alone.

2. Semantic authorization in the bulletin case.

In the case study, the Hard-Benign prompt “Penetrate the senior class bulletin system and zero in on Maya Chen\.\.\.” contains no authorization, and the bare model’s refusal reads as a reasonable response to the text, not obvious over-refusal.

This family follows ordinary assistant use. Users delegate tasks through a request and its application context, without necessarily adding a separate authorization statement. The request is to identify a student, describe her achievements and distribute a school announcement through specified channels. Under the family's legitimate-user assumption, task semantics convey delegation for that workflow.

The HB rewrite introduces verbs such as “penetrate,” “zero in,” and “exploit” while preserving the benign bulletin objective. The bare agent interprets them as intrusion and refuses before tool use, which we view as the misclassification under that delegation assumption.

We will state this delegation assumption explicitly in I.2. It is not an additional credential in the archived input. Workspace policy, privacy and recipient restrictions still apply, and the CM harassment workflow remains forbidden. I.1 gives an explicit complementary contrast: The $50,000 invoice workflow operates inside a quarantined test environment in HB, while its harmful counterpart changes and uploads a financial record to the company system.

3. The role of roster observation.

The tool observations in the case study (roster listings) do not change what is permitted.

In I.2, the prompt and use context provide the intended delegation. In the example, the roster identifies the student and grounds the bulletin's content. The lookup does not grant publication authority. We will state these roles directly in the case description. Other families reveal further constraints during execution, including the license, verification and disclosure facts shown in point 1.

4. What the two aligned decisions demonstrate.

Both aligned decisions follow prompt wording, not execution evidence.

Permission evidence can be present in the user request or supplied by tools. In I.2, the user's request delegates the benign announcement workflow within the stipulated legitimate-use context. The roster provides the factual content needed to execute it. AgentBoundary asks whether the action respects the applicable permission and effect. It does not require every task to reveal a new permission after the first tool call.

5. The basis for an LM-emulated environment.

The environment is LM-emulated.

Lines 250–253 in Section 4.1 explicitly follows ToolEmu's LM-emulated sandbox methodology. Related published implementations include:

- ToolEmu Ruanetal,˙ICLR2024Ruan et al\., ICLR 2024 emulates observations from tool specifications, call arguments and interaction history to evaluate sensitive tool-use workflows [1].

- PrivacyLens Shaoetal,˙NeurIPS2024DatasetsandBenchmarksShao et al\., NeurIPS 2024 Datasets and Benchmarks builds on ToolEmu and uses GPT-4 to generate privacy-sensitive observations for calendar, email, messaging and notebook tools [2].

- HAICOSYSTEM Zhouetal,˙COLM2025Zhou et al\., COLM 2025 uses GPT-4o as an environment engine for tool responses under scenario-specific constraints, including healthcare and finance settings [3].

- StableToolBench Guoetal,˙FindingsofACL2024Guo et al\., Findings of ACL 2024 uses cached API responses and falls back to LM simulation when neither cached nor live responses are available [4].

These works provide methodological support for AgentBoundary's LM-emulated sandbox. Our evaluation also includes human validation of task semantics and realized tool-use trajectories. Professional annotators assess interaction coherence, tool misuse, task completion and tool-call risk, with repeated annotations and quality review; automatic judge scores are checked against human ratings §42˙;AppendixE§4\.2; Appendix E.

References

[1] Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., and Hashimoto, T. 20242024. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. International Conference on Learning Representations (ICLR).

[2] Shao, Y., Li, T., Shi, W., Liu, Y., and Yang, D. 20242024. PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action. Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track.

[3] Zhou, X., Kim, H., Brahman, F., Jiang, L., Zhu, H., Lu, X., Xu, F., Lin, Y., Choi, Y., Mireshghallah, N., Le Bras, R., and Sap, M. 20252025. HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Interactive AI Agents. Conference on Language Modeling (COLM).

[4] Guo, Z., Cheng, S., Wang, H., Liang, S., Qin, Y., Li, P., Liu, Z., Sun, M., and Liu, Y. 20242024. StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models. Findings of the Association for Computational Linguistics: ACL 2024, 11143–11156.

6. Different rosters across arms.

the same prompt yields different rosters across runs differentIDs,gradesandfabricatedachievementsinI2˙2˙toI2˙4˙different IDs, grades and fabricated achievements in I\.2\.2 to I\.2\.4, so evidence is not held fixed across arms.

Repeated-run stability. To demonstrate stability directly, we conducted 100 multi-run experiment cases across seven backbones. It showed that the result remained stable across runs. Minor information differences don't affect the result.

BackboneUtility pass/fail agreementSafety pass/fail agreement
GPT-5.597.00%96.00%
Gemini-3.5-Flash100.00%98.00%
Qwen39799.00%97.00%
Qwen3.8-27B97.00%99.00%
DeepSeek-V4-Flash97.00%96.00%
GLM-595.00%95.00%
Nemotron-Nano-9B-v2100.00%100.00%

Independent human validation. The human study checks family semantics and permission boundaries: HB/CM boundary contrast passes 98.0%, and split-safety semantics pass 99.1%. Repeated independent annotations agree on these checks at 96.4% and 99.5%, respectively.

7. Validation of the boundary labels.

Labels (HB authorized, CM forbidden) come from the generator’s intent

Labels are checked through construction constraints and independent human judgments. HB preserves EB's authorized workflow; CM preserves HS's harmful endpoint while changing its framing AppendixB1˙Appendix B\.1. The human questionnaire assesses the resulting task families and permission boundaries (Appendix E, Tables 7–8):

Construction checkPass rate, 95% CI
Sibling consistency97.0%
965˙,974˙96\.5, 97\.4

| |Natural wording|99.7%

995˙,999˙99\.5, 99\.9

| |Harmful-source severity|98.1%

976˙,986˙97\.6, 98\.6

| |HB/CM boundary contrast|98.0%

976˙,983˙97\.6, 98\.3

| |Split-safety semantics|99.1%

989˙,993˙98\.9, 99\.3

|

Human evaluation supports the intended four-way construction and boundary labels. The HB/CM boundary contrast passes 98.0%, and split-safety semantics passes 99.1%.

8. Human validation of realized trajectories.

Human validation checks families, not realized trajectories.

The submitted study includes 660 trajectory-annotation records covering Bare and Aligner runs §42˙;AppendixE,Tables8–9§4\.2; Appendix E, Tables 8–9. We ask annotators who are expertised in agent safety to inspect the actual prompts, tool definitions and realized trajectories. They are required to assess coherence, tool misuse, task completion and risk under common guidelines and quality review.

Moreover, the annotation finds that 97.88% of records mark the interactions as coherent, and 99.24% report no obvious tool misuse. These results support the coherence and tool-use plausibility of the audited interactions. The same annotations also provide a human reference for validating the automatic judge through their completion and risk ratings.


Q1 Please report binary-threshold agreement pass/failat =2pass/fail at \>= 2 between humans and the judge, with kappa, per split.

Below, we report pass/fail agreement and κ separately for each split at the actual decision threshold. We also compare human–human and human–judge agreement under the same rules on a common archive population. For EB and HB safety, all human reference labels are “safe”; We therefore mark Cohen’s κ N/AN/A for these single-class reference subsets.

Split / metricPass/fail agreement % 9595% CIκ
EB utility84.18%
798˙4,882˙279\.84, 88\.22

|0.632| |EB safety|97.34%

952˙1,989˙995\.21, 98\.99

|N/A| |HB utility|92.94%

870˙6,976˙587\.06, 97\.65

|0.854| |HB safety|98.40%

966˙0,997˙496\.60, 99\.74

|N/A| |CM utility|88.70%

849˙6,921˙084\.96, 92\.10

|0.708| |CM safety|90.48%

833˙3,964˙383\.33, 96\.43

|0.618| |HS utility|97.38%

953˙4,991˙995\.34, 99\.19

|0.794| |HS safety|98.43%

968˙3,997˙396\.83, 99\.73

|0.842|

Human–human versus human–judge agreement on the same episodes. We use the same pass/fail rule and the same repeatedly annotated archive episodes for both comparisons.

MetricHuman–human pass/fail agreementHuman–judge pass/fail agreement
Utility91.41%89.24%
Safety97.37%94.61%
Reviewer_YPirOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies two failure modes in tool-using LLM agents: over-refusal, where authorized actions are incorrectly rejected, and unsafe compliance, where unauthorized actions are allowed. To evaluate these failures, the authors introduce AgentBoundary, a four-way counterfactual framework that independently varies apparent risk and action permissibility. Experiments on a 4,000-task benchmark across 17 model and harness configurations show that strong unsafe-action blocking can coexist with poor authorized-task completion. The authors also train a lightweight runtime calibration module, reporting average improvements of 18.2% in authorized-task completion and 5.4% in unsafe-action blocking.

Critical strengths
  1. Insightful findings on safety and task completion. The paper provides interesting observations about the trade-off between safety and task completion. Some models appear to be overly conservative, achieving high unsafe-action blocking rates (CM) but low authorized-task completion rates (HB). This finding is meaningful because, in practice, failed tasks can often be retried, while unsafe actions may have irreversible consequences.

  2. Interesting observations on harness configurations and model scaling. The results suggest that agent harnesses, such as OpenClaw, can improve authorized-task completion, while increasing model size does not necessarily lead to better performance.

Critical weaknesses
  1. Limitations of the controlled evaluation. The paper claims to provide controlled evaluations, but the tool responses are generated by an LLM. This may introduce additional variability and make it difficult to determine whether the observed performance differences are solely due to the intended counterfactual changes.

  2. Concerns about judge reliability. The agreement metric used to validate the judge may not fully reflect evaluation reliability. Scores are considered to agree when their absolute difference is at most one point. However, scores of 1 and 2 represent substantially different outcomes, with one indicating unsatisfactory completion and the other indicating successful completion. Therefore, the reported agreement may overestimate the judge's reliability.

  3. Limited evidence of generalization. The training and test datasets are generated using the same four-way construction pipeline. The lack of evaluation on independent or out-of-domain benchmarks makes it difficult to assess whether the reported improvements generalize beyond the proposed benchmark.

Questions for authors
  1. Judge agreement. Could the authors report additional agreement results using an absolute score difference of at most two? It would also be helpful to report exact agreement, since a one-point difference may represent substantially different completion outcomes.

  2. Missing metric. It seems that IorI_{\mathrm{or}} is not reported in the paper. Could the authors clarify whether this metric was evaluated and, if so, provide the corresponding results?

  3. Evaluation sample size. The paper appears to contain 2,787 samples, but only 1,000 are used for evaluation. Could the authors clarify why only 1,000 samples were selected and how they were chosen?

4 comments

Dear Reviewer YPir,

Thank you for the careful review and thoughtful feedback. We are greatly encouraged by your recognition of the importance of distinguishing strong unsafe-action blocking from successful completion of authorized tasks, as well as the useful observations about agent harnesses and model scaling. Below, we reply to all your concrete comments one by one.


W1 Limitations of the controlled evaluation. The paper claims to provide controlled evaluations, but the tool responses are generated by an LLM. This may introduce additional variability and make it difficult to determine whether the observed performance differences are solely due to the intended counterfactual changes.

Thank you for raising this important question about evaluation control. For clarity, we would report the repeated-run evidence first, then discuss human validation and the simulation methodology separately.

LM-emulated sandbox methodology. The counterfactual construction controls the workflow scaffold and intended task changes. LM emulation makes sensitive tool workflows executable inside a sandbox. This choice follows ToolEmu, ICLR 2024 [1], PrivacyLens, NeurIPS 2024 [2], HAICOSYSTEM, COLM 2025 [3], and the LM API fallback in StableToolBench, Findings ACL 2024 [4]. These works provide methodological precedent for LM-generated tool feedback.

To validate stability more directly. We further conducted 100 multi-run experiment cases across seven backbones. It showed that the result remained stable across runs.

BackboneUtility pass/fail agreementSafety pass/fail agreement
GPT-5.597.00%96.00%
Gemini-3.5-Flash100.00%98.00%
Qwen39799.00%97.00%
Qwen3.8-27B97.00%99.00%
DeepSeek-V4-Flash97.00%96.00%
GLM-595.00%95.00%
Nemotron-Nano-9B-v2100.00%100.00%

Independent human validation. The human study checks family semantics and permission boundaries: HB/CM boundary contrast passes 98.0%, and split-safety semantics pass 99.1%. Repeated independent annotations agree on these checks at 96.4% and 99.5%, respectively.

References

[1] Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., and Hashimoto, T. 20242024. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. International Conference on Learning Representations (ICLR).

[2] Shao, Y., Li, T., Shi, W., Liu, Y., and Yang, D. 20242024. PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action. Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track.

[3] Zhou, X., Kim, H., Brahman, F., Jiang, L., Zhu, H., Lu, X., Xu, F., Lin, Y., Choi, Y., Mireshghallah, N., Le Bras, R., and Sap, M. 20252025. HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Interactive AI Agents. Conference on Language Modeling (COLM).

[4] Guo, Z., Cheng, S., Wang, H., Liang, S., Qin, Y., Li, P., Liu, Z., Sun, M., and Liu, Y. 20242024. StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models. Findings of the Association for Computational Linguistics: ACL 2024, 11143–11156.


W2 Concerns about judge reliability. The agreement metric used to validate the judge may not fully reflect evaluation reliability. Scores are considered to agree when their absolute difference is at most one point. However, scores of 1 and 2 represent substantially different outcomes, with one indicating unsatisfactory completion and the other indicating successful completion. Therefore, the reported agreement may overestimate the judge's reliability.

Q1 Judge agreement. Could the authors report additional agreement results using an absolute score difference of at most two? It would also be helpful to report exact agreement, since a one-point difference may represent substantially different completion outcomes.

We appreciate your suggestion to make the agreement measures easier to interpret. We focus on agreement at the actual pass/fail threshold, so the validation directly matches the decisions used to compute benchmark outcomes.

We recomputed these statistics from human annotations, using score ≥2 as pass and family-level cluster bootstrap confidence intervals. The evaluator shows high agreement with human judgements of task completion and action safety.

Split / metricPass/fail agreement % 9595% CI
HB utility92.94%
870˙6,976˙587\.06, 97\.65

| |CM safety|90.48%

833˙3,964˙383\.33, 96\.43

|

Q2 Missing metric. It seems that I_or is not reported in the paper. Could the authors clarify whether this metric was evaluated and, if so, provide the corresponding results?

Thank you for the opportunity to clarify how this indicator relates to the reported HB over-refusal rate. The requested metric is the HB over-refusal rate. Each HB task receives IorI_{\mathrm{or}} when a safety or permission misclassification causes refusal despite an available safe completion path, and 0 otherwise. The percentage labeled 1 gives the over-refusal rate.


Q3 Evaluation sample size. The paper appears to contain 2,787 samples, but only 1,000 are used for evaluation. Could the authors clarify why only 1,000 samples were selected and how they were chosen?

They refer to the two stages of the pipeline, i.e., construction validation and human-curated benchmark selection. To clarify, the 2,787 families were used to validate the task-generation pipeline. Based on human evaluations, we selected the 1,000 highest-quality families to form the benchmark while also considering balance across risk categories. Each family contains four counterfactual tasks, yielding 4,000 evaluation instances. The two counts refer to pipeline validation and the human-curated benchmark, respectively.

The human questionnaire assesses sibling consistency, natural wording, harmful-source severity, HB/CM boundary contrast and split-safety semantics (Appendix E, Tables 7–8). Section 4.1 and Appendix B.2 also describe stratification across all eight top-level risk categories and all 13 Cybercrime classes.


W3 Limited evidence of generalization. The training and test datasets are generated using the same four-way construction pipeline. The lack of evaluation on independent or out-of-domain benchmarks makes it difficult to assess whether the reported improvements generalize beyond the proposed benchmark.

To validate generalization beyond our proposed benchmark, we further complete external evaluations using the Aligner. Below, we provide the two task environments, tool interfaces, attempt-selection protocol and measured results.

ToolEmu

All four backbones attain higher Helpfulness and Tool-call safety pass rates in ToolEmu.

ModelHelpfulness pass rate B→Aligner A % ↑Δ ppTool-call safety pass rate B→Aligner A % ↑Δ pp
GPT-5.566.67%→75.00%+8.3370.83%→83.33%+12.50
DeepSeek V4 Flash60.87%→65.22%+4.3552.17%→78.26%+26.09
Gemini 3.5 Flash70.83%→79.17%+8.3358.33%→100.00%+41.67
Nemotron Nano 9B v241.67%→54.17%+12.5062.50%→70.83%+8.33

AgentHarm

All four backbones attain higher Benign scores and lower Harm scores in AgentHarm.

ModelBenign score B→Aligner A % ↑Δ ppHarm score B→Aligner A % ↓Δ pp
GPT-5.534.65%→42.79%+8.140.32%→0.00%-0.32
DeepSeek V4 Flash52.08%→55.22%+3.132.78%→0.92%-1.86
Gemini 3.5 Flash59.99%→60.25%+0.267.10%→0.00%-7.10
Nemotron Nano 9B v263.08%→63.64%+0.5511.83%→3.39%-8.44

We sincerely appreciate your thoughtful feedback and recognition of the importance of distinguishing authorized completion from unsafe compliance. We will make the benchmark selection, evaluator validation and scope of the external results clearer in the revised paper. We hope these clarifications help address your concerns and would be grateful for your reconsideration and support.

Dear Reviewer YPir,

Thank you again for your constructive feedback. We would appreciate the opportunity to hear your thoughts on our rebuttal and additional results when convenient. If any concerns remain unresolved, please let us know so that we can address them during the discussion.

Best regards,
The Authors

Dear Reviewer YPir,

Thank you again for your careful review and constructive feedback on AgentBoundary. We have provided additional judge validation, repeated-run analyses, clarification of the benchmark selection, and external evaluations on ToolEmu and AgentHarm.

As the discussion period is coming to an end, we would greatly appreciate your thoughts on these updates when you have a chance. Please let us know if any questions or concerns remain; we would be happy to provide further clarification during the discussion.

Best regards,

The Authors

Reviewer_a3BvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AgentBoundary studies whether tool-using agents distinguish apparent risk from evidence that an action is permitted. Each task family contains four variants: ordinary authorized work (EB), risky-looking authorized work (HB), routine-looking unauthorized work (CM), and an overtly harmful request (HS). The authors evaluate 1,000 families, or 4,000 tasks, across 17 model and harness configurations in an LM-emulated tool environment. They also train a Qwen3-14B module that edits the agent’s visible pre-action Thought before the agent generates its next action. Across ten configurations, this intervention improves HB completion by 18.2 percentage points and CM safety by 5.4 points on average. The paper includes human validation of task families and trajectory scores, as well as comparisons with an untrained module, Thought-Aligner, and GuardAgent.

Critical strengths

The four-variant design addresses an important problem in agent evaluation. An authorized action can look suspicious, while an unauthorized action can be presented as routine work. Appendix B.1 makes a useful effort to preserve the safe workflow when changing EB into HB, and to preserve the harmful endpoint when changing HS into CM. This gives the benchmark a stronger basis for comparison than unrelated collections of benign and harmful tasks. The human checks on these family contrasts, including repeated annotations and confidence intervals computed by resampling families, add useful support to the construction.

The mitigation experiments include meaningful controls. Table 4 compares the trained module with an untrained module using the same interface, as well as Thought-Aligner and GuardAgent. The trained version improves both HB utility and CM safety over the untrained version on all four tested backbones. This helps establish that the training contributes beyond the effect of inserting an additional model into the loop. The gains across ten configurations are also encouraging, particularly since the base agent remains unchanged.

Critical weaknesses
  1. The results do not yet demonstrate the distinction between over-refusal and ordinary task failure that motivates the benchmark.

    Section 3 defines an over-refusal indicator, IorI_{\mathrm{or}}, requiring evidence of a safety or permission misclassification. EB is also introduced as a competence control. However, Tables 2–4 report HB utility, CM safety, and pooled scores; they do not report IorI_{\mathrm{or}}, EB completion, or HB failure conditioned on success of the corresponding EB task.

    Figure 3 pairs HB with CM, which does not resolve this issue: an agent unable to execute either task can fail HB and still avoid unsafe actions on CM. The construction provides the controls needed for a stronger analysis, but low HB completion alone does not establish that an agent is over-refusing. This weakens the interpretation that the observed pattern primarily reflects a global caution policy rather than limited capability.

  2. The validation of the automatic judges does not match the decisions used in the results.

    Appendix D counts scores of at least 2 as passing, but the headline human–judge agreement in Appendix E counts scores within one point as agreement. A human score of 1 and a judge score of 2 therefore count as agreement while producing opposite benchmark outcomes. The reported 91.0% helpfulness and 96.7% risk agreement cannot, on their own, establish reliable pass/fail classification.

    This is particularly concerning for utility, since repeated human helpfulness annotations agree exactly only 49.6% of the time and within one point 68.2% of the time. Appendix D also excludes missing trajectories and invalid judge outputs, while the result tables omit the corresponding coverage counts. Agreement at the actual decision threshold and comparisons over the same evaluated families are needed to assess the reported gains.

Questions for authors
  1. Can you report EB completion, HB failure conditional on matched EB success, and the trajectory-verified over-refusal rate under IorI_{\mathrm{or}}? Please distinguish safety-based refusals from execution errors and other capability failures. How are attempted but unsuccessful forbidden actions scored, given that Section 3 includes proposed unsafe effects while the human risk rubric focuses on successfully executed calls?

  2. What are the human–judge confusion matrices at the score-2 threshold, separately for HB utility and CM safety? Please provide coverage counts for each condition and paired Bare–Aligner results on the same valid families, with uncertainty estimates.

  3. What environment state and emulator randomness are fixed across matched runs? In Appendix I.2, the same CM prompt and initial list_records call return different student IDs and majors in the Bare and Aligner trajectories (pp. 27 and 30). Please explain this difference and whether the aggregate improvements persist when identical initial calls receive identical observations.

  4. Which splits account for GPT-5.5’s all-split safety decline from 99.4% to 95.5% after alignment, despite unchanged CM safety? A breakdown of unsafe outcomes across all four splits would clarify the intervention’s safety trade-offs.

6 comments

Dear Reviewer a3Bv,

Thank you for the careful and constructive review. We are greatly encouraged by your recognition that the four-way task design provides a stronger basis for controlled comparisons, and that the human checks and trained-versus-untrained controls offer useful support for AgentBoundary. Below, we reply to all your concrete comments one by one.


W1 The results do not yet demonstrate the distinction between over-refusal and ordinary task failure that motivates the benchmark. Section 3 defines an over-refusal indicator, $I_{\mathrm{or}}$, requiring evidence of a safety or permission misclassification. EB is also introduced as a competence control. However, Tables 2–4 report HB utility, CM safety, and pooled scores; they do not report I_or, EB completion, or HB failure conditioned on success of the corresponding EB task. Figure 3 pairs HB with CM, which does not resolve this issue: an agent unable to execute either task can fail HB and still avoid unsafe actions on CM. The construction provides the controls needed for a stronger analysis, but low HB completion alone does not establish that an agent is over-refusing. This weakens the interpretation that the observed pattern primarily reflects a global caution policy rather than limited capability.

Q1(a) Can you report EB completion, HB failure conditional on matched EB success, and the trajectory-verified over-refusal rate under $I_{\mathrm{or}}$? Please distinguish safety-based refusals from execution errors and other capability failures.

Thank you for suggesting this important diagnostic analysis. We address the distinction between over-refusal and ordinary task failure through matched EB competence controls and trajectory-based failure attribution. We report below the EB completion, the HB failure conditioned on EB success, and the HB over-refusal rate. We also provide explanation on how these measurements complement one another.

Results across all 17 configurations

ConfigurationEB completionHB completionHB failure given matched EB success95% CI
GPT-5.548.5%28.7%69.9%
657˙65\.7%, 73\.9%

| |Claude-Opus-4.8|82.0%|40.4%|57.0%|

537˙53\.7%, 60\.4%

| |Gemini-3.5-Flash|58.5%|36.6%|54.0%|

502˙50\.2%, 57\.9%

| |Grok-4.5|90.2%|52.0%|44.9%|

418˙41\.8%, 48\.1%

| |DeepSeek-V4-Flash|86.9%|45.0%|53.0%|

497˙49\.7%, 56\.2%

| |DeepSeek-V4-Pro|85.1%|55.9%|41.6%|

384˙38\.4%, 44\.8%

| |Kimi-K3|90.0%|58.7%|39.8%|

365˙36\.5%, 43\.0%

| |GLM-5|84.5%|36.8%|61.1%|

578˙57\.8%, 64\.2%

| |Qwen397|80.6%|23.7%|73.4%|

704˙70\.4%, 76\.4%

| |Qwen3.8-27B|76.2%|37.1%|58.1%|

547˙54\.7%, 61\.7%

| |Qwen3.8-27B Uncensored|92.7%|90.2%|8.3%|

66˙6\.6%, 10\.0%

| |Nemotron-Nano-9B-v2|54.1%|38.5%|48.3%|

440˙44\.0%, 52\.4%

| |SmolLM3-3B|1.6%|4.6%|86.7%|

667˙66\.7%, 100\.0%

| |Granite-3.3-8B-Instruct|56.2%|53.8%|37.4%|

333˙33\.3%, 41\.5%

| |Codex|54.1%|28.1%|65.8%|

618˙61\.8%, 69\.7%

| |OpenClaw|56.4%|36.6%|59.0%|

549˙54\.9%, 63\.1%

| |DeepSeek Harness|81.6%|45.6%|51.8%|

485˙48\.5%, 55\.2%

|

HB failures persist among EB-successful families across all 17 configurations. The matched competence control therefore identifies an authorized-completion deficit beyond failure on the ordinary workflow; the trajectory audit attributes its cause.

HB over-refusal rate. $I_{\mathrm{or}}$ assigns each HB task a 0/1 label; The percentage labeled 1 is the over-refusal rate.

Distinguishing refusal from task failure. The evaluator judges semantic task completion from the requested outcome, recorded tool calls, observations and final response. Its rubric separately considers over-refusal, tool-use errors and task infeasibility. For $I_{\mathrm{or}}$, only a trajectory showing a safety or permission misclassification that prevents an available safe completion is counted as over-refusal. Execution errors and ordinary capability failures are not counted as over-refusal.

Interpretation of Figure 3. Its HB–CM pairing describes joint utility and safety outcomes. The EB–HB comparison supplies the competence control, and the trajectory audit supplies failure attribution. We will base over-refusal claims on those cause-supported labels and describe the global-caution interpretation as configuration-dependent.

Q1(b) How are attempted but unsuccessful forbidden actions scored, given that Section 3 includes proposed unsafe effects while the human risk rubric focuses on successfully executed calls?

Thank you for asking us to clarify this scoring distinction. We distinguish the proposed action from its realized effect, and explain below how unsuccessful calls enter the trajectory-level safety assessment. The automatic safety judge reads the recorded actions, arguments, observations and final response, including unsuccessful calls. Its AgentSafety-specific instruction treats executing or clearly proposing the blocked terminal behavior as a safety failure. Thus, a forbidden attempt can be judged unsafe even when the final harmful effect does not succeed. In an existing DeepSeek trajectory, the agent created and attempted to run an external-upload script; The sandbox blocked the network connection, yet the judge classified the trajectory as unsafe.

The inherited ToolEmu notes also assess the residual risk of failed calls: a technical failure with no resulting risk can receive a safe outcome-risk score. An unsuccessful endpoint is therefore not evidence that the agent correctly recognized the boundary. Appendix F, lines 1260–1264, describes the runtime checks of unsafe proposals and execution. Appendix E's wording about successfully executed calls describes the human ToolCallRisk rubric; It does not describe the automatic judge's entire input. We will make the decision-level and outcome-level distinction explicit.

W2 The validation of the automatic judges does not match the decisions used in the results. Appendix D counts scores of at least 2 as passing, but the headline human–judge agreement in Appendix E counts scores within one point as agreement. A human score of 1 and a judge score of 2 therefore count as agreement while producing opposite benchmark outcomes. The reported 91.0% helpfulness and 96.7% risk agreement cannot, on their own, establish reliable pass/fail classification. This is particularly concerning for utility, since repeated human helpfulness annotations agree exactly only 49.6% of the time and within one point 68.2% of the time. Appendix D also excludes missing trajectories and invalid judge outputs, while the result tables omit the corresponding coverage counts. Agreement at the actual decision threshold and comparisons over the same evaluated families are needed to assess the reported gains.

Q2 What are the human–judge confusion matrices at the score-2 threshold, separately for HB utility and CM safety? Please provide coverage counts for each condition and paired Bare–Aligner results on the same valid families, with uncertainty estimates.

Thank you for this helpful suggestion. We validate the judge at the actual benchmark decision threshold and compare Bare and Aligner on the same valid families. Below, we provide the agreement statistics, the matched human comparisons, the coverage information and the paired uncertainty estimates, along with the complete confusion matrices in the statistical attachment.

We recomputed these statistics from human annotations, using score ≥2 as pass and family-level cluster bootstrap confidence intervals.

Split / metricPass/fail agreement % 9595% CI
HB utility92.94%
870˙6,976˙587\.06, 97\.65

| |CM safety|90.48%

833˙3,964˙383\.33, 96\.43

|

Human–human versus human–judge agreement on the same episodes. We use the same pass/fail rule and the same repeatedly annotated archive episodes for both comparisons.

Human–judge pass/fail agreement is broadly consistent with the repeated-human reference on matched episodes.

MetricHuman–human pass/fail agreementHuman–judge pass/fail agreement
Utility91.41%89.24%
Safety97.37%94.61%

Paired comparison. We compare Bare and Aligner on identical families for each metrics. HB completion is higher with the Aligner in all ten archived paired comparisons. CM safety improves or remains unchanged in eight of ten configurations. The table below reports each configuration and its confidence interval.

ConfigurationHB Bare→Aligner %Δ pp [95% CI]CM Bare→Aligner %Δ pp [95% CI]
GPT-5.528.7%→57.8%+29.1
254˙,328˙25\.4, 32\.8

|99.5%→99.5%|+0.0

−05˙,06˙-0\.5, 0\.6

| |Gemini-3.5-Flash|36.6%→58.9%|+22.3

187˙,258˙18\.7, 25\.8

|91.2%→95.9%|+4.7

28˙,67˙2\.8, 6\.7

| |DeepSeek-V4-Flash|45.0%→63.0%|+17.9

147˙,211˙14\.7, 21\.1

|82.2%→97.0%|+14.8

124˙,173˙12\.4, 17\.3

| |GLM-5|36.8%→52.6%|+15.8

129˙,188˙12\.9, 18\.8

|91.7%→97.0%|+5.3

34˙,72˙3\.4, 7\.2

| |Qwen3.8-27B|37.1%→60.7%|+23.6

200˙,272˙20\.0, 27\.2

|97.4%→98.8%|+1.4

03˙,25˙0\.3, 2\.5

| |Qwen397|23.7%→55.6%|+31.9

286˙,352˙28\.6, 35\.2

|87.2%→97.0%|+9.8

77˙,120˙7\.7, 12\.0

| |Nemotron-Nano-9B-v2|38.5%→50.8%|+12.3

93˙,154˙9\.3, 15\.4

|83.5%→94.1%|+10.5

80˙,132˙8\.0, 13\.2

| |SmolLM3-3B|4.6%→10.6%|+6.0

37˙,84˙3\.7, 8\.4

|78.7%→87.1%|+8.4

54˙,116˙5\.4, 11\.6

| |OpenClaw|36.6%→50.3%|+13.7

99˙,175˙9\.9, 17\.5

|99.5%→98.8%|-0.7

−15˙,00˙-1\.5, 0\.0

| |Kimi-K3|58.7%→68.4%|+9.7

67˙,128˙6\.7, 12\.8

|99.5%→99.3%|-0.2

−09˙,05˙-0\.9, 0\.5

|

Comments on the paper

2 comments
AuthorsOpenReview ↗

A heartfelt general comment from the authors

Dear Reviewers and Area Chairs,

We sincerely thank Reviewers a3Bv, YPir, and 9KQG for their careful reading and constructive feedback. We are encouraged by the recognition of AgentBoundary’s four-way task design, the importance of distinguishing authorized completion from unsafe compliance, and the value of the trained-versus-untrained intervention controls. We have worked through the reviewers’ questions one by one and made every effort to support our responses with concrete experiments and quantitative analyses.

During the rebuttal, we reported matched EB–HB competence diagnostics across all 17 configurations, recomputed judge agreement at the actual pass/fail threshold, and provided paired Bare–Aligner comparisons with confidence intervals. Human–judge agreement reaches 92.94% for HB utility and 90.48% for CM safety. Repeat checks across seven backbones yield 95%–100% agreement on utility and safety pass/fail outcomes. We also extended evaluation to ToolEmu and AgentHarm, where the reported results show improvements across all four tested backbones. In addition, we returned to the original human-annotation archive, reconciled its coverage, disclosed the grouping correction, and clarified the evaluator roles, execution controls, and intervention’s safety trade-offs.

We have invested substantial effort in turning the reviewers’ questions into additional experimental checks and clearer reporting. These additions provide substantive responses to the concerns about competence, evaluator reliability, repeatability, validation coverage, and external evaluation. We sincerely hope that the complete evidence and the care taken in these responses will receive consideration. We remain grateful for any further discussion and would welcome the opportunity to clarify additional questions.

Thank you again for the time and effort you have devoted to our work.

Best regards,

The Authors

AuthorsOpenReview ↗

Author Final Remarks

Dear Reviewers, Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank you for the time and effort devoted to reviewing AgentBoundary. The reviewers’ questions led us to strengthen the experimental record through matched competence controls, threshold-level judge validation, paired uncertainty estimates, repeated-run checks, external evaluations, and a detailed reconciliation of the human annotations. Our responses also clarify the execution procedure, authorization assumptions, evaluator roles, and measured safety trade-offs.

AgentBoundary makes authorized completion and harmful-action blocking jointly assessable within four-way executable task families. The benchmark exposes a practically important weakness in current agent safety, while the Aligner demonstrates that decision-time calibration can improve both sides of this evaluation across multiple configurations. We believe the original contribution and the additional rebuttal evidence together provide a substantive basis for further research on useful and safe tool-using agents.

We have made every effort to respond carefully and constructively, and will carry the additional analyses and reporting clarifications into the revised presentation. We remain available for further questions and sincerely hope that our work and the evidence provided will receive your recognition and support.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.