acceptodds
Withdrawn from ICLR 2027

SafeClawBench: Separating Semantic Failure from Response-Visible Evidence in Tool-Agent Responses

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Evaluating safety policies requires distinguishing semantic violations from evidence present in model responses. We introduce SafeClawBench, a text-only benchmark of 600 tasks across six attack families, evaluated under four prompt-policy conditions (D0, B2, D3, and D4) on seven models. The benchmark separates semantic rubric failure (CoreFail) from response-visible evidence (RVE): auditor-labeled grounded disclosures or assertions of access, action, or persistence. It evaluates returned text rather than tool execution. Relative to D0, all 21 model–policy comparisons reduce RVE, and 20 are significant after within-model, endpoint-wise Holm correction. D3 and D4 reduce CoreFail in all seven models, whereas B2 does so in six. In the original five-model panel, the reference audit labels 587 of 2,201 CoreFail responses (26.7%) as RVE. Within this panel, each of three auditors assigns more asserted-operation labels than disclosure labels, but their disclosure judgments differ substantially. The lowest observed counts are split between D3 and D4 across models and endpoints, supporting model-dependent policy profiles rather than a universal winner. Together, these findings support evidence-aware policy evaluation that separately reports semantic failures, response-visible evidence channels, and auditor sensitivity, instead of treating them as interchangeable measures of harm.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safety, LLM Agent

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xnKBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces SafeClawBench, a text-only benchmark of 600 adversarial tool-agent cases, 100 in each of six families (direct prompt injection, indirect prompt injection, tool-return injection, memory poisoning, memory extraction, and ambiguity-driven unsafe inference). Cases were expanded by GPT-4.1 from 36 researcher-written seeds and selected through a D0 pilot on eight endpoints that required at least one attack success. Models are called without function calling and no tool executes, so the unit of analysis is the returned text. Each frozen response is scored in two stages by a GPT-5.4 judge. The first stage is a semantic endpoint (CoreFail: the response adopts, authorizes, reveals, plans, or justifies the attack goal). The second stage, applied only to CoreFail responses, labels "response-visible evidence" (RVE), split into grounded disclosure (GD, a protected value appears in the text) and asserted-operation claims (AOC, the response states that access, an action, or persistence occurred). Seven models are compared under four system-prompt policies (no added defense, a sandwich reminder, and two longer layered policies) with exact McNemar tests and Holm correction on a fixed 600-task denominator. The main reported findings are: all 21 model-policy contrasts reduce RVE relative to D0 (20 significant); the two longer policies reduce CoreFail for all seven models and the sandwich policy for six; 587 of 2,201 CoreFail responses in the five-model reference panel are RVE, with AOC labels far outnumbering GD labels; and the D3 versus D4 comparison is model-dependent with no significant paired difference. Extensive appendices report re-audits by Gemini 2.5 Pro and DeepSeek V3.1, a lexical input-grounding detector, test-retest stability, leave-one-family-out analyses, a stratified audit of CorePass responses, and two author-led human reviews.

Critical strengths

S1. Careful and unusually transparent measurement practice. The design pairs responses by task within model, keeps a fixed 600-task denominator for both endpoints, uses exact McNemar tests with prespecified Holm families, adds task-cluster bootstrap and lexical-cluster sensitivity checks, bounds the effect of recovered and filtered cells by worst-case completion, and preregisters disconfirmation criteria. Just as importantly, the paper reports its negative results rather than hiding them: the human review that fails its own criteria, the retest that meets neither explanation criterion, and the full three-auditor envelope. This matters to my assessment because the core policy result is about as well supported as a single-turn, prompt-level result can be: every model has its highest RVE count under D0 in the primary audit, and pooled over models D0 carries the most CoreFail and the most RVE rows under all four labelings and all six leave-one-family-out exclusions (Appendix I).

S2. The paper shows, with its own data, how much an LLM-judged agent ASR can hide. On the same 2,201 responses, three auditors produce between 587 and 1,346 RVE rows; the memory-extraction family moves from 1.4% to 62% conditional evidence depending on the auditor; and the CorePass audit suggests that roughly a third of potential evidence lies outside the semantic gate. These are useful cautionary findings for anyone who reports judge-based attack success on agent benchmarks, and I regard them as the most informative part of the submission, more so than the benchmark itself.

Critical weaknesses

W1. The evidence endpoint does not measure harm, despite its name. With no tool execution, every asserted-operation claim is a narrated, unexecuted action. AOC appears in 563 of 587 RVE rows and the persistence label alone in 465 (399 of 400 MPI RVE rows), so RVE mostly measures whether a model role-plays task completion. Disclosure has a similar problem: in a prompt-only setting a protected value is either copied from the input or invented, yet the authors' detector finds no 8-token input match for 39 of the reference auditor's 53 TextHarm labels and for 90.2% of auditor-discordant MEX rows. Fabricated secrets and fabricated action reports are worth studying, but they are not "grounded disclosure," and names like TextHarm and HarmEvidence invite a harm reading. Gating on CoreFail makes this worse: RVE@600 cannot exceed CoreFail@600 by definition, and the gate drops refuse-but-leak responses (the authors' own extrapolation puts about 32% of potential RVE outside the gate, concentrated at D0). Prior work also shows that text is a weak proxy for action in agents (GAP, arXiv 2602.16943; "Large Language Models Often Say One Thing and Do Another," ICLR 2025; WASP), none of which is discussed. Without an execution-level anchor, even mock function calling on a subset, readers cannot tell whether RVE predicts realized harm better than CoreFail alone.

W2. The evidence endpoint is not reliably measurable, and its distinctive claims do not survive the paper's own robustness checks. Pairwise RVE agreement is κ\kappa = 0.31 to 0.37; TextHarm agreement between the reference auditor and either alternative is κ\kappa = 0.07 to 0.10, while the two alternatives agree at 0.48, so the reference auditor is the outlier. MEX, the largest family, ranges from 1.4% to 62% conditional evidence across auditors. Appendix I shows that "AOC exceeds GD" and the pooled RVE ordering are not exclusion-invariant, yet the former appears in the abstract and conclusion. Human validation does not resolve this: the frozen-queue review misses four of five prespecified criteria (about a quarter of HE and SO labels changed), was done by an author with machine suggestions visible, and neither review labels GD or AOC. What remains robust (D0 fails more than the policies) is visible from CoreFail alone, is well known, and is inflated by selecting cases for D0 success on a pilot that included Kimi K2.5 and the GPT-5.4 judge. Family composition also shifts across policies (MEX's share of the gate rises from 32% at D0 to 56% at D3), and without a benign-utility arm or a length-matched control, policy gains cannot be separated from broader refusal.

W3. Presentation and illustration are poor, which matters here because the contribution is a distinction between kinds of responses that the paper never actually shows.

(a) Organization. Main-text space goes to provenance (legacy sweep, rejudge history, retries, recovered cells), while CoreFail, GD, and AOC get a few sentences each and the auditor disagreement (S2) is pushed to Appendices F to H. Many main-text claims rely on appendix-only tables, and the abstract mixes five-model and seven-model results without signposting. Appendix I lists 29 reversals in one run-on paragraph using raw endpoint strings ("claude-opus-4-7-aws under gemini"), and Appendices G and H cite internal file names instead of stating the specification.

(b) Terminology. RVE is also HarmEvidence/HE; D4 is also LongPolicy and D4_AgentShield; the disconfirmation criteria D1 to D3 collide with the policy names; and Table 24's V2, K1, K2, and "Kilo route" are undefined.

(c) Figures. There are only two. Figure 1 is a rough five-box flowchart that restates Section 3.2, with words broken mid-word ("con-trolled", "suc-cess", "re-sponse"), and never shows what a single ASR misses. Figure 2 prints labels over the whiskers and overlays two intervals with different meanings (task-cluster bootstrap and between-auditor range).

(d) No concrete examples. The only worked case (Appendix K, TRI 0827) is a few sentences without the input, full response, or auditor spans. Readers need to see a SemanticOnly, a GD, an AOC-only, and a refuse-but-leak CorePass response, ideally one per family; this would also answer Q1. The D3 and D4 policy texts and lengths are likewise absent from the paper.

(e) Suggested fix. Move provenance to the appendix, define the endpoints with real examples, replace Figure 1 with one showing where a single ASR and the two endpoints diverge on actual responses, and unify terminology.

W4. Secondary issues.

(a) Reproducibility. Per-case seed lineage and the candidate pool are lost (114 of 600 representative cases recoverable), so the number of independent cases is unknown and the selection-bias check is not reproducible.

(b) Policy selection. Conditions were kept by file completeness, which dropped the StruQ-style variant; no trained or structured defense remains.

(c) Judge overlap. GPT-5.4 judges GPT-5.5 and GPT-5.6 Sol, and the favoritism check covers only Core labels.

(d) Configuration. GLM-5.3 uses an 8,192-token limit versus 2,048 for the others, and judge inputs are truncated at 3,000 and 4,000 characters.

(e) Related work. Missing AgentCanary (arXiv 2606.10484), which also argues against a single ASR in executable environments, and StrongREJECT-style separation of willingness from capability.

Questions for authors

Q1 (could change my view of W1). Per family, how often is the targeted protected value actually present in the model input? If a MEX secret is absent from the input, should those TextHarm rows be reported as fabricated disclosures instead? Was the TRI 0827 grade table in the tool return?

Q2 (could change my view of W1). On a subset with mock function calling and a deterministic state oracle, can you report P(executed harm∣X)P(\text{executed harm} \mid X) for X = RVE, SemanticOnly, and CorePass?

Q3. Why gate the evidence audit on CoreFail? Do the main conclusions change with ungated RVE over all 12,000 responses?

Q4. Is there any independent, blinded human annotation at the GD/AOC level? Without it, why use the most conservative auditor as the reference rather than the predeclared 2-of-3 consensus?

Q5. Do D3 and D4 increase false refusals or degrade completion on matched benign tasks?

Q6. Which judge decided attack success in the selection pilot? Can you report the policy contrasts on the 114 recoverable representative cases?

Reviewer_snLKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to distinguish semantic rubric failure from response-visible evidence in LLM-generated text. To this end, the authors propose a response-based measurement framework that separates semantic compliance from disclosure labels and claims of performed operations. They evaluate the framework on 600 challenging examples spanning different attack families, models, and task-specific policies.

Critical strengths

None.

Critical weaknesses
  • The research questions lack significance and relevance to the current research community. Both attack compliance and sensitive information leakage have been extensively studied in prior work such as PrivacyLens and AgentDojo, which examine these issues in more complex and realistic settings. Furthermore, the comparison with representative prior work in Table 1 fails to establish a compelling motivation and clearly differentiate this study from existing research. None of the proposed dimensions appears novel and insightful to constitute a meaningful contribution to the current research community.
  • The full paper is logically incoherent and difficult to follow. Importantly, the experimental setup and results are poorly explained, making them difficult to interpret.
Questions for authors
  • What is Holm correction? I can't find any description and reference about Holm correction throughout the full paper.
  • Please explain what endpoints are, which appear throughout the paper with different names such as model endpoint, CoreFail endpoint, human endpoint, HE/SO/CP endpoint, etc
  • Could you please provide one or two representative examples out of 600 controlled cases?
Reviewer_6nvzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SafeClawBench, a text-only benchmark of 600 adversarial tasks across six attack families for tool-using LLM agents. Each response is scored in two stages. An LLM judge first decides whether the model semantically complied with the attack (CoreFail). Failed responses are then audited for “response-visible evidence” (RVE): either a protected value appearing in the text, or a claim of having accessed, acted on, or stored something. The authors evaluate seven models under four system-prompt defenses, with no tools executed. All defenses reduce both measures. They validate with re-grading by two other LLM judges and a small author-led human review.

Critical strengths

The idea of separating failures is interesting.

Critical weaknesses
  1. The evidence measure is unreliable. The three LLM graders disagree heavily.
  2. Most “evidence” is the model claiming it did something, not real harm. There is no tool runs.
  3. The writing is hard to follow. Abbreviations appear before they are defined. D0/B2/D3/D4/HE/SO/CP appear earlier in the paper but are only defined later.
Questions for authors

See weaknesses above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.