- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how untrusted repository content can influence reusable permission requests in coding agents such as Claude Code and Codex CLI. It proposes a benchmark and a two-stage attack framework that first induces an overly broad allow-rule request and then, conditional on user approval, reuses that rule to carry out harmful actions without further approval prompts. The evaluation measures the scope of the acquired permissions and verifies their downstream consequences.
Critical strengths
-
I appreciate that the paper develops both a benchmark for evaluating permission escalation and an automated two-stage attack method for this setting. The benchmark provides a structured way to assess the capabilities covered by acquired permissions and their downstream consequences, while the method gives a concrete way to investigate these risks in coding agents.
-
It is a hot topic and emerging security risk.
-
The trace analysis offers useful observations about how harness-specific permission rules and repository workflows influence attack outcomes.
Critical weaknesses
-
I think the paper identifies an interesting problem, but the distinction between attacker-induced over-authorization and the expected dual-use risk of an approved rule needs more discussion. If a user knowingly approves a broad rule such as
curl *, allowing later matching commands is consistent with that authorization. The more important issue is how the permission request comes to exceed the user's intended task scope, and how this mismatch could be reduced. Since the downstream evaluation is conditional on approval, it does not establish whether users would approve these rules or understand their consequences. I would have liked a more systematic analysis of permission granularity, for example by resource, operation, or task scope, and how these choices affect security and approval burden. This could provide more actionable insight for designing safer permission mechanisms. -
I am not yet fully convinced by the additional contribution over existing permission-gate studies. AmPermBench [1] already studies authorization-scope failures in Claude Code's Auto Mode, and a public security report [2] has demonstrated harmful execution through apparently benign workflows despite the permission classifier. The specific contribution here appears to be inducing a reusable allow-rule and then exploiting it after approval. Although the related-work section makes this distinction, I would have liked a more direct comparison explaining what additional security insight this setting provides.
-
I find the attack-entry scenario somewhat underspecified. If the benign task takes place in the user's own repository or branch, the attacker's ability to modify the relevant content needs a clearer justification. The threat model assumes adversary-controlled repository content, but I would have liked a more concrete account of how that content enters the user's workflow and which files the attacker can change. For example, an external contributor's branch or a pull request would provide a clearer untrusted entry point. Evaluating the attack under the modification constraints of such a contribution, and explaining how the later repository revision reaches the agent, would make the scenario more practical and clarify which trust boundary is being crossed.
-
From a practical perspective, users may continue to rely on Claude Code or Codex for their convenience even when they know that the permission mechanisms carry risks. I would therefore have liked the paper to provide a clearer path toward making this continued use safer. The current results offer limited actionable guidance on how to reduce unsafe permission expansion while preserving the usefulness of these tools. A concrete method addressing this trade-off would have made the work much more impactful to me, and I would have been more willing to champion it.
[1] Zimo Ji et al. Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode. arXiv:2604.04978, 2026.
[2] Johann Rehberger. Breaking Claude Code Opus 5 Auto Mode. Embrace The Red, 2026.
Questions for authors
How were the target command prefixes selected, and could this selection introduce bias? The evaluation uses familiar command families such as curl, wget, grep, and python. Would agent behavior and attack success change if less common prefixes with comparable capabilities were used? Were the prefixes chosen independently of observed attack performance, or were they tuned for the tested agents' permission-extraction behavior?
Suggestion: I see this as a largely engineering-driven, practical security study, with the vulnerability mainly rooted in the surrounding harness and permission-management design. Given this emphasis, the authors could also consider a security conference, where the practical findings may be a stronger fit.
2 comments
Part 2
W3: Clarification on the attack-entry scenario
Thank you for pointing this out. The task does not take place in the user's own private repository: the adversary publishes a repository containing legitimate functionality on a public platform (e.g., GitHub), and the victim downloads it and asks the agent to review/run it.
Revision: We clarify this in Section 2.1 (Lines 140–149).
W4 (W1): Lack of discussion of actionable guidance for defense
We thank the reviewer for this constructive advice. To provide a clearer and more actionable path toward safer continued use of coding agents such as Claude Code and Codex CLI, we evaluate two practical defense mechanisms alongside dicussion of concrete future directions for making these execution harnesses safer:
-
Runtime Drift Evaluation and Re-prompting:
We evaluate a reference-based runtime defense that retains the original command that generated an allow-rule and re-prompts the user whenever a subsequent matching command's lexical drift exceeds a threshold . Across thresholds, this mechanism exposes a direct trade-off between security and approval burden: tightening substantially suppresses silent attack execution at the cost of more frequent benign re-prompts, whereas relaxing reduces user interruptions while allowing more attacks to proceed silently. -
Permission Granularity and Approval Burden:
We systematically compare four rule-scoping policies on attack executions blocked versus benign approval burden relative to the current harness default:Scope Blocked Approvals Burden Prefix (default) - 1.0× Resource 96% 1.5× Operation 62% 1.8× Exact 97% 2.3×
As shown in the table, Resource scoping (binding reusable rules to the approved target file path or network host) achieves the best security–usability trade-off: it blocks 96% of attacks—nearly matching Exact matching's 97%—at only a 1.5× approval burden, whereas Operation scoping blocks fewer attacks (62%) at higher cost (1.8×) and Exact matching more than doubles approval prompts (2.3×). This works because benign workflows repeatedly access the same repository paths and trusted endpoints, whereas downstream exploits pivot to distinct sensitive resources (e.g., ~/.ssh/id_ed25519 or external sinks).
Overall, scoping reusable allow-rules to the approved resource provides a practical way to defend against cross-resource privilege escalation without sacrificing usability.
- Future Work for Safer Harnesses:
Beyond runtime drift checks and resource-scoped rules, we outline two directions to harden the interactive approval loop itself: (i) replacing fixed syntactic or history-based heuristics with a learned rule-proposer model (trained via SFT and RL to balance task sufficiency, low approval burden, and narrow capability scope), and (ii) pairing each proposed rule with a natural-language scope explanation clarifying the resources, operations, and downstream capabilities it grants. controlled HCI user studies are needed to test human understanding.
Revision: We have already added these defenses, the granularity comparison, and the future-work discussion as Section 6 (along with additional drift sensitivity analysis in Appendix D.1) in the revised paper. Please refer to Section 6 of the paper for the full detailed discussion.
Q:How were the target command prefixes selected?
The target prefixes (Appendix A.3, Table 6) were selected a priori based on their familiarity in routine developer workflows and their functional alignment with the five adversarial goals (curl, wget, grep, unlink, sed, and python), rather than being tuned on observed attack performance.
Under our threat model (Section 2.1), the adversary controls the repository context and constructs an artificial task need to induce a desired capability; because the attacker chooses which tool to embed, targeting familiar commands reflects realistic attacker behavior rather than evaluation bias.
However, if less common or non-preferred prefixes were used instead, attack success would likely be lower because agent behavior depends on learned tool-use priors and safety alignment (e.g., an agent trained to favor rm over unlink, or to avoid unfamiliar binaries, is harder to steer toward the less preferred command)—which is precisely why a rational attacker targets standard, high-prior tools.
Part 1
We thank the reviewer for the valuable and constructive comments. We provide our point-by-point responses below.
W1: Attacker-Induced Over-Authorization vs. Dual-Use Risk, and Conditional Approval
Attacker-induced scope mismatch vs. natural dual-use: Thank you for this distinction. We would like to clarify that the rule itself is attacker-induced rather than needed by the user's legitimate task in our case. We do not study broadening permissions that a benign workflow already requires. Instead,
Advintroduces an artificial dependency (e.g., a missing fixture) and shapes the command to trigger the harness's generalization heuristic. In our paper, the Benign Control baseline show 0 reusable rules were elicited across every model, confirming these permissions are never organically proposed by the normal task. Thus, both the rule's creation and its over-broad scope are attacker-chosen rather than a natural dual-use need of the user.Conditional evaluation & user comprehension: We agree this boundary should be stated explicitly. Our evaluation intentionally isolates the system exposure conditional on approval. This is motivated by Anthropic's study showing severe approval fatigue (86.4% approval of dangerous commands, rejection falling to ~5% after 50+ prompts), while noting that empirically studying users' approval decisions and scope understanding remains an important complementary direction for future HCI research.
Revision: We have updated Section 2.1 (Lines 157–164) in the revised manuscript to clarify the "conditional on approval" assumption.
W2: Contribution Over
AmPermBench[1] and the Public Security Report [2]We thank the reviewer and agree the comparison should be explicit.
In short: [1] and [2] show that a classifier can misjudge an action (target component: the per-action classifier). We show that untrusted context can steer the agent into proposing an overly broad, unnecessary reusable rule, which, once approved, authorizes later harmful actions without any further check (target component: the rule-proposal llm module).
We also stress that inducing such a rule is non-trivial. Unmodified tasks never elicit the target rules (0/25 on all four configurations), and explicitly asking the agent to request the rule mostly fails to compete with our method. The attacker must make a broad rule look like a natural need of the task and match how each harness generalizes rules (as shown in Table 2).
Additional insights this setting provides:
The two lines of work cover complementary parts of the permission design space.