acceptodds
Under review as a conference paper at ICLR 2027

One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents

Abstract

The adoption of AI agents is increasing rapidly. Terminal AI agents, i.e., AI agents that run in terminal environments, are widely used. Terminal AI agents rely heavily on shell commands to interact with host systems. They adopt denylists to mitigate the security risks introduced by executing arbitrary commands, which serve as the last line of defense against dangerous and harmful operations. However, denylists can be fragile, as the same goal (e.g., performing a harmful operation) can often be achieved by many commands. This paper presents the first systematic characterization of command denylist fragility in terminal AI agents. The paper formalizes the command denylist fragility problem and proposes an LLM-driven pipeline, ShellSieve, to detect such fragility. It prompts the LLM to propose possible bypasses and iteratively repairs them using feedback from a validator that executes them in a sandbox. In the evaluation, we applied ShellSieve to 4,668 real-world command denylists (containing 36,846 denylist rules) collected from GitHub. The evaluation shows several key findings, including that 69.5-99.8% of the denylists are fragile, that this fragility occurs consistently across projects and agents, and that one cause of this fragility stems from denylist authors overlooking less-known commands. Our pipeline and findings will hopefully facilitate future research and practice regarding the command denylists used by AI agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Agent, Terminal Agent, Agent Security, Access Control, Denylist, Shell

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jnPhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies if command denylists used by terminal AI agents miss alternative commands that would perform the same file system operations. It introduces a system that uses an LLM that is given command documentation to suggest operation-command pairs, checks the effects through sandbox execution, and repairs unsuccessful candidates with execution feedback. The authors collect denylists with rules from GitHub configurations for three agents. For several file system operation types, they report operation-specific incompletely blocked rates. The paper studies differences in agents and repository popularity and studies explanations that include command popularity and versatility. It contributes an empirical characterisation of the limitations of command-based restrictions.

Critical strengths

The inconsistency between command names and the effects from commands is important for agents that execute shell commands. Studying thousands of configurations could help in finding how that problem behaves. I would regard a well-supported characterisation as a useful contribution, even without a new defensive mechanism.

Also, ShellSieve uses the LLM to explore possibilities and execution-based oracles to check the suggested effects. The canary-based checks in their appendix and the feedback-given repair loop should give a great basis for validating individual examples. This seems like a promising design for a systematic way of looking into heterogeneous command capabilities.

Critical weaknesses

There seem to be some central quantitative results that have inconsistencies, which make it less possible to assess their magnitude reliably. Section 5.2 states that 2,718 denialists cover at least one studied operation, while Section 5.3.3 states that 3,091 denial lists incompletely block at least one operation. With their definitions: the latter should be a subset of the former. Section 5.3.3 also reports 214 distinct bypass commands overall, while Table 3 reports an average of 334 and a maximum of 346 distinct read bypass commands per analyst. These cannot use the same counting unit and population as currently described. Are these reporting errors?

It also seems that a command that escapes a denylist may still require approval or be stopped by another control (figure 2 includes that distinction). Appendix A describes another difference in that its auto mode example concerns commands that should be routed to a classifier and not commands that must always be denied. Meaningful findings, but they should be distinguished from successful unauthorised execution through a deployed agent's complete policy.

Questions for authors
  1. Can you explain the issue with 2,718 versus 3,091 denial lists and 213 distinct bypass commands versus Table 3's read counts? It would be great if you could please specify the counting units and populations and give per-operation numerators and denominators.

  2. For a smaller representative sample, could you show the original rule and protected operation with its specific arguments, the validated alternative command, and the actual agent decision: deny, request approval/classify evaluation, or allow? For Appendix A, what agent version and permission configuration were used? This would help make clear what a gap is in the policy component from a complete approval bypass.

  3. How do you check that an alternative command performs the same restricted action on the same protected files while getting past the agent's denylist? Execution on one substituted path establishes that instance, but does not by itself establish all generalised instances.

Reviewer_dFZiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to mitigate agent security risks, by proposing ShellSieve, an LLM-driven pipeline to detect security issues by iteratively prompt LMs to propose bypasses and fix them via execution feedback. The paper shows extensive fragility across projects and agents, to facilitate future ai agent development.

Critical strengths
  1. Complete and informative experimental design: This paper not only proposes the method to block unsafe behaviors, but also conducts additional studies revealing how this method reacts to different cases and the root cause of issues, all of which provide useful insights for future research.
Critical weaknesses
  1. Lack of baseline approaches and technical novelty: the proposed method leverages execution-based feedback, and ask an LM to process it. Long from coding LM/agents, this is a standard approach, so (i) it seems like a lack of baselines that represent the approaches of solving the target problem, and (ii) lack/lack of discussion of the technical novelty of the proposed approach.
Questions for authors
  1. The motivation (in abstract and intro) frame the problem that “the same goal can be achieved by many commands”. But the proposed method (section 3) only processes one command, and not looking at the relationship across commands. Is there a mismatch between?
Reviewer_PfLGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper measures how often the command denylists used by terminal AI agents (Claude Code, Codex, OpenCode) miss other commands that achieve the same operation. It formalizes denylists, operations, and incomplete blocking, and proposes SHELLSIEVE: an LLM proposes which file-system operations (read, write, create, delete, chmod, chown) commands and deny rules perform, each candidate is validated by execution in a sandbox with canary oracles, and failures go through an LLM repair loop. On 4,668 GitHub denylists (36,846 rules), the paper reports that 69.5% to 99.8% incompletely block the operations they target, consistently across star tiers and agents; that less popular commands are more likely to be bypasses, while versatile commands and a semantic gap do not explain the fragility; and that fully blocking op_read would require 334 additional commands on average.

Critical strengths

S1. The problem is timely, and the measurement uses real configurations: 4,668 denylists (36,846 rules) from Claude Code, Codex, and OpenCode settings files. Every reported bypass is execution-validated with canary or flag-file oracles (Appendix E), so the bypass sets are not LLM claims, and the concrete cases (cp .env /dev/stdout against Listing 1, AWK against Claude Code's auto-mode list) are convincing and were responsibly disclosed. As W1 argues, however, this validates that a binary can perform an operation on the reference host, not that the agent would execute it.

S2. The versatile-command analysis carries a useful practical message. Removing all 82 interpreters, shells, and wrappers still leaves 68.2% to 99.8% of denylists incomplete, because nearly every denylist they bypass is also bypassed by a single-purpose utility (lines 415-421). Listing every interpreter would not fix a denylist, which, together with Table 3, suggests that command-name denylists are the wrong abstraction for file-level goals. W2 explains why this does not amount to a statistical rejection of H2.

Critical weaknesses

W1. The measured quantity is not agent exposure, and this is the main reason for my rating. What is measured is whether some binary, run alone in the validator, performs an operation the LLM attributes to the denylist without matching its command patterns. Three things are left out. First, the rest of the gate: a command that misses the denylist still faces the allowlist, the ask stage, and optional sandboxing, all present in the collected settings files but unused; cp .env /dev/stdout against Listing 1 triggers a permission prompt unless an allow rule matches it, and even the AWK case escapes the auto-mode classifier only if the user allowlists awk. Second, path-level rules: Claude Code's documentation (Anthropic, 2026b) states that a Read deny rule also stops shell commands that name the file, so a Read(./.env) rule next to Listing 1 may deny the counted bypass. Third, the targeted operation is inferred by the LLM and generalized as much as possible (Figure 6), so an rm -rf rule plausibly makes any file removal or rename, including mv (Table 2), a bypass, and op_delete dominates Figure 3 without any human check of author intent. These gaps cut against LLM recall, so the lower-bound claim in §6 does not hold.

W2. The root-cause and repair-burden analyses do not support their conclusions. For H1, the outcome is bypassing any denylist, which nearly every capable command satisfies (Table 6: 97.9% for op_read), so Figure 5 mostly tracks capability against popularity; popcon also ranks packages, not commands, and the p-values ignore that clustering. For H2, IBR only records whether one bypass exists, so excluding versatile commands has little power to move it; for H3, commands whose TLDR omits an operation mostly cannot perform it, so 88.7% versus 1.9% is expected without any semantic gap. Several counts also do not reconcile: 147 op_read bypass commands and 214 in total (lines 403 and 420) versus 307 to 827 in Table 3 (lines 461-463), and 2,718 denylists covering an operation (line 335) versus 3,091 incompletely blocking one (line 419). Finally, RQ4 prices a repair no defender should use, since path-level rules or sandbox policies close the op_read gap with a constant number of entries.

Other issues (not decisive): related work omits classic denylist incompleteness (Saltzer and Schroeder, 1975), living-off-the-land measurement (Barr-Smith et al., IEEE S&P 2021), and agent policy enforcement (e.g., CaMeL, Progent); the LLM component has no recall estimate or ablations; RQ2 rests on small cells (11 repositories with at least 1,000 stars) without confidence intervals; and there are typos (e.g., "conduct Appx. F", line 336; "codespanstdout", §4.4).

Questions for authors

Q1. Can you run a stratified sample of denylist-bypass pairs through the actual agents with each repository's full settings (allow, ask, mode, sandbox, and path-level rules), and report how many execute, how many are prompted, and how many are blocked? What fraction of the collected settings files would auto-approve the bypasses?

Q2. Can you redo the H1 to H3 analyses on (denylist, command) pairs restricted to commands validated to perform the operation, using count-based outcomes, a command-level familiarity measure, and package-clustered standard errors? Do the conclusions on H1, H2, and H3 still hold?

Q3. Please reconcile the 147 and 214 bypass commands in §5.3.3 with Table 3 (307 to 827), and the 2,718 denylists in §5.2 with the 3,091 in §5.3.3. What unit does Table 3 count? In §4.5, is blocking checked per denylist, or are host commands first filtered to those not blocked by any denylist in the dataset?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.