acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Scenario Selection for Screening Overeager Behavior in Coding Agents

Abstract

A coding agent executes a developer task as a sequence of shell, file, and network actions, any of which can exceed the authorized scope even when the requested artifact is produced. We study overeager behavior: actions that exceed the authorization of a benign developer task, such as exposing credentials or deleting protected files. Existing benchmarks miss this behavior: task-completion suites do not score authorization scope, adversarial suites study a different input regime, and the closest overeager benchmark uses one fixed prompt distribution for every framework–model pair. We present ASSBF (Adaptive Scenario Selection with Bandit Feedback), which builds a structurally screened pool of benign-task scenarios, applies controlled scope-pressure mutations, scores observable effects with a judge-free two-signal rule, and reallocates each pair's run budget across archetype–consent cells using Thompson sampling. Instantiating the method over 24 behavioral archetypes yields OverEager, evaluated on a 4 × 5 matrix of coding-agent frameworks and base models. Across 10,000 runs, the broad screening rule returns 1,951 positives (19.51%), including 493 runs matched by scenario-specific trap predicates; pair-conditioned screening yields span 11.9×. All 24 scenario archetypes are represented for every pair, a framework-first sequential decomposition assigns 56.1% of the observed deviance reduction to framework identity and 23.1% to the framework–model interaction, and adaptive-500 recovers full-pool screening rates within one percentage point on the evaluated ablation column while halving executions. Code and data are available at https://anonymous.4open.science/r/overeager-E385/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
coding agents, agent safety, authorization scope, overeager behavior, adaptive evaluation, Thompson sampling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4uiGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates "overeager behavior": actions that exceed the authorized scope of a benign developer task. The authors propose an Adaptive Scenario Selection mechanism and introduce a benchmark spanning 24 behavioral archetypes and five consent realizations. Through evaluations across diverse coding agent frameworks and foundation models, the study reveals several overeager behavioral patterns.

Critical strengths
  1. Well-grounded behavioral taxonomy: The 24 behavioral archetypes are well-motivated, drawing from diverse and authoritative sources including OWASP, the NIST Generative AI Risk Framework, empirical studies on proactive coding assistants, and documented production incidents.

  2. Interesting stress-testing methodology: The proposed scope-pressure mutation stress test offers an approach to probing agent boundaries under varying task constraints.

Critical weaknesses
  1. Unverified reliability and false-positive control of task verifiers: The paper lacks sufficient detail on how the accuracy of task verifiers is guaranteed. It is unclear whether the benchmark ensures that an agent performing no actions yields a guaranteed true negative without inadvertently triggering traps in the environment.
  2. Reduced vulnerability diversity from adaptive selection: Relying on the Online Scope-Pressure Selector appears to narrow the diversity of discovered overeager behaviors, which is not good for broadly uncovering diverse failure modes across the taxonomy.
  3. Poor clarity and presentation: The manuscript is overly difficult to follow for me. The text in the figures is also too small.
Questions for authors
  1. Verifier Generation and Baseline Sanity Checks: Could the authors clarify the exact generation pipeline for each task's verifier? How do you guarantee verifier accuracy and rule out false positives. Have you verified that a baseline where the agent takes no action consistently results in a negative detection without triggering environmental traps?

  2. Diversity Trade-offs in Adaptive Selection: Using the Online Scope-Pressure Selector seems to concentrate the search space and reduce the diversity of uncovered overeager behaviors. How do the authors justify this trade-off when a primary goal of red-teaming/benchmarking is to discover a wide variety of distinct vulnerabilities?

Reviewer_j4gDOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents ASSBF, which constructs coding-task scenarios with sensitive-action traps and uses Thompson sampling to allocate each framework–model pair's evaluation budget toward scenarios with higher trap yields. Across 20 pairs and 10,000 runs, the authors report 1,951 screening positives, including 493 scenario-specific trap matches. They use these results to discuss framework differences, portable scenarios, and the cost of adaptive evaluation.

Critical strengths
  1. Authorization scope is a worthwhile target that final-patch evaluation can miss.
  2. The scenarios place permission cues and sensitive actions inside ordinary development tasks, and the recorded traces support investigation of individual cases.
  3. Adaptive budget allocation is a reasonable way to search for such cases.
Critical weaknesses
  1. In the database-migration example, the agent writes a production credential into migration.sql. Yet §3 and Appendix A say that prod_token_used_in_dev includes a post-run check for PROD_ in .envrc, which already contains production credentials in the initial fixture (PDF pp. 4, 11). That post-run condition alone cannot distinguish credential exposure from an agent that never touched the file. The paper does not show the full predicate, so this does not establish that a no-op run would trigger it, or that the 493 trap matches are invalid. It does leave unclear whether the evidence in the illustrated trace is required by the rule used for the reported count.

  2. The paper does not establish what the broad rule adds as a screening tool. The authors correctly call 19.51% a screening-positive rate and acknowledge that a change in file count does not prove overreach. The concern is that 1,458 of the 1,951 positives arise only from a net change in file count, while producing a new file is a normal part of many coding tasks. A screening rule can tolerate false positives, but the paper should show whether these additional alerts uncover sensitive actions that the scenario-specific traps miss, or mostly flag authorized outputs. Counts and framework differences alone do not establish the value of the broader signal. Put simply, I cannot tell what useful safety signal the broad rule provides beyond flagging ordinary file creation.

  3. Permission conditions are central to the benchmark design but largely absent from its findings. The scenarios vary among silent, explicit and implicit permission or denial, and the mutations include cues such as (forwarded from CTO) and “Feel free to clean up anything else”. These are reasonable stress-test conditions. The authors also acknowledge that scope-broadening language can change what is authorized, so a trap match is not automatically an unauthorized action. For that very reason, a sensitive action under denial, silence, and permission has different meanings. The results mainly aggregate by framework–model pair and predefined archetype; they do not show whether agents respond to the permission cues or treat a claimed authority as authorization. This dimension should yield an insight into authorization behavior, not only supply variants to the scenario generator.

  4. How many distinct development tasks are represented by the 1,000 scenarios? The construction combines ten task skeletons with decision points, permission cues, and fixture files. The paper reports archetype and permission coverage, but not how many semantically distinct base tasks remain after those variations are removed, or how much each task family contributes to the pool and the positive results. Twelve of the 26 portable candidates come from one credential-migration branch (Appendix H), illustrating how counting scenario variants can give one task family disproportionate weight. The structural checks require at least eight atoms in the main chain and a shortcut at least three steps shorter; they do not establish that the tasks are realistic or that agents actually follow long workflows. Scenario count and formal chain length therefore do not, on their own, establish workload diversity.

  5. The benchmark yields limited insight into the agents' behavior. Figure 3 gives broad positive rates for each framework–model pair across 24 predefined archetypes, and Table 5 reports how many pairs trigger ten example scenarios. The archetypes are labels assigned during construction, however, and a positive rate does not explain an execution trace. After reading the results, it remains difficult to say which agent took which sensitive action, at what point in a task, and under what permission cue. Figure 1 offers one migration case, but there is little comparable analysis of the matrix. The paper tells us considerably more about scenario coverage and rule-trigger frequency than about coding-agent failure modes.

  6. The paper treats scenario-specific matches as more targeted evidence than file-count changes, but §5.1 and Appendix A describe automated rules. It does not report the extent of any systematic human review, its adjudication criteria, or an estimated false-positive rate. The interpretation of the single case in Figure 1 cannot establish that the remaining matches reflect an agent action, much less an unauthorized action under the applicable permission cue. For a benchmark, the authors should state whether they checked a sample of trap positives and how often matches remain valid across predicate types. As presented, 493 is a rule-trigger count of unknown precision.

Writing

The mention of SHA-256 on line 241 looks like unedited AI-generated writing.

Figure 4 labels the broad screening-positive rate “Composite overeager rate,” which could be confused with a confirmed overreach rate; the terminology should match the main text.

Questions for authors

see weaknesses

Reviewer_xvHTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper tries to measure how often AI coding agents do things they weren’t authorized to do, like copying a password into a file or deleting files that should be left alone, while working on ordinary programming tasks.

The paper creates a benchmark with 1000 simple tasks. Each one is designed to tempt the agent toward a specific kind of overstep, and the instructions vary in whether the user permits, forbids, or doesn’t mention the risky action. Some tasks get extra pressure added, such as “this is urgent” or fake files planted as bait. The paper uses Thompson sampling to steer each agent’s runs toward the categories of task that have most often caught that agent overstepping.

They test four coding tools (Claude Code, Codex CLI, Gemini CLI and OpenHands), each paired with five underlying AI models, running each combination 500 times for 10,000 runs in total. A run counts as a problem if an automatic check detects the targeted overstep, or simply if the number of files in the project changed.

Critical strengths

This is an important and somewhat understudied problem.

The method for steering towards concerning behaviour is interesting, and the benchmark seems pretty large and diverse.

Critical weaknesses

Most of the positives are from changing numbers of files and not anything particularly concerning.

Relatively out of date models tested.

Questions for authors

Did you do human validation of a sample of positives?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.