acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Stress Testing of Behavioral Invariants in LLM Agents: A Study of Dynamic Authorization

Abstract

Safety evaluations of large language model (LLM) agents often summarize performance through violation rates, which alone do not explain how failures arise or whether legitimate tasks remain achievable. We ask whether agents voluntarily recheck authorization when permission becomes invalid between planning and execution, and how their behavior differs when current evidence is supplied. We develop counterfactual invariant stress testing, a paired evaluation protocol that expresses behavioral invariants as executable safety constraints. Five matched conditions vary authorization state, evidence availability, and reread requirements; replayable tool traces distinguish evidence acquisition, unauthorized effects, and task outcomes. We evaluate five models on 30 human-reviewed synthetic tasks. GPT-5.4 completes 98.9% of stable-authority trials, yet commits unauthorized actions in 44.4% of silent-revocation trials; when authenticated revocation evidence is supplied, it achieves successful safe recovery in every evaluated trial. Across endpoints, all observed silent-revocation violations occur without a fresh pre-commit read. Supplying revocation evidence yields no observed unauthorized effects, but does not uniformly improve safe recovery. These results illustrate that successful behavior with supplied evidence does not establish reliable voluntary revalidation, and that absence of violations does not establish successful task handling. By making these distinctions observable, the protocol supports evaluating authorization failures alongside task utility rather than treating violation rates as a sufficient account of agent safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, behavioral invariants, dynamic authorization, trace-based auditing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ecs3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper evaluates whether LLM agents voluntarily recheck authorization when permission changes between preparation and an irreversible action. Its five matched conditions distinguish stable authority, silent revocation, supplied revocation evidence, and explicit requirements to reread authorization. Deterministic tool traces separately measure evidence acquisition, unauthorized effects, successful task handling, and execution failures. The evaluation covers five model endpoints, 30 human-reviewed synthetic tasks, and three repetitions per condition. GPT-5.4 completes 89/90 stable-authority trials but produces unauthorized effects in 40/90 silent-revocation trials; supplied revocation evidence yields successful safe recovery in 90/90 trials. Across endpoints, all 94 observed silent-revocation violations occur without a fresh read, although the paper correctly treats this as an association rather than proof of a causal effect.

Critical strengths
  1. The experimental design separates meaningfully different authorization behaviors. The five conditions distinguish voluntarily acquiring current evidence, using supplied evidence, following an explicit reread instruction, and completing legitimate tasks. The GPT results demonstrate why these distinctions matter: high stable-task competence and successful use of revealed evidence coexist with frequent failures to revalidate voluntarily. This provides a useful diagnostic beyond aggregate violation rates.

  2. The evaluation carefully distinguishes safety from execution failure. Invalid calls, abandonment, and exhausted budgets do not receive successful-recovery credit, and the appendix reports their concrete causes. Task-level bootstrapping also respects the distinction between 30 independent tasks and repeated trajectories. This reporting materially strengthens confidence in the central findings and prevents unreliable endpoints from appearing safe merely because they fail to act.

Critical weaknesses
  1. Scaffold compatibility substantially affects some behavioral comparisons. The one-tool-call rule accounts for 141 of Claude’s 173 invalid executions, while 64 of Kimi’s 77 unsuccessful stable-required trajectories contain a server-reported length termination under the 512-token response limit. These observations limit how confidently recovery differences and required-read utility losses can be attributed to authorization behavior, even though the authors appropriately acknowledge the confounding. Interface and response-budget sensitivity controls would strengthen the interpretation.

  2. The evidence covers a narrow model of dynamic authorization. The benchmark contains one synthetic task per mechanism–effect cell, uses an exact trusted ledger, and schedules a single atomic authorization change after preparation; no further revocation occurs between a fresh read and commit. "Safe recovery" also means grounded denial, without requiring cleanup or an alternative authorized workflow. These choices provide strong experimental control but limit conclusions about ongoing authorization changes and realistic recovery, so broader workflow validation would improve the contribution’s practical reach.

Questions for authors
  1. Was the one-tool-call restriction enforced through provider API settings or checked only after generation? Given Claude’s 141 multiple-call failures, this distinction materially affects the interpretation of its results.

  2. Do the saved traces show how many revealed/required-condition failures occur after receiving current evidence, versus before reaching the relevant transition? Separating these would sharpen the distinction between evidence-use failures and workflow failures.

  3. For four-step preparation tasks, how much of the eight-request budget remains for revalidation, committing, and recording the final decision? Please clarify whether the reference-policy validation also checks this execution-budget margin.

Reviewer_69kMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a controlled evaluation protocol for studying how LLM agents handle authorization changes between preparation and execution. Five matched conditions distinguish voluntary acquisition of current authorization evidence, use of supplied evidence, compliance with explicit rechecking instructions, and legitimate task utility. The authors evaluate five model endpoints on 30 synthetic tasks using executable traces and outcome-based scoring. The results show that successful behavior with supplied evidence does not imply reliable voluntary revalidation, and that the absence of unauthorized effects does not necessarily indicate successful task handling.

Critical strengths

The evaluation clearly distinguishes different aspects of agent behavior under changing authorization. The five controlled conditions separate voluntary acquisition of current evidence, use of supplied evidence, compliance with explicit rechecking instructions, and legitimate task execution. In particular, the protocol empirically evaluates the distinction between acting correctly when current evidence is provided and independently obtaining that evidence.

The study pays careful attention to experimental control and statistical analysis. For matched action histories, the stable and silently revoked conditions provide identical observations until a fresh read, while evidence provision and procedural instructions are manipulated separately. Repeated runs are aggregated and resampled at the task level rather than treated as independent tasks, and human reviewers validate tasks before observing model outcomes. The paper also appropriately limits its interpretation of small mechanism-level samples and internal model mechanisms.

Critical weaknesses

The incremental novelty over prior work is not sufficiently clear. The prior studies [1, 2]discussed in the paper already address authorization becoming invalid over time, provision of current authorization information, and voluntary versus forced verification of evidence. The distinguishing contribution appears to be combining these elements into a five-condition protocol for rechecking action-specific authorization. However, the paper needs to explain more clearly what new diagnoses or insights this combination enables that existing evaluations cannot readily provide.

The practical importance of the evaluation target is not sufficiently established. The experiments assume a trusted ledger that can verify exact authorization status and action arguments, while removing mandatory authorization checks at execution time. In this setting, enforcing authorization checks within the execution tool is a direct way to prevent unauthorized actions. An evaluation paper need not propose a solution, but the authors should explain what additional value measuring voluntary revalidation without this safeguard provides for real-world deployment decisions.

Task success conflates authorization-related capabilities with constraints of the execution environment. Some models have low success rates even under stable authorization and frequently exhibit execution errors or abandonment. In particular, Kimi’s performance degradation under required rechecking is accompanied by records indicating response-length limits. Although the authors acknowledge this issue, the absence of follow-up experiments with larger response budgets or adjusted tool interfaces makes it difficult to distinguish difficulties in processing authorization information from compatibility issues with the evaluation environment.

[1] Igor Santos-Grueiro. Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents. arXiv:2607.10487, 2026. https://arxiv.org/abs/2607.10487

[2] Moonwon Choi et al. ResidualAuth: What Authorization State Must Language Agents Preserve under Revocable Delegation? arXiv:2609.08062, 2026. https://arxiv.org/abs/2609.08062

Questions for authors
  • Compared with ResidualAuth and prior work on voluntary versus forced evidence verification, what specific diagnostic capability is unique to this protocol? Could its additional evaluation value be demonstrated through comparisons with existing approaches on identical or similar tasks?

  • Do the model differences identified by this protocol translate into differences in task success or recovery quality when an execution-time authorization checker is present? Could unauthorized attempts, repeated attempts after blocking, and correct termination be evaluated in that setting?

  • To what extent do stable-task success rates and differences between conditions persist when the response-token budget or tool-use step limit is increased? In particular, does this alleviate Kimi’s substantial performance degradation under required rechecking?

  • If the central contribution is the decomposition of failure mechanisms, why was the primary analysis defined as the difference in violation rates between stable authorization and silent revocation, rather than the effects of supplying evidence or requiring rechecking?

Reviewer_HfSgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the security behavior of large language model (LLM) agents in dynamic authorization environments. It focuses on whether agents proactively recheck current authorization when permissions are revoked, expire, or change between task planning and execution, and whether they make correct execution decisions after obtaining updated authorization evidence. To address the limitations of traditional agent security evaluations, which often rely on single metrics such as violation rates and struggle to distinguish failures in proactive evidence acquisition, evidence utilization, task execution, and genuine security recovery, the paper proposes Counterfactual Invariant Stress Testing, a counterfactual stress-testing protocol based on executable behavioral invariants. The protocol uses five controlled conditions to examine stable authorization, silent revocation, visible revocation, forced rereading, and the corresponding stable control condition. By combining replayable tool traces with final states, it measures fresh authorization reads, actual unauthorized operations, correct decisions, task completion, and security recovery.

Critical strengths
  1. The research problem has important security implications. The paper investigates how agents recheck permissions when authorization states change dynamically. This problem is closely related to time-of-check-to-time-of-use (TOCTOU) risks and the principle of least privilege in agent tool use, making it an important security research problem.

  2. The paper adopts a relatively clear controlled experimental protocol. The paper establishes five experimental conditions that systematically vary authorization states, evidence availability, and rereading requirements, while comparing model behavior under consistent tasks, tool interfaces, and deterministic environments. The 30 synthetic tasks were cross-validated by two reviewers, and a total of 2,250 execution traces were collected across five models. Overall, the paper adopts a relatively clear controlled experimental protocol.

Critical weaknesses
  1. The reliability of cross-model comparisons of security capabilities is insufficient. The paper uniformly limits each response to at most one tool call, each execution to at most eight tool-use steps, and each output to 512 tokens. However, different models have different tool-calling mechanisms and reasoning requirements. In the experiments, Claude produced 173 invalid executions, while Kimi abandoned 171 tasks. Some failures were related to tool-call formatting and output-length restrictions. Although the authors distinguish execution failures from security successes, they do not provide sensitivity analyses of tool-calling restrictions, execution-step limits, or output budgets.

  2. The independent contribution of the evaluation protocol has not been sufficiently validated. The proposed five-condition counterfactual stress-testing protocol can distinguish evidence acquisition, evidence utilization, procedural compliance, and task utility. However, it is closely related to existing research on TOCTOU, dynamic authorization, and agent state verification. Although the paper discusses recent related work, it lacks direct comparisons with existing evaluation protocols or simplified combinations of conditions. Consequently, it does not sufficiently demonstrate what additional diagnostic capabilities are provided by the complete five-condition design.

  3. The core experiments lack sufficient ablation and generalization validation. Although the paper conducts controlled comparisons across five experimental conditions, it lacks systematic ablations of key evaluation factors, including authorization tool interfaces and execution budgets. This makes it difficult to assess the robustness of the main conclusions to changes in experimental configurations. Furthermore, the experiments include only 30 semantically independent synthetic tasks, with just one instance for each combination of authorization failure mechanism and operation type. This is insufficient to fully validate cross-task generalization.

Questions for authors

Q1. The paper limits each response to at most one tool call, each execution to at most eight tool-use steps, and each output to 512 tokens, while some models exhibit large numbers of invalid executions or task interruptions. Could the authors provide sensitivity experiments under different tool-calling rules, execution-step limits, and output budgets? After relaxing these restrictions, do the authorization read rates, violation rates, and security recovery rates of the different models still exhibit consistent trends? How can the authors rule out the influence of differences in models' tool-execution capabilities on the security evaluation results?

Q2. Compared with existing research on TOCTOU, ResidualAuth, and related topics, what additional types of security failures can the proposed five-condition counterfactual testing protocol identify? Could the authors demonstrate the independent necessity of each experimental condition through comparisons with existing evaluation methods or simplified combinations of conditions?

Q3. The current experiments are primarily based on 30 synthetic tasks and lack systematic ablations of key factors such as authorization tool interfaces and execution budgets. Could the authors provide additional controlled experiments to validate the robustness of the main conclusions to changes in experimental configurations? What are the applicability boundaries of the proposed evaluation protocol when dealing with authorization update delays, repeated permission revocations, and complex task workflows in real-world environments? Do the current experiments support conclusions about the dynamic authorization security capabilities of agents more broadly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.