acceptodds
Under review as a conference paper at ICLR 2027

Only What Was Asked: Overcoming Effect Blindness of LLM Agents in Stateful Environments

Abstract

Language-model agents are increasingly operating in real-world, stateful environments, where their actions modify persistent records that others depend on, making unintended changes costly to detect and correct. A run can complete its task while leaving such changes, and official success does not detect them. We identify effect blindness: the information available to a check before execution does not determine whether a candidate action stays within the request, because the request leaves unstated what must remain unchanged or the agent's observations omit records that decide the outcome. To measure the resulting intent deviation and collateral effects, we propose intent-consistent completion (ICC), a metric that requires official success and the satisfaction of independently evaluated forbidden-effect and preservation constraints. We then propose LEDGER, which overcomes effect blindness before execution. LEDGER extracts an effect contract stating what the request authorizes and what must remain unchanged, predicts what the candidate would change, and verifies prior records through direct reads. The contract and the prediction are produced in model contexts that carry none of the executor's history, and code compares them to decide whether the candidate stays within the request. We evaluate LEDGER with three backbones on four settings from AppWorld, AgentDojo, and SWE-bench, against Self-critique and two published guard agents. On the two AppWorld settings, where agents frequently change records the request does not authorize, LEDGER attains the highest ICC, 27.0 and 12.5 points above the strongest baseline, with the lowest or tied-lowest violation rate, and on SWE-bench it attains the highest ICC without any out-of-scope change. The artifacts are available at https://figshare.com/s/bb39c21953502d77c986

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, over-execution, intent consistency, effect prediction

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_Y8CvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at agents that complete a task but change records the user did not ask to change, which the official evaluators fail to notice. The authors argue that pre-execution checks are "effect blind" because the request does not say what must stay unchanged, or because the agent's observations omit records that decide the outcome. The paper proposes intent-consistent completion (ICC), which adds both independently evaluated forbidden-effect and preservation constraints to official success, plus LEDGER, a method that extracts an effect contract along with a record-level effect prediction in contexts isolated from the executor and checks prior records by reading off their receipt identifiers. LEDGER then reconciles these in code, deciding if it should execute, revise or stop. Evaluation uses three backbones on 21 AppWorld-Amazon order tasks, 8 AppWorld-Splitwise continuations, 97 AgentDojo user tasks and 43 SWE-bench tasks, with Self-critique, GuardAgent and AGrail serving as its baselines.

Critical strengths
  • The problem the paper targets is real, but official success hides it, as the paper shows. On AppWorld-Amazon, Self-critique, with the best official success of all methods, has just 28.6% ICC, since its runs drop items from the cart or buy unrelated ones. This makes ICC a good metric in its own right, independent of LEDGER.
  • The explanation that observations can miss records is supported by both an ablation and paired tests. With no verified prior records, 13 of 24 Splitwise runs record one expense twice. On Amazon, the gain over the No check baseline is also nearly entirely one-sided over paired runs. Reading receipts gives a check some information it would otherwise lack, so LEDGER's contribution goes beyond a change to the prompt.
Critical weaknesses
  • For the unstated-preservation source, LEDGER does not overcome the ambiguity formalized in Equation 3; it resolves it with a default, given in Section 2.2 as K "preserves every record the request does not address", said by the Limitations to match the evaluated constraints. None of the evaluated tasks has an unrequested adjacent change that the user would want, so ICC cannot tell a method inferring intent from one applying this convention, and it treats any harmless unrequested change as a violation.
  • The central claim, "overcoming effect blindness" in the title and abstract, is therefore supported only where observations miss records.
  • As far as I can tell, the headline margins over baselines are mainly due to the extracted contract. Contract critic, a free-text reviewer that gets LEDGER's contract without prediction or code reconciliation, isn't significantly different from LEDGER on Amazon ICC, with 11 against 8 discordant pairs, and even gets more ICC runs on Splitwise. Prediction and reconciliation do reduce cart loss further.
  • The margins over the strongest baseline are measured against baselines given no scope statement specific to the request, so they mostly reflect giving the check a request-specific contract.
  • The supporting evidence is also thin: the Splitwise margin comes to just three of 24 runs, and on AgentDojo LEDGER clearly loses to No check in a paired comparison, mostly through abstentions from contracts that omit a requested write, so the general conclusion that a safe check must compare effects with the authorized request rests mainly on the two AppWorld settings.
Questions for authors
  1. How does ICC behave on tasks where an unrequested adjacent change is acceptable, and has a human judged a sample of runs that ICC flags?
  2. Could Contract critic be reported in Table 2 with the same ICC and violation metrics? What about simply telling the executor, in its prompt, to preserve unrelated records?
  3. Would AppWorld's built-in state-diff evaluation already flag the lost cart and wish-list records in the 21 Amazon tasks, and how do your forbidden-effect and preservation constraints differ from its collateral-damage checks?
  4. Would running Progent, Task Shield, or a simple hand-written preservation rule for each adapter on the AppWorld settings show if an LLM-extracted contract with effect prediction is needed over simpler request-derived policies?
  5. What fraction of AgentDojo abstentions would a fix for contracts omitting a requested write recover, and would it change the AppWorld results? Work on out-of-scope actions in benign tasks, such as "Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks", may also help place ICC in context.
Reviewer_xWD1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces effect blindness, that an LLM agent can cuase unintended state changes while completing tasks. It proposes a new metric ICC to measure the intent-consistent task success rate, and a noval framework LEDGER, that can extract effect contract, predict action effect, and verify them before execution. The experimental results show that the method can improve the task reliability, and reduce unauthorized side effects.

Critical strengths
  1. The problem identified by the paper is important and practical in a real agentic system.
  2. The proposed method LEDGER is clear and interpretable, and the separation of intent extraction, effect prediction, and verfication make the decision process more transparent.
  3. The experimental results are sound.
Critical weaknesses
  1. The novalty of the method LEDGER is limited, and mainly a comination of existing ideas.
  2. The generalization of the method is limited, and relies on the prefefined effect definition and env-specific adapters.
  3. LEDGER does not consistentently outperform all the baselines.
Questions for authors

Does the effect blindness only contains two types (intent deviation and collateral effect)? It would be helpful to discuss other potential failure mode.

Reviewer_nMJ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a potential risk in LLM agents, which may modify the environment beyond the users' request because of intent deviation or collateral effects. The authors design an ICC metric to evaluate the percentage of tasks completed successfully without out-of-scope modifications. Afterwards, the authors propose LEDGER, which reconciles an effect contract, predicted record-level effects, and verified prior records in code. Experimental results show that LEDGER achieves the highest ICC on AppWorld and SWE-bench.

However, the proposed framework has limitations in prediction reliability, generalizability and the trade-off between safety and task success. Moreover, the evaluation sets are relatively small to support the effectiveness and robustness of the method. The component-wise ablation is also missing.

Critical strengths
  1. Existing AI agent benchmarks mainly focus on task success rate, with less attention to the side effects of agent executions. This paper identifies effect blindness and proposes a new metric, ICC, to evaluate task success without out-of-scope modifications, which is meaningful.
  2. This paper presents a coherent narrative, with a clear connection between the identified problem, the proposed metric and method, and the evaluation. The writing is well-organized and easy to follow.
Critical weaknesses
  1. Predicting record-level effects with the same model may have limited benefits. If the model doesn't understand the side effects of an API, invoking it again doesn't improve its understanding. Even with a different context, the same model may make the same mistakes. Moreover, if the Predictor doesn't know what is in the shopping cart, for example, it can't accurately predict the outcome of a purchase either.
  2. LEDGER converts natural language expressions into rigid structured representations to enable programmatic comparison. This process can introduce failures. An LLM judge could instead directly compare the natural language descriptions, potentially reducing unnecessary rejections caused by formatting mismatches and eliminating the need for predefined customized vocabularies and normalization. The results in App. C also demonstrate that vocabulary design has a substantial impact on performance. Therefore, replacing code-based reconciliation with a LLM judge seems to be a reasonable alternative.
  3. LEDGER relies on environment-specific schemas, matching rules, and acceptance conditions. As a result, applying it to a new environment requires substantial adaptation.
  4. The Amazon and Splitwise experiments use only small subsets of AppWorld, so does the SWE-bench. How these tasks were selected is unclear. The experimental results are not representative due to the small evaluation sets.
  5. Tab.2 reveals a clear trade-off in LEDGER: while it improves ICC, it may sacrifice task success rate. On Amazon, the improvement in ICC comes at the cost of a lower success rate (-11.1pp). On SWE-bench, ICC only improves by 0.8pp, while the success rate drops by 3.9pp. On AgentDojo, both the success rate and ICC are lower than some baselines.
  6. The paper doesn't provide a component-wise ablation study. LEDGER consists of 3 core components: effect contract extraction, effect prediction and code reconciliation. However, the authors don't independently evaluate the contribution of each component.
  7. Tab.3 only reports the violation rate, without presenting the other metrics used in Tab.2 (success rate and ICC).
Questions for authors
  1. How did the authors select the evaluation sets? Why were these benchmarks selected, and how were the sample sizes determined?
  2. Can the authors provide a component-wise ablation study and analysis?
  3. Can the authors provide the success rate and ICC in Tab.3?
Reviewer_D6FwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

There is a silent failure in our task-based benchmarks, where we don't measure if something has been "broken", but only the parameters that were being measured for success. Authors create a metric that can measure these undesired or harmful changes that don't align to the task, and also a pre-execution task that can outline it as well

Critical strengths
  • Interesting problem, contextualized it well with e.g., self-critique and identifies what the ICC score would be if you also accounted for extraneous actions.
Critical weaknesses
  • Most benefits seem to come from what is allowed. In the appendix, there's a table with "what's allowed" to an ordinary reviewer, and skip the prediction and code checking step. It seems to do as well as the Amazon tasks. What do the prediction and code checking steps add?
  • How were each of the tasks selected from the entire test suite for some of theese benchmarks?
  • If the competing harnesses weren't run with the built in assumption of the penalization of any changes to other parameters, and also got fewer changes to fix their plans, it seems the comparison isn't even. What if you told the agent to not change anythign else in the instruction as well? Does LEDGER also account for changes that would not have made a negative change?
Questions for authors

Please see above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.