acceptodds
Under review as a conference paper at ICLR 2027

Only What Was Asked: Overcoming Effect Blindness of LLM Agents in Stateful Environments

Abstract

Language-model agents are increasingly operating in real-world, stateful environments, where their actions modify persistent records that others depend on, making unintended changes costly to detect and correct. A run can complete its task while leaving such changes, and official success does not detect them. We identify effect blindness: the information available to a check before execution does not determine whether a candidate action stays within the request, because the request leaves unstated what must remain unchanged or the agent's observations omit records that decide the outcome. To measure the resulting intent deviation and collateral effects, we propose intent-consistent completion (ICC), a metric that requires official success and the satisfaction of independently evaluated forbidden-effect and preservation constraints. We then propose LEDGER, which overcomes effect blindness before execution. LEDGER extracts an effect contract stating what the request authorizes and what must remain unchanged, predicts what the candidate would change, and verifies prior records through direct reads. The contract and the prediction are produced in model contexts that carry none of the executor's history, and code compares them to decide whether the candidate stays within the request. We evaluate LEDGER with three backbones on four settings from AppWorld, AgentDojo, and SWE-bench, against Self-critique and two published guard agents. On the two AppWorld settings, where agents frequently change records the request does not authorize, LEDGER attains the highest ICC, 27.0 and 12.5 points above the strongest baseline, with the lowest or tied-lowest violation rate, and on SWE-bench it attains the highest ICC without any out-of-scope change. The artifacts are available at https://figshare.com/s/bb39c21953502d77c986

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.