acceptodds
Under review as a conference paper at ICLR 2027

ALFRED: Governing Knowing-doing gap in LLM Agents

Abstract

LLM agents tasked with delegated work can state a correct plan yet execute another: they skip the authoritative sources, confirmations, and designated entry points they themselves promised, and pass controllable substitutes off as uncontrollable deliverables. This knowing–doing gap has been repeatedly documented by independent measurements, but measurement alone does not stop it, and unfaithful delivery undermines the evaluations meant to detect it. We present ALFRED, a governance framework that makes an agent's knowing–doing unity enforceable at the moment of delivery. ALFRED closes the loop with four components: selective isolation of roles and information surfaces, undisclosed heterogeneous review, contract-based acceptance at submission time, and deterministic failure routing, with only semantically undecidable cases escalated to the human owner. We operationalize the phenomenon into two independent evaluations: task completion, judged by deliverables and environment end-states against frozen task requirements, and knowing–doing failure, judged by the executor's self-stated plan and delivery commitments against its actual execution path, adjudicated by an independently validated judge. On a frozen 45-task set spanning real engineering sessions, established benchmarks, and a bandit decision task, we compare nine conditions: an ungoverned baseline, four external governance methods under their native protocols, ALFRED, and three mechanism ablations that remove process evidence, active verification, or enforced disposal. [Results sentence to be filled after the experiments…] We will open-source the framework, the measurement protocol, and the task instances.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.