Certified Feasibility Reporting Under Partial Observability: Distinguishing Unreachable from Unresolved in Tool-Using Agents
Abstract
We introduce a mechanically certified evaluation of feasibility reporting: whether a tool-using agent can report what its observations establish about task reachability. The benchmark separates hidden-world reachability from the epistemic status induced by an agent's observation history, and it constructs paired worlds whose reachability differs while every permitted observation available to the agent agrees. A finite-system proposition identifies the regime in which such a pair cannot exist: deterministic monotone resource dynamics, complete Boolean requirement probes, and a budget sufficient to saturate the operation catalogue. The benchmark then breaks that completeness assumption with an opaque resource field while keeping the budget ample. The resulting evaluation separates resource accounting from reasoning about observability. In a matched experiment that holds the model, prompt wording, report menu and family set fixed and moves only the budget and the observation channel, budget-horizon twins are reported correctly on 58/59 runs and opaque-field twins on 28/60 runs (46.7%; 95% CI [34.6%, 59.1%]): a 52-point separation between two conditions whose licensed report is the same word. An ample budget on its own disturbs nothing, with 60/60 where committing is licensed. The same ordering appears across three models in source blocks collected earlier under different prompt revisions (360/360 against 90/300), which we report as descriptive. A controller that preserves an unresolved observation scores 120/120 on the opaque condition, while the ablation that collapses unresolved into absent emits the wrong state on 120/120 of the same runs. The reporting distinction transfers to ten real-execution twin pairs, from 0/590 under the base menu to 589/590 under the disambiguated menu. These results establish feasibility identifiability as a separate evaluation target for tool-using agents.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.